MCollective provides a framework for server orchestration or parallel job execution. It is commonly used to synchronize changes across clusters of servers in near realtime. It is not entirely inaccurate to imagine the classic marionette controller with puppets dancing on strings. (Yes it’s a pun, but it is more apt than you may realize.)

MCollective is an adjunct tool in your toolbox that cooperates and enhances the capabilities of configuration management tools like Puppet, Chef, and Salt. Whereas these tools ensure configuration consistency, MCollective provides orchestration for parallel action.

Let’s talk about the difference between MCollective and using Chef, Puppet, Capistrano, Salt Overstack, or hand-built tools for orchestration. You may have used or built some home-built command-and-control systems before. You may have built or improved one yourself. And you know their limitations well - any or all of the following:

MCollective differs from these tools in a wide variety of ways:

Let’s talk about how MCollective’s design makes this possible.

All of us have spent time trying to make something happen at the same time on a number of systems. How many times have you done something like this?

$ for host in bunch of hosts
  do
     scp config-file $host:/some/path
     ssh $host "service apache restart"
  done

You tell yourself that this is happening all at once, but you know better. Even with more complex parallel execution processors like pssh, ClusterSSH, or Fabric the sequential ordering ensures a delay that creates drift between hosts.

If you have more than a few hosts, the output from the commands drifts off the screen. Did you notice that the 15th host command failed? The better parallel SSH processors will keep the output from each host, but you need to examine it for errors. None of them can identify an error in the middle of a multi-command sequence and bring it to your attention.

When it’s time to do something on a lot of hosts, you want it to happen fast, and you need to know that it succeeded.

MCollective was designed from the ground up to achieve true parallel execution with consistent and repeatable results. MCollective avoids the use of a centralized master for command and control, thus avoiding centralized resource problems. It also doesn’t reach out to the clients in an ordered loop, thus avoiding drift between each of the systems.

MCollective uses Publish-Subscribe Middleware to transport requests to servers. Any node you want to control by MCollective is an MCollective server running mcollectived. Any system which has the mco command installed is an MCollective client which can issue commands.

On each server mcollectived registers with the middleware broker and remains in a listening or IDLE state. Whenever a client sends a request to the middleware, each server receives and evaluates the request immediately and independently. If mcollectived has an agent installed capable of process the request, it executes it immediately and sends back the results. In this model you can have a command execute on tens, hundreds, or thousands of hosts at exactly the same time. This Publish-Subscribe infrastructure delivers a scalable and fast parallel execution environment.

Figure 1-1. One-to-Many Publishing Model used by MCollective

Now you might be thinking to yourself, “What if I only want the command executed on a subset of hosts?” MCollective provides a rich language for describing which systems should execute the commands. You can send filters based on hostname, operating system, packages installed, processes running, and many other criteria. Best of all, new criteria custom to your environment will be available when you create your own agent.

As we have discussed above, MCollective provides a well-designed infrastructure for orchestrating changes on large numbers of systems simultaneously. Here are a few reasons to use it:

While MCollective plays very well with configuration management systems, it works above and outside of them. I’ve used MCollective to control and manage systems in hundreds of locations around the world without any configuration management available. I’ve seen MCollective used for multi-continent distributed data collection.

Don’t get tied up thinking of the control MCollective provides you as only puppets dancing on your strings. Consider a fishing model where the marionette holds the strings cautiously, waiting for the strings to go taut. I’ve built auto-healing components which listen passively to server inputs and take action to correct a problem without any human involvement.

There are far more ways to use MCollective than I can make marionette and string metaphors for. After reading this book you’ll likely have thought of a way to use it even the developers didn’t imagine. You’ll find that MCollective’s framework not only supports but encourages creativity.

Some sites don’t succeed at using MCollective.

Wow. You may be surprised to a statement this strong at the front of this book. However I have found examining sites which fail with MCollective to be instructive in how to succeed with MCollective. So we’re going to evaluate some reasons I have seen MCollective not be widely deployed at sites.

Posting to the help channels with specific information like this allows people to quickly determine if they can help you, and whether they have seen the problem before. Even a busy person might be able to point you in the right direction.

In summary, MCollective provides a flexible framework for server orchestration. If it isn’t doing what you expect, read through this book and see if your question is already answered. Reach out to the support resources provided in the book with specific questions about what you are trying to accomplish. Other people may have solved problems just like yours.

Developing new functionality with MCollective is a creative endeavor. If nothing available today meets your needs, this book provides you with the technical bits necessary to create your own plugins. When you are done reading this book, you’ll have all of the tools at your disposal. You’ll only fail if you don’t reach out and use them.

As we proceed, this book will show you how MCollective can help you do more, do it faster, and yet more precisely than ever before. You’ll learn how to extend MCollective to meet your specific needs:

By the time you finish this book you will understand not just how powerful MCollective is, but you’ll know exactly how MCollective works. You’ll have the knowledge and understanding to debug problems within any part of the infrastructure. You’ll know what to tune as your collective grows. You’ll have a resource to return to as your knowledge and experience expands.

Let’s get moving. Your servers are marionettes waiting to dance for you. Time for you to take hold of the strings.