6. Configuration Management

Search in book...
Toggle Font Controls
Create new playlist

Name your new playlist

Playlist description (optional)
Sign In

Email address

Password

Forgot Password?

or

Continue with Facebook

Continue with Google
Sign Up

Full Name

Email address

Confirm Email Address

Password

or

Continue with Facebook

Continue with Google

Chapter 6. Configuration Management

By now you have to be thinking to yourself, Wow, this is a lot of configuration data. Yes, you are absolutely right.

It is best to use Puppet, Chef, or another configuration management system to deploy and maintain MCollective. As you proceed through this book, you will be constantly tweaking the MCollective configuration and adding new plugins. Every change will need to be synchronized across servers, yet many servers will also have customized settings. In previous sections we have gone over how to set up MCollective by hand, but across many systems it becomes a lot of work. It’s best to manage your MCollective installation with configuration management.

As the installation documentation on the Puppet Labs website says:

[MCollective] is the textbook example for why you need config management:
It has multiple components that run on many different machines.
It has pieces of global configuration that must be set harmoniously, everywhere.
Most of its settings are identical for machines in a given role (e.g. every server), but some of its settings have per-system differences. This is easy to manage with a template, and incredibly frustrating to manage by hand.
Its configuration will change over time, and the changes affect many many systems at once.
New/updated agents must be deployed to all servers; when a new admin user is introduced, every server must be made aware of their permissions.
In summary, its configuration requirements are strict, and configuration drift will cause it to stop working. Use Puppet.
— http://docs.puppetlabs.com/mcollective/deploy/index.html#best-practices

Tip

For what it is worth, you could just as easily use Chef, Salt, Cfengine, Ansible, or another configuration management tool to install and configure MCollective. We’re going to discuss using Puppet since MCollective and Puppet are both maintained by Puppet Labs, and their integration is something you can get support for.

If you are already heavily invested in a different configuration management tool, there is no need to worry. MCollective does not depend on Puppet. You can either adapt the configuration recipes from this book to your chosen tool, or look to see if someone has already provided a plugin for you.

Installing the Puppet Module

So there are two different ways to install the Puppet module to manage MCollective.

Using `r10k`

If you already use r10k you can get a configuration and Puppetfile from http://github.com/jorhett/learning-mcollective to install the modules for you. As this is not really an MCollective concern, documentation for the r10k setup can be found in Using r10k to install Puppet Modules.

Straight from GitHub

If you don’t use r10k, you can pull the module down from Github like so:

$ git clone git://github.com/jorhett/puppet-mcollective.git
Cloning into 'puppet-mcollective'...
remote: Counting objects: 65, done.
remote: Compressing objects: 100% (56/56), done.
remote: Total 65 (delta 11), reused 59 (delta 5)
Receiving objects: 100% (65/65), 34.83 KiB, done.
Resolving deltas: 100% (11/11), done.

If you have your own Puppet module forge internally, the following optional step will build a module package appropriate for uploading to the forge.

$ cd puppet-mcollective
$ puppet module build
Notice: Building /home/jrhett/src/puppet-mcollective for release
Module built: /home/jrhett/src/puppet-mcollective/pkg/jorhett-mcollective-0.1.0.tar.gz

Or you can move the module directly into your modulepath:

$ mv puppet-mcollective /etc/puppet/modules/mcollective

In the next section you’ll define your local setup in the manifests and/or Hiera data.

Configuring MCollective using Puppet

The following puppet manifest will perform a basic MCollective setup and initialization for you. If you don’t yet use puppet, look at how easy it is to replace the entire Chapter 2 section of the book with these small recipes.

This module contains all of the manifests and templates shown in this book, plus examples of ActionPolicy configurations.

Note

This module could be extended to cover a great many more middleware configurations if it relied on other Forge modules for managing ActiveMQ and RabbitMQ configurations. For the purposes of simple, one-shot setup, I wanted to focus on a simplified configuration that serviced only MCollective clients and servers. If you prefer to manage the middleware configuration yourself, simply don’t use the mcollective::middleware module in this book.

For a simple example, if you wanted to install a machine that provided ActiveMQ and had the server and client installed as well, you would use:

manifests/site.pp:
node default {
  class { '::mcollective':
    hosts           => ['activemq.example.net'],
    client_password => 'Client Password ',
    server_password => 'Server Password',
    psk_key         => 'Pre-Shared-Key',
  }
  include mcollective::server
}
node 'activemq.example.net' inherits default {
  include mcollective::activemq
  include mcollective::client
}

For most nodes on your network you would use only the mcollective and mcollective::server classes. You should put the mcollective::client class on any hosts from which you wish to submit requests.

Hiera Configuration Data

You can supply the following parameters to each of the classes above, or better yet you can simply define all of the settings in Hiera. Following is the most minimal set of parameters you’ll need to supply.

manifests/site.pp:
hiera_include('classes')

hieradata/common.yaml:
classes:
  - mcollective::server

mcollective::hosts:
  - 'activemq.example.net'
mcollective::client_password: 'Client Password '
mcollective::server_password: 'Server Password'
mcollective::psk_key        : 'Pre-Shared-Key'

Note

Hiera is a key/value lookup tool for configuration data, built to let you define node-specific data without repeating yourself.

If you are not familiar with using Hiera, the best reference can be found at Hiera Overview. In particular read carefully Usage with Puppet for how Puppet modules automatically include Hiera data.

There are dozens of parameters available to tweak the default settings. This module can be used to create high-security configurations, or build a large distributed network of brokers. Here is a much more complex example that does the following:

Directs clients to their local broker (depending on the Hiera hierarchy)
Sets up site→site links between the brokers
Uses SSL encryption for all connections
Sets up file logging at debug loglevel
Loads up package, service, and puppet agents at both sites
Installs the client programs only on machines at site1

The Hiera files below break up the configuration between the London and Philadelphia offices. Each office has their own ActiveMQ server. All of the admins are in London so the client packages are loaded there. All other parameters are the same for the MCollective framework.

london.yaml.

mcollective::hosts:
  - 'activemq.london.example.net'

mcollective::plugin::clients:
  package:
    version: 'latest'
  service:
    version: 'latest'
  puppet:
    version: 'latest'

philadelphia.yaml.

mcollective::hosts:
  - 'activemq.philadelphia.example.net'

common.yaml.

mcollective::hosts:
  - 'activemq.london.example.net'
  - 'activemq.philadelphia.example.net'
mcollective::collectives:
  - 'mcollective'
  - 'site1'
  - 'site2'
mcollective::client_password: '<replaceable>Client Password </replaceable>'
mcollective::server_password: '<replaceable>Server Password</replaceable>'
mcollective::broker_password: '<replaceable>Broker Password</replaceable>'
mcollective::psk_key        : '<replaceable>Pre-Shared-Key</replaceable>'

mcollective::connector         : 'activemq'
mcollective::connector_ssl     : true
mcollective::connector_ssl_type: 'anonymous'

# Server configuration
mcollective::server::logger_type: 'file'
mcollective::server::log_level  : 'debug'

# Client configuration
mcollective::client::package_ensure: 'absent'
mcollective::client::unix_group    : 'wheel'

# Middleware configuration
mcollective::middleware::keystore_password  : '<replaceable>Keystore Password</replaceable>'
mcollective::middleware::truststore_password: '<replaceable>Truststore Password</replaceable>'

# Plugins to install
mcollective::plugin::agents:
  package: {}
  service: {}
  puppet :
    version: 'latest'

Don’t worry about understanding this complex example just yet, this is skipping way ahead into chapters much later in the book! We’ll be spending the remainder of this book walking you through different things you can achieve with MCollective, and each and every section will contain Puppet/Hiera parameters you can apply to enable those features.

Note

If you are already deeply involved with Puppet, you might notice that there is a puppetlabs-mcollective module available on the Puppet Forge. We are not using this module for teaching MCollective for several reasons:

If you don’t override them, the setup will use well-known usernames and passwords. A mistype would make your setup vulnerable to attack.
It doesn’t separate client and server permissions which creates a security problem if any server is compromised.
It doesn’t separate client permissions and broker link permissions.
The Puppet Labs module has numerous external dependencies which could be distracting to set up when trying to follow the book.

The module provided in this book allows a simple setup to work immediately, and then for you to add more and more as you read each chapter in the book.

Sharing Facts with Puppet

As discussed in the Filters section above, one of the most flexible ways to match related groups of systems is by facts. The easiest way to get factor facts for mcollective to use is to let Puppet supply them for you. This is much better than the simple cron method we showed you above, as it allows you to add both custom puppet facts and specific puppet variables to the list.

To get this more flexible usage, add the following to your Puppet manifests:

node myhost.example.net {
  include mcollective::server
  include mcollective::facts
}

Or when using Hiera with:

hieradata/common.yaml:
classes:
  - mcollective::server
  - mcollective::facts

Warning

Did you enable fact creation using the cron script we documented in Facts?

If so, you’ll want to remove the /etc/cron.d/facts.sh cron script or Puppet and cron will keep overwriting each other’s results. Puppet has more facts available, and more flexibility in adding new facts from Puppet variables, so this is the better of the two choices.

Installing Agents with Puppet

Here are some example manifests for adding agents to your systems, assuming the agent packages are available in a repository your package system uses. For the first one we show you how to define an explicit version, but this is not necessary.

node nodename {
  mcollective::plugin::agent { 'filemgr':
    $version => '1.0.1-1', # specific version for this
  }
  mcollective::plugin::agent { 'nettest': }
  mcollective::plugin::agent { 'package': }
  mcollective::plugin::agent { 'service': }
  mcollective::plugin::agent { 'puppet':
    version      => latest,
    dependencies => [ Package[ $puppet::client::package_name ] ],
  }
}

As you can see from above, we have set a dependency on another module named puppet::client. On hosts from which you execute the client commands you can likewise install the clients:

node nodename {
  mcollective::plugin::client { 'filemgr': }
  mcollective::plugin::client { 'nettest': }
  mcollective::plugin::client { 'package': }
  mcollective::plugin::client { 'service': }
  mcollective::plugin::client { 'puppet':  }
}

If you use Hiera this can be done even easier. The module we provided looks for agent and client names to be defined in YAML and installs them using the parameter hash supplied. The following Hiera is identical to the puppet policy we defined above.

mcollective::plugin::agents:
  filemgr:
    - version: 1.0.1-1
  nettest: {}
  package: {}
  service: {}
  puppet:
    version     : latest
    dependencies:
      - Package[ %{puppet::client::package_name} ]

mcollective::plugin::clients:
  filemgr: {}
  nettest: {}
  package: {}
  service: {}
  puppet:
    version: latest

Note that the Hiera uses a plural name but the individual invocation uses a single name. Fun with Hiera variables.

..................Content has been hidden....................

You can't read the all page of ebook, please click here login for view all page.

Table of Contents for 6. Configuration Management

Create new playlist

Sign In

Sign Up

Chapter 6. Configuration Management

Tip

Installing the Puppet Module

Using r10k

Straight from GitHub

Configuring MCollective using Puppet

Note

Hiera Configuration Data

Note

Note

Sharing Facts with Puppet

Warning

Installing Agents with Puppet

Table of Contents for
6. Configuration Management

Using `r10k`