Environment Configuration File

Within each environment directory, create an environment.conf file. This has the same configuration format as other Puppet configuration files, but does not support sections. The following settings are allowed in this file:

Environment configuration files are parsed after the main configuration, so Puppet configuration options can be used within the environment configuration. The following settings are allowed in this file:

config_version

A script that can be run to provide a configuration version number. If this is not specified, the current epoch time will be used as a configuration version.

manifest

The manifest file or directory that will be read before catalogs are built for this environment. It defaults to the manifests/ directory within the environment.

modulepath

A list of directories to load modules from. Defaults to the modules/ directory within the environment, followed by $basemodulepath. The module path will be searched until the first location is found that contains a directory with that module name. Modules that exist in multiple directories are never merged.

environment_timeout

This sets a per-environment cache timeout. This can be useful for testing, but will impact performance on production instances.

Environment configuration files are parsed after the main configuration, so Puppet configuration options can be interpolated within the environment configuration.

Here is an example environment configuration file. This file shows the default values, as if the configuration file did not exist. Any relative paths within the environment configuration file are interpreted relative to the environment directory:

# Defaults to $environmentpath/$environment/manifests
manifest = manifests

# Defaults to $environmentpath/$environment/modules:$basemodulepath
modulepath = modules:$basemodulepath

# Cache until API call to clear cache is received
environment_timeout = unlimited

# A script to supply a configuration version
#config_version = (a script)

You can review a specific environment’s settings using the following command:

$ puppet config print --environment environment

It is entirely possible (albeit unintuitive) for you to use a different environment path for puppet apply versus puppet agent when using a server. You do so by changing environmentpath in the [agent] section of the Puppet configuration file. You can check the agent’s settings like so:

$ puppet config print --section agent --environment environment

Choosing a Manifest Path

The manifest path can point to a specific file or a directory. If the target is a directory, every manifest .pp file in the directory will be read in alphabetical order. It can also point to a specific file, in which case only that file will be read. It defaults to the manifests/ directory within the environment directory.

In older versions of Puppet, it was common or necessary to assign classes to nodes in a manifests/site.pp file. While still supported, it is no longer a required practice. Assign classes to nodes using Hiera data or with a node terminus (ENC).

The global configuration setting disable_per_environment_manifest overrides the environment setting for manifest. If it’s set to true, then the target of global configuration option default_manifest is used regardless of the client environment. This can be used to centralize resource defaults across all environments.

Disabling the Environment Cache

It is possible to disable Puppet Server’s caching of the environment data for development and testing. Changes will be visible immediately for successive catalog compilations, but has a negative performance impact that makes it unsuitable for high-performance environments. An inconsistent catalog may be returned if new code is deployed at the same time as a catalog is built.

Use a value of 0 (no caching) for environments undergoing active development with a single client, such as a temporary bugfix or dynamic test environment. Stable environments should use a value of unlimited.

Removing Environment Paths from Global Data

In order versions of Puppet there was only a single Hiera configuration file, In order to use environment-specific data sources, it was necessary to interpolate the $environment variable into the data path, like so:

# old Hiera v3 format
:yaml:
  :datadir: /etc/puppetlabs/code/environments/%{::environment}/hieradata
:json:
  :datadir: /etc/puppetlabs/code/environments/%{::environment}/hieradata

A frequently asked question is how to define a separate Hiera hierarchy per environment. This wasn’t possible at the time. From Puppet 4.4 onward, it is possible to supply a hiera.yaml file in each environment, allowing a custom Hiera hierarchy for each environment.

The global Hiera configuration should not source data from an environment directory. Search for and remove any environment directory paths in the global Hiera configuration file, to avoid redundant lookups in the same data sources. This is a simple hierarchy appropriate for the global Hiera configuration:

hierarchy:
  - name: "Global Data"
    datadir: /etc/puppetlabs/code/hieradata
    path: "common.yaml"
    data_hash: yaml_data

Utilizing Hiera Hierarchy in Environments

As mentioned in “Configuring Hiera”, data from the environment is queried after the global data. Environment-specific data sources can be especially useful when environments are used by distinct teams with their own data management practices.

Each environment contains its own hiera.yaml file, which allows each environment to have a unique lookup hierarchy. Puppet 4.3 through 4.8 used a Hiera v4 configuration file in the environment. Puppet 4.9 and higher uses the Hiera version 5 file format for all data sources: global, environment, and module. This file format was described in “Configuring Hiera”.

Create a hiera.yaml file within the environment. The following example shows a minimal use case:

---
version: 5
defaults:        # for any hierarchy level without these keys
  datadir: data  # directory name inside the environment
  data_hash: yaml_data

hierarchy:
  - name: "Hostname YAML"
    path: "hostname/%{trusted.hostname}.yaml"

  - name: "Common Environment Data"
    path: "common.yaml"

Create the datadir directory data/ within the environment, and populate environment-specific Hiera data files there.

Using Custom Backends in Environments

As discussed in “Configuring Hiera”, it is possible to define a custom backend for Hiera for use in the environment data hierarchy. This is done in exactly the same way shown in “Backend configuration”.

You can configure Hiera to query a Puppet function at any level in the lookup hierarchy. Specify the backend with lookup_key for the key, and the function name as the value.

  - name: 'Environment-specific user account'
    lookup_key: environment::my_local_user

Puppet 4.9 introduced the new Hiera v5 hierarchy and a new custom backend architecture. This is evolving quickly. Track the latest changes at “Hiera: How custom backends work”.

Puppet::Functions.create_function(:'environment::local_user') do
  def local_user()
    # the following is just example code to be replaced
    # Return a hash with parameter name to value mapping for the user class
    return {
      'user::id'   => 'Default for parameter $id',
      'user::name' => 'Default for parameter $name',
    }
  end
end

Querying Environment Data from Hiera

No changes to the Puppet manifests are required to enable environment Hiera data. Once you have created the hiera.yaml file, lookups not satisfied by the global data source will automatically query the environment data. Removing the Hiera configuration file will disable lookups of environment data.

Promoting Change Through Layers

The most common development strategy is to develop changes to a module on a feature branch. Merge that module’s changes to master for testing. Then tag a new branch or merge to an existing release branch to push the changes into production.

It is easy to support this development workflow with a fixed set of environments. The environments would be structured and used as shown in Table 29-1.

While the branch names and the steps involved differ slightly from team to team, this is a well-known strategy that many development teams use, and every code management tool is capable of implementing. In a small environment, or with a small DevOps team, this might be sufficient.

A small variation, outlined in Table 29-2, can support many more engineers, including feature and hotfix branches working on the same module at the same time. A new environment is created for each feature branch. After testing is complete, the change is merged to test for testing against a production-like environment. The feature branch environment is destroyed when the change is merged to master and pushed to production.

You can utilize any method that supports your workflow. Tables 29-1 and 29-2 show two basic and common structures.

Whichever strategy you use, with a small set of environments that track a single common master branch, you can use Puppet with “out of the box” defaults.

Solving One-Off Problems Using Environments

It is very common to have a test environment where changes to modules are tried out. That’s exactly what we created in Chapter 10. This might serve your needs for testing new modules if you’re part of a small team.

It is often necessary to test changes in isolation, avoiding other modules in a different stage of development, when you’re fixing a production problem. In this scenario, it’s a good practice to use a one-off test environment.

Simply go into the $codedir/environments/ directory and create a new environment named for the problem. I tend to use the ticket number of the problem I’m working on, but any valid environment name, such as your login name or even annoying_bug/, will work.

There are two strategies for doing this. One is to check out everything from the production environment into this environment’s diretory. Another is to set modulepath to include the production environment’s modules directory ($environmentpath/production/modules/), and then check out only the problematic module into this environment’s modules/ directory. Here is an environment.conf file that implements that strategy:

environment_timeout = 0
manifest = $environmentpath/production/manifests
modulepath = ./modules:$environmentpath/production/modules:$basemodulepath

Check out only the modules related to this bug in the testing environment’s module directory. When looking for modules, the server will find the module you are working on first. All other modules will be found in the production module directory. This allows you to work on a quick fix without pulling in dozens or hundreds of unrelated modules.

When the problem is solved and merged back to the production code base, rm -r the one-off environment and go grab a cup of your favorite poison.

Supporting Diverse Teams with Environments

Environments allow diverse teams to segregate their code and data according to their own standards. This allows for the freedom of self-direction, while also letting both teams share common code and organization data.

To give each team freedom with its own modules and data, start with the following configuration and adjust as necessary for your teams:

1. Put shared modules in a directory specified in basemodulepath, usually /etc/puppetlabs/code/modules/.
2. Put global Hiera data in the /etc/puppetlabs/code/hieradata directory.
3. Put team modules in the /etc/puppetlabs/code/environments/team/modules directory.
4. Enable hiera as the environment data provider in /etc/puppetlabs/code/environments/<team>/environment.conf.
5. Define a team-specific Hiera hierarchy in /etc/puppetlabs/code/environments/<team>/hiera.yaml.
6. Put team data in the /etc/puppetlabs/code/environments/<team>/hieradata/ directory.

The per-environment Hiera hierarchy should contain only data specific to this team’s nodes. There’s no performance impact because even if Puppet doesn’t cache the environment data, this data is used for every catalog build so these files always reside in the filesystem cache.

Listing Modules in the Puppetfile

Within each environment directory, create a file named Puppetfile. Following is an example that should be pretty easy to understand:

# Puppet forge
forge "http://forge.puppetlabs.com"
moduledir = 'modules'   # relative path to environment (default)

# Puppet Forge modules
mod "puppetlabs/inifile", "1.4.1"   # get a specific version
mod "puppetlabs/stdlib"             # get latest, don't update thereafter
mod "jorhett/mcollective", :latest  # update to latest version every time

# track master from GitHub
mod 'puppet-systemstd',
   :git => 'https://github.com/jorhett/puppet-systemstd.git'

# Get a specific release from GitHub
mod 'puppetlabs-strings',
   :git     => 'https://github.com/puppetlabs/puppetlabs-strings.git',
   #:branch => 'yard-dev'           # an alternate branch

   # Define which version to install using one of the following
   :ref    => '0.2.0'               # a specific version
   #:tag    => '0.1.1'              # or specific tag
   #:commit => '346832a5f88a0ec43d' # or specific commit

As you can see, this format lets you control the source and the version on a module-by-module basis. Development environments might track the master branch to get the latest module updates, whereas production environments can lock in to release versions. This allows each environment individual levels of specification and control.

You can validate the Puppetfile syntax using the puppetfile check command. This only works when you are in the environment directory containing the Puppetfile and modules/ subdirectory:

[vagrant@client ~]$ cd /etc/puppetlabs/code/environments/test
[vagrant@client test]$ r10k puppetfile check
Syntax OK

r10k can remove any module not specified in the Puppetfile. This can be especially useful after a major update has been merged, where some modules are no longer necessary:

[vagrant@client ~]$ cd /etc/puppetlabs/code/environments/test
[vagrant@client test]$ r10k puppetfile purge -v
INFO -> Removing unmanaged path
  /etc/puppetlabs/code/environments/test/modules/now-obsolete

To prevent r10k from wiping out a module you are building, you can add an entry to the Puppetfile that tells r10k to ignore it:

# still hacking on this
mod 'my-test', :local => true

Creating a Control Repository

Create a new repository to contain a map of environments in use. This is called a control repository.

Add the following files from your production environment:

• environment.conf
• A Puppetfile
• The manifests/ directory and everything in it
• The hieradata/ directory (optional)

If you want to import from your existing setup, the process could look something like this:

[vagrant@client ~]$ cd /etc/puppetlabs/code/environments/production
[vagrant@client production]$ git init
[vagrant@client production]$ git checkout -b production
Switched to a new branch 'production'
[vagrant@client production]$ git add environment.conf
[vagrant@client production]$ git add Puppetfile
[vagrant@client production]$ git add manifests
[vagrant@client production]$ git commit -m "production environment"
[production (root-commit) 7204191] production environment
 3 files changed, 71 insertions(+)
 create mode 100644 environment.conf
[vagrant@client production]$ git remote add origin 
    https://github.com/example/control-repo.git
[vagrant@client production]$ git push --set-upstream origin production
Counting objects: 3, done.
Compressing objects: 100% (2/2), done.
Writing objects: 100% (3/3), 657 bytes | 0 bytes/s, done.
Total 3 (delta 0), reused 0 (delta 0)
To https://github.com/jorhett/control-repo-test.git
 * [new branch]      production -> production
Branch production set up to track remote branch production from origin.

If you want to start from an existing example, clone the Puppet Labs control repository template.

Configuring r10k Sources

You’ll need to create an r10k configuration file at /etc/puppetlabs/r10k/r10.yaml. You can install one from the learning repository we’ve been using with this command:

$ mkdir /etc/puppetlabs/r10k
$ cp /vagrant/etc-puppet/r10k.yaml /etc/puppetlabs/r10k/

Edit the file to list your control repository as a source. The file should look like this:

# The location to use for storing cached Git repos
cachedir: '/var/cache/r10k'

# A list of repositories to create
sources:
  # Clone the Git repository and instantiate an environment per branch
  example:
    basedir: '/etc/puppetlabs/code/environments'
    remote: '[email protected]:jorhett/learning-mcollective'
    prefix: false

# An optional command to be run after deployment
#postrun: ['/path/to/command','--opt1','arg1','arg2']

You can list multiple sources for teams with their own control repositories. Enable the prefix option if the same branch name exists in multiple sources. As almost every control repo has the production branch, this will almost always be necessary:

sources:
  storefront:
    basedir: '/etc/puppetlabs/code/environments'
    remote: '[email protected]:example/storefront-control'
    prefix: true

  warehouse:
    basedir: '/etc/puppetlabs/code/environments'
    remote: '[email protected]:example/warehouse-control'
    prefix: true

Adding New Environments

Create a branch of this repository for each environment in the $codedir/environments directory, as shown here:

[vagrant@client ~]$ cd /etc/puppetlabs/code/environments
[vagrant@client environments]$ git clone production test
[vagrant@client production]$ cd test
[vagrant@client test]$ git checkout -b test

If the environment directory already exists inside the specified basedir, the clone command will fail. You may want to rename the existing environment directory, then move the files to the newly created repository:

[vagrant@client production]$ mv test test.backup
[vagrant@client environments]$ git clone production test
[vagrant@client production]$ cd test
[vagrant@client test]$ git checkout -b test
[vagrant@client test]$ mv ../test.backup/* ./

Make any changes that are specific to the test environment, then commit the repository back to the control repo with the following commands:

[vagrant@client production]$ git add changed files
[vagrant@client production]$ git commit -m "adding test environment"
[vagrant@client production]$ git push --set-upstream origin test

Follow this process any time you wish to create a new environment. No changes to the r10k configuration are necessary, as each branch is automatically checked out into an environment path of the same name.

Populating a New Installation

Now that you have configured everything, use the display command to see what r10k would do for you:

[vagrant@puppetmaster environments]$ r10k deploy display -v
example (/etc/puppetlabs/code/environments)
    - test
        modules:
            - inifile
            - stdlib
            - mcollective
            - systemstd
            - strings
    - production
        modules:
            - inifile
            - stdlib
            - mcollective
            - systemstd
            - strings

That looks exactly like what we have configured. Now you can either move to another server that doesn’t have this environment data, or you can move it aside:

[vagrant@client ~]$ cd /etc/puppetlabs/code
[vagrant@client code]$ mv environments environments.backup

Now redeploy your environments using r10k, as shown here:

[vagrant@client ~]$ r10k deploy environment -p -v
Deploying environment /etc/puppetlabs/code/environments/test
Deploying module /etc/puppetlabs/code/environments/test/modules/inifile
Deploying module /etc/puppetlabs/code/environments/test/modules/stdlib
Deploying module /etc/puppetlabs/code/environments/test/modules/mcollective
Deploying module /etc/puppetlabs/code/environments/test/modules/systemstd
Deploying module /etc/puppetlabs/code/environments/test/modules/strings
Deploying environment /etc/puppetlabs/code/environments/production
Deploying module /etc/puppetlabs/code/environments/production/modules/inifile
Deploying module /etc/puppetlabs/code/environments/production/modules/stdlib
Deploying module /etc/puppetlabs/code/environments/production/modules/mcollective
Deploying module /etc/puppetlabs/code/environments/production/modules/systemstd
Deploying module /etc/puppetlabs/code/environments/production/modules/strings

It’s that easy to deploy your environments on a Puppet server or on a Vagrant instance for testing purposes. Anyone with access to the control repo can re-create the known state of every environment in a minute. You can use this approach for building new Puppet servers, staging infrastructure, or development setups on your laptop. Seriously, it doesn’t get easier than this.

When a module is added or upgraded, the changes can be tested in one environment and then merged from the development branch to production.

Updating a Single Environment

To check out or update the files in an environment (environment.conf, Puppetfile, modules, etc.), use this command:

[vagrant@puppetmaster ~]$ r10k deploy environment test -v

To update the modules specified by an environment’s Puppetfile, use this command:

[vagrant@puppetmaster ~]$ r10k deploy environment test --puppetfile -v

You can update a specific module in all environments, or just one environment:

[vagrant@puppetmaster ~]$ r10k deploy module stdlib -v
[vagrant@puppetmaster ~]$ r10k deploy module -e test mcollective -v

When you are comfortable using r10k, you may wish to utilize r10k commands within Git post-commit hooks, in Jenkins jobs, or through MCollective to automatically push out changes to environments.

Replicating Hiera Data

When using environment-specific Hiera data, you’ll want to deploy the Hiera data along with the modules. In that situation, add the hiera.yaml and hieradata/ directory to the control repo. This is a common strategy when the same people edit the code and the data.

When the data in every environment is similar, an alternative strategy is to place all Hiera data within its own repo, and utilize r10k to install the data separately from the code. This is useful when you have different permissions or deployment processes for data changes and code changes.

To enable automatic data checkout, add the hiera Hepo as a source to the r10k configuration file /etc/puppetlabs/r10k/r10.yaml:

sources:
  control:
    basedir: '/etc/puppetlabs/code/environments'
    remote: '[email protected]:example/control'
    prefix: false

  hiera:
    basedir: '/etc/puppetlabs/code/hieradata'
    remote: '[email protected]:example/hieradata'
    prefix: false

Then add data_provider = hiera to environment.conf, or add the environment directory to the global data provider by using the $::environment variable in /etc/puppetlabs/code/hiera.yaml:

:yaml:
  :datadir: /etc/puppetlabs/code/environments/%{::environment}/hieradata