Default node statement

If any node statements exist, every node must match a node statement, or the catalog compilation will fail. Therefore, if you utilize even a single node statement, make certain to have a node default statement that will be applied to nonmatching nodes, as shown in the following:

node default {
  # this can be empty
}

Regular expression node statements

Node statements permit the use of regex statements to match groups of nodes. This is commonly used to classify nodes using a semantic hostname pattern. This can be useful for easily adding nodes of the same type without having to create node statements for each new node. The following example shows the code for this:

node /^webd+[13579]/ {
  # odd numbered web servers are Blue pod
}

Because the regex language can be difficult to read, always put a comment in or above a regex explaining what you expect the regex to match.

Because regexes are checked only after a complete name match is not found, create a node statement for a node’s complete name to exclude it from a group that matches a regex.

Partial node match

Node statements permit the use of partial FQDN statements to match nodes. During this step, Puppet will strip the rightmost sections of the name (up to the next period) and then check for a complete match. This is commonly used to classify nodes using a hostname pattern where the domain name could differ, as shown here:

node 'puppet-server' {
  # will match 'puppet-server.example.com', 'puppet-server.example.net', etc
}

As a partial match intends to match against a missing part of the data, these statements are inherently unreadable. Always put a comment in or above the match reminding the reader that it is intended to match multiple domains. Otherwise, it might be either misunderstood as a mistake and fixed or copied when creating another node because the person copying it doesn’t realize the consequences of a partial match.

Node classification with inheritance

The most common inheritance design pattern is to specify a base node, a series of node statements that extend the base with specific services (such as an Apache node, a Tomcat node, a MySQL node), and a node that performs the final service specific configuration tasks, as shown in Example 8-1.

# Base used by numerous profiles
node base {
  include 'ntp'
  include 'security'
}

# Web node profile extends base
node web inherits base {
  include 'apache'
  include 'php'
}

# Database node profile extends base
node db inherits base {
  include 'mysql'
}

# These nodes use existing node groups
node www1.example.com www2.example.com inherits web { }
node db1.example.com db2.example.com inherits db { }

# This node inherits web, but has to add db manually
node www-dev1.example.com inherits web {
  include 'mysql'
}

Even though this example was greatly simplified to keep it on a single page, in practice this often becomes a large sprawling mess of inconsistent updates. Besides becoming difficult to maintain, it has a few major limitations:

• Node definitions do not support multiple inheritance, so it isn’t possible to create web and db profiles and include them both.

• Inheritance also affects variable lookup scope, causing node assignment changes to affect code evaluation unpredictably.

There is nothing worse than unpredictable variable values and resource defaults that leak in unpredictable manners. Seriously, you’d be staring at the same code applied to two identical nodes trying to figure out why they got different catalogs. It was always a rabbit-chasing exercise and never lent itself to being fixed permanently. That’s why this approach has been abandoned.

Node classification using roles and profiles

The roles and profiles design pattern allows you to assign multiple profiles to the node without changing the code’s scope or parent module. You can easily convert any node inheritence design to the roles and profiles pattern by following these steps:

1. Convert each node inheritence statement (something named inherits name) to a profile.

2. Convert each node matching statement (something that matched a given certname) to a role.

3. Replace the node matching statement with a role selection.

So, the first thing we do is convert each inherited node statement to a profile:

class profiles::base {
  include 'security'
  include 'ntp'
}

class profiles::web {
  include 'apache'
  include 'php'
}

class profiles::db {
  include 'mysql'
}

Next, we convert each matching node statement to a role:

class roles::webserver {
  include 'profiles::base'
  include 'profiles::web'
}

class roles::devserver {
  include 'profiles::base'
  include 'profiles::web'
  include 'profiles::db'
}

With this approach you can simply apply roles::devserver to a node, making use of any other profiles. There’s no linear map of single inclusion to maintain or work around. Each role implements a site-specific configuration by declaratively including the relevant profiles.

Finally, we use node statements to assign roles to the nodes to complete the conversion:

node 'www1.example.com', 'www2.example.com' {
  inlude 'roles::webserver'
}

node 'www-dev1.example.com' {
  inlude 'roles::devserver'
}

In several easy steps, you’ve just converted the old node inheritence pattern into the modern roles and profiles best practice. Further evolutions like removing kitchen-sink base profiles will be easy to accomplish after you finish this migration.

Mitigating the risk by using trusted facts

If you carefully control how node certificates are signed, you can employ the example shown in Example 6-2 to utilize a role stored in the node’s signed Puppet certificate. This would be a very effective way to decentralize role assignment without an ENC.

If you make use of certificate authorities with autosign enabled, you’ll gain nothing from this change. Anybody can generate a certificate with any role they want and get it signed.

Customizing Hiera hierarchy using facts

This topic was already covered in depth in Chapter 6, but let’s refer to the following example that uses both default and custom facts:

:hierarchy:
  - name: "Node-specific data"
    path: "fqdn/%{facts.fqdn}.yaml"

  - name: "Role data"
    path: "roles/%{facts.role}.yaml"

  - name: "Site data"
    path: "sites/%{facts.site}.yaml"

  - name: "OS-specific data"
    path: "os/%{facts.os.family}.yaml"

Listing classes in Hiera data

We can add classes to the preceding example at multiple levels of the hierarchy.

1. Classes for the specific node in the fqdn/ data file

2. Classes for the node’s role (from a custom fact) in the roles/ data file

3. Classes for the node’s site (from a custom fact) in the sites/ data file

4. Classes for every node of a given OS family in the os/ data file

Following is the basic role definition in YAML format:

---
# classes for the webserver role
classes:
  - 'profiles::base'
  - 'profiles::web'

Declaring classes from Hiera data

The typical approach is to use the include() function call on the results of a Hiera lookup of classes, like so:

lookup('classes', Array, 'unique', []).include

Always supply a default value in case the lookup() call fails.

Class list (node role)

The most common purpose of an ENC is to assign classes to a node. As the usage of ENCs has evolved with practice, the recommended approach is to assign a single role class as described in Chapter 7; however, a classifier can also assign individual classes a-la-carte if so desired.

Node parameters

There are often a number of node-specific properties that cannot be provided by node facts because they might not be available or knowable by the node. As with node facts, you can use these parameters for Hiera hierarchy and Puppet catalog customization. The more commonly used values include the following:

• Tier, such as whether it’s a prod or dev node

• Site, the physical or virtual location of the node

• The state of the node for deployment purposes

• Contact information for node maintenance

You’ll notice that most of this information is tied to inventory or resource management.

Puppet environment

The node classifier can override the Puppet environment requested by the node. This allows Puppet Enterprise or infrastructure management tools like Foreman to dynamically assign groups of nodes without making configuration changes on each node.

There won’t be a best-practice choice of whether to assign environments from the ENC because this will differ greatly depending on your infrastructure and tools available. Concerns about environment management and how to handle testing are discussed in depth in Chapter 9.

Puppet Enterprise console

The Puppet Enterprise product offers the following capabilities:

• Provides both an ENC and a Puppet-supported console for Puppet node management. This is one of the prime differentiators between Puppet Community and Enterprise versions.

• Authorizes users based on granular RBAC that integrates cleanly with Active Directory and Lightweight Directory Access Protocol (LDAP) authentication.

• Uses a group model for node classification: classes and node parameters can be assigned to every node, to a group of nodes, to a subgroup, or directly to a single node. Additionally, a rules-based node classifier enables classification based on node facts or exported resources to assign classes.

• Generates a console overview of nodes in the infrastructure, including last run status and inventory. Node events can be inspected. Reports can be generated based on data from the Puppet convergence reports.

• Provides a robust RESTful API for managing nodes, making it fairly simple to integrate with external automation systems and node inventories.

• Includes a GUI for ad hoc execution and exploration of nodes using Puppet. The Puppet agent on a node can be invoked at any time from the Console.

With Puppet version 3, there were significant differences between the way Puppet Enterprise and Open Source Puppet were deployed to your infrastructure. Puppet 4 Unified Packaging uses the same basic installation layout as Puppet Enterprise, making conversion from and use of open source code much simpler.

We encourage you to consider licensing Puppet Enterprise. It’s a great product, and purchasing a license helps to support ongoing development of Puppet for both Enterprise and Open Source users.

Puppet Dashboard

The Puppet Dashboard was an early proof of concept for what evolved to become the Puppet Enterprise console. Development was discontinued years ago, and community support has stopped as the ENC features no longer matched well with Puppet feature development. You should consider one of the other console solutions instead.

Foreman

Foreman is a complete lifecycle management tool for physical and virtual servers. In addition to providing an external node classifier for Puppet, Foreman provides inventory management, physical machine provisioning (PXE), virtual machine provisioning, cloud provisioning, and reporting services.

Its node classification system is robust. Foreman supplies a group-based class and node parameter inheritance model, similar to the model offered by Puppet Enterprise. Classes are autodiscovered but you can hide them. Parameterized class assignment is supported. You can pass array and hash data structures to the node. Unlike Puppet Enterprise, Foreman does not currently provide a rules-based classifier.

Because Puppet node management is just a part of what Foreman does, configuration is somewhat more complex than Puppet Enterprise or any Puppet-oriented ENC solutions. Foreman has a strong focus on provisioning, and provides PXEboot automation, DNS management, DHCP management, and an IPAM solution. As a result, its RESTful API is somewhat heavier than other solutions, though it is well documented and straightforward to use. Even though most of the components are optional, a full-featured environment management solution needs to integrate with your hypervisors, cloud providers, DNS, DHCP, TFTP, and Puppet infrastructure.

Foreman is a RedHat–supported project that works with community contributors, much as Puppet does. Upstream, it’s incorporated into Katello and sold as part of the Red Hat Satellite product.

Cobbler

Cobbler was originally used as part of Red Hat Satellite provisioning services but was replaced by the more full-featured Foreman. Cobbler is a good choice if you’re looking for a simple, lightweight provisioning system and ENC. It is not as full-featured as the other solutions, but it does provide platform-, group-, and node-based classification. You must supplement Cobbler with another console for reporting.

Cobbler’s strength is that it’s fast, well understood, and very simple to set up. Cobbler integrates with DNS and DHCP by writing templates for each. Its APIs are fairly straightforward, and KOAN provides a simple method to provision KVM and Xen virtual machines. It maintains a node inventory and provides an ENC for Puppet, but does not offer reporting features.

It stores node data in a simple JSON document, and supports a number of replication topologies out of the box. Using replication, you can easily scale out your DHCP and DNS infrastructure, as well. The storage backend is pluggable, and MongoDB, CouchDB, and MySQL are also supported out of the box.

Cobbler is a good choice if you need a simple provisioning solution for a small environment without node reporting requirements.

Build your own ENC

You can use virtually anything that stores node data as an ENC. The Puppet node terminus is fairly simple to extend using Puppet’s plugin architecture, but if you’re uncomfortable with that approach you can simply write an external node classifier to be invoked by the exec terminus.

Although we generally advise against creating anything from scratch when a suitable solution is publicly available, node classification logic can be incredibly site specific. The creation of an external classifier to use an existing configuration management database or inventory management database could be done fairly easily.

If you do decide to generate your own, be sure to consider the demand load from Puppet and reliability requirements. An ENC lookup failure will prevent the node catalog from being built. Here’s some things to consider in an ENC design:

• The data source should be designed for availability not less than the Puppet servers.

• The data source must be capable of handling spiky bursts of traffic such as an orchestrated push, or every node restarting at the same time.

• Are read-only slaves available to handle traffic while maintenance is done on the database?

If you do implement your own ENC, it’s a good idea to review Puppet’s internal indirectors and termini and consider how your classifier can use them. In particular, the facts indirector and the report indirector can be queried for information about the nodes, such as the agent_specified_environment fact available on newer releases of Puppet.

Your console can also perform queries against PuppetDB, which makes a huge amount of node data available via the API. If your console relies on PuppetDB, be sure that it does so in a way that meets your availability and failover requirements because this approach can potentially create another point of failure.