Node-provided facts

The most common interpolation is the use of node-provided facts to select the data source.

:hierarchy:
  - name: "Per-node data"
    path: "nodes/%{facts.hostname}.yaml"

The hierarchy level shown in this example will search a filename specific to the hostname provided by the node.

Use trusted facts when available

Using node facts is sufficient and necessary for most use cases, but it can be a security concern. You should prefer any fact available in the $trusted hash over node-supplied facts. There are few trusted facts, but they cover most essential use cases.

Example 6-2 demonstrates the use of the safer server-validated node certificate name.

:hierarchy:
  - name: "Per-node data"
    path: "nodes/%{trusted.certname}.yaml"

Any custom attributes from the node’s signed certificate are available in the $trusted['extensions'] hash. For example, if the certificate was built to contain a customer name you can retrieve this and use it in the hierarchy as shown in the following code:

:hierarchy:
  - name: "Per-customer data"
    path: "customers/%{trusted.extensions.customer}.yaml"

To learn more about storing custom attributes in the node’s certificate, refer to the documentation.

Remove environment interpolation

Hiera v3 (used in Puppet 3) had only a single layer, and thus only a single hierarchy. This led many people to develop complex environment interpolation in the global hierarchy to access environment and module data. Now that Hiera has separate layers for environment and module data, we can remove that unnecessary complexity.

Label node classifier-provided data

The hierarchy can and often should make use of data provided by a node classifier such as Puppet Enterprise Console or Foreman. Which values are provided by the ENC is specific to the implementation. Unless labeled clearly, they are indistinguishable from variables set in Puppet code.

To make the data source apparent to the viewer, place the ENC-provided data in its own bespoke hash, as shown in the following code:

:hierarchy:
  - name: "Site data"
    path: "sites/%{foreman.location}.yaml"

Avoid using Puppet variables

The hierarchy can make use of variables set by Puppet modules and manifests. You should avoid this in the global and environment hierarchies, because it ties the deployment or an environment to a specific module or manifest. It is also parse-order dependent, so any queries performed before the variable is defined will not resolve to the correct path.

This is useful and effective only when you use it to customize lookups within a certain module. For example, a service profile might use a mandatory parameter in the hierarchy layer for queries done within the profile.

To make the data source readily apparent to the viewer, place module-provided data in its own bespoke hash, as shown in the following:

:hierarchy:
  - name: "Data from service module"
    path: "services/%{service.cluster}.yaml"

As discussed in Chapter 1, use only variables explicitly documented as public by the module author. Otherwise, code refactoring of that module might rename, eliminate, or change the purpose and behavior of the variable used. It’s best to avoid using module-sourced values whenever possible.

Use explain to debug lookup interpolation

As discussed in Chapters 3 and 7, it’s easiest and most intuitive to follow variable references when those references strictly flow upward from your node’s facts and the ENC data. By interpolating module-specific variables into higher-level Hiera hierarchies, the flow of data through your site becomes somewhat cyclical and difficult to debug and understand.

The puppet lookup --explain will get node facts from --node hostname or --facts facts.yaml and provide a detailed analysis of the Hiera hierarchy and in which level of which layer a specific value was found. This is an incredibly powerful and useful tool. The following example shows it in action:

$ puppet lookup --node node_name --explain classes --merge unique
Searching for "classes"
  Global Data Provider (hiera configuration version 5)
    Using configuration "/etc/puppetlabs/puppet/hiera.yaml"
    Hierarchy entry "Per-node data"
      Path "/etc/puppetlabs/code/hiera_global/nodes/testy.example.om.yaml"
        Original path: "nodes/%{trusted.certname}.yaml"
        Path not found
...each level of each layer will be shown...

By keeping to the aforementioned recommendations, the puppet lookup command makes it trivial to debug. If any part of the hierarchy is interpolated from Puppet code, those values won’t be available. You’ll be forced to read and analyze the code yourself to figure out what happened.

You can read about how to use the explain feature at Using Puppet Lookup and the Puppet Lookup man page.

This is by no means an exhaustive list; facts are arbitrary and treated as top-level variables. Any top-level variable can easily be spoofed unless it’s defined explicitly by your Puppet code or ENC.

Facts are by default presented as top-level variables. Global variables and data from your ENC have priority over facts. However, values not supplied by your ENC can be spoofed using facts. For example, if your hierarchy uses a tier parameter from your ENC to provide production specific values and there is a case in which an agent can invoke a run without the ENC supplying a tier parameter, a malicious user might be able to recover data from that hierarchy using a static fact.

You should use caution with Hiera interpolation functions, as well. Although Hiera does provide the lookup() recursive lookup function, you should not use it as a hierarchy interpolation token because doing so might create an infinite recursive lookup loop.

Most specific to general—top to bottom

Because first-found answers override lower answers, the Hiera hierarchy should always query node-specific files first, with each successive layer containing more general data. The final layer should contain fallback values common to all nodes. This allows overrides to be applied at the appropriate level for clarity and debugging. Example 6-3 shows a simple, flat hierarchy demonstrating the idea.

:hierarchy:
  - name: "Per-node data"
    path: "nodes/%{trusted.certname}.yaml"

  - name: "Per-site data"
    path: "sites/%{facts.site}/test.yaml"

  - name: "Per-OS defaults"
    path: "os/%{facts.os.family}.yaml"

  - name: "Common data"
    path: "common.yaml"

In this hierarchy, values for every node are applied in the common level. Values that are specific to the operating system (OS), site, and node are stored in their specific files. The first level is the most specific, and the final level is the most general. Even though most hierarchies will have more levels, this concept holds true regardless.

Avoid unnecessary deep hierarchies

In our experience, it’s best to keep your hierarchy as flat as possible. In a flat hierarchy, levels of the hierarchy are near the root and have no children of their own. A flat hierarchy tends to improve data organization and tends to reduce the amount of duplicated data you need to manage.

A deep hierarchy is one shaped like a tree, where each node in the hierarchy is a child of a higher priority node. The example shown below was a flat hierarchy with a clear separation of data type. The best way to identify a problematic deep hierarchy is the use of unrelated data in subdirectories below data. The following type of hierarchy will create many layers of duplication:

:hierarchy:
  - name: "Node data"
    path: "%{facts.tier}/%{facts.location}/%{facts.os.family}/%{facts.fqdn}.yaml"

  - name: "OS data"
    path: "%{facts.tier}/%{facts.location}/%{facts.os.family}.yaml"

  - name: "Location data"
    path: "%{facts.tier}/%{facts.location}.yaml"

A deep hierarchy will increase data redundency the deeper you look. In the preceding example, there is likely to be a lot of commonality in your os.family hierarchy, but because this is nested under location and tier, it will be duplicated in each location tree. If you have three tiers (dev, stage, prod) and four locations (America, Europe, Asia, Africa) you’ll end up duplicating the same value in 12 places. Larger structures could have hundreds of duplicates.

Deep hierarchies should contain data relevant to the top level

The benefit of deep hierarchies is that they facilitate flexibility in your values. Because nothing is shared between branches of the structure, you are free to set whatever value you want in each branch.

For cases in which a deep hierarchy is necessary, the subsections should reflect most specific to most general. This will use Hiera’s lookup functionality and decrease duplicate data. Here’s what the code looks like:

:hierarchy:
  - name: "Per-node data"
    path: "nodes/%{trusted.certname}.yaml"

  - name: "Per-site data"
    paths:
      - "sites/%{facts.site}/%{facts.service}/%{facts.cluster}.yaml"
      - "sites/%{facts.site}/%{facts.service}.yaml"
      - "sites/%{facts.site}.yaml"

  - name: "Per-OS defaults"
    path: "os/%{facts.os.family}.yaml"

There are benefits and drawbacks to both approaches, but there’s a simple golden rule for evaluating the need for depth in a hierarchy: the branches should be refinements of their parent. If the data is not related to the data higher on the branch, it should be stored in a more general location.

The accounts hierarchies

Most organizations keep user account data separate from service data. Account data rarely needs to be platform, location, or tier specific, and it is often managed by a completely different team.

The example that follows will handle not only project-specific and shared-account lists, but also the few cases in which an account might be platform specific: your Linux root and Windows administrator accounts will have platform-specific properties.

  - name: "Account data"
    paths:
      - "accounts/%{facts.project}.yaml"
      - "accounts/%{facts.os.family}.yaml"
      - "accounts/common.yaml"

Data in the accounts hierarchy can be stored as serialized data in a form ready to be consumed by the create_resources() function or a defined type you’ve created for account management. For more details, see “Converting Serialized Hiera Data into Resource Declarations”.

Packages hierarchy level

Some people create complex hierarchies to track package information for each OS. We recommend that you place this data as close to the relevant branch as possible. The following example shows a quite powerful example:

  - name: "Service data"
    paths:
      - "os/%{facts.service}/%{facts.os.family}.yaml"
      - "os/%{facts.service}/common.yaml"

  - name: "OS data"
    paths:
      - "os/%{facts.os.family}/common.yaml"
      - "os/%{facts.os.family}/packages.yaml"

You can use this hierarchy with the create_resources() function call to install OS base packages and service-specific packages. You don’t need to write platform-specific code for that purpose since the platform differences (data) are entirely contained in Hiera.

If multiple modules need to install the same packages, you can store the packages in a common hash and then create a list of packages used by each module. This ensures that all necessary packages are installed without platform-specific data (package names) in the code.

Team hierarchy level

In some cases, it can be useful to have a hierarchy to manage data based on the team that owns a particular node or the service it provides. This tends to be common in large environments that provide Infrastructure as a Service (IaaS) and allow self-service provisioning. This can be a useful hierarchy level for storing team-specific security data, tags for data gathering, and monitoring alarm recipient information.

Team is a property that is difficult to determine programmatically, and will usually be maintained as a node property in your configuration management database or determined from node configuration and provided as a fact as shown here:

  - name: "Team owner"
    path: "project/%{facts.project}/team.yaml"

This hierarchy has some overlap with the accounts hierarchy. Keep account data out of the owner hierarchy. Instead, use the owner hierarchy to list what accounts should be added to a particular host.

Hierarchy design

We already discussed hierarchy design at length. It’s important to emphasize here that a good design will limit, not cause, data duplication. Any design that makes copypasta problems more likely needs to be revisited.

Package management

A common use for Hiera is to store package version information. If your organization already uses a package management solution such as Pulp, Katello, or Spacewalk, use Puppet to configure that data source rather than recreate the same data in Hiera.

User management

Users are another common source of data lists in Hiera. Deploying a directory service provides security and usability benefits over maintaining user accounts as static data in Hiera.

Service discovery

Service information tends to make up the bulk of data in Hiera, and it tends to create the most change requests in a complex environment. Questions such as what servers are currently available in my web pool? are better answered by service registries such as Consul or etcd.

Defined types

Defined types do not support automatic parameter lookups. This is because classes are singletons and use only a single set of parameters. Defined types (and any other resource type) can be called with unique parameters multiple times.

You can still get data from Hiera by interpolating a parameter within a lookup key, as shown here:

define example_module::example_type(
  id      => lookup("example_module::service_hash::${title}::id'),
  version => lookup("example_module::service_hash::${title}::version'),
) {
  #...
}

Key naming conventions

When performing explicit Hiera lookups, it’s a good idea to follow the same key naming conventions as the automatic parameter lookup system. To have profile-specific class parameters, you could use the following style:

profiles::apache {
  class { 'apache':
    docroot => lookup('profiles::apache::docroot'),
  }
}

This approach helps keep the key in Hiera tightly associated with the location that calls the key. Being able to quickly identify and review the context in which a value is being used is invaluable when attempting to update the key. In this example, we named the key as if it were a parameter within the profiles::apache class before being assigned to the docroot parameter of the Apache module. Even though no such parameter exists, the intent and context where the key is used is clear at a glance in our Hiera data. For a contrasting example, look at this key and value combination:

wordpress_docroot: "/var/www/wordpress"

Do you notice in this case how much context and clarity of purpose is lost by a simple change of key name? Although the variable appears to be named clearly, it would be difficult to determine where the value is being used without performing an exhaustive search of the code. With the profiles::apache::docroot name, it’s immediately clear which class is using the variable, making it trivial to track down the code it might affect.

By carefully selecting key names, you can significantly ease troubleshooting and maintenance of your hierarchy data.

Hash and array data merges

When retrieving a value from Hiera using the lookup() function call or automatic parameter lookup, Hiera will normally return the first key encountered. For queries of Hash and Array values, you can choose a merge behavior. When a merge attribute is specific, Hiera will retrieve keys and values from every level of each hierarchy and merge all keys into a single data structure. The merge function selected controls how the results are combined into a single data structure.

Data-driven class assignment

You can pass a Hiera lookup that retrieves a unique array of class names to the include() function for Hiera-driven class assignment:

lookup('classes', Array, 'unique', []).include

This code looks up array of entries for classes from every level of each layer of the hierarchy, flattens the results into a single array, and removes duplicates. The results will be exactly the same as calling include module_name for each array entry.

Automatic parameter lookup ensures that each class’ parameters will also be looked up from Hiera. This allows for great flexibility with design of your roles and profiles.

Encrypted Hiera is not filesystem security

When working with either of these backends, it’s important to understand how the data is handled. Hiera’s encrypted backends solve the problem of storing secrets in shared source-code repositories. They do not keep the value encrypted in all parts of the process.

When a request is made to Hiera, the value is decrypted and returned to Puppet for use in the catalog build. In many cases, the decrypted value will be inserted into a catalog, either as a class parameter, a resource parameter, or the contents of a file.

There are many situations in which your encrypted data is stored to disk and available in decrypted form, including at least the following:

• The node caches the catalog to the disk containing the decrypted values.

• The values might be written out by Puppet to files that contain the decrypted value.

• The values might be used to execute commands (and available in memory).

In short, you cannot keep encrypted Hiera values secret from someone who has root or administrator access to the node that uses the decrypted values.

This is not a complete data-security solution. Use these tools to facilitate revision control of your sensitive data, but do not expect them to provide anything greater than protection from people reading the secrets from your source tree. The value will not be logged or included in the Puppet report unencrypted. But it’s not possible to prevent a user with access to a signed Puppet client certificate of the target node from accessing the decrypted values.