Appendix A. Installing and configuring OpenShift

The purpose of this appendix is to help you get a multiple-node OpenShift cluster up and running. OpenShift runs on CentOS 7 or Red Hat Enterprise Linux 7 systems. It can run on physical servers, virtual machines (VMs), or VMs in a public cloud like Amazon Web Services (AWS) EC2. This installation should take approximately an hour, depending on your internet connection speed.

A.1. Prerequisites

This section covers what you’ll need to have access to or control of to build out your OpenShift cluster.

A.1.1. Available systems or creating virtual machines

This appendix starts during the OS installation process. The primary assumption we’re making is that you have a place to house two servers. These servers can be physical or VMs.

A.1.2. Administrator or root access

For many of the examples, you’ll use the oc command-line client to control your OpenShift cluster. This appendix will cover its installation.

oc is run from your laptop or workstation. To install oc, you need administrator access if you’re using a Windows computer, or root access if you’re using Linux or macOS. It’s a robust application; full documentation is available on GitHub at http://mng.bz/2s9U.

A.1.3. Internet access

For the configuration described here and many of the examples in the book, your OpenShift cluster must be able to access the internet to download software updates and example files.

A.1.4. Access to the servers

You’ll need several types of access to these systems to install the OS and perform the examples throughout the book.

Console access

For either a physical or virtual system, you’ll need some sort of console access to install the OS. This could be direct console access on a physical system or some sort of remote console solution like VNC (www.realvnc.com/en/connect/download/vnc) for physical or virtual systems.

ssh access

To install OpenShift and perform some of the examples, you’ll need to have SSH access to all the systems you create. SSH is a widely used remote access protocol that’s used to manage systems over a network connection. The default SSH port is TCP port 22.

On Linux and macOS, there’s a built-in command-line SSH client that you can use to access your systems once the OS installation is complete. For Windows, you can use an SSH client application like PuTTY (www.putty.org).

HTTPS access

OpenShift’s default configuration uses TCP port 8443 to access its web interface as well as its API. You’ll need to be able to browse to your master server on this port.

A.1.5. Communication between servers

To ensure that your OpenShift cluster can communicate properly, several TCP and UDP ports need to be open on the master and nodes. You can find more details at http://mng.bz/gjy1, but we usually pare this down to a simpler configuration for the sort of lab installation that you’ll be creating. If you’re building this cluster on an isolated network such as your laptop, you can leave all connectivity between your cluster servers open.

If your cluster will have any sort of inbound connectivity from the internet, table A.1 provides a list of ports to keep open for communication among the cluster members.

Table A.1. Ports to keep open between hosts

A.1.6. DNS resolution

In OpenShift, the hostnames for all nodes must have a DNS record. This allows encrypted traffic between nodes to work properly.

You need to configure a wildcard DNS record (https://tools.ietf.org/html/rfc4592) that points to your OpenShift cluster to access the applications you deploy. If you have a DNS server that you can control, you can go that route. But as long as you have internet access, you can use the nip.io domain.

Note

If you have experience with Linux, you may be asking yourself, “Why can’t I just use the /etc/hosts file in my OpenShift cluster?” The answer is that you could use /etc/hosts, but only for your server hostnames. OpenShift also can use a wildcard DNS domain for all the applications it deploys. This has some helpful advantages that we’ll discuss later, and it’s how you’ll configure this cluster.

Understanding the nip.io domain

The nip.io domain (http://nip.io/) is a wonderful little service. Instead of having to configure and manage a DNS server, if you have access to the internet, you can create hostnames and DNS records that resolve to any IP address you choose. It works by taking any DNS record that ends with any IP address and .nip.io and returning that IP address.

Here are some examples of DNS lookups of records using the nip.io domain:

dig +short anything.192.168.12.150.nip.io
192.168.12.150
dig +short anything-else.192.168.65.200.nip.io
192.168.65.200

The only requirement is that your servers all be able to access a public DNS server.

Deciding on hostnames

The hostnames that we used in our cluster and will use in the examples are outlined in table A.2. We’re using the nip.io domain, so the corresponding IP address is important as well. You’ll configure the IP address on your servers later in this appendix.

Table A.2. Hostnames and IP addresses for our OpenShift example cluster

A.1.7. Networking information

The servers in your OpenShift cluster will have static IP addresses to ensure that the DNS and hostnames that you configure work consistently. If you didn’t use static IP addresses, you’d need to be able to manage a DHCP server in your environment.

To configure a static IP address in the CentOS installer, you’ll need to know some information about the network your systems will be connected to (see table A.3). You’ll use this information when you configure your server’s network interface.

Table A.3. Networking information needed for server installation

Note

The DNS server we’re using is 8.8.8.8, which is one of Google’s public DNS servers. You can use any DNS server you wish, but in order to work, it must resolve public DNS queries for the nip.io domain.

Next, we’ll walk through the resource requirements for the nodes.

A.2. Machine resource requirements

OpenShift Origin’s official hardware requirements are published at http://mng.bz/gjy1. These are based on the premise of running a large, production-ready cluster. You’re welcome to follow these guidelines, if you have those resources available. But we tested the examples in this book with smaller virtual systems, as outlined in table A.4.

Table A.4. Resources used for our example cluster

You can always create systems with more resources. These are just values we tested with, and we confirmed that they work.

These server values allow you to have a fully operational OpenShift cluster in VMs on a smaller, portable laptop. This configuration won’t let you run as many applications in OpenShift, but for the examples we’ll go through in this book, these systems should get the job done.

There are a few nondefault configurations you need to use when you’re configuring your OpenShift cluster. Let’s go through those next.

A.3. Installing CentOS 7

CentOS 7 has a graphical installer you can use if you’re installing it using an installation DVD image. In addition, cloud images are available at www.centos.org/download for multiple platforms like VMWare, OpenStack, and Amazon. We’ll go through the installation ISO method using the standard installation DVD image, because it can be used on any platform, including bare-metal systems.

You should be able to use this guide to provision both servers and install OpenShift. If you have another process or tool you use to provision servers, it should work as well. This approach isn’t exclusive, by any means. The goal of this section is to demonstrate the most universal installation method.

A.3.1. Launching the installer

When you boot your system using the installation DVD, you’ll see the screen shown in figure A.1. You can press Enter, or the process will start automatically after 60 seconds.

Figure A.1. The CentOS installation startup

The installer boots into the installation process. There’s a media check that takes a minute or so, which confirms that the DVD image is fully functional and there were no errors during the download.

Next, you arrive at the home screen of the graphical installer, shown in figure A.2. The graphics are buttons for various installation options. For example, if you need to adjust the time zone, you can change it on the Date & Time screen.

Figure A.2. The CentOS installation main screen opens immediately after the installer launches.

The first customization that must happen to install OpenShift is selecting the disk configuration for your server.

A.3.2. Configuring the disk setup

This step is the same for master and node servers. In section A.2, we noted that the OpenShift nodes need two disks. The first disk in these systems will be for the OS. The second disk won’t be provisioned by CentOS: you’ll take care of it on the node in a subsequent step so you can use it for container image storage. On the master, this disk will be used in appendix B to set up an NFS server.

From the Installation Destination on the main screen, click the smaller first disk, and be sure to leave the second disk unselected (see figure A.3). Then, click Done to return to the installer main screen.

Figure A.3. The CentOS disk setup screen

The next step in getting CentOS configured properly is to set up networking for both nodes.

A.3.3. Setting up networking

The networking configuration process is the same for the master and node servers. On the main screen, click Network & Host Name to access the networking configuration. You’ll go through several screens, beginning by setting the hostname and enabling networking.

Hostname resolution

The Host Name field is in the lower-left corner of the Network & Host Name screen (see figure A.4). Your server hostnames need to be resolvable on your network. Earlier, we mentioned the nip.io domain and how you can use it to resolve anything you need to any IP address you want—and you can use this service for your server hostnames. To set a hostname for your server, fill in the Host Name field, and click Apply. The current hostname appears at lower right after you apply the new hostname.

Figure A.4. The Network & Host Name screen

Also on this screen, you need to enable the server’s network. By default, the network interface isn’t turned on. To turn it on, click the slider button at upper right, as shown in figure A.4.

Next, you need to configure the interface on your server after the installation is complete.

A.3.4. Setting the permanent configurations on the servers

This step is the same on both the Master and the Node servers. To start, click Configure on the Network & Host Name screen to open the screen shown in figure A.5.

Figure A.5. The initial network configuration screen

Don’t make any changes—this is a tab-oriented screen. The first tab you need to configure is the General tab.

The General configuration tab

On the General tab, select the Automatically Connect to This Network When It Is Available check box, as shown in figure A.6. This will make sure the interface is brought up when the server is booted.

Figure A.6. Network configuration General tab configuration

Next, you need to configure the IPv4 Settings tab.

IPv4 settings

This is the last network configuration you need to make on your OpenShift servers. These systems will use a static IP address to ensure that the hostnames always resolve properly; you’ll configure the static IP address on the screen shown in figure A.7.

Figure A.7. Network IPv4 Settings screen configuration

In section A.1, we pointed out the hostnames we’re using for the master and node servers. These are the hostnames and IP addresses we use for the examples throughout the book.

Select Manual from the Method drop-down list; this option lets you enter static IP addresses for your servers. To fill in the values, use the networking information you gathered in section A.1.5, including the IP address, netmask, and default gateway addresses. Then, click Save to return to the main installation screen.

A.3.5. Starting the installation

Once the disk and networking configurations are complete, the Begin Installation button becomes available on the main screen. Click it, and you’re on your way.

You’ll be prompted to set a password for your root user. Click the Root Password button shown in figure A.8, and you’ll be prompted to enter and confirm a root password. Type a password, and click Done. If it’s a weaker password, the installer may warn you and force you to either change it or click Done again to confirm that’s the password you want to use. For the examples, you won’t have to create any additional users.

Figure A.8. The CentOS User Settings screen

Now you can sit back and wait for the installer to finish doing its job.

A.3.6. Wrapping up and rebooting

When the installer signals that it’s finished, it will prompt you to reboot the server, as shown in figure A.9. Click Reboot to restart your server so that it can confirm everything was laid down correctly.

Figure A.9. A completed CentOS installation that’s ready to be rebooted

You’re now ready to connect to your server, configure it, and install OpenShift.

A.4. Preparing to install OpenShift

All the commands in this section will be executed using an SSH connection to the proper server. If you have console access for your systems, you can use that to execute these commands. But in our experience, using an SSH client makes it possible to copy and paste text. This simple feature can save a lot of time when provisioning a system.

OpenShift has a few dependencies that you need to get out of the way before launching the installer. Just as when you built out your master and node OSs, we’ll point out if the steps are different for the master and node servers here.

A.4.1. Software prerequisites

You need to install a few software packages. The software that’s specific to OpenShift comes from a specialized CentOS repository, which you must install and configure on all servers.

Configuring the OpenShift repositories on all servers

The first step is to install the Extra Packages for Enterprise Linux (EPEL) and OpenShift Origin repository RPMs. These configure your systems to access those software repos to pull down the needed files. To do so, run the following yum command:

# yum -y install epel-release centos-release-openshift-origin36

Once that command completes, several software packages will be common to both master and node servers; they come from the two newly enabled repositories. You can install these packages with the following command:

# yum -y install origin origin-clients vim-enhanced atomic-openshift-utils
 NetworkManager python-rhsm-certificates

With those packages installed, there’s one more command to run on both systems: you need to enable and start NetworkManager. NetworkManager is a tool on Linux servers that configures and maintains all aspects of a server’s networking stack. OpenShift uses NetworkManager to manage the network configurations of all the servers in the cluster. You can enable and start NetworkManager by running the following command:

# systemctl enable NetworkManager --now

In the next section, you’ll make a few changes to your server’s DNS configurations.

A.4.2. Configuring DNS resolution on both servers

Because DNS is used by OpenShift for everything from encrypted traffic to communication between deployed services, configuring DNS on your nodes is essential.

Note

The steps in this section apply only if you’re using nip.io for your hostnames. If your server’s hostnames are all on the same domain, this step isn’t required.

Configuring /etc/resolv.conf

If you’re using the nip.io domain for your server hostnames, you need to make an additional configuration to NetworkManager. In Linux, specifically CentOS 7, NetworkManager is the utility that manages the configuration for all the networking components on a server.

One of the components that NetworkManager controls is DNS resolution. In Linux, the configuration file that sets up how a server resolves DNS requests is /etc/resolv.conf.

The first line in resolv.conf lets you know that this file was created by NetworkManager. The nameserver parameter is the DNS server that your server will connect to for DNS queries. You can have up to three nameserver parameters listed in resolv.conf.

$ dig +short *.apps.jeduncan.com
12.207.21.2

$ dig +short app1.apps.jeduncan.com
12.207.21.2
$ dig +short someother.apps.jeduncan.com
12.207.21.2

The other parameter that’s included by default in resolv.conf is search. This value defaults to the domain name for your server’s hostname. The search value is used for any DNS queries that aren’t fully qualified domain names (FQDN). FQDNs are DNS records that are complete—that means they have a hostname, domain name, and top-level domain. For example, server.example.com is an FQDN, and server isn’t.

Here’s how resolv.conf looks immediately after installation:

# Generated by NetworkManager
search 192.168.122.100.nip.io         1
nameserver 8.8.8.8                    2

• 1 search parameter
• 2 Listed nameserver

Using the nip.io domain, each octet in the IP address is separated by a period. That means each number in the IP address is a level in the domain, with nip.io being the top-level domain. Because of some configurations OpenShift adds to each container, this can cause confusion when pulling down container images from the OpenShift integrated registry. You must edit the search parameter to have only the nip.io top-level domain, as shown in the next listing.

Listing A.1. Edited resolv.conf

# Generated by NetworkManager
search nip.io                    1
nameserver 8.8.8.8               2

• 1 Edited search parameter
• 2 Listed nameserver

This configuration will work fine until you reboot your servers. That’s because NetworkManager controls /etc/resolv.conf, and it will add the search parameter back when the service is restarted after a reboot. To stop this from happening, you need to configure NetworkManager to not make any further changes to /etc/resolv.conf.

Configuring NetworkManager

To configure NetworkManager, you need to edit its default configuration file. On CentOS 7, the NetworkManager configuration file is located at /etc/NetworkManager/NetworkManager.conf. The default configuration shown in the following listing is an example of a standard Python configuration file. It has two sections, defined by the lines encased in square brackets: [main] and [logging].

Listing A.2. Example of the default /etc/NetworkManager/NetworkManager.conf

# Configuration file for NetworkManager.
#
# See "man 5 NetworkManager.conf" for details.
#
# The directory /etc/NetworkManager/conf.d/
      can contain additional configuration
# snippets. Those snippets override the settings from this main file.
#
# The files within conf.d/ directory are read in asciibetical order.
#
# If two files define the same key, the one that is read afterwards will
 overwrite
# the previous one.

[main]
plugins=ifcfg-rh

[logging]
#level=DEBUG
#domains=ALL

You need to add a line to the [main] section of this configuration file, to tell NetworkManager not to make changes to /etc/resolv.conf. Add the line dns=none as shown here:

[main]
plugins=ifcfg-rh
dns=none

After you restart NetworkManager, the change you made to /etc/resolv.conf will persist across server reboots. To restart the NetworkManager, run the following systemctl command:

systemctl restart NetworkManager

This should only take a second or so. Once it completes, confirm that NetworkManager is running using systemctl status:

systemctl status NetworkManager
? NetworkManager.service - Network Manager
 Loaded: loaded (/usr/lib/systemd/system/NetworkManager.service; enabled;
 vendor preset: enabled)
   Active: active (running) Because Sat 2017-05-13 17:05:12 EDT; 6s ago
...

The final confirmation is to check /etc/resolv.conf and make sure the search parameter hasn’t been added back to the file by the freshly restarted NetworkManager service. If there’s no search parameter, then everything is as it should be, and you’re ready to move forward.

Now you’re ready to set up software that’s specific to the master and node servers. You’ll start with the master server in the next section.

A.4.3. Installing software on the master server

Several packages need to be installed only on the master server. This primarily has to do with installing Ansible and ensuring that it’s the correct version. Ansible (www.ansible.com) is an open source automation and deployment tool. The installation process for OpenShift is written using Ansible. One of the key features of Ansible is that it uses YAML, a human-readable data-transport language, to describe all the work it does.

To install OpenShift, you’ll create a configuration file written in YAML. This file will be read by the Ansible engine to deploy OpenShift exactly as you want. You’ll be creating a relatively simple configuration file; for more advanced installations, the full set of options is documented at http://mng.bz/CD7l.

The OpenShift installer is written and tested against a specific version of Ansible. That means you’ll need to make sure that’s the version of Ansible that’s installed on your master. You only need to worry about Ansible on the master server, because there’s no agent on the node. Ansible doesn’t use an agent on the systems it’s controlling; instead, it uses SSH as a transport mechanism and to execute remote commands. Start this process by running the following yum command:

yum -y install httpd-tools gcc python-devel python-pip

The python-pip package installs the Python application package manager named pip. It’s used to install applications written in Python and available on the Python Package Index (www.pypi.org). With pip installed, you can use it to install Ansible and ensure that you install version 2.2.2.0, which is tested with OpenShift 3.6:

# pip -v install ansible==2.2.2.0

After Ansible is installed, you’ll need to set up SSH keys from your master server to the node, and to the master as well. That’s what you’ll do next.

Setting up SSH access from the master server

To get the OpenShift installer to function correctly, you need to create an SSH key pair on your master server and distribute the public key to your node. To create a new SSH key pair on your master server, you can use the ssh-keygen command as in this example:

ssh-keygen -f /root/.ssh/id_rsa -t rsa -N ''

This command creates an SSH key pair in the root user’s home directory, /root, in the .ssh subfolder. In Linux, this is the default location for a user’s SSH keys.

Next, run the following ssh-copy-id command to distribute your newly created SSH public key to your OpenShift node (if you used a different IP addresses for your master and node, adjust the command accordingly):

for i in 192.168.122.100 192.168.122.101;do ssh-copy-id root@$i;done

You’ll be prompted for the root password for your OpenShift master and node. Once you confirm each password, this command will add the public key from your SSH key pair to the authorized_keys file in /root/.ssh on the OpenShift node.

This will allow the OpenShift installer to connect to the master and node to perform the installation steps. It may seem a little odd to specify the IP address of the master when you’ll be running the installer from that server. This is because the OpenShift installer is designed so that it can be run from outside your OpenShift cluster if desired. For your first lab, and because we aren’t 100% sure what type of computer you’re using daily, it’s easier to satisfy the dependencies for the installer on the master node.

This should meet all the dependencies to run the OpenShift installer on the master server. Next, you’ll configure your application node.

Installing software on the application-node server

The software requirements for the nodes are a little different. The biggest difference is that this is where docker will be installed. The libcgroup-tools package provides utilities that you’ll use to inspect how applications are isolated using kernel control groups in chapter 9. To install these packages, run the following yum command:

# yum -y install docker libcgroup-tools

Once this is complete, you’re ready to configure your container engine’s storage on your OpenShift nodes.

A.4.4. Configuring container storage for application nodes

An application called docker-storage-setup is packaged with docker. It configures the desired storage for docker to use when it creates containers for OpenShift.

Note

The examples in this book use a logical volume management (LVM) configuration. This setup creates an LVM volume for each container on demand. These are small initially but can grow to the maximum size configured in OpenShift for your containers.

You can find additional details about the storage setup in the OpenShift documentation at http://mng.bz/hR82.

The first step in this process is to create a configuration file for docker-storage-setup on your OpenShift node, as shown in the next listing. The disk you specify in /etc/sysconfig/docker-storage-setup is the second disk you created for your VM. Depending on your choice of platform for your servers, the OS’s name for this disk (/dev/vdb in our example) may vary, but the operation won’t.

Listing A.3. Creating the docker-storage-setup configuration file

cat <<EOF > /etc/sysconfig/docker-storage-setup
DEVS=/dev/vdb                                     1
VG=docker-vg
EOF

• 1 /dev/vdb is the 20 GB volume you added to your nodes.

# lsblk
NAME                               MAJ:MIN RM  SIZE RO TYPE MOUNTPOINT
vda                                253:0    0    8G  0 disk          1
??vda1                             253:1    0    8G  0 part /
vdb                                253:16   0   20G  0 disk          2

After you’ve created this file, you’re ready to run the storage-setup utility. This command should run in a couple of seconds, and the output should look like this:

docker-storage-setup

Checking that no-one is using this disk right now ...
OK

Disk /dev/vdb: 41610 cylinders, 16 heads, 63 sectors/track
...
  Rounding up size to full physical extent 24.00 MiB
  Logical volume "docker-pool" created.
  Logical volume docker-vg/docker-pool changed.

A.4.5. Enabling and starting docker on your OpenShift nodes

With your container storage configured, it’s time to start up the docker service on your OpenShift node. This is the container runtime that will start and stop containers in the OpenShift workflows. It needs to be enabled and running prior to installing OpenShift.

To enable docker to start at boot time, and to start the service initially, run the following systemctl command:

systemctl enable docker.service --now

Like NetworkManager, you can confirm that docker started correctly by running systemctl status docker:

[root@ocp2 ~]# systemctl status docker
? docker.service - Docker Application Container Engine
   Loaded: loaded (/usr/lib/systemd/system/docker.service; enabled;
 vendor preset: disabled)
  Drop-In: /etc/systemd/system/docker.service.d
           ??custom.conf
   Active: active (running) since Fri 2017-11-10 18:45:12 UTC;
 12 secs ago                                                   1
     Docs: http://docs.docker.com
 Main PID: 2352 (dockerd-current)
   Memory: 121.4M
   CGroup: /system.slice/docker.service
...

• 1 The active (running) status confirms the docker service is up and functioning.

The next step is to modify SELinux to allow OpenShift to connect to NFS as a persistent storage source.

A.4.6. Configuring SELinux on your OpenShift nodes

Throughout this book, OpenShift applications will need NFS volumes to act as persistent storage. To be able to do this successfully, you have to tell SELinux on your nodes to allow containers to use NFS. You do so using the setsebool command-line utility:

# setsebool -P virt_use_nfs 1
# setsebool -P virt_sandbox_use_nfs 1

A.5. Installing OpenShift

OpenShift is installed using an Ansible playbook: a collection of tasks and configuration parameters needed to perform a task. To execute an Ansible playbook, three things must be present on your server:

• Ansible engine— Executes the playbook code. You installed the Ansible engine earlier in this appendix.
• Playbook— The code that’s executed. When you installed the OpenShift packages, the deployment playbooks were included.
• Inventory— The list of hosts against which to run a playbook. Inventories can be divided into groups and contain any variables needed for the hosts in the inventory for the playbook to run.

To deploy OpenShift, you need to configure an Ansible inventory file for your cluster. That’s what you’ll do next.

A.5.1. Creating the OpenShift inventory

The Ansible inventory for OpenShift contains information about your two hosts and specifies which roles each node will have in your cluster. If you’re using the IP addresses and hostnames we’re using in this appendix, you can download a prepared inventory to your master node from the OpenShift in Action organization on GitHub:

# curl -o /root/hosts
 https://raw.githubusercontent.com/OpenShiftInAction/AppendixA/master/hosts

For those of you who need to customize your installation, let’s go through the inventory components and how they’re designed.

Ansible inventories are divided into groups. Each group consists of hosts that are defined by either hostname or IP address. In an inventory, a group can also be defined by listing child groups that belong to it using the :children syntax. In the following example, the group master_group is made up of the hosts in group1 and group2. You’ll use this capability in your OpenShift inventory:

[master_group:children]         1
group1
group2

[group1]                        2
host1
host2

[group2]                        3
host3
host4

[group3]                        4
host5
host6

• 1 master_group, defined by the child groups of group1 and group2
• 2 group1 hosts
• 3 group2 hosts
• 4 group3 hosts, which aren’t part of master_group

Another capability of Ansible inventories that you’ll use in your OpenShift inventory is defining variables for hosts and host groups. You can define variables for an entire group using a group heading and the :vars syntax. To define a variable for a single host, add it to the same line you use to define the host in a group:

[group1]
host1 var2=False var3=42        1
host1 foo=bar

[group1:vars]                   2
var1=True

• 1 var2 and var3 are defined only for host1.
• 2 var1 will be defined and available for all hosts in group1.

Your initial OpenShift inventory uses several groups and lots of variables:

• OSEv3—The group that represents your entire cluster. It’s made up of the child groups nodes, masters, nfs, and etcd.
• nodes—All groups in your cluster, including all masters and all application nodes.
• masters—The nodes in your cluster that will be designated as masters.
• nfs—Nodes used to provide NFS shared storage for several services on the master nodes. This is required if you have multiple master servers. We aren’t taking advantage of multiple masters in this initial cluster, but the group is still required for deploying OpenShift.
• etcd—The nodes where etcd will be deployed. etcd is the database for Kubernetes and OpenShift. Your cluster will use the master server to house the etcd database. For larger clusters, etcd can be separated into its own cluster nodes.

For the nodes and masters groups, you’ll disable a few of the system checks the deployment playbook runs prior to deployment. These checks verify the amount of free space and memory available on the system; for an initial cluster, you can use smaller values than the recommendations verified by these checks. (You can learn more about these checks at http://mng.bz/r0dI.) To disable the checks, you set variables for each of these groups:

[nodes:vars]                                                               1
openshift_disable_check=disk_availability,memory_availability,docker_storage

[masters:vars]                                                             2
openshift_disable_check=disk_availability,memory_availability,docker_storage

• 1 Disables storage and memory checks for the nodes group
• 2 Disables storage and memory checks for the masters group

Your deployment inventory will also set variables for most of the hosts. The ansible_connection variable is specific to Ansible, telling the Ansible engine to connect to the host from the local system where the playbook is running. Additional Ansible variables are discussed at http://mng.bz/g3g0.

Note

The IP addresses and hostnames used in this inventory are specific to the example cluster. If your IP addresses and hostnames are different, you’ll need to change them in the inventory to successfully deploy OpenShift.

The rest of the variables are specific to the OpenShift playbook, and documented in the following listing, which is a full example of the OpenShift inventory.

Listing A.4. OpenShift inventory file

[OSEv3:children]                                                          1
nodes
nfs
masters
etcd

[OSEv3:vars]                                                              2
openshift_master_cluster_public_hostname=None
openshift_master_default_subdomain=apps.192.168.122.101.nip.io
ansible_ssh_user=root
openshift_master_cluster_hostname=None
openshift_override_hostname_check=true
deployment_type=origin

[nodes:vars]                                                              3
openshift_disable_check=disk_availability,memory_availability,docker_storage

[masters:vars]
openshift_disable_check=disk_availability,memory_availability,docker_storage

[nodes]
192.168.122.100  openshift_public_ip=192.168.122.100
 openshift_ip=192.168.122.100
 openshift_public_hostname=ocp1.192.168.122.100.nip.io
 openshift_hostname=ocp1.192.168.122.100.nip.io
 connect_to=192.168.122.100 openshift_schedulable=False
 ansible_connection=local                                               4
192.168.122.101  openshift_public_ip=192.168.122.101
 openshift_ip=192.168.122.101
 openshift_public_hostname=ocp2.192.168.122.101.nip.io
 openshift_hostname=ocp2.192.168.122.101.nip.io
 connect_to=192.168.122.101 openshift_node_labels="{'region': 'infra'}"
 openshift_schedulable=True

[nfs]                                                                     5
192.168.122.100 connect_to=192.168.122.100 ansible_connection=local

[masters]                                                                 6
192.168.122.100  openshift_public_ip=192.168.122.100
 openshift_ip=192.168.122.100
 openshift_public_hostname=ocp1.192.168.122.100.nip.io
 openshift_hostname=ocp1.192.168.122.100.nip.io
 connect_to=192.168.122.100 ansible_connection=local

[etcd]                                                                    7
192.168.122.100  openshift_public_ip=192.168.122.100
 openshift_ip=192.168.122.100
 openshift_public_hostname=ocp1.192.168.122.100.nip.io
 openshift_hostname=ocp1.192.168.122.100.nip.io
 connect_to=192.168.122.100 ansible_connection=local

• 1 OSEv3 group, made up of all groups in the inventory
• 2 Cluster-wide variables applied to all nodes in the OSEv3 group
• 3 Variables for the nodes group
• 4 Host entry for ocp1, using its IP address; includes host-specific variables
• 5 nfs group
• 6 masters group
• 7 etcd group

Note

The ocp1 node has a variable named openshift_node_labels. Node labels are arbitrary values you can apply to nodes on your cluster. The label applied during deployment, region = infra, tells OpenShift the correct node to deploy the container that runs the router to handle internet requests. You’ll work with node labels in chapter 10.

After making any inventory edits required to match your environment, save your inventory on your master node as /root/hosts. The next step is to start your OpenShift deployment.

A.5.2. Running the deployment playbook

It’s time to deploy your OpenShift cluster. Ansible uses SSH to log in to each node and perform the tasks to deploy OpenShift, so this command needs to be executed as the root user on the master, which has SSH access keys on each node. To run the proper Ansible playbook, run the ansible-playbook command, specifying the inventory file and the deploy playbook installed at /usr/share/ansible/openshift-ansible/playbooks/byo/config.yml:

# ansible-playbook -i /root/hosts 
/usr/share/ansible/openshift-ansible/playbooks/byo/config.yml

This launches the deployment process. Depending on your internet connection speed, the deployment could take 30-45 minutes. If everything is successful, you should see output indicating that the playbook completed. If you see a red Ansible error on the command line, it should give you an indication of what to look at.

A.6. Installation complete

Once the installation is complete, you should be able to browse to the hostname of your master server on port 8443 with HTTPS. In our example, the URL is https://ocp1.192.168.122.100.nip.io:8443 (see figure A.10). Before the page loads, you’ll probably get a warning about the site being insecure because the SSL certificate hasn’t been properly signed. Don’t worry about this—OpenShift created its own SSL certificates as part of the installation process. In our configuration, because the cluster is deployed with VMs on a laptop, the cluster is available only from the laptop the VM nodes are installed on.

Figure A.10. The OpenShift login page after a successful installation

If your OpenShift server looks like figure A.10, you’ve successfully installed OpenShift! You’re ready to go through the examples beginning in chapter 2.

Many of the examples you’ll go through involve using the OpenShift oc command-line tool. The next section will help you get oc installed and ready to use on Windows, macOS, or Linux.

Tip

OpenShift has a robust diagnostics toolset integrated into the oc command-line tool. Documentation for running diagnostics and checks on your OpenShift cluster is available at http://mng.bz/Zo4P.

A.7. Installing the oc OpenShift command-line utility

The oc application has ready-to-use binaries for Linux, Windows, and macOS. They’re available on the Releases GitHub page for OpenShift Origin at https://github.com/openshift/origin/releases. We’re using the latest release for oc; you can find it by browsing directly to https://github.com/openshift/origin/releases/latest.

Note

Because OpenShift Origin is a fast-moving project, there are usually one or more pre-release versions of software available for download. These versions are a little too new and untested to ensure that you have a good experience with OpenShift.

The downloadable files are listed in the Downloads section for each software release. For each OS, there’s a corresponding openshift-origin-client-tools archive that you can download. These are the files you’ll be using to install oc.

A.7.1. Installing oc on Windows

To install oc on Windows, download the windows Zip archive from the Downloads section of the Latest Release page. After unzipping the archive, move the oc binary in it into a directory in your Windows PATH.

You can see the directories that are in PATH on your Windows system from the Windows command line by running the following command:

C:> path

A.7.2. Installing oc on macOS

For macOS, the binary can also be downloaded from the Releases page on GitHub. The downloaded file is a Zip binary. Extract the downloaded oc binary file, and then copy it to a directory in your PATH. To see all the directories in your PATH, you can use the Terminal application to print out the variable using the following command:

echo $PATH

With oc in your PATH, you can execute the command from the Terminal application by typing oc.

A.7.3. Installing oc on Linux

There are 32- and 64-bit versions of the oc client available for download. If you’re unsure of the type of Linux you have installed on your system, you can confirm whether you have a 32- or 64-bit system by running the following uname command:

uname -i

The output should contain either x86_64 for 64-bit systems or i686 for 32-bit installations. Be sure to download the appropriate openshift-origin-client-tools archive for your Linux architecture—they aren’t cross-compatible. To download the Linux archive, run the following curl command:

curl -L https://github.com/openshift/origin/releases/download/v3.6.0/
 openshift-origin-client-tools-v3.6.0-c4dd4cf-linux-64bit.tar.gz

After you download the archive you plan to use, extract the archive using the tar utility:

tar -xzf /tmp/openshift-origin-client-tools.tar.gz

After you extract the archive, move the oc file that was in it into a directory that’s listed in the PATH variable on your system. If you aren’t sure about the directories in your PATH, you can confirm them by echoing the variable out to your command line:

$ echo $PATH
/usr/lib64/qt-3.3/bin:/usr/local/bin:/usr/local/sbin:/usr/bin:/usr/sbin:/home/
 jduncan/.local/bin:/home/jduncan/bin

Note

The OpenShift Origin documentation website has a section dedicated to getting oc up and running. It’s available at http://mng.bz/iM9L if you’d like to take a look.

A.7.4. Confirming that oc is installed and functioning correctly

Now that you’ve installed oc on your favorite OS, you need to confirm that it’s functional. You can do this by asking oc for its version information on the command line. Your output should look like the following example. For Linux, use your favorite terminal emulator. macOS uses the Terminal application, and Windows has the command prompt. This example is from a Linux workstation:

$ oc version
oc v1.3.1
kubernetes v1.3.0+52492b4
features: Basic-Auth GSSAPI Kerberos SPNEGO
error: You must be logged in to the server (the server has asked for the
 client to provide credentials)

If this executes cleanly, your system should be ready to connect to OpenShift and get work done once you have OpenShift installed and configured. You can add additional nodes to an existing OpenShift cluster using the next section as a guide.

A.8. Adding an OpenShift node

Scaling your OpenShift cluster is essential as you deploy more application workloads. This section walks you through the steps required to add a second application node to your cluster. First, you need to provision a new server and install the prerequisites.

A.8.1. Preparing the new application node

OpenShift manages the resources available on a node and allocates resources to it accordingly. Your second application node’s resources don’t have to match your original node.

For our new node, we’re using an additional VM with similar resources, hostname, and IP address (see table A.5). Any additional server that can run CentOS 7 will work if it has network connectivity to your existing cluster.

Table A.5. New application node specs

Configure your new node using the same steps as the original application node covered in this appendix. These steps are summarized in table A.6.

Table A.6. Summary of steps in configuring an application node

After your new application node is provisioned and configured, reboot it to make sure all changes take effect. The server is now ready to be added to your cluster. The workflow to add your node is executed on your master server, like deploying your original cluster. Let’s discuss that next.

A.9. Configuring the master node

The first step in configuring your master node is to add the root user’s SSH to your new application node. Doing this allows the Ansible playbook that adds your node to access your new application node. To do this, run the following ssh-copy-id command. If you used a different IP address for your new node, be sure to use it when running the command:

ssh-copy-id [email protected]

You’re prompted for the root user’s password that you set during the CentOS installation process. This command sets up the SSH key from your master to provide access to the root user on your new node. With this complete, make sure the installer packages are up to date for your master server.

A.9.1. Updating OpenShift playbooks

To make sure you have the most up-to-date version of the OpenShift installer, run the following yum command on the master node:

yum update atomic-openshift-utils

If there are updated playbooks for your version of OpenShift, this command will install them on your master, notifying you when complete. Next, add your new node to the OpenShift installer’s inventory.

A.9.2. Updating your OpenShift inventory

When you deployed OpenShift, you created an Ansible inventory file at /root/hosts on the master node. To add the new node to OpenShift, you need to edit this inventory before running the Ansible playbook that scales your cluster.

Note

Red Hat supports OpenShift clusters as large as 2,000 application nodes. For clusters that large, services on the master, like etcd, are separated onto their own servers to provide better performance and scalability.

To add your new node, create a group that contains the information for your node by editing your inventory file directly, as shown in listing A.5. Be sure to name this group new_nodes. Also include any labels it needs (none in this case), and tell the installer to make this node schedule-able for workloads.

Listing A.5. new_nodes group to contain your new node’s information

[new_nodes]
192.168.122.102 openshift_node_labels="{}" openshift_schedulable=True

Next, you need to tell the OpenShift playbook that the new_nodes group is part of your OpenShift cluster. To do this, add the new_nodes group to the [OSEv3:children] group, as shown in the following listing.

Listing A.6. new_nodes group added to the OSEv3:children group

[OSEv3:children]
nodes
nfs
masters
etcd
new_nodes

The final step of editing your inventory is to define the variables your new_nodes group needs to function properly. You need to create this group so you can define the variables to disable a few of the resource checks that the installer performs by default because of the limited resources on the nodes. You can copy the [nodes:vars] group and change the name to [new_nodes:vars], shown in the following listing.

Listing A.7. Creating the new_nodes:vars group

[new_nodes:vars]                                                           1
openshift_disable_check=disk_availability,memory_availability,docker_storage
openshift_deployment_type=origin

• 1 Copy the content from nodes:vars and use it to create a new group called new_nodes:vars.

After creating the new_nodes and new_nodes:vars groups, and adding new_nodes to the OSEv3:children group, you’re ready to scale your OpenShift cluster.

A.10. Adding the node

There’s an Ansible playbook on your master node that will add your new node to your existing OpenShift cluster. You run it using the ansible-playbook command, specifying your newly edited inventory with the -i parameter. To start this process, run the following command on your master server:

ansible-playbook -i /root/hosts  /usr/share/ansible/openshift-
 ansible/playbooks/byo/openshift-node/scaleup.yml

Once scaleup.yml completes successfully, your new node has been added to your OpenShift cluster. You can confirm this by running oc get nodes while being logged in as the admin user:

# oc get nodes
NAME                          STATUS                     AGE
 VERSION
ocp1.192.168.122.100.nip.io   Ready,SchedulingDisabled   3d
 v1.6.1+5115d708d7
ocp2.192.168.122.101.nip.io   Ready                      3d
 v1.6.1+5115d708d7
ocp3.192.168.122.102.nip.io   Ready                      2d
 v1.6.1+5115d708d7                                             1

• 1 The new ocp3 node is visible to OpenShift and ready to handle workloads.

This process is repeatable as your OpenShift cluster grows over time, to make sure your OpenShift cluster always has resources to handle your workloads.