The purpose of this appendix is to help you get a multiple-node OpenShift cluster up and running. OpenShift runs on CentOS 7 or Red Hat Enterprise Linux 7 systems. It can run on physical servers, virtual machines (VMs), or VMs in a public cloud like Amazon Web Services (AWS) EC2. This installation should take approximately an hour, depending on your internet connection speed.
This section covers what you’ll need to have access to or control of to build out your OpenShift cluster.
Minishift (https://github.com/Minishift/minishift) is a single-node installation of OpenShift that can be stood up in a few minutes on just about any OS as a VM. As a development platform, it’s a very useful tool.
We recommend going through the process of installing a full OpenShift cluster. You can run most of the examples in this book on Minishift. But you’re going to run into trouble when you start working with persistent storage, metrics, complex application deployments, and networking.
We love Minishift and use it daily, but the focus of this book is to work with you to deploy an OpenShift cluster that can be the prototype for a larger cluster that’s ready to do meaningful work. Minishift has a different goal in mind.
In addition to OpenShift in your datacenter, or even your laptop, hosted versions of OpenShift are available. Red Hat has a hosted version of OpenShift available at www.openshift.com that has a free usage tier. There’s also an interactive learning portal at https://learn.openshift.com with guided scenarios that give you access to a single-node installation of OpenShift.
This appendix starts during the OS installation process. The primary assumption we’re making is that you have a place to house two servers. These servers can be physical or VMs.
For many of the examples, you’ll use the oc command-line client to control your OpenShift cluster. This appendix will cover its installation.
oc is run from your laptop or workstation. To install oc, you need administrator access if you’re using a Windows computer, or root access if you’re using Linux or macOS. It’s a robust application; full documentation is available on GitHub at http://mng.bz/2s9U.
For the configuration described here and many of the examples in the book, your OpenShift cluster must be able to access the internet to download software updates and example files.
You’ll need several types of access to these systems to install the OS and perform the examples throughout the book.
For either a physical or virtual system, you’ll need some sort of console access to install the OS. This could be direct console access on a physical system or some sort of remote console solution like VNC (www.realvnc.com/en/connect/download/vnc) for physical or virtual systems.
To install OpenShift and perform some of the examples, you’ll need to have SSH access to all the systems you create. SSH is a widely used remote access protocol that’s used to manage systems over a network connection. The default SSH port is TCP port 22.
On Linux and macOS, there’s a built-in command-line SSH client that you can use to access your systems once the OS installation is complete. For Windows, you can use an SSH client application like PuTTY (www.putty.org).
OpenShift’s default configuration uses TCP port 8443 to access its web interface as well as its API. You’ll need to be able to browse to your master server on this port.
To ensure that your OpenShift cluster can communicate properly, several TCP and UDP ports need to be open on the master and nodes. You can find more details at http://mng.bz/gjy1, but we usually pare this down to a simpler configuration for the sort of lab installation that you’ll be creating. If you’re building this cluster on an isolated network such as your laptop, you can leave all connectivity between your cluster servers open.
If your cluster will have any sort of inbound connectivity from the internet, table A.1 provides a list of ports to keep open for communication among the cluster members.
Port number |
Network protocol |
Reason |
---|---|---|
22 | TCP | SSH access |
1936 | TCP | OpenShift router statistics |
8053 | TCP and UDP | Internal DNS management |
4789 | UDP | Software-defined networking communication |
443 | TCP | SSL communications |
8443 | TCP | Web and API services |
10250 | TCP | Kubernetes communication |
9200 | TCP | Aggregated logging collection |
9300 | TCP | Aggregated logging collection |
In OpenShift, the hostnames for all nodes must have a DNS record. This allows encrypted traffic between nodes to work properly.
You need to configure a wildcard DNS record (https://tools.ietf.org/html/rfc4592) that points to your OpenShift cluster to access the applications you deploy. If you have a DNS server that you can control, you can go that route. But as long as you have internet access, you can use the nip.io domain.
If you have experience with Linux, you may be asking yourself, “Why can’t I just use the /etc/hosts file in my OpenShift cluster?” The answer is that you could use /etc/hosts, but only for your server hostnames. OpenShift also can use a wildcard DNS domain for all the applications it deploys. This has some helpful advantages that we’ll discuss later, and it’s how you’ll configure this cluster.
The nip.io domain (http://nip.io/) is a wonderful little service. Instead of having to configure and manage a DNS server, if you have access to the internet, you can create hostnames and DNS records that resolve to any IP address you choose. It works by taking any DNS record that ends with any IP address and .nip.io and returning that IP address.
Here are some examples of DNS lookups of records using the nip.io domain:
dig +short anything.192.168.12.150.nip.io 192.168.12.150 dig +short anything-else.192.168.65.200.nip.io 192.168.65.200
The only requirement is that your servers all be able to access a public DNS server.
The hostnames that we used in our cluster and will use in the examples are outlined in table A.2. We’re using the nip.io domain, so the corresponding IP address is important as well. You’ll configure the IP address on your servers later in this appendix.
Hostname |
IP address |
OpenShift role |
---|---|---|
ocp-1.192.168.122.100.nip.io | 192.168.122.100 | Master |
ocp-2.192.168.122.101.nip.io | 192.168.122.101 | Node |
The servers in your OpenShift cluster will have static IP addresses to ensure that the DNS and hostnames that you configure work consistently. If you didn’t use static IP addresses, you’d need to be able to manage a DHCP server in your environment.
To configure a static IP address in the CentOS installer, you’ll need to know some information about the network your systems will be connected to (see table A.3). You’ll use this information when you configure your server’s network interface.
Network parameter |
Description |
Example values |
---|---|---|
Network mask (netmask) | Network mask for your network | 24 |
Gateway | Default gateway for your network | 192.168.122.1 |
DNS servers | DNS server(s) your systems will use | 8.8.8.8 |
The DNS server we’re using is 8.8.8.8, which is one of Google’s public DNS servers. You can use any DNS server you wish, but in order to work, it must resolve public DNS queries for the nip.io domain.
Next, we’ll walk through the resource requirements for the nodes.
OpenShift Origin’s official hardware requirements are published at http://mng.bz/gjy1. These are based on the premise of running a large, production-ready cluster. You’re welcome to follow these guidelines, if you have those resources available. But we tested the examples in this book with smaller virtual systems, as outlined in table A.4.
Server type |
CPU/vCPU |
RAM |
Storage |
---|---|---|---|
Master | 2 | 8 GB | 2 disks: 10 GB for OS, 20 GB for persistent storage |
Node | 2 | 8 GB | 2 disks: 10 GB for OS, 20 GB for container storage |
You can always create systems with more resources. These are just values we tested with, and we confirmed that they work.
These server values allow you to have a fully operational OpenShift cluster in VMs on a smaller, portable laptop. This configuration won’t let you run as many applications in OpenShift, but for the examples we’ll go through in this book, these systems should get the job done.
There are a few nondefault configurations you need to use when you’re configuring your OpenShift cluster. Let’s go through those next.
CentOS 7 has a graphical installer you can use if you’re installing it using an installation DVD image. In addition, cloud images are available at www.centos.org/download for multiple platforms like VMWare, OpenStack, and Amazon. We’ll go through the installation ISO method using the standard installation DVD image, because it can be used on any platform, including bare-metal systems.
You should be able to use this guide to provision both servers and install OpenShift. If you have another process or tool you use to provision servers, it should work as well. This approach isn’t exclusive, by any means. The goal of this section is to demonstrate the most universal installation method.
When you boot your system using the installation DVD, you’ll see the screen shown in figure A.1. You can press Enter, or the process will start automatically after 60 seconds.
The installer boots into the installation process. There’s a media check that takes a minute or so, which confirms that the DVD image is fully functional and there were no errors during the download.
Next, you arrive at the home screen of the graphical installer, shown in figure A.2. The graphics are buttons for various installation options. For example, if you need to adjust the time zone, you can change it on the Date & Time screen.
The first customization that must happen to install OpenShift is selecting the disk configuration for your server.
This step is the same for master and node servers. In section A.2, we noted that the OpenShift nodes need two disks. The first disk in these systems will be for the OS. The second disk won’t be provisioned by CentOS: you’ll take care of it on the node in a subsequent step so you can use it for container image storage. On the master, this disk will be used in appendix B to set up an NFS server.
From the Installation Destination on the main screen, click the smaller first disk, and be sure to leave the second disk unselected (see figure A.3). Then, click Done to return to the installer main screen.
The next step in getting CentOS configured properly is to set up networking for both nodes.
The networking configuration process is the same for the master and node servers. On the main screen, click Network & Host Name to access the networking configuration. You’ll go through several screens, beginning by setting the hostname and enabling networking.
The Host Name field is in the lower-left corner of the Network & Host Name screen (see figure A.4). Your server hostnames need to be resolvable on your network. Earlier, we mentioned the nip.io domain and how you can use it to resolve anything you need to any IP address you want—and you can use this service for your server hostnames. To set a hostname for your server, fill in the Host Name field, and click Apply. The current hostname appears at lower right after you apply the new hostname.
Also on this screen, you need to enable the server’s network. By default, the network interface isn’t turned on. To turn it on, click the slider button at upper right, as shown in figure A.4.
Next, you need to configure the interface on your server after the installation is complete.
This step is the same on both the Master and the Node servers. To start, click Configure on the Network & Host Name screen to open the screen shown in figure A.5.
Don’t make any changes—this is a tab-oriented screen. The first tab you need to configure is the General tab.
On the General tab, select the Automatically Connect to This Network When It Is Available check box, as shown in figure A.6. This will make sure the interface is brought up when the server is booted.
Next, you need to configure the IPv4 Settings tab.
This is the last network configuration you need to make on your OpenShift servers. These systems will use a static IP address to ensure that the hostnames always resolve properly; you’ll configure the static IP address on the screen shown in figure A.7.
In section A.1, we pointed out the hostnames we’re using for the master and node servers. These are the hostnames and IP addresses we use for the examples throughout the book.
Select Manual from the Method drop-down list; this option lets you enter static IP addresses for your servers. To fill in the values, use the networking information you gathered in section A.1.5, including the IP address, netmask, and default gateway addresses. Then, click Save to return to the main installation screen.
Once the disk and networking configurations are complete, the Begin Installation button becomes available on the main screen. Click it, and you’re on your way.
You’ll be prompted to set a password for your root user. Click the Root Password button shown in figure A.8, and you’ll be prompted to enter and confirm a root password. Type a password, and click Done. If it’s a weaker password, the installer may warn you and force you to either change it or click Done again to confirm that’s the password you want to use. For the examples, you won’t have to create any additional users.
Now you can sit back and wait for the installer to finish doing its job.
When the installer signals that it’s finished, it will prompt you to reboot the server, as shown in figure A.9. Click Reboot to restart your server so that it can confirm everything was laid down correctly.
You’re now ready to connect to your server, configure it, and install OpenShift.
All the commands in this section will be executed using an SSH connection to the proper server. If you have console access for your systems, you can use that to execute these commands. But in our experience, using an SSH client makes it possible to copy and paste text. This simple feature can save a lot of time when provisioning a system.
OpenShift has a few dependencies that you need to get out of the way before launching the installer. Just as when you built out your master and node OSs, we’ll point out if the steps are different for the master and node servers here.
You need to install a few software packages. The software that’s specific to OpenShift comes from a specialized CentOS repository, which you must install and configure on all servers.
The first step is to install the Extra Packages for Enterprise Linux (EPEL) and OpenShift Origin repository RPMs. These configure your systems to access those software repos to pull down the needed files. To do so, run the following yum command:
# yum -y install epel-release centos-release-openshift-origin36
Once that command completes, several software packages will be common to both master and node servers; they come from the two newly enabled repositories. You can install these packages with the following command:
# yum -y install origin origin-clients vim-enhanced atomic-openshift-utils NetworkManager python-rhsm-certificates
With those packages installed, there’s one more command to run on both systems: you need to enable and start NetworkManager. NetworkManager is a tool on Linux servers that configures and maintains all aspects of a server’s networking stack. OpenShift uses NetworkManager to manage the network configurations of all the servers in the cluster. You can enable and start NetworkManager by running the following command:
# systemctl enable NetworkManager --now
In the next section, you’ll make a few changes to your server’s DNS configurations.
Because DNS is used by OpenShift for everything from encrypted traffic to communication between deployed services, configuring DNS on your nodes is essential.
The steps in this section apply only if you’re using nip.io for your hostnames. If your server’s hostnames are all on the same domain, this step isn’t required.
If you’re using the nip.io domain for your server hostnames, you need to make an additional configuration to NetworkManager. In Linux, specifically CentOS 7, NetworkManager is the utility that manages the configuration for all the networking components on a server.
One of the components that NetworkManager controls is DNS resolution. In Linux, the configuration file that sets up how a server resolves DNS requests is /etc/resolv.conf.
The first line in resolv.conf lets you know that this file was created by NetworkManager. The nameserver parameter is the DNS server that your server will connect to for DNS queries. You can have up to three nameserver parameters listed in resolv.conf.
The application domain you use needs to point to your node server. This is because OpenShift uses HAProxy to route traffic properly between your DNS entry and the proper containers in OpenShift. HAProxy is a popular, open source, software-based load balancer. In OpenShift, it runs in a container on a specific host in your cluster. For this installation, it will run on the single application node.
For your cluster, the domain name you specify could be your application node’s hostname. But for larger clusters, you can specify which node or nodes run your HAProxy container. It’s a good practice to get used to specifying a domain here; we’ll use it for examples in the book.
In DNS, a wildcard domain means any host for this domain will automatically point to the same IP address. Let’s look at a couple of examples. First, here’s an actual wildcard domain that we set up on a sample domain:
$ dig +short *.apps.jeduncan.com 12.207.21.2
You can look up any other record ending in .apps.jeduncan.com, and it will return the same record:
$ dig +short app1.apps.jeduncan.com 12.207.21.2 $ dig +short someother.apps.jeduncan.com 12.207.21.2
OpenShift uses this same principle. Each application is given a domain that’s a member of the wildcard domain that’s configured. That way, all the DNS entries for your applications work without any additional configuration.
The other parameter that’s included by default in resolv.conf is search. This value defaults to the domain name for your server’s hostname. The search value is used for any DNS queries that aren’t fully qualified domain names (FQDN). FQDNs are DNS records that are complete—that means they have a hostname, domain name, and top-level domain. For example, server.example.com is an FQDN, and server isn’t.
Here’s how resolv.conf looks immediately after installation:
# Generated by NetworkManager search 192.168.122.100.nip.io 1 nameserver 8.8.8.8 2
Using the nip.io domain, each octet in the IP address is separated by a period. That means each number in the IP address is a level in the domain, with nip.io being the top-level domain. Because of some configurations OpenShift adds to each container, this can cause confusion when pulling down container images from the OpenShift integrated registry. You must edit the search parameter to have only the nip.io top-level domain, as shown in the next listing.
# Generated by NetworkManager search nip.io 1 nameserver 8.8.8.8 2
This configuration will work fine until you reboot your servers. That’s because NetworkManager controls /etc/resolv.conf, and it will add the search parameter back when the service is restarted after a reboot. To stop this from happening, you need to configure NetworkManager to not make any further changes to /etc/resolv.conf.
To configure NetworkManager, you need to edit its default configuration file. On CentOS 7, the NetworkManager configuration file is located at /etc/NetworkManager/NetworkManager.conf. The default configuration shown in the following listing is an example of a standard Python configuration file. It has two sections, defined by the lines encased in square brackets: [main] and [logging].
# Configuration file for NetworkManager. # # See "man 5 NetworkManager.conf" for details. # # The directory /etc/NetworkManager/conf.d/ can contain additional configuration # snippets. Those snippets override the settings from this main file. # # The files within conf.d/ directory are read in asciibetical order. # # If two files define the same key, the one that is read afterwards will overwrite # the previous one. [main] plugins=ifcfg-rh [logging] #level=DEBUG #domains=ALL
You need to add a line to the [main] section of this configuration file, to tell NetworkManager not to make changes to /etc/resolv.conf. Add the line dns=none as shown here:
[main] plugins=ifcfg-rh dns=none
After you restart NetworkManager, the change you made to /etc/resolv.conf will persist across server reboots. To restart the NetworkManager, run the following systemctl command:
systemctl restart NetworkManager
This should only take a second or so. Once it completes, confirm that NetworkManager is running using systemctl status:
systemctl status NetworkManager ? NetworkManager.service - Network Manager Loaded: loaded (/usr/lib/systemd/system/NetworkManager.service; enabled; vendor preset: enabled) Active: active (running) Because Sat 2017-05-13 17:05:12 EDT; 6s ago ...
The final confirmation is to check /etc/resolv.conf and make sure the search parameter hasn’t been added back to the file by the freshly restarted NetworkManager service. If there’s no search parameter, then everything is as it should be, and you’re ready to move forward.
Now you’re ready to set up software that’s specific to the master and node servers. You’ll start with the master server in the next section.
Several packages need to be installed only on the master server. This primarily has to do with installing Ansible and ensuring that it’s the correct version. Ansible (www.ansible.com) is an open source automation and deployment tool. The installation process for OpenShift is written using Ansible. One of the key features of Ansible is that it uses YAML, a human-readable data-transport language, to describe all the work it does.
To install OpenShift, you’ll create a configuration file written in YAML. This file will be read by the Ansible engine to deploy OpenShift exactly as you want. You’ll be creating a relatively simple configuration file; for more advanced installations, the full set of options is documented at http://mng.bz/CD7l.
The OpenShift installer is written and tested against a specific version of Ansible. That means you’ll need to make sure that’s the version of Ansible that’s installed on your master. You only need to worry about Ansible on the master server, because there’s no agent on the node. Ansible doesn’t use an agent on the systems it’s controlling; instead, it uses SSH as a transport mechanism and to execute remote commands. Start this process by running the following yum command:
yum -y install httpd-tools gcc python-devel python-pip
The python-pip package installs the Python application package manager named pip. It’s used to install applications written in Python and available on the Python Package Index (www.pypi.org). With pip installed, you can use it to install Ansible and ensure that you install version 2.2.2.0, which is tested with OpenShift 3.6:
# pip -v install ansible==2.2.2.0
After Ansible is installed, you’ll need to set up SSH keys from your master server to the node, and to the master as well. That’s what you’ll do next.
To get the OpenShift installer to function correctly, you need to create an SSH key pair on your master server and distribute the public key to your node. To create a new SSH key pair on your master server, you can use the ssh-keygen command as in this example:
ssh-keygen -f /root/.ssh/id_rsa -t rsa -N ''
This command creates an SSH key pair in the root user’s home directory, /root, in the .ssh subfolder. In Linux, this is the default location for a user’s SSH keys.
Next, run the following ssh-copy-id command to distribute your newly created SSH public key to your OpenShift node (if you used a different IP addresses for your master and node, adjust the command accordingly):
for i in 192.168.122.100 192.168.122.101;do ssh-copy-id root@$i;done
You’ll be prompted for the root password for your OpenShift master and node. Once you confirm each password, this command will add the public key from your SSH key pair to the authorized_keys file in /root/.ssh on the OpenShift node.
This will allow the OpenShift installer to connect to the master and node to perform the installation steps. It may seem a little odd to specify the IP address of the master when you’ll be running the installer from that server. This is because the OpenShift installer is designed so that it can be run from outside your OpenShift cluster if desired. For your first lab, and because we aren’t 100% sure what type of computer you’re using daily, it’s easier to satisfy the dependencies for the installer on the master node.
This should meet all the dependencies to run the OpenShift installer on the master server. Next, you’ll configure your application node.
The software requirements for the nodes are a little different. The biggest difference is that this is where docker will be installed. The libcgroup-tools package provides utilities that you’ll use to inspect how applications are isolated using kernel control groups in chapter 9. To install these packages, run the following yum command:
# yum -y install docker libcgroup-tools
Once this is complete, you’re ready to configure your container engine’s storage on your OpenShift nodes.
An application called docker-storage-setup is packaged with docker. It configures the desired storage for docker to use when it creates containers for OpenShift.
The examples in this book use a logical volume management (LVM) configuration. This setup creates an LVM volume for each container on demand. These are small initially but can grow to the maximum size configured in OpenShift for your containers.
You can find additional details about the storage setup in the OpenShift documentation at http://mng.bz/hR82.
The first step in this process is to create a configuration file for docker-storage-setup on your OpenShift node, as shown in the next listing. The disk you specify in /etc/sysconfig/docker-storage-setup is the second disk you created for your VM. Depending on your choice of platform for your servers, the OS’s name for this disk (/dev/vdb in our example) may vary, but the operation won’t.
cat <<EOF > /etc/sysconfig/docker-storage-setup DEVS=/dev/vdb 1 VG=docker-vg EOF
If you’re unsure about the name of the disk to use for your container storage, the lsblk command gives you a list of all disks on your server. The output is in an easy-to-understand tree diagram, as in the following example:
# lsblk NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINT vda 253:0 0 8G 0 disk 1 ??vda1 253:1 0 8G 0 part / vdb 253:16 0 20G 0 disk 2
Your second disk will have no partitions, because it hasn’t been formatted yet.
After you’ve created this file, you’re ready to run the storage-setup utility. This command should run in a couple of seconds, and the output should look like this:
docker-storage-setup Checking that no-one is using this disk right now ... OK Disk /dev/vdb: 41610 cylinders, 16 heads, 63 sectors/track ... Rounding up size to full physical extent 24.00 MiB Logical volume "docker-pool" created. Logical volume docker-vg/docker-pool changed.
With your container storage configured, it’s time to start up the docker service on your OpenShift node. This is the container runtime that will start and stop containers in the OpenShift workflows. It needs to be enabled and running prior to installing OpenShift.
To enable docker to start at boot time, and to start the service initially, run the following systemctl command:
systemctl enable docker.service --now
Like NetworkManager, you can confirm that docker started correctly by running systemctl status docker:
[root@ocp2 ~]# systemctl status docker ? docker.service - Docker Application Container Engine Loaded: loaded (/usr/lib/systemd/system/docker.service; enabled; vendor preset: disabled) Drop-In: /etc/systemd/system/docker.service.d ??custom.conf Active: active (running) since Fri 2017-11-10 18:45:12 UTC; 12 secs ago 1 Docs: http://docs.docker.com Main PID: 2352 (dockerd-current) Memory: 121.4M CGroup: /system.slice/docker.service ...
The next step is to modify SELinux to allow OpenShift to connect to NFS as a persistent storage source.
Throughout this book, OpenShift applications will need NFS volumes to act as persistent storage. To be able to do this successfully, you have to tell SELinux on your nodes to allow containers to use NFS. You do so using the setsebool command-line utility:
# setsebool -P virt_use_nfs 1 # setsebool -P virt_sandbox_use_nfs 1
OpenShift is installed using an Ansible playbook: a collection of tasks and configuration parameters needed to perform a task. To execute an Ansible playbook, three things must be present on your server:
To deploy OpenShift, you need to configure an Ansible inventory file for your cluster. That’s what you’ll do next.
The Ansible inventory for OpenShift contains information about your two hosts and specifies which roles each node will have in your cluster. If you’re using the IP addresses and hostnames we’re using in this appendix, you can download a prepared inventory to your master node from the OpenShift in Action organization on GitHub:
# curl -o /root/hosts https://raw.githubusercontent.com/OpenShiftInAction/AppendixA/master/hosts
For those of you who need to customize your installation, let’s go through the inventory components and how they’re designed.
Ansible inventories are divided into groups. Each group consists of hosts that are defined by either hostname or IP address. In an inventory, a group can also be defined by listing child groups that belong to it using the :children syntax. In the following example, the group master_group is made up of the hosts in group1 and group2. You’ll use this capability in your OpenShift inventory:
[master_group:children] 1 group1 group2 [group1] 2 host1 host2 [group2] 3 host3 host4 [group3] 4 host5 host6
Another capability of Ansible inventories that you’ll use in your OpenShift inventory is defining variables for hosts and host groups. You can define variables for an entire group using a group heading and the :vars syntax. To define a variable for a single host, add it to the same line you use to define the host in a group:
[group1] host1 var2=False var3=42 1 host1 foo=bar [group1:vars] 2 var1=True
Your initial OpenShift inventory uses several groups and lots of variables:
For the nodes and masters groups, you’ll disable a few of the system checks the deployment playbook runs prior to deployment. These checks verify the amount of free space and memory available on the system; for an initial cluster, you can use smaller values than the recommendations verified by these checks. (You can learn more about these checks at http://mng.bz/r0dI.) To disable the checks, you set variables for each of these groups:
[nodes:vars] 1 openshift_disable_check=disk_availability,memory_availability,docker_storage [masters:vars] 2 openshift_disable_check=disk_availability,memory_availability,docker_storage
Your deployment inventory will also set variables for most of the hosts. The ansible_connection variable is specific to Ansible, telling the Ansible engine to connect to the host from the local system where the playbook is running. Additional Ansible variables are discussed at http://mng.bz/g3g0.
The IP addresses and hostnames used in this inventory are specific to the example cluster. If your IP addresses and hostnames are different, you’ll need to change them in the inventory to successfully deploy OpenShift.
The rest of the variables are specific to the OpenShift playbook, and documented in the following listing, which is a full example of the OpenShift inventory.
[OSEv3:children] 1 nodes nfs masters etcd [OSEv3:vars] 2 openshift_master_cluster_public_hostname=None openshift_master_default_subdomain=apps.192.168.122.101.nip.io ansible_ssh_user=root openshift_master_cluster_hostname=None openshift_override_hostname_check=true deployment_type=origin [nodes:vars] 3 openshift_disable_check=disk_availability,memory_availability,docker_storage [masters:vars] openshift_disable_check=disk_availability,memory_availability,docker_storage [nodes] 192.168.122.100 openshift_public_ip=192.168.122.100 openshift_ip=192.168.122.100 openshift_public_hostname=ocp1.192.168.122.100.nip.io openshift_hostname=ocp1.192.168.122.100.nip.io connect_to=192.168.122.100 openshift_schedulable=False ansible_connection=local 4 192.168.122.101 openshift_public_ip=192.168.122.101 openshift_ip=192.168.122.101 openshift_public_hostname=ocp2.192.168.122.101.nip.io openshift_hostname=ocp2.192.168.122.101.nip.io connect_to=192.168.122.101 openshift_node_labels="{'region': 'infra'}" openshift_schedulable=True [nfs] 5 192.168.122.100 connect_to=192.168.122.100 ansible_connection=local [masters] 6 192.168.122.100 openshift_public_ip=192.168.122.100 openshift_ip=192.168.122.100 openshift_public_hostname=ocp1.192.168.122.100.nip.io openshift_hostname=ocp1.192.168.122.100.nip.io connect_to=192.168.122.100 ansible_connection=local [etcd] 7 192.168.122.100 openshift_public_ip=192.168.122.100 openshift_ip=192.168.122.100 openshift_public_hostname=ocp1.192.168.122.100.nip.io openshift_hostname=ocp1.192.168.122.100.nip.io connect_to=192.168.122.100 ansible_connection=local
The ocp1 node has a variable named openshift_node_labels. Node labels are arbitrary values you can apply to nodes on your cluster. The label applied during deployment, region = infra, tells OpenShift the correct node to deploy the container that runs the router to handle internet requests. You’ll work with node labels in chapter 10.
After making any inventory edits required to match your environment, save your inventory on your master node as /root/hosts. The next step is to start your OpenShift deployment.
It’s time to deploy your OpenShift cluster. Ansible uses SSH to log in to each node and perform the tasks to deploy OpenShift, so this command needs to be executed as the root user on the master, which has SSH access keys on each node. To run the proper Ansible playbook, run the ansible-playbook command, specifying the inventory file and the deploy playbook installed at /usr/share/ansible/openshift-ansible/playbooks/byo/config.yml:
# ansible-playbook -i /root/hosts /usr/share/ansible/openshift-ansible/playbooks/byo/config.yml
This launches the deployment process. Depending on your internet connection speed, the deployment could take 30-45 minutes. If everything is successful, you should see output indicating that the playbook completed. If you see a red Ansible error on the command line, it should give you an indication of what to look at.
OpenShift is a complex application, so we can’t point you to a single link of troubleshooting tips if you run into issues with your deployment. Copying the most important parts of the error and searching for those terms on the internet is always a good starting point. Also, please feel free to contact us on the Manning forum for OpenShift in Action at https://forums.manning.com/forums/openshift-in-action. We’ll do our best to help you get up and running!
Once the installation is complete, you should be able to browse to the hostname of your master server on port 8443 with HTTPS. In our example, the URL is https://ocp1.192.168.122.100.nip.io:8443 (see figure A.10). Before the page loads, you’ll probably get a warning about the site being insecure because the SSL certificate hasn’t been properly signed. Don’t worry about this—OpenShift created its own SSL certificates as part of the installation process. In our configuration, because the cluster is deployed with VMs on a laptop, the cluster is available only from the laptop the VM nodes are installed on.
If your OpenShift server looks like figure A.10, you’ve successfully installed OpenShift! You’re ready to go through the examples beginning in chapter 2.
Many of the examples you’ll go through involve using the OpenShift oc command-line tool. The next section will help you get oc installed and ready to use on Windows, macOS, or Linux.
OpenShift has a robust diagnostics toolset integrated into the oc command-line tool. Documentation for running diagnostics and checks on your OpenShift cluster is available at http://mng.bz/Zo4P.
The oc application has ready-to-use binaries for Linux, Windows, and macOS. They’re available on the Releases GitHub page for OpenShift Origin at https://github.com/openshift/origin/releases. We’re using the latest release for oc; you can find it by browsing directly to https://github.com/openshift/origin/releases/latest.
Because OpenShift Origin is a fast-moving project, there are usually one or more pre-release versions of software available for download. These versions are a little too new and untested to ensure that you have a good experience with OpenShift.
The downloadable files are listed in the Downloads section for each software release. For each OS, there’s a corresponding openshift-origin-client-tools archive that you can download. These are the files you’ll be using to install oc.
To install oc on Windows, download the windows Zip archive from the Downloads section of the Latest Release page. After unzipping the archive, move the oc binary in it into a directory in your Windows PATH.
You can see the directories that are in PATH on your Windows system from the Windows command line by running the following command:
C:> path
For macOS, the binary can also be downloaded from the Releases page on GitHub. The downloaded file is a Zip binary. Extract the downloaded oc binary file, and then copy it to a directory in your PATH. To see all the directories in your PATH, you can use the Terminal application to print out the variable using the following command:
echo $PATH
With oc in your PATH, you can execute the command from the Terminal application by typing oc.
There are 32- and 64-bit versions of the oc client available for download. If you’re unsure of the type of Linux you have installed on your system, you can confirm whether you have a 32- or 64-bit system by running the following uname command:
uname -i
The output should contain either x86_64 for 64-bit systems or i686 for 32-bit installations. Be sure to download the appropriate openshift-origin-client-tools archive for your Linux architecture—they aren’t cross-compatible. To download the Linux archive, run the following curl command:
curl -L https://github.com/openshift/origin/releases/download/v3.6.0/ openshift-origin-client-tools-v3.6.0-c4dd4cf-linux-64bit.tar.gz
After you download the archive you plan to use, extract the archive using the tar utility:
tar -xzf /tmp/openshift-origin-client-tools.tar.gz
After you extract the archive, move the oc file that was in it into a directory that’s listed in the PATH variable on your system. If you aren’t sure about the directories in your PATH, you can confirm them by echoing the variable out to your command line:
$ echo $PATH /usr/lib64/qt-3.3/bin:/usr/local/bin:/usr/local/sbin:/usr/bin:/usr/sbin:/home/ jduncan/.local/bin:/home/jduncan/bin
The OpenShift Origin documentation website has a section dedicated to getting oc up and running. It’s available at http://mng.bz/iM9L if you’d like to take a look.
Now that you’ve installed oc on your favorite OS, you need to confirm that it’s functional. You can do this by asking oc for its version information on the command line. Your output should look like the following example. For Linux, use your favorite terminal emulator. macOS uses the Terminal application, and Windows has the command prompt. This example is from a Linux workstation:
$ oc version oc v1.3.1 kubernetes v1.3.0+52492b4 features: Basic-Auth GSSAPI Kerberos SPNEGO error: You must be logged in to the server (the server has asked for the client to provide credentials)
If this executes cleanly, your system should be ready to connect to OpenShift and get work done once you have OpenShift installed and configured. You can add additional nodes to an existing OpenShift cluster using the next section as a guide.
Scaling your OpenShift cluster is essential as you deploy more application workloads. This section walks you through the steps required to add a second application node to your cluster. First, you need to provision a new server and install the prerequisites.
OpenShift manages the resources available on a node and allocates resources to it accordingly. Your second application node’s resources don’t have to match your original node.
OpenShift can run on any x86_64 server that can run CentOS 7 or Red Hat Enterprise Linux 7. In theory, your OpenShift cluster can stretch across multiple platforms like these:
Communication between OpenShift nodes requires a quality connection with relatively low latency. You’ll get the best performance if you locate your cluster on a single platform. For building out larger clusters across multiple types of server infrastructure, you should consult a company like Red Hat (or at least its documentation) for best practices and example architectures.
For our new node, we’re using an additional VM with similar resources, hostname, and IP address (see table A.5). Any additional server that can run CentOS 7 will work if it has network connectivity to your existing cluster.
Value |
|
---|---|
VCPU | 2 |
RAM | 8192 MB |
OS disk | 10 GB |
Data disk | 20 GB |
Hostname | ocp3.192.168.122.102.nip.io |
IP address | 192.168.122.102 |
Configure your new node using the same steps as the original application node covered in this appendix. These steps are summarized in table A.6.
Command |
Description |
---|---|
yum -y install origin origin-clients vim-enhanced atomic-openshift-utils NetworkManager python-rhsm-certificates | Installs base OpenShift packages |
systemctl enable NetworkManager --now | Enables and starts NetworkManager |
setsebool -P virt_use_nfs 1 setsebool -P virt_sandbox_use_nfs 1 | Sets SELinux to allow NFS persistent storage |
yum -y install docker libcgroup-tools | Installs docker and cgroup management tools |
cat <<EOF > /etc/sysconfig/docker-storage-setup DEVS=/dev/<CONTAINER_STORAGE_DISK> VG=docker-vgEOF | Creates the docker-storage-setup configuration file |
docker-storage-setup | Runs the docker-storage-setup utility |
After your new application node is provisioned and configured, reboot it to make sure all changes take effect. The server is now ready to be added to your cluster. The workflow to add your node is executed on your master server, like deploying your original cluster. Let’s discuss that next.
The first step in configuring your master node is to add the root user’s SSH to your new application node. Doing this allows the Ansible playbook that adds your node to access your new application node. To do this, run the following ssh-copy-id command. If you used a different IP address for your new node, be sure to use it when running the command:
ssh-copy-id [email protected]
You’re prompted for the root user’s password that you set during the CentOS installation process. This command sets up the SSH key from your master to provide access to the root user on your new node. With this complete, make sure the installer packages are up to date for your master server.
To make sure you have the most up-to-date version of the OpenShift installer, run the following yum command on the master node:
yum update atomic-openshift-utils
If there are updated playbooks for your version of OpenShift, this command will install them on your master, notifying you when complete. Next, add your new node to the OpenShift installer’s inventory.
When you deployed OpenShift, you created an Ansible inventory file at /root/hosts on the master node. To add the new node to OpenShift, you need to edit this inventory before running the Ansible playbook that scales your cluster.
Red Hat supports OpenShift clusters as large as 2,000 application nodes. For clusters that large, services on the master, like etcd, are separated onto their own servers to provide better performance and scalability.
To add your new node, create a group that contains the information for your node by editing your inventory file directly, as shown in listing A.5. Be sure to name this group new_nodes. Also include any labels it needs (none in this case), and tell the installer to make this node schedule-able for workloads.
[new_nodes] 192.168.122.102 openshift_node_labels="{}" openshift_schedulable=True
Next, you need to tell the OpenShift playbook that the new_nodes group is part of your OpenShift cluster. To do this, add the new_nodes group to the [OSEv3:children] group, as shown in the following listing.
[OSEv3:children] nodes nfs masters etcd new_nodes
The final step of editing your inventory is to define the variables your new_nodes group needs to function properly. You need to create this group so you can define the variables to disable a few of the resource checks that the installer performs by default because of the limited resources on the nodes. You can copy the [nodes:vars] group and change the name to [new_nodes:vars], shown in the following listing.
[new_nodes:vars] 1 openshift_disable_check=disk_availability,memory_availability,docker_storage openshift_deployment_type=origin
After creating the new_nodes and new_nodes:vars groups, and adding new_nodes to the OSEv3:children group, you’re ready to scale your OpenShift cluster.
There’s an Ansible playbook on your master node that will add your new node to your existing OpenShift cluster. You run it using the ansible-playbook command, specifying your newly edited inventory with the -i parameter. To start this process, run the following command on your master server:
ansible-playbook -i /root/hosts /usr/share/ansible/openshift- ansible/playbooks/byo/openshift-node/scaleup.yml
Once scaleup.yml completes successfully, your new node has been added to your OpenShift cluster. You can confirm this by running oc get nodes while being logged in as the admin user:
# oc get nodes NAME STATUS AGE VERSION ocp1.192.168.122.100.nip.io Ready,SchedulingDisabled 3d v1.6.1+5115d708d7 ocp2.192.168.122.101.nip.io Ready 3d v1.6.1+5115d708d7 ocp3.192.168.122.102.nip.io Ready 2d v1.6.1+5115d708d7 1
This process is repeatable as your OpenShift cluster grows over time, to make sure your OpenShift cluster always has resources to handle your workloads.