Chapter 6. Network Design
OpenStack provides a rich networking environment, and this chapter details the requirements and options to deliberate when designing your cloud.
If this is the first time you are deploying a cloud infrastructure in your organisation, after reading this section, your first conversations should be with your networking team. Network usage in a running cloud is vastly different from traditional network deployments, and has the potential to be disruptive at both a connectivity and a policy level.
For example, you must plan the number of IP addresses that you need for both your guest instances as well as management infrastructure. Additionally, you must research and discuss cloud network connectivity through proxy servers and firewalls.
Management Network
A management network, typically consisting of a separate switch and separate NICs, is a recommended option. This segregation prevents system administration and monitoring system access from being disrupted by traffic generated by the guests themselves.
Consider creating other private networks for communication between internal components of OpenStack, such as the Message Queue and OpenStack Compute. VLANs are great for these scenarios.
Public Addressing Options
There are two main types of IP addresses for guest virtual machines: Fixed IPs and Floating IPs. Fixed IPs are assigned to instances on boot, whereas Floating IP addresses can change their association between instances by action of the user. Both types of IP addresses can either be public or private, depending on your use case.
Fixed IP addresses are required, whereas it is possible to run OpenStack without Floating IPs. One of the most common use cases for Floating IPs is to provide public IP addresses to a private cloud, where there are a limited number of IP addresses available. Another is for a public cloud user to have a “static” IP address that can be reassigned when an instance is upgraded or moved.
Fixed IP addresses can be private for private clouds, or public for public clouds. When an instance terminates, its Fixed IP is lost. It is worth noting that newer users of cloud computing may find their ephemeral nature frustrating.
IP Address Planning
An OpenStack installation can potentially have many subnets, and different types of services in each. An IP address plan can assist with a shared understanding of network partition purposes and scalability. Control services can have public and private IP addresses, and as noted above there are a couple of options for instance’s public addresses.
An IP address plan might be broken down into the following sections:
|
subnet router |
Packets leaving the subnet go via this address, which could be a dedicated router or a nova-network service. |
|
control services public interfaces |
Public access to |
|
Object Storage cluster internal communications |
Traffic amongst object/account/container servers and between these and the proxy server’s internal interface uses this private network. |
|
compute and storage communications |
If ephemeral or block storage is external to the compute node, this network is used. |
|
out-of-band remote management |
If a dedicated remote access controller chip is included in servers, often these are on a separate network. |
|
in-band remote management |
Often, an extra (such as, 1 GB) interface on compute or storage nodes is used for system administrators or monitoring tools to access the host instead of going through the public interface. |
|
spare space for future growth |
Adding more public-facing control services, or guest instance IPs should always be part of your plan. |
For example, take a deployment which has both OpenStack Compute and Object Storage, with private ranges 172.22.42.0/24 and 172.22.87.0/26 available. One way to segregate the space might be:
172.22.42.0/24 172.22.42.1 - 172.22.42.3 - subnet routers 172.22.42.4 - 172.22.42.20 - spare for networks 172.22.42.21 - 172.22.42.104 - Compute node remote access controllers (inc spare) 172.22.42.105 - 172.22.42.188 - Compute node management interfaces (inc spare) 172.22.42.189 - 172.22.42.208 - Swift proxy remote access controllers (inc spare) 172.22.42.209 - 172.22.42.228 - Swift proxy management interfaces (inc spare) 172.22.42.229 - 172.22.42.252 - Swift storage servers remote access controllers (inc spare) 172.22.42.253 - 172.22.42.254 - spare 172.22.87.0/26: 172.22.87.1 - 172.22.87.3 - subnet routers 172.22.87.4 - 172.22.87.24 - Swift proxy server internal interfaces (inc spare) 172.22.87.25 - 172.22.87.63 - Swift object server internal interfaces (inc spare)
A similar approach can be taken with public IP addresses, taking note that large, flat ranges are preferred for use with guest instance IPs. Take into account that for some OpenStack networking options, a public IP address in the range of a guest instance public IP address is assigned to the nova-compute host.
Network Topology
OpenStack Compute provides several network managers, each with their own strengths and weaknesses. The selection of a network manager changes your network topology, so the choice should be made carefully.
| Type | Strengths | Weaknesses |
|---|---|---|
|
Flat |
Extremely simple. No DHCP broadcasts. |
Requires file injection into the instance. Limited to certain distributions of Linux. Difficult to configure and is not recommended. |
|
FlatDHCP |
Relatively simple to setup. Standard networking. Works with all operating systems. |
Requires its own DHCP broadcast domain. |
|
VlanManager |
Each tenant is isolated to their own VLANs. |
More complex to set up. Requires its own DHCP broadcast domain. Requires many VLANs to be trunked onto a single port. Standard VLAN number limitation. Switches must support 802.1q VLAN tagging. |
|
FlatDHCP Multi-host HA |
Networking failure is isolated to the VMs running on the hypervisor affected. DHCP traffic can be isolated within an individual host. Network traffic is distributed to the compute nodes. |
More complex to set up. By default, compute nodes need public IP addresses. Options must be carefully configured for live migration to work with networking. |
VLANs
VLAN configuration can be as simple or as complicated as desired. The use of VLANs has the benefit of allowing each project its own subnet and broadcast segregation from other projects. To allow OpenStack to efficiently use VLANs, you must allocate a VLAN range (one for each project) and turn each compute node switch port into a trunk port.
For example, if you estimate that your cloud must support a max of 100 projects, pick a free VLAN range that your network infrastructure is currently not using (such as, VLAN 200 - 299). You must configure OpenStack with this range as well as configure your switch ports to allow VLAN traffic from that range.
Multi-NIC
OpenStack Compute has the ability to assign multiple NICs to instances on a per-project basis. This is generally an advanced feature and not an everyday request. This can easily be done on a per-request basis, though. However, be aware that a second NIC uses up an entire subnet or VLAN. This decrements your total number of supported projects by one.
Multi-host and Single-host Networking
The nova-network service has the ability to operate in a
multi-host or single-host mode. Multi-host is when each compute node
runs a copy of nova-network and the instances on that compute node use
the compute node as a gateway to the Internet. The compute nodes also
host the Floating IPs and Security Groups for instances on that node.
Single-host is when a central server, for example, the cloud controller,
runs the nova-network service. All compute nodes forward
traffic from the instances to the cloud controller. The cloud controller
then forwards traffic to the Internet. The cloud controller hosts the
Floating IPs and Security Groups for all instances on all compute nodes
in the cloud.
There are benefits to both modes. Single-node has the downside of a single point of failure. If the cloud controller is not available, instances cannot communicate on the network. This is not true with multi-host, but multi-host requires that each compute node has a public IP address to communicate on the Internet. If you are not able to obtain a significant block of public IP addresses, multi-host might not be an option.
Services for Networking
OpenStack, like any network application, has a number of the standard considerations to apply, such as DNS and NTP.
NTP
Time synchronisation is a critical element to ensure continued operation of OpenStack components. Correct time is necessary to avoid errors in instance scheduling, replication of objects in the object store, and even matching log timestamps for debugging.
All servers running OpenStack components should be able to access an appropriate NTP server. You may decide to set one up locally, or use the public pools available from http://www.pool.ntp.org/
DNS
OpenStack does not currently provide DNS services, aside from the
dnsmasq daemon which resides on nova-network hosts. You
could consider providing a dynamic DNS service to allow instances to
update a DNS entry with new IP addresses. You can also consider making a
generic forward and reverse DNS mapping for instance’s IP addresses,
such as vm-203-0-113-123.example.com.