Chapter 3
The Management Layer
In this chapter, we'll discuss the points you should take into account when you're designing your management layer. We'll examine the components of this layer and how you incorporate them into your design.
The management layer comprises several components. In this chapter, we'll address what you should consider for your design regarding these items, among others:
What is the management layer? It's definitely not the executives or the board of directors of your company. We're talking about the components you use to manage your entire virtual infrastructure on a day-to-day basis. In this section, we'll provide a quick overview of vSphere's management components. We'll start with the main component, vCenter Server.
vCenter Server (which was once known as VirtualCenter Server) is one of the most centrally critical elements of your virtual infrastructure. It's the management application you'll likely use to manage your virtual datacenter. You'll create datacenters, clusters, resource pools, networks, and datastores; assign permissions; configure alerts; and monitor performance. All of this functionality is centrally configured in the vCenter Server. You should therefore dedicate part of your design to building a robust and scalable vCenter Server.
As we'll show you shortly, vCenter Server comes in a couple of different forms (either as a Linux-based virtual appliance or as a Windows application you install on Windows Server). In either case, you can download the most up-to-date version of vCenter from the VMware site, but be warned: depending on which version of vCenter Server you're downloading, it can be a pretty large download (almost 4 GB for the Linux-based virtual appliance, for example).
Prior to vSphere 5.1, vCenter Server was largely a monolithic application. There were really only three major components involved with vCenter Server in vSphere 4.x and vSphere 5.0:
With the introduction of vSphere 5.1, VMware has broken the monolithic vCenter Server into a number of different components. In addition to the three components listed previously, vCenter Server 5.1 also includes
The introduction of these additional components in vSphere 5.1 means you also have to decide if you'll run all these components on a single computer or break the roles apart on separate computers.
Let's take a closer look at each of the components in vCenter Server.
We've grouped vCenter Server and the OS instance together because these two components are directly linked to each other. vCenter Server comes in two basic flavors: as an installable application running on an instance of Windows, or as a preinstalled application running on Linux as part of the vCSA. As you can see, choosing Windows as the OS on which to run vCenter Server means you're choosing the installable application version. Choosing Linux as the OS, on the other hand, means you're choosing the preinstalled version of vCenter Server in the virtual appliance. At the time of this writing, there was no option for a Windows-based virtual appliance or an installable version of vCenter Server that ran on Linux.
Later in this chapter, we'll discuss the considerations that go into selecting either the installable version of vCenter Server or the vCSA; we'll also discuss the considerations for choosing a version of Windows on which to run vCenter Server. (Note that you can't choose a Linux distribution or version; this is preinstalled as part of the virtual appliance.)
Regardless of which form of vCenter Server you choose, you'll need a back-end database. One question that always comes up during the design phase is, “Do I use a central database server or install the database locally on the vCenter Server?” This question ranks right up there with similar questions like, “Should I use full-blown Microsoft SQL Server or Oracle as my database engine?”
We'll examine these questions and others in greater detail later in this chapter. In the section “Examining Key Management Layer Design Decisions,” we'll discuss the various reasons you might choose one database over another (vCenter Server supports several different database servers) and whether you should host the database locally or remotely.
At this stage, though, you just need to know that vCenter Server requires a back-end database. As a result, you'll need to account for the additional resources that a back-end database requires, and you'll need to plan for the maintenance and operation of the back-end database server. In the section “Creating the Management Layer Design,” we'll review how you can ensure the proper availability, manageability, performance, recoverability, and security for the back-end database. (Recall that these principles—introduced in Chapter 1, “An Introduction to Designing VMware Environments,” and referred to as AMPRS—help guide your design decisions.)
With the release of vSphere 5.1, VMware introduced a new component to vCenter Server known as vCenter Single Sign On. This component introduces a centralized authentication service that vCenter Server uses to allow for authentication against multiple back-end services, such as Active Directory and LDAP. For smaller environments, vCenter Single Sign On can be installed on the same system as the rest of the vCenter Server components; for larger environments, it can be installed on a separate system. vCenter Single Sign On also supports a variety of topologies, including a single server, a cluster of servers, and a multisite topology.
To enable greater scalability for vCenter Server, in vSphere 5.1 VMware also split off the inventory portion of vCenter Server into a separate component. The vCenter Inventory Service now supports the discovery and management of inventory objects across multiple linked vCenter Server instances. As with vCenter Single Sign On, you can install vCenter Inventory Service on the same system as the other components (in what is called a Simple Install), or you can split the Inventory Service onto a separate system for greater scalability (and perhaps high availability/redundancy).
To enable the next-generation vSphere Web Client—something we discuss in the next section—vCenter requires a server-side component referred to as vSphere Web Client. In reality, it should be called the vSphere Web Client server, because this is the server-side component that enables the use of a web browser to manage vSphere environments. This component was introduced in vSphere 5.0, but in vSphere 5.1 it takes on much greater importance because some features and tasks in vSphere 5.1 are accessible only from the next-generation web client.
Now, let's shift our focus to the client side.
Traditionally, VMware administrators have used a Windows-based application known as the vSphere Client to perform the vast majority of the day-to-day management tasks. This is true for VMware vSphere 4.x as well as VMware vSphere 5.0. In vSphere 5.0, VMware introduced a rudimentary Web Client; in vSphere 5.1, the vSphere Web Client (sometimes referred to as the Next-Generation Client [NGC]) takes its first steps toward becoming the primary way of managing VMware vSphere environments.
The Windows-based vSphere Client can be installed on almost any Windows OS:
The minimum resources required for the Windows-based vSphere Client are as follows:
For the vSphere Web Client, the following browsers are supported:
As we stated earlier, the vSphere Web Client takes on significant new importance in vSphere 5.1 environments. To take advantage of vCenter Single Sign On, for example, you must use the vSphere Web Client. If you use the traditional Windows-based vSphere Client, authentication won't use the new vCenter Single Sign On component. You'll note that throughout this book we make an effort to show all screenshots from the new vSphere Web Client as opposed to the older, Windows-based vSphere Client (which remains largely unchanged from previous versions).
These components make up the core of the management layer for a vSphere implementation. However, VMware also offers a number of optional components that you can also choose to include in your design. The first of these that we'll examine is vSphere Update Manager.
vSphere Update Manager (VUM) is an add-on that VMware provides in order to update your ESXi hosts and VMs. VUM (www.vmware.com/products/update-manager) is VMware's patch-management solution for your virtual environment. It's included with most tiers of vSphere.
In versions of vSphere prior to vSphere 5.0, VUM provided a patch-management solution not only for your ESXi hosts but also for the OSes and certain applications in supported VMs. With the release of vSphere 5.0, VMware discontinued the patch-management component for the OSes and applications in the VMs. This allows organizations to use existing patch-management solutions in place to patch their VMs the same way they patch their physical machines.
A VUM installation requires a few components to get started:
You can install VUM on the same instance of Windows as vCenter Server, if you like; or, for greater scalability and security, you can install VUM on a separate instance of Windows. Note that although VUM requires a 64-bit instance of Windows, VUM itself is a 32-bit application.
With regard to the VUM database, VUM can reside side-by-side with the vCenter database on the same database server, but it can't be the same database as the vCenter Server. In addition to setting up a separate database server (or using an existing separate database server), you also have the option of using an embedded SQL Server 2008 R2 Express database on the VUM server. As soon as you exceed 5 hosts and 50 VMs, though, you'll need to step up to a separate SQL Server or Oracle server for the VUM database.
When you install vCenter Server, you create a system data source name (DSN) entry for the database. You need to create an additional DSN entry for the VUM database as well, as you can see in Figure 3.1. Note that because VUM is a 32-bit application, this needs to be a 32-bit DSN.
Once you've installed VUM, you configure it for the updates you want to download, schedule the automatic download of patches, set up notifications, scan your ESXi hosts, and update them. Optionally, if the environment necessitates, you can set up a dedicated download server to download patches and distribute them to your VUM server(s). This is particularly applicable in high-security environments, where only certain server are permitted to access external resources on the Internet.
Now let's look at some other applications that are also part of the management layer.
Logging in to each host to perform a management task—such as configuring a network vSwitch or setting the maximum number of NFS mounts you can connect on an ESXi host—can be a tiresome and repetitive task. It may become a nightmare if (and when) your infrastructure grows.
Let's start with the basics. ESXi does have a management console, although its use is generally discouraged in favor of other solutions (which we'll discuss later in this section). Prior to vSphere 5.0, vSphere included both ESX (which offered a Service Console based on a customized Linux kernel) and ESXi (which offers a management console based on a BusyBox environment). With the release of vSphere 5.0, vSphere now only includes ESXi.
To connect these consoles remotely, you need an SSH client. Most administrators prefer using PuTTY (www.chiark.greenend.org.uk/∼sgtatham/putty). Several other tools provide the same functionality, and every administrator has a preference. Administrators using Mac OS X or Linux can also use the preinstalled SSH client that is available via their native terminal application. In vSphere 5.0 and 5.1, you'll also need to enable SSH access to the ESXi management console (it's disabled by default). This is done from the Direct Console User Interface (DCUI).
Management consoles aren't the end of the story, though. ESXi and vCenter have an extensive API you can use to perform remote management. Building on this API, VMware provides several tools to manage vCenter and your ESXi hosts remotely:
Note that we haven't included orchestration tools such as vCenter Orchestrator in this list, nor is vCloud Director included. Although they could be considered management tools, our focus here is on day-to-day management tools at the vSphere level. Note that we do discuss some vCloud Director design considerations in Chapter 12, “vCloud Design.”
The good thing about the different tools is that you don't have to choose between them: you can use them all. For example, both the Perl and PowerShell platforms have dedicated followers who constantly expand the ways you can use these tools in your environment.
Let's start with a look at the vCLI.
The vCLI command set allows you to run common system-administration commands against ESXi systems from any machine with network access to those systems. You can run most vCLI commands against a vCenter Server instance and target any ESXi system that the vCenter Server instance manages. Because vSphere 5.0 doesn't include ESX, vCLI commands are especially useful for passing commands to ESXi hosts where direct use of the management console is generally discouraged.
vCLI commands run on top of the vSphere SDK for Perl. vCLI and the vSphere SDK for Perl are included in the same installation package.
The supported platforms for vCLI are as follows:
The commands you run on vCLI for management aren't exactly the same as the commands you can run on the ESXi console (or the commands you might have used in previous versions with the ESX Service Console), so you may have to adjust to the difference. Table 3.1 shows a couple of examples.
vCLI | ESXi Shell |
vicfg-vmknic | esxcfg-vmknic |
vicfg-nics | esxcfg-nics |
VMware KB Article 1008194 (http://kb.vmware.com/kb/1008194) gives a list of some of the differences/similarities between vCLI commands and ESX/ESXi console commands.
When you're connecting to a remote host, you must—at minimum—specify the following parameters: server, username, password, and command to perform. Authentication precedence is in the order described in Table 3.2.
Authentication Method | Description |
Command line. | Password (--password), session file (--sessionfile), or configuration file (--config) specified on the command line. |
Configuration file. | Password specified in a .visdkrc configuration file. |
Environment variable. | Password specified in an environment variable. |
Credential store. | Password retrieved from the credential store. |
Current account (Active Directory). | Current account information used to establish an SSPI connection. Available only on Windows. |
Prompt the user for a password. | Password isn't echoed to the screen. |
Let's consider an example to see how these work. The environment is built as detailed in Table 3.3.
vCenter Server | vcenter.design.local |
Username | viadmin |
Password | a:123456 |
You need to create a configuration file to define these settings. The file must be accessible while running your vCLI session. A session file for the configuration in Table 3.3 looks like Figure 3.2.
You can run your commands against the hosts defined in the configuration file without having to input the credentials each time you connect to a server. Here's an example of using the vCLI on a Windows-based system:
vicfg-nas.pl --config c:usersadministratorvcli.txt --vihost esxi51-01.design.local -a -o storage1.design.local -s /shared NFS_datastore1
PowerShell is becoming the default scripting language for all Windows applications. Many articles explain why it's easier, better, and simpler to use PowerShell to manage your environment. The good thing is that VMware has made a strategic decision to follow suit and provide a PowerShell management environment for vCenter and ESX: PowerCLI.
PowerCLI has hundreds of cmdlets (pronounced “command-lets”) that deal with practically all the elements of your infrastructure. You can configure and manage ESXi hosts, VMs, the OSes of the VMs—more or less anything. A vibrant community is constantly developing ways you can use PowerCLI to allow administrators to perform their jobs more easily.
For a list of all the cmdlets, see the online cmdlet reference:
http://pubs.vmware.com/vsphere-50/topic/com.vmware.powercli.cmdletref.doc_50/Overview.html
What can you do with PowerCLI? In addition to making your toast in the morning, pretty much anything. Seriously, though, anything that is exposed in the vSphere SDK, you can access (and therefore manipulate).
Just as you create in vCLI a configuration file to store your credentials so you don't have to enter them each time you connect to your vCenter/ESXi host, you can do the same in PowerCLI. This isn't a unique feature of PowerCLI—it's more a PowerShell feature—but because you're using PowerShell, you can use it. Let's see how. First, let's look at the code for saving the credentials:
$vicredential = get-credential $vicredential.Password | ConvertFrom-SecureString | Out-File c: empviadmin.txt $VIcred = New-Object System.Management.Automation.PsCredential “[email protected]”, (Get-Content c: empviadmin.txt | ConvertTo-SecureString) Connect-VIServer -Server vcenter.design.local -Credential $VIcred
Get-Credential is a PowerShell cmdlet that lets you store credentials in an object for use without having to expose the contents of the password. Here, you store it in a variable named $vicredential. Next, you export the password from the variable into a text file. Don't worry, unlike in vCLI, the password isn't in plain text; rather, it looks something like this:
01000000d08c9ddf0115d1118c7a00c04fc297eb01000000c39f99b56e206a40a56c6e8e4ebc6ec00000000002000000000003660000c000000010000000395d0ba992b59f39e42e30a30e9c972b0000000004800000a0000000100000008deee87fd1ebd9d74990d6ed44d984d018000000494b8c989a3f55018cd9b7a743450f1d09a214843fda25b4140000005226217e4587317d235557ad8e5177541859094b
That is pretty hard to decipher. You export the password because you'll use this credential more than once—instead of having to insert the password each time, you can create a variable to store it. This is done in line 3: a variable named $VIcred is created with an already known username, but the password is imported from the file you saved. This is what the variable holds:
UserName Password -------- -------- [email protected] System.Security.SecureString
Last but not least, you connect to your vCenter Server with the credentials supplied in the variable.
After you've connected, let's use the same example as before with vCLI:
New-Datastore -Nfs -VMHost (get-vmhost) -Name NFS_datastore1 -Path “/shared” -NfsHost $storage1.design.local
vMA is a Linux-based virtual appliance provided by VMware that includes prepackaged software; specifically, it includes a preinstalled version of vCLI, which in turn includes the vSphere SDK for Perl; an authentication component (vi-fastpass) that allows a centralized direct connection to established target servers; and a logging component (vi-logger) that lets you collect logs from ESX/ESXi and vCenter Server systems and store the logs on vMA for analysis. Administrators and developers can use vMA to run scripts and agents to manage both ESX/ESXi and vCenter Server systems.
With the full transition to ESXi only in vSphere 5.0, vMA is much more relevant than it might have been in past versions of vSphere. Although ESXi includes a console shell, users are strongly encouraged to use vMA instead of the ESXi shell. In some cases, customers may prefer to use vCLI installed on their own Linux distribution; in other cases, deploying vMA might be easier. Both options allow administrators and other users access to run management and configuration commands without needing the ESXi shell.
Now, let's shift our focus to how you assemble these components to satisfy the requirements of the design. To help you better understand what's involved in creating the management layer design, we'll first look at key design decisions involved in creating a management layer design.
In this section, we'll examine some—but not all—of the key decisions involved in crafting the management layer design for your VMware vSphere implementation. Specifically, we'll examine four key decisions:
Let's kick things off by tackling the decision whether to run vCenter Server on a physical system or in a VM.
There is a long-standing discussion in the VMware world about whether you should install vCenter Server in a VM or on a physical server. In the following section, we'll cover the support for both sides of the argument.
This book is about designing a virtual infrastructure environment, so why are we talking about physical hardware? Well, a certain amount of hardware has to be involved: your ESXi hosts, at a minimum. But why would you consider installing vCenter on a physical server? Here are some possible reasons.
We've discussed why it could be necessary, or better, for your environment to have vCenter on physical hardware. Now, let's look at the other side of the coin: why you could choose to go for vCenter as a VM. Note that VMware now lists the use of vCenter as a VM as a best practice; however, as we stated in Chapter 1, it's imperative you understand why something is recommended as a best practice:
Potential Risk | Mitigation Action |
vCenter is lost when the database fails. | Separate the SQL instance on a different server to enable the use of high-availability features such as clustering. |
Both vCenter and SQL are VMs on the virtual infrastructure. | Place them on separate hosts with anti-affinity rules. |
Both vCenter and SQL are on the same storage. | Place the VMs on different LUNS or different storage devices. |
SQL data corruption occurs on the LUN. | Plan for database snapshots at regular intervals. |
vCenter/SQL suffers a performance hit due to other VMs. | Ensure that your vCenter/SQL has guaranteed resources with reservations. |
vCenter completely crashes. | Install a new VM to replace the failed VM and attach to the database, and you're back in business. |
This question is a relatively new addition to the list of commonly asked management design questions. With vSphere 5.0, VMware introduced the vCenter Server virtual appliance (vCSA), a Linux-based virtual appliance that comes with vCenter Server preinstalled. Prior to the introduction of the vCSA, you had only a single choice: a Windows-based installation of vCenter Server. Now you have to decide: Windows-based vCenter Server or vCSA? Let's look at a few reasons in favor of each option.
Here are some reasons you might deploy the Windows-based vCenter Server into your design:
Now that we've discussed some reasons for deploying the Windows Server–based version of vCenter Server, what are some reasons for deploying the vCSA? Here are a few that you might consider:
Here's a topic for endless discussion: “Where do I install the database: on the vCenter Server or on a remote server?” Let's go into the rationale behind both options.
Having your database local means you've installed it on the same OS as your vCenter Server. Here are some of the benefits of having all the components on one server:
As opposed to local, here we're talking about installing vCenter software on one server and connecting to a database that doesn't reside on the same machine. The reasons to choose this option are as follows:
The major database vendors—Microsoft, Oracle, and IBM—provide recommended resource configurations for optimal performance of their database servers. For example, Microsoft recommends a minimum of 1 GB of RAM and recommends 4 GB for optimal performance for SQL Server. In addition, take into account the resources needed for the vCenter service, web services, and plug-ins—you're looking at another 2–4 GB RAM for the vCenter host. It's pretty likely that in an enterprise environment, you have a properly sized database server that can accommodate the vCenter database without any major issues.
The following article provides more information about the recommended resources for SQL Server 2012:
http://msdn.microsoft.com/en-us/library/ms143506.aspx
This article explains that you need at least 1 GB RAM with at least one 1.4 GHz CPU. But more realistically, the recommended resources are as follows:
Taking this into account, if you install both the vCenter and the database server on the same box, you need approximately 8 GB RAM and at least 4 CPUs (either a quad core physical machine or a VM with four vCPUs configured). Later in this chapter, we'll discuss some sizing recommendations, and you'll see more information about the resources that would be required for vCenter based on the anticipated size of the environment it will manage.
We'll get into this topic a bit later in the chapter. But in short, there's no point in providing a clustered solution for only one database. If you're setting up a cluster to protect the database, you may as well protect more databases at the same time. Thus it's useless to install the database locally.
You don't see this question as much, but it's a valid design consideration. Naturally, vCenter is a server and should be installed on a server OS. But which server OS?
As of vSphere 4.1, the requirement for the Windows-based version of vCenter Server is a 64-bit build of Windows Server. This aligns well with Microsoft's stated OS strategy, which declares that Windows 2008 R2 and all subsequent server OSes will be 64-bit only. Given that you have to go with a 64-bit version of Windows Server, then the consideration of which edition of Windows Server is essentially irrelevant—the 64-bit Standard Edition of Windows Server 2008 and Windows Server 2008 R2 are both capable of addressing up to 32 GB of RAM. (32-bit versions of Windows Server 2008 Standard were limited to 4 GB of RAM, which didn't make it an ideal platform for vCenter Server.)
More information on the memory limits for various versions of Windows can be found here:
http://msdn.microsoft.com/en-us/library/windows/desktop/aa366778(v=vs.85).aspx
With these considerations in mind, it seems reasonable that selecting Windows Server 2008 R2 Standard Edition should be fine for deploying the Windows-based version of vCenter Server.
If you've chosen to go down the vCSA route, then this question doesn't apply to you because the vCSA comes with Linux preinstalled. At the time of this writing, there was no supported way to roll your own Linux-based vCenter Server installation.
There are many, many more potential management layer design questions, but in the interest of time (and pages!), let's move on to a discussion of actually creating the management layer design.
In this section, we'll focus on creating the design for the management layer. To structure the discussion, we'll use the basic principles of design as described in Chapter 1—availability, manageability, performance, recoverability, and security—as a framework around which we'll organize the information. We'll start with a discussion of availability.
The first basic principle of design to consider when creating the management layer design is availability. Availability, as we discussed in Chapter 1, encompasses metrics like uptime and, in some cases, performance (if the application is so slow as to be unusable, is it still available?).
There are a couple of ways to provide a level of availability for your vCenter Server:
Before we talk about how to provide availability, you need to understand the implications of not having your vCenter available. Table 3.5 lists the functions that will or won't be affected.
As you can see from the table, most of the management tasks related to vCenter won't work or will only partially function. Although a number of management functions are impacted, the good news is that not all VMs are affected.
Availability should be divided into two parts: the vCenter application and the database. Let's examine how you can provide availability for vCenter—first, the easiest way.
The first component you'll be protecting is vCenter. You should consider several options; it all depends on what level of redundancy is necessary for your environment and how critical it is to have your vCenter available the entire time.
By running vCenter as a VM on a vSphere HA-enabled cluster, you automatically improve availability by protecting against hardware failure. This is true for both the Windows version of vCenter Server as well as the vCSA. If your ESXi host fails, then vCenter will automatically restart on another host within a short period of time and reconnect to the database. This is a very acceptable solution in the case of host hardware failure; you shouldn't have any major downtime.
As we mentioned earlier in this chapter, current limitations on vSphere FT (limited to a single vCPU only) prevent its use with vCenter Server, given that vCenter would typically require more than a single vCPU.
If you're running vCenter Server on a physical system, then vSphere HA isn't an option, and you'll have to look at vCenter Server Heartbeat instead.
vCenter Server Heartbeat is a relatively new product from VMware that's specifically targeted at organizations that can't afford even the slightest downtime or failure of their vCenter Server. The product provides automatic failover and failback of VMware vCenter Server using failure detection and the failover process, making manual recovery and failover process definitions unnecessary. It enables administrators to schedule maintenance windows and maintain availability by initiating a manual switchover to the standby server. You can find vCenter Server Heartbeat at
www.vmware.com/products/vcenter-server-heartbeat/
The product also provides protection for the vCenter database, even if the database is installed on a separate server. The technology is based on a product from Neverfail (www.neverfailgroup.com). Figure 3.3 shows the product design.
The great thing about vCenter Server Heartbeat is that it can be placed across the WAN. If you have two datacenters (US and Europe), you can place each node in a different location to provide potentially greater levels of availability.
You'll need to address certain prerequisites and limitations:
You can install the configuration in three ways:
We won't go into the details of how to install, because this isn't a step-by-step book (if you haven't noticed by now).
At this point we do need to point out a few things. First, the product is relatively new in comparison to the rest of the vSphere product suite. We haven't met anyone who has implemented it in their environment, because it's targeted at the highest percentile of customers who can't—at any cost—have their vCenter down. It also isn't a cheap solution: it starts at approximately $12,000 per license. Finally, vCenter Server Heartbeat is only supported for the Windows-based version of vCenter Server, not the vCSA.
Also, judging from the community activity (or lack thereof), vCenter Server Heartbeat hasn't been implemented widely. Or perhaps it works like a charm, so no one has any issues with it. See
http://communities.vmware.com/community/vmtn/mgmt/heartbeat
Now, on to the second component: the database.
There are several ways to protect the database for your vCenter Server. Database protection is important because the database is the heart and soul of your virtual environment. All the settings (vSphere DRS, vSphere HA, permissions—basically, everything) are stored in the database.
The vCenter Server can be protected with vSphere HA, and the same goes for your database server. But the chance of data corruption is much higher with a database server. If the server crashes during a write operation, the data can become corrupt, and your database may be rendered unusable. In this case, a single database on a vSphere HA-enabled cluster might cause you problems.
Both Microsoft and Oracle provide their own solution for active-active and active-passive clusters for databases. The great thing is that you can combine these two options: vSphere HA and Microsoft/Oracle Cluster.
You can create a highly available database on vSphere. VMware has best practices for both platforms:
Here are a few considerations and best practices you should take into account when virtualizing these applications, distilled from the documents just referenced:
As noted earlier, this product provides redundancy for an SQL database only. That database must be the only one on the server in order for this to work.
Availability is obviously a key principle to keep in mind when creating the management layer design. The second principle we're going to discuss is manageability, and that's the topic of the next section.
In Chapter 1, we stated that the principle of manageability includes such concepts as compatibility, usability, interoperability, and scalability. Some of these concepts are outside of your control. For example, you can't control the usability of most portions of the management layer, because VMware drives that. If your users find the vSphere Web Client to be unintuitive or the syntax of the vCLI to be unintelligible, there's very little you can do to alter or address those complaints. Other aspects of manageability, however, are under your control:
Scalability is an aspect that can be addressed in a couple of different ways. One of these ways, Linked Mode, is a solution that will have a number of different design impacts and therefore deserves a bit more consideration.
In this section, we'll go into detail about vCenter Linked Mode as one potential method of providing scalability to the management layer design. VMware added this feature in the release of vSphere 4. It solves the issue of very large environments that need multiple vCenters to manage the infrastructure due to the sheer size and number of hosts and VMs.
The limit on the number of ESXi hosts that one vCenter can manage is 1,000; the limit for powered-on VMs is 10,000 per vCenter (15,000 total). For many organizations, this is more than sufficient—they can only dream of reaching that number of VMs. But for others, this limit is too small.
In comes Linked Mode. By joining your vCenter instances together, you can expand your infrastructure to 10 vCenter Servers, 3,000 hosts, and 30,000 powered-on VMs (50,000 total).
To use Linked Mode for additional scalability to the management layer design, keep in mind the following requirements. These requirements apply to each vCenter Server system that will be a member of a Linked Mode group:
You should take into account several considerations before you configure a Linked Mode group:
With these considerations in mind, let's take a peek under the covers of vCenter Server's Linked Mode groups.
How does Linked Mode work? VMware uses Active Directory Lightweight Directory Services (AD LDS, previously known as Active Directory Application Mode [ADAM]).
Instead of using your organization's Active Directory database to store the directory-enabled application data, you can use AD LDS to store the data. AD LDS can be used in conjunction with AD DS so you can have a central location for security accounts (AD DS) and another location to support the application configuration and directory data (AD LDS). By using AD LDS, you can reduce the overhead associated with Active Directory replication, you don't have to extend the Active Directory schema to support the application, and you can partition the directory structure so the AD LDS service is deployed only to the servers that need to support the directory-enabled application.
Linked Mode behaves like an Active Directory application. Let's connect to it and see what information is inside. Figure 3.4 shows an example of using ldp.exe (http://support.microsoft.com/kb/224543) to connect to a vCenter Server on port 389 (unless you've changed the default port).
It's easier to use Active Directory Services Interfaces (ADSI) Edit to see all the properties. Connect to your vCenter Server with the correct credentials, using the naming context DC=virtualcenter,DC=vmware,DC=int as shown in Figure 3.5 and Figure 3.6.
You can connect to three naming contexts:
It's a good idea to get acquainted with the structure of the schema and the configuration of the LDS instance. They aren't documented well, and the information may come in handy one day.
You need to know several things about roles when joining two or more servers in Linked Mode:
Linked Mode offers a great way to improve the manageability of a large vSphere environment by both enabling greater scalability as well as aggregating information from multiple vCenter Server instances. In fact, prior to vSphere 5.1, the only way to see information from multiple vCenter Server instances at the same time from the vSphere Client was using Linked Mode.
With the release of vSphere 5.1, though, architectural changes in vCenter Server—specifically, splitting the vCenter Inventory Service into a separate component that can service multiple vCenter Server instances—and the introduction of the new vSphere Web Client now allow environments running vSphere 5.1 to aggregate information from multiple vCenter Server instances in the next-generation Web Client without having to use Linked Mode groups.
Let's move on to the third principle of design, which is performance.
Invariably, most aspects of performance come down to sizing the components properly. So, as you consider the principle of performance when creating the management layer design, sizing the components properly will be a primary concern. With that in mind, let's discuss how you size the various management components.
You need to consider the following elements while sizing your vCenter Server:
Let's look at each of these elements.
As we described earlier in the section “Which Operating System for vCenter Server?” vCenter Server (if you're using the Windows version) must be installed on a 64-bit version of Windows. To avoid potential memory constraints, our recommendation is to use Windows Server 2008 R2.
To recap the reasons behind this recommendation, recall that one limitation of a Windows 32-bit OS is that it can only natively support 4 GB of RAM. Many vCenter Server instances were installed in the past with Enterprise Edition OSes to overcome that boundary. With a Standard license of Windows Server 2008 R2, you're limited to 32 GB of RAM, which should be more than sufficient for the vast majority of vCenter deployments. In addition, licensing for the higher versions of both the Windows OS and SQL Server is substantially more expensive than a standard license.
As we explained earlier in the section “Local or Remote Database Server?” the placement of the database for vCenter Server (and related components like vSphere Update Manager) has a significant impact on the resources that must be allocated to vCenter Server. In that section, we stated that the recommendations for Microsoft SQL Server, for example, are 4 GB of RAM and a 2.0 GHz or higher CPU. When you add these requirements to the requirements described next based on the number of managed objects, you can see that accounting for database placement is critical. If you're going to run the database local to vCenter Server, be sure to allocate appropriate resources. For maximum performance, we recommend splitting the database server onto a separate system.
VMware's documentation for installing and setting up vCenter provides the numbers in Table 3.6 as recommendations for optimal performance.
You can clearly see that at most, you need four CPUs and 8 GB RAM. The sizing of your vCenter will depend on how big a deployment you're planning.
Will you be using your vCenter Server as a VUM server as well? Why does this make a difference?
In terms of CPU or RAM resources, you won't need additional resources if you don't have the SQL Server running on the vCenter Server. What you do need is disk space. VMware provides a tool that assists in sizing your installation: the VMware vCenter Update Manager Sizing Estimator, found at this URL:
www.vmware.com/support/vsphere4/doc/vsp_vum_41_sizing_estimator.xls
At the time this book was written, the VUM 4.1 document listed here was the latest version available, and it should be applicable to newer versions of VUM as well. Note that there are also VUM patch-management guidelines included in VMware's “Configuration Maximums” documentation for each vSphere release.
The vCenter Update Manager Sizing Estimator is an Excel spreadsheet that requests information about your environment:
The results you get from the spreadsheet will look like Table 3.7.
The biggest resource you need for VUM is disk space. Because of all the patches you'll download, the required disk space will increase depending on how many versions of the software you're downloading for (either version of ESX, and the different OS service packs).
It's recommended that you not install the patch repository in the default location provided during the installation (C:Documents and SettingsAll UsersApplication DataVMwareVMware Update ManagerData). Most administrators don't notice this question during the installation; then, somewhere along the way, they begin to run out of space on the C: drive of their vCenter Server, and they wonder why. It's better to allocate a separate partition and folder for the downloaded updates, for these reasons:
In addition to sizing vCenter Server itself, it's also critical to properly size the database, as we describe in the next section.
When you follow the previous recommendations and decide to use a database server that isn't on the same system as vCenter Server, you next have to plan the size of the database. Before we get into sizing, let's review the purpose of the database in a vCenter installation.
The database is the central repository of the logic and structure of your virtual infrastructure: resource pools, permissions on each item in vCenter, alarms, thresholds, the cluster structure, distributed resource scheduling (DRS), and, of course, the biggest consumer—statistics. All the statistics for every object and counter in your environment are stored in the database, including CPU, RAM, disk, network, and uptime. And each category has multiple counters associated with it.
VMware also provides the vCenter Server 4.x Database Sizing Calculator for Microsoft SQL Server (this was the latest version available at the time of writing):
www.vmware.com/support/vsphere4/doc/vsp_4x_db_calculator.xls
It's very similar to the calculator mentioned earlier for VUM. But in this case, a larger number of parameters are required in addition to those we've already covered:
And most important, how long will you keep each level of statistics? You configure this setting in vCenter as shown in Figure 3.7.
The higher the level of statistics collected for each interval, the larger your database will become. VMware's report “VMware vCenter 4.0 Database Performance for Microsoft SQL Server 2008” includes the recommendations in Table 3.8.
Statistics Interval | Statistics Level |
Past day | Level 2 |
Past week | Level 2 |
Past month | Level 1 |
Past year | Level 1 |
The document also provides a few additional recommendations, most of which we've already discussed or are general rules of thumb. You need to allocate enough RAM for your database server. One guideline for all relational databases is to allocate the server as much memory as necessary to allow for the caching of all the needed data in memory. You should refer to the documentation for the amount of memory to give to the OS and any other applications that run on the server you use for your database. You should go with a 64-bit server for the same reason we mentioned earlier regarding the 4 GB RAM memory limitation on Windows 32-bit OSes.
Another area to consider when performing sizing calculations for vCenter Server is the database recovery model. For example, with Microsoft SQL Server the default recovery model is Full. This means the database logs will grow endlessly if no backup is performed. If your organization doesn't have a DBA on staff to manage backups of the SQL Server database, then we recommend changing this to the Simple recovery model. If you still prefer the Full recovery model, be sure to account for the additional disk space that is required to store the logs between database backups.
Plug-ins are great. They allow you to perform all the tasks you need in one management console without having to use a multitude of tools to manage your environment. As an example, most of the major storage vendors (NetApp, EMC, Dell, HP, and IBM) already have plug-ins that let you configure the virtualization portions of the storage array. Using these plug-ins, you can create new LUNs/datastores, view statistics of the VMs in correlation to the storage back-end, optimize the ESXi hosts according to vendor best practices, and more.
Unfortunately, with the advantages come some disadvantages that you didn't plan for—especially the additional resources required to run the plug-ins. You need to take into account the following resources:
You must also consider whether these plug-ins are client-side plug-ins (designed to work with the Windows-based vSphere Client) or server-side plug-ins (designed to work with the next-generation vSphere Web Client). As you can probably surmise, client-side plug-ins have a resource impact on the clients where the plug-ins are installed, and server-side plug-ins have a resource impact on the vCenter Server or the vCenter Web Client server. Because the resource requirements vary greatly among plug-ins, we can't provide any specific recommendations here other than to consult with the plug-in provider to get complete details on the resource impact, configuration impact, and operational impact of the plug-ins.
All this talk of sizing and estimators is great, but ultimately it comes down to hardware resources (physical or virtual). According to the VMware documentation, the minimum resources required for vCenter are as follows:
These minimum requirements are fine for a test environment and for kicking the tires. But if you already know that you'll be deploying a large number of hosts and VMs, you'll need more than the minimums. In fact, we've already shared with you some recommendations based on a few key factors: OS, database placement, number of objects managed, and the presence (or absence) of VUM. In selecting or allocating hardware resources for vCenter Server, you shouldn't plan for the immediate need, but rather for the expected or required expansion.
We've now considered three of the five principles of design: availability, manageability, and performance. Our next principle is recoverability.
When you design the management layer with recoverability in mind, you're designing with failure in mind. How quickly will you be able to recover a failed vCenter Server instance? How quickly will you be able to recover from a failed vCenter database? As we described in Chapter 1, recoverability describes several different ideas, including well-known metrics like Recovery Time Objective (RTO) and Recovery Point Objective (RPO) in addition to other measurements like Mean Time To Repair (MTTR). Concepts of disaster recovery and business continuity (DR/BC) are also included in this principle of design.
How does all this apply to the design of the management layer? As with all areas of vSphere design, the ideas of this principle are closely related to and heavily intertwined with the ideas in the other principles of design:
Backups are a key part of ensuring the recoverability of the management layer, and you should be sure that you're properly incorporating backup of the management layer into the design. You'll need to account for backups of all the various management layer components—vCenter Server, vSphere Update Manager (if present), other management applications, and the appropriate database servers—when crafting the backup strategy for the management layer.
Finally, complete and comprehensive documentation is key to recoverability. A measure of recoverability is not only the time required to recover from a failure or other event, but also the amount of effort required to recover. A well-crafted set of documentation that is both comprehensive and up to date can greatly reduce the amount of time required to recover the environment.
The fifth and final principle of design that you need to apply to the creation of the management layer design is security, and it's the focus of the next section.
Perhaps this part of the chapter should have come first, but its current position has no reflection on its importance in the design process. Nine out of ten administrators would rank security as one of the most important considerations in virtual infrastructure design.
How can you design your vCenter Server for security? Here are three considerations you should keep in mind:
Let's look at each of these considerations in a bit more detail.
Your vCenter Server, from a security perspective, is probably the most important component of your infrastructure. If someone compromises the vCenter Server, they can cause immense damage, from powering off VMs, to deleting data from a VM, to deleting a VM or—worse—a complete datastore.
Your vCenter Server shouldn't be accessible from all workstations in your corporate network. You can achieve this by using any of the following measures:
By default, the Administrators Security group on a vCenter Server has full permissions to the entire environment. Do you always want the administrators on the vCenter to control every VM? This isn't always the best idea. Limiting the default Administrators group's access to the full vCenter can be achieved by doing the following:
In addition, you should always follow the model of least-privilege permissions. Only give a user account the minimum required permissions needed to perform a task. There is no need to give a user the Administrator role if all they need is to access the VM console. It's a better idea to create a custom role that contains only the relevant permissions needed for the task and then assign that permission to that user.
One excellent example is VM access. Consider this: would you give any user who asked you the code for the server room so they could power on a server if they wanted to? That wouldn't be wise.
You should view your vCenter Server as a secure area of your datacenter. Not everyone should have access to the vSphere infrastructure. If you want to give users access to the server, they should access it through either Remote Desktop or an SSH session (depending on the OS).
It isn't recommended that you enable a VNC client on a VM; this requires additional configuration on each and every ESXi to allow for such remote management. In such a case, you're better off with a custom role.
Client sessions with vCenter Server may be initiated from any vSphere API client, such as vSphere Client and PowerCLI. By default, all traffic is protected by SSL encryption, but the default certificates aren't signed by a trusted certificate authority and therefore don't provide the authentication security you may need in a production environment. These self-signed certificates are vulnerable to man-in-the-middle attacks, and clients receive a warning when they connect to the vCenter Server.
If you intend to use encrypted remote connections externally, consider purchasing a certificate from a trusted certificate authority, or use your own PKI infrastructure in your domain to provide a valid certificate for your SSL connections.
You can locate the official “Replacing vCenter Server Certificates” guidelines here (this was the latest version available at the time of writing):
www.vmware.com/files/pdf/vsp_4_vcserver_certificates.pdf
The factors involved in the design of your vCenter Server shouldn't be taken lightly. You'll need to design the management layer and account for all five of the design principles: availability, manageability, performance, recoverability, and, last but not least, security.
Take into account how many hosts and VMs you have and what kind of statistics you'll need to maintain for your environment.
Size your server correctly from the ground up, so you won't need to redeploy when you outgrow your environment.
Remember that your vCenter Server should be separated from your database server, providing a separation of duties for the different components of the infrastructure. You should plan for redundancy for both components and take into account what kind of outage you can afford to sustain. When you have that information, you can plan the level of redundancy you need.
Don't be afraid to run your vCenter as a VM. In certain cases, you can provide a greater level of resilience, with more ease, than you can with a physical server.
Your vCenter is the key to your kingdom. Leaving it exposed places your kingdom out in the open. A great number of attacks today are carried out from within the network, for a number of reasons: disgruntled employees, malicious software, and so on. Protect that key with the methods we've discussed.