1.4.1. Central processing unit

The processor, also referred to as the CPU, is one of the greatest sources of heat generation within a server. Aside from the basic processing of data and instructions to provide an output result, processors may also have many more features for managing data and power throughout a system. The processor die is generally housed in a package that includes a substrate (i.e. a small printed circuit board, or PCB, for bringing out signals) and a lid, as shown in Figure 2.1. The lid, or case, more evenly distributes heat to an attached cooling component such as a heat sink (air-cooled) or cold plate (liquid-cooled), as shown in Figure 2.1. In most cases, a socket is used to enable removal and replacement of the processor on the motherboard. Some lower powered processors are lidless using direct attachment of a heat sink on top of the die. In the volume server segment, limitations of low-cost thermal solutions and a greater focus on performance per watt have slowed the generational increases in thermal design power (TDP). The transition to multicore processors has maintained Moore’s law (a doubling of the transistors on a chip every 18 months), an improvement even within this limited power envelope. However, recent trends, which include the integration of functions previously implemented in a chipset or external devices, greater numbers of high-performance I/O interfaces and memory channels, larger internal caches and incorporation of specialty processing units, are pushing power up despite significant improvements in power management features. Although integration of additional features into the processor provides an overall improvement in server efficiency, a greater percentage of the overall server power is concentrated in a smaller volume, creating new cooling challenges. Over the last several years, customer demand and regulatory requirements have driven vendors to increase server efficiency across their portfolios. Power management features within the processor have become prevalent, with increasing capability to optimize performance within power, acoustical and thermal limits. Processor, subsystem and system level power-capping features are also expected to continue improving, enabling more options for system designers to tailor servers to the customer’s power and cooling constraints. Microserver processors have appeared with power and die area is optimized for smaller, simpler and targeted/limited workloads. As this technology has evolved, it has improved its performance, capacity and reliability, availability, serviceability (RAS) features required for enterprise applications. These processors use finegrain power optimizations developed for the mobile market and as such can be computationally efficient. Although the power of these individual microservers is quite low, system designers typically place many of them on a single board or in a single chassis (tens to hundreds). From a total power-per-unit-volume standpoint, aggregate power may be equivalent or higher than a traditional volume server but with a more even power distribution (versus concentration of power in traditional volume server CPU sockets). CPU power continues to slowly increase from generation to generation. Higher performance processors, typically used in high-performance computing applications, have TDP ranging from 100 to 200 W, with the prospect that over the next few years it will increase. For telecommunications and lower performance volume servers, the TDP has historically been in the 50–100 W range and will likely increase to the 150 W range over the same period of time. Historically, it has been difficult to extrapolate today’s processor power into the future, and this remains the case today.

Table 1.1 presents a power summary of different Intel Xeon Scalable Skylake processors available in mid-2018.

1.4.2. Graphics processing unit

Graphics processing units (GPUs), also called general purpose GPUs (GPGPUs) in the industry, enable heterogeneous computing (multiple CPU types within one server) when installed into a server. A common example of this system is a server that has both GPUs, which have their own processor, and the server’s CPU(s). This application is used for high-throughput computing with a more mainstream CPU for latency-sensitive computing. This model offers significant performance and capability improvements in computing while creating challenges in data center thermal management and planning.

Originally designed for computer graphics, GPUs are increasingly being used for other computational purposes that benefit from the massively parallel architectures of these devices. Today GPUs are designed to address parallel, high-throughput computing problems (e.g. rendering a screen’s worth of data). This is similar to single instruction, multiple data (SIMD) vector processors used in old supercomputers. The aggregate compute capability of these small processors exceeds that of general-purpose processors. Memory bandwidth is also higher in GPUs compared to CPUs. Several companies have invested in this new paradigm, with GPU products currently available. Others are pursuing a somewhat different heterogeneous computing strategy, offering many integrated core (MIC) processors using a core architecture more similar to standard x86 CPUs. While both GPUs and MIC processors are similar in that they consist of a much higher number of smaller cores than in a typical CPU, there are also significant differences in their specific architecture and programming models. Each has its particular advantages and disadvantages and computational applicability, but these differences are beyond the scope of this book and will not be covered.

A downside to increased compute capability is a surge in power usage. GPUs and MICs that are relevant to servers typically draw between 75 W and 300 W, depending upon the application, with future solutions likely to draw greater than 400 W. While GPUs and MICs are usually more efficient in terms of performance per watt, the large increase in performance often results in server designs with different and larger power requirements compared to CPU-only nodes. Thus, while the overall server count and power draw of the solution may decrease, the power draw of individual servers and racks may increase, therefore making it more challenging to cool.

These GPUs can typically be deployed in a peripheral component interconnect express (PCIe) form factor; however, many manufacturers are developing products that integrate these processors on the main system board. They can have thermal solutions that are passive (no dedicated fans, instead relying on the base server air movers and heat sinks) or active (dedicated fans onboard the PCIe card itself). An active solution generally supports increased processor power and performance and has a lower platform fan power requirement for cooling than a passive solution. However, an active GPU/MIC thermal solution typically does not support fan redundancy, may increase airflow resistance and may not integrate as tightly with the host server base board thermal management strategy. Passive thermal solutions have been developed to support fan redundancy requirements and provide better integration with server management. Airflow and temperature requirements for passive solutions require custom shrouding/baffling to better manage airflow.

Table 1.2 presents a power summary of different GPUs available in mid-2018.

1.4.3. Volatile memory

Memory temporarily stores data that have been processed or are to be processed. Multiple dynamic random-access memory (DRAM) chips are packaged with many devices on one PCB. An example of form factor is the dual in-line memory module (DIMM). These PCBs have card edge connectors that allow them to be installed in sockets mounted on the board. The memory hierarchy is driven by the latency to access the data and the capacity to store the data. Onboard memory (i.e. DIMMs) is one step removed from the onprocessor memory that stores data more directly in the execution path (with lower latency) and may be integrated on the processor die or packaged on the processor substrate. As processor core count and performance continue to increase, memory channel count has also been increasing. Each memory channel provides a link to a group of connected DIMMs on that channel and can effectively expand both throughput and capacity. DRAM device density has been doubling every 24–30 months for over a decade, but that trend is expected to start to slow over the coming decade. Despite the doubling of the number of storage cells and incremental frequency for each density generation, DRAM manufacturers have previously been able to hold per-device power relatively stable through process shrinks, improved power management features and lower core and I/O operating voltages. This power is now less likely to stay the same due to less efficient process scaling. As double data rate (DDR) memory operating frequency has increased, the quantity of DIMMs per channel (DPC) is more difficult to maintain, potentially decreasing capacity. To counter the loss of supportable DPC, server DIMMs have been increasing the number of ranks per DIMM, where a rank is a set (row) of DRAMs that are accessed simultaneously to provide the channel’s full width in bits of data. To accommodate so many DRAMs, manufacturers have resorted to stacking multiple DRAM dies per DRAM package. Today’s commodity DRAMs have a single die package (SDP), and premium packaging provides a dual die package (DDP). In the future, four and eventually eight DRAM stacking will be possible using a method called 3D stacking thru-silicon-via (3DS, TSV), where vertical die-to-die connections are made directly in the silicon. These stacked die packages will be roughly of the same height as an SDP but will consume greater power. Although only one rank per DIMM can be active at a time, all of the inactive ranks still consume idle power. DIMM buffer devices (mounted on each DIMM) are also expected to consume higher power as the operating frequency increases over time. Thus, as the number of ranks per DIMM increases, the worst case and typical power of the DIMM will increase. The expected trend is that single- and dual-rank DIMMs will gradually increase their current power envelopes with DRAM frequency increases and DIMMs with four or more ranks will increase in power, introducing new cooling challenges. DIMM volatile memory power typically ranges from 4 to 15 W, depending on the number of DIMMs per channel, capacity and speed.

Table 1.3 presents a power summary of different DIMMs available in mid-2018.

1.4.4. Non-volatile memory

Non-volatile-memory-based (NVM) DIMMs are becoming more commonplace, demonstrating the crossover between memory and storage and the potential for more than one usage for the same device. These devices, coupled with a system backup battery, have the capability to provide the server with the ability to capture volatile memory data, despite system power loss, to be saved to adjacent NVM. The downside to this added capability is higher power consumption than volatile memory. An example of such technology is Intel Optane DC Persistent Memory, which will be supported in Cascade Lake, the 14 nm evolution of Skylake that will be described in Chapter 4. Optane DIMMs should be pin compatible with DDR4 pins and provide capacity of 128, 256 and 512 GB.

1.4.5. Non-volatile storage

Solid-state devices (SSD) use the NVM technology and are incorporated into several different form factors. One common size is the spinning disk’s small form factor (SFF). This SFF SSD typically consumes 9 to 12 W, but some models have the capability to reach 25 W. An alternative form factor is a PCIe card. These cards can vary in length, which will determine how many components can be placed on its printed circuit board assembly (PCBA), and will range from less than 25 W up to 300 W. Yet another more thermally challenging form factor is the M.2. There are two types of M.2 devices. Currently, the serial advanced technology attachment (SATA) M.2 has a maximum consumption of 5 W, while the PCIe M.2 consumes up to 8.25 W. At first, these power levels appear to be small until their power density is examined. Comparing the storage power as a function of length, M.2 easily doubles power density when compared to an SFF SSD drive. To compound the problem, these small devices are located in low airflow areas within the server, making them more challenging to cool. Historically, any components over 5 W are likely to have a heat spreader/sink, but these M.2s often do not have space to add heat sinks. M.2 devices come in multiple lengths and widths, with some of the more common examples being 60 mm, 80 mm and 110 mm lengths. The power that a NVM device consumes in operation is a function of its performance design point, capacity and the workload placed upon it. Ranking the performance of these storage solutions, the classical SATA and the serial attached SCSI (SAS) products would fall on the low end, the emerging PCIe and MultiLink SAS SFF products would fall in the mid-range and PCIe add-in cards would align with the high end. Finally, power will vary widely based on the workload as well, and some general rules of thumb apply:

• – Sustained 100% write operations generate the maximum sustained power profile.
• – Sustained 100% read operations generate the lowest sustained power profile, which in general will be about 60–80% of the maximum write power profile.
• – Mixed workloads will range somewhere in-between, proportional to the read-to-write ratio.

Table 1.4 presents a power summary of different M.2 and SSD-NVMes available in a server in mid-2018.

In the years ahead, as modern manycore processors evolve, providing adequate I/O performance will remain a challenge. One can expect enterprise NVM solutions to remain focused first on performance, thereby saturating the power of target form factors. This will lead to some unpredictability in projecting overall NVM subsystem trends at the server and data center levels.

1.4.6. Spinning disks and tape storage

While NVM offers low-latency permanent storage, spinning disks (hard disk drives, or HDDs) provide data storage for several years. Tape is the only medium rated for archival data storage (30 years).

Over the past 55 years, HDD size has reduced and settled on two common form factors. The large form factor (LFF) is also known as the 3.5′′ drive, while the SFF is referred to as the 2.5′′ drive.

An HDD is an electromechanical device containing two motors, resulting in a strong mechanical influence on power consumption. Operation at high I/O per second (IOPS) performance levels and/or high data transfer rates can cause power consumption to increase above idle.

The HDD is made up of several components. Internal to the HDD are a set of one or more rotating disks mounted on a spindle motor at a constant speed. While a second motor (voice coil motor, or VCM) drives the magnetic heads back and forth from the disk inner to outer diameter reading and writing data. The PCBA contains the firmware to control drive operation. The host interface is either SAS or SATA.

Over the past decade, HDD power has remained fairly constant. As shown in Table 1.5, LFF, 7,200 rpm drives have a measured idle power that ranges from 7 to 9 W, while when active the drive power ranges from 10 to 13 W. An SFF, 10,000 rpm measured idle power varies from 4 to 6 W, while in an active state these drives can consume between 6 and 9 W. All power consumed by the drive is eventually converted to heat and must be removed through the IT equipment cooling system.

In an effort to reduce HDD power consumption, SAS and SATA standards have been created. With these standards, these modes allow underused drives (idle or standby) to reach lower power states with short, but non-zero additive response times. These methods include management of the host interface, moving the actuator to the unloaded position and reducing spindle revolutions per minute to a fractional value.

Tape drives offer an inexpensive, long-term storage solution that is still being used today. The most common tape format is linear tape-open (LTO). This tape format is optimized for high capacity and performance with high reliability. Available in either single or multiple drive configurations, the LTO format is ideally suited for enterprise-level backup, restore and archive applications. LTO is the open standard alternative to proprietary tape technologies, which results in interoperability across vendors and can be a more economical product line for the user.

Table 1.5 presents a power summary of different HDDs available in a server in mid-2018.

1.4.7. Motherboard

The board (or motherboard) provides interconnections between the various active components of the servers. Typically, the boards themselves are multilayered with interconnects and circuitry residing on power and signal layers separated using dielectric layers.

VRs that convert power supply voltages to the required silicon voltages are placed on the board. With Haswell, Intel introduced an on-die (or onpackage) VRM, which is usually referred to as a fully integrated voltage regulator (FIVR) or simply an integrated voltage regulator (IVR).

Table 1.6 presents a power summary of the different components on a Xeon board available in a server in mid-2018.

1.4.8. PCIe I/O cards

Network adapter and storage controller cards are projecting significant power increases over the next several years. For those not familiar with a network adapter card have media choices of either Ethernet, twisted-pair copper, or fiber. Their speed capability ranges from 1 to 100 GB/s. If fiber media is used, an active optical connector can be used. These connectors will drive up power as their performance increases, as much as 100% over the next couple of years.

Table 1.7 presents a power summary of different PCIe network cards available in a server in mid-2018.

GPUs can also be installed in the PCIe slots and are projected to increase in power. These cards, also known as accelerators, are expected to increase in power by 75% over the next few years. Table 1.8 shows examples of existing accelerators’ power consumption available in mid-2018.

Given that these cards are increasing in power while also being rearmounted within the server with potentially temperature sensitive optical connectors, careful consideration must be given during server configuration. This will limit supported server configurations to support these cards’ maximum local inlet air temperature of 55°C (131°F) when these cards are located in the rear of the system.

1.4.9. Power supplies

The majority of mainstream IT equipment is designed for connection to an alternating current (AC) source via a power supply. The IT equipment’s power supply converts the input AC to lower voltage direct current (DC) for use by the subsystems in the computer. Power supplies are typically designed to handle a wide range of input voltage levels, enabling use around the world with a single supply that can handle input frequencies in the 50–60 Hz range. IT equipment power supplies have come under great scrutiny by IT manufacturers, their customers and regulatory agencies due to the view that the energy consumed by them does not provide any compute capability. The efficiency of a power supply (ratio of output power delivered vs. input power) depends on its load as illustrated in Figure 1.2.

As shown, a PSU can be quite inefficient at low power load. That is why the 80 Plus certification has been created (80 Plus 2011). It states that the PSU will be at least 80% efficient at given power load. Table 1.9 presents the power efficiency required to achieve the different certification levels (80 Plus, Bronze, Silver, Gold, Platinum, Titanium) at different power loads.

1.4.10. Fans

Fans are a critical part of IT devices to transport the heat from the internal components of the devices (as described above) to the front or back of the server where it will be captured by the data center. Details on the different cooling techniques are described in Chapter 2.

Table 1.10 presents a power summary of different fans in a server today in mid-2018.