Recommended Hardware Configuration
The Firm decided to deploy a Sun Cluster 3.0 environment on a pair of Sun Enterprise™ 220R servers using a Sun StorEdge™ D1000 array. These low-cost, workgroup components provide ample capacity for the planned NFS service. A Netra T1 AC200 server provides the management infrastructure services. A terminal concentrator (TC) allows easy, remote access to server consoles. For customer convenience, Sun mounts all of the components in a single rack. The paragraphs that follow describe the hardware features that the Firm considered in making this decision. FIGURE 5-1 shows the logical configuration.
Figure 5-1. NFS Cluster Logical Configuration
Management Server
Chapter 4 describes the management server in the SunTone Cluster Platform 220/1000 server. In this configuration, a low-cost, single-processor Netra T1 AC200 server is ideal.
The management server connects to a TC that provides access to the console for the management server and the server consoles of the cluster nodes.
The management server also acts as backup-and-restore server. A differential UltraSCSI card in the single PCI slot drives a Sun StorEdge™ L9 tape autoloader library. “Backup, Restore, and Recovery” describes this library in more detail. TABLE 5-3 lists the parts for the management server and other support components.
| Quantity | Description |
|---|---|
| 1 | 72-inch StorEdge expansion rack. Includes power sequencers with independent power cables and individual peripheral power cables (distributed among the two power sequencers) |
| 1 | Netra T1 AC, one 500 MHz UltraSPARC processor, 1 Gbyte of memory, and internal 18-Gbyte UltraSCSI drive for the management server |
| 1 | Optional internal 18.2-Gbyte UltraSCSI drive for the management server |
| 1 | RJ45/DB25 Netra serial port adapter for the management server |
| 1 | Terminal concentrator kit |
| 1 | Terminal concentrator mounting bracket |
| 3 | 5-meter terminal concentrator RJ45/DB25 cable to access management server and cluster node consoles |
| 2 | 436 Gbytes (12 × 36.4-Gbyte 10K RPM disks) Sun StorEdge D1000 with two power supplies, two fan trays, four UltraSCSI differential host ports and rackmount rails |
| 4 | 4-meter SCSI 68 TO VHDC differential UltraSCSI cable |
Options
The Netra T1 AC200 management server offers little room for expansion or options. The only hardware option is to add more memory, up to two gigabytes.
Nodes
The cluster nodes are two CPU workgroup servers. These servers provide a cost- effective cluster solution with limited expansion capability. The number of PCI slots limits the amount of I/O devices. Also, only two CPUs can be added without upgrading to a larger server.
The Sun Enterprise 220R is a workgroup server optimized for rackmounting. The key features are:
One or two UltraSPARC II processors running at 450 MHz with 4 Mbytes of E-cache
Up to 2 Gbytes of ECC protected main memory
One 5.25-inch peripheral bay that can house a CD/DVD, 4mm, or 8mm tape drive
Two serial ports. One port is used as a console device.
One parallel port
One Quad FastEthernet port with unshielded twisted pair (UTP) and media independent interface (MII)
Four full-length PCI slots. One PCI bus has a dedicated slot, another PCI bus is shared among three slots.
Rack-optimized 4U design
One or two load-sharing, hot-swappable power supplies
The cluster nodes are built to provide sufficient redundancy for the cluster. All four PCI slots have Sun Quad FastEthernet™ cards and dual UltraSCSI cards for the interconnects, public networks, and shared storage. TABLE 5-4 lists the parts for each node.
Options
The nodes have no hardware expansion options or unused PCI, CPU, or memory slots.
Options Considered But Discounted
The Sun Enterprise 220R can support a keyboard and graphics card. However, the Firm sees little need to add such devices to a rackmounted server.
An alternate node choice is the Sun Enterprise™ 420R server. This server has room for more processors, up to four, and memory, up to 4 Gbytes. While the extra computing power and memory might be useful for some applications, they do little to enhance the performance of the target environment, workgroup-sized NFS service. This server also has four PCI slots. However, one of the PCI slots is a short slot because of mechanical restrictions. The short slot changes the I/O configuration so that instead of two Sun Quad FastEthernet cards, only one will fit physically. Two of the long slots contain the shared storage HBAs. The remaining slots, one long and one short, are available for cluster interconnects and public networks. The redundant interconnect would have to use a single FastEthernet card in the short slot. The loss of three FastEthernet interfaces reduces the capability of providing multiple, redundant public network interfaces. Given the network limitations and the fact that additional computing power is not needed, the Firm does not believe the Sun Enterprise 420R is a viable alternative.
Boot Environment
The boot environment consists of two, hot-pluggable, 36.4-Gbyte UltraSCSI disks. These disks share a common UltraSCSI bus that is built into the motherboard. The factory mirrors the boot disks with Solstice DiskSuite™. Although the common UltraSCSI bus is a single point of failure (SPOF), it is not critical to the overall cluster services because they failover to the other node in case of a SCSI bus failure.
Shared Storage
The shared storage is low-cost, UltraSCSI disks in a Sun StorEdge D1000 array. Sun can rackmount these with the cluster nodes. The storage uses Solstice DiskSuite software instead of a hardware RAID controller.
This array does not have an embedded hardware RAID controller. Combining this array with the VERITAS Volume Manager software or with the RAID capabilities that Sun embeds with the Solaris operating environment achieves software RAID solutions. This array has two UltraSCSI channels (four UltraSCSI connections). The the Sun StorEdge D1000 backplane is split into two 4- or 6-drive segments. Two of the UltraSCSI connections can be jumpered to create a single 8- or 12-drive segment. This array is a software RAID solution that offers many capabilities and features:
Slots for 12 single-ended, hot-pluggable, UltraSCSI drives. Configurations are available with 72 to 436 Gbytes of storage, using 1-inch high, 18- or 36-Gbyte, 10K RPM disk drives.
Two independent UltraSCSI buses with differential interface to a host. The buses can be daisy-chained to form one physical bus. The differential UltraSCSI interface allows total cable length up to 25 meters.
Redundant, hot-swappable, load-sharing power supplies. Power supply fault sensing and failover.
Redundant, hot-swappable cooling fan trays with two fans each. Fan tray fault sensing and failover
Environmental sensing
Software RAID using Solstice DiskSuite software
Higher RAS than a Sun StorEdge™ MultiPack disk drive
Compact, 4U height, mountable in standard 19-inch racks
Options
The cluster nodes have two dual differential UltraSCSI adapters. Each Sun StorEdge D1000 array uses only one of the UltraSCSI buses per node. A minimum of two Sun StorEdge D1000 arrays is required to eliminate all SPOFs in the cluster. The cluster can support up to two additional Sun StorEdge D1000 arrays before reaching capacity. This support would provide approximately 436 Gbytes of file system space, which is more than the Firm anticipates requiring for this project.
Options Considered But Discounted
Sun can upgrade the Sun StorEdge D1000 array to include a hardware RAID controller, the Sun StorEdge™ A1000 array. However, Sun Cluster 3.0 currently does not support the Sun StorEdge A1000 array.
The Sun StorEdge MultiPack disk drive is another low-cost storage option for clusters. However, the StorEdge MultiPack cannot be rackmounted. This limitation exposes the system to mechanical packaging risks.
The Sun StorEdge T3 Array provides FC-AL hardware RAID capabilities. However, this option adds significant costs. FC-AL host bus adapters (HBAs) are more expensive than UltraSCSI HBAs, and the Sun StorEdge T3 Array is significantly more expensive than the Sun StorEdge D1000 Array on both a unit and cost-per-gigabyte basis. Also, the potential storage capacity for the StorEdge T3 array (nine disks) is less than that of the Sun StorEdge D1000 (twelve disks).
The VERITAS Volume Manager was not used because the extra cost of a license would increase the cost of the cluster, which is against the design priorities.
Network and Interconnects
FastEthernet is the chosen technology for the cluster interconnect, cluster administration network, and public network.
Options
The Firm can connect the cluster administration network directly to the public network. This would facilitate access to the console serial ports if the management server is down. Alternatively, the Firm can connect the cluster administration network to other devices, such as the system administrator's desktop workstation. TABLE 5-5 lists the network port assignments for the FastEthernet interfaces on each cluster node.
| Port | Description |
|---|---|
| Onboard FastEthernet | Cluster administration network, NAFO group 1 |
| QFE No.1, port 0 | Cluster interconnect |
| QFE No.1, port 1 | Public network, NAFO group 2 |
| QFE No.1, port 2 | Unused |
| QFE No.1, port 3 | Unused |
| QFE No.2, port 0 | Cluster interconnect |
| QFE No.2, port 1 | Public network, NAFO group 2 |
| QFE No.2, port 2 | Unused |
| QFE No.2, port 3 | Unused |
Options Considered But Discounted
Gigabit Ethernet is an option for the cluster interconnects. However, adding Gigabit Ethernet requires the removal of the Sun Quad FastEthernet cards. In such a configuration, the public network has to use the FastEthernet interface on the motherboard. This reduces the potential availability for the public network because there is no redundant public network interface. The public network would be a NAFO group with one interface. The internode traffic the Sun Cluster 3.0 HA-NFS generates is minimal and does not warrant the performance gains achieved by using Gigabit Ethernet and the subsequent loss of NAFO on the public network.
FIGURE 5-2 shows the NFS cluster network configuration options selected.
Figure 5-2. NFS Cluster Network Configuration Options
Environmental
The cluster uses components designed for office environments. The Firm can supply 110 or 220 VAC and 50 or 60 Hz power. The audible noise generated by the fans is quiet enough to be comfortable in an office setting. These considerations are important to the Firm, because they lack a raised floor data center environment. However, the Firm decided to place the components in a single rack, such as the Sun StorEdge expansion cabinet, to use floor space efficiently, and to reduce the risk of accidental physical disruption. Also, the Firm decided to install a rackmountable uninterruptible power supply (UPS) to provide power in the case of a blackout. FIGURE 5-3 shows the NFS cluster rack configuration.
Figure 5-3. NFS Cluster Rack Configuration
Options
The cabinet has some room for expansion. This area could be used in the future for additional disk capacity or other components.
Options Considered But Discounted
A four-rail, Telco-style rack is another possibility. However, these racks do not provide protection from accidental mechanical disruption. Also, the Firm must bolt these racks to the floor, making deployment in their office environment difficult.
Backup, Restore, and Recovery
File server users expect their production data to be guaranteed by a backup-and- restore solution. At a minimum, the system administrator should back up the user files daily, using the incremental backup option, which backs up only files that have changed since the last backup to the tape media. The proposed backup-and-restore solution for the HA-NFS server is the Solstice Backup™ software, which executes on the management server and provides automatic backup and ad hoc restore services to file server users of the HA-NFS logical host. The logical host abstraction makes current ownership of the HA-NFS service disk data irrelevant.
The Solstice Backup 6.0 software is Sun's branded version of Legato NetWorker®×6.0 software, which provides backup and recovery capabilities for midsize to large heterogeneous UNIX, Microsoft Windows, Novell, and Linux environments. The Solstice Backup 6.0 software provides a system administrator's interface to implement backup and recovery policies and to establish an unattended, incremental backup schedule for the user files on the HA-NFS server. The Firm can group file server users according to different file access needs, and create separate backup schedules to implement a continuous backup strategy. File server users can do ad hoc backups and recover their own files using a graphical online index tool.
The management server has a dual-channel differential ultrawide SCSI PCI card that provides connectivity to a Sun StorEdge L9 Tape Autoloader. The tape autoloader consists of a robotic system that manipulates data cartridges from nine separate slots into a single DLT™ 8000 tape drive. Each tape cartridge has a native (no software or hardware compression) capacity of 40 Gbytes, which provides an accumulated capacity of 360 Gbytes of storage.
The native tape drive transfer rate is 6 Mbytes/sec and the transfer rate for the Ethernet network connection between the management server and the Sun Cluster 3.0 nodes is 100 Mbytes/sec (approximately 8 Mbytes/sec). It is safe then to assume that the transfer rate between the Sun Cluster 3.0 nodes and the tape media is between 4 to 6 Mbytes/sec. (The 4 Mbytes/sec figure assumes that the production network is busy). FIGURE 5-4 shows the expected backup times.
Figure 5-4. Expected Backup Times
Assuming a native transfer rate of 4 to 6 Mbytes/sec for the tape connection, and assuming that the entire file user community generates 5 Gbytes of new and modified files every day, the daily incremental backup will take 14 to 20 minutes.
where:
IBT = incremental backup time
Assuming a native transfer rate of 4 to 6 Mbytes/sec for the tape connection, a full backup of the entire shared disk (218 Gbytes) will take 10 to 15 hours.
where:
FBT = full backup time
Note
Chapter 4 of the Sun BluePrint Backup and Restore Practices for Sun Enterprise Servers [SKB00] describes a model for sizing distributed tape backup systems.
Justification
A basic service level agreement (SLA) for file server users involves daily backups. To guarantee the validity of the data, the system administrator makes the daily backups when file server users are idle, to be certain the data is not changing while the tape backup is occurring.
Options
A backup solution alternative is the Veritas NetBackup software.