Storage
Storage should be a key discussion point when considering the integration of IBM PureApplication System or IBM PureApplication Software into a data center. Both PureApplication products support different storage configurations for various architectural reasons.
This chapter provides descriptions about basic storage concepts and how those concepts are applied to PureApplication System and PureApplication Software. The first part of the chapter describes basic concepts such as storage volumes, block storage, and file-level storage. There is an additional description about how these concepts are implemented and used in PureApplication System.
The second part of the chapter describes external storage options, in particular by using an IBM SAN Virtualization Controller to connect an existing SAN. The other external storage option that is described is how to add storage capacity to the IBM Storwize V7000 storage system.
The final part of the chapter provides a description about IBM General Parallel File System (GPFS) topologies.
This chapter covers the following topics:
5.1 Storage principles
The following sections are provided as the background and foundation for this chapter. It is important to understand several basic storage concepts before proceeding with this chapter. The sections that follow describe basic concepts such as storage volumes, block storage, and file-level storage. There are additional descriptions about how they are implemented and used in PureApplication System.
5.1.1 What a storage volume is
When a pattern is deployed by using PureApplication System, the result is one or more virtual machines (VMs). Each VM has at least one or more volumes attached. A volume is a logical disk drive that typically represents a single partition of a drive.
With PureApplication System, all VMs start from a volume that is internal to the rack. Secondary volumes can be RAW, VMFS, Block, or Block Shared. For more information about these secondary volumes, see 5.1.5, “Block storage and file-level storage in PureApplication System” on page 107.
On a POWER-based PureApplication System, either RAW or block volumes are available. Both RAW and block volumes are implemented as individual Logical Unit Numbers (LUNs). Boot volumes are individual LUNs.
On an Intel based PureApplication System, RAW, VMFS, Block, or Block Shared volumes are available. Each VMware data store is a LUN. Individual VMFS volumes are stored as a vmdk inside a data store. Just like POWER-based systems, RAW, Block, and Block Shared volumes are implemented as individual LUNs. Boot volumes are always VMFS.
|
Definition: A Logical Unit Number (LUN) is a unique identifier to designate an individual or collection of physical or virtual storage devices that run input/output (I/O) commands with a host computer, as defined by the Small System Computer Interface (SCSI) standard
|
5.1.2 Base OS images
Base OS images that are shipped with PureApplication System use Logical Volume Manager (LVM) to manage volumes. Administrators can use the LVM to manage volumes and file systems more independently. This capability provides more flexibility in how volumes are grouped and managed. This capability applies to both deployment and operations throughout the lifecycle of a deployed VM. Figure 5-1 on page 105 shows the block LVM definition in the default Red Hat OS image.
Figure 5-1 Block LVM definition in the default Red Hat OS image
The default layout of the IBM OS Image base OS is a single volume. On Intel, the layout is /dev/sda. This volume is split into two partitions: sda1 and sda2. sda1 is the /boot partition and sda2 is used for LVM. sda2 is part of the volume group rhel. Volume group rhel (see Figure 5-2 for the default volume group definition) is split into two logical volumes: rhel-root and rhel-swap.
Figure 5-2 Default volume group information
Figure 5-3 shows details of the block device layout for the default Linux base OS.
Figure 5-3 Block device layout for the default Linux base OS
5.1.3 What block storage and file-level storage are
There are two main types of storage technology in use today: block storage and file-level storage. Nearly every time a file is accessed, data is accessed at the file level and at the block level. The amount of abstraction between the OS and the storage media where files are stored varies widely. There can be multiple physical devices and different protocols that are involved when getting files from storage.
In essence, block storage is the physical storage and file-level storage is how the physical storage is organized and accessed. The next two sections take a closer look at block storage and file-level storage.
Block storage
Block storage is raw volumes that are exposed to the OS. They must be partitioned and formatted before they can be used for file storage. Block storage is exposed to an operating system either as a direct connection or through a storage protocol. Examples of block storage protocols are Fibre Channel, iSCSI, and FCoE.
In a SAN solution, block devices are exposed directly to the operating system. The SAN administrator creates a LUN and makes it available to one or a group of hosts through the storage fabric.
File-level storage
File-level storage is familiar to anyone who uses a computer. The folders, paths, and permissions of the file system are all concepts that are part of file-level storage.
File-level storage is exposed in the OS where the file system is. Network-attached storage solutions such as NAS, NFS, and Samba all present file-level storage to the user. In an NAS solution, the block device is abstracted from the user. The RAID level, partitioning, and formatting are all handled by the host device. The user is not aware of this setup.
|
Definitions:
Network-attached storage (NAS) is a type of dedicated file storage device that provides local area network (LAN) nodes with file-based shared storage through a standard Ethernet connection.
Network File System (NFS) is a distributed file system protocol that allows a user on a client computer to access files over a network much like local storage is accessed.
Samba is an open source, no-charge software suite that provides file and print services to SMB/CIFS clients. Samba is freely available and allows for interoperability between Linux and UNIX servers and Windows based clients.
|
5.1.4 How block storage and file-level storage are used
This section covers block and file storage, and the difference between the two types.
Your notebook
A typical notebook has a single hard disk drive (HDD) that is connected to the rest of the computer through the PCIe bus. When the HDD is new, it first must be partitioned. Partitioning creates the C: and perhaps D: and E: drives.
Although the HDD has the C:, D:, and E: partitions, this does not mean that they are usable for file storage. They must be formatted with a file system such as FAT32 or NTFS. After they are formatted, the partitions are ready to be used as C: D: and E: drives.
To summarize, the HDD is the block device. The file system contains the file devices. The bus in between them implements the storage protocol that connects the block device to the OS and thus makes the file systems available.
|
Definitions:
FAT32 is the most current version of the File Allocation Table that the operating system uses to find files on a disk. Due to fragmentation, a file may be divided into many sections that are scattered around the disk. The FAT tracks all these pieces.
New Technology File System (NTFS) is a high-performance and self-healing file system that is proprietary to the Windows platform. It supports file-level security, compression, and auditing. It also supports large volumes and storage solutions, such as RAID. Its most important features are data integrity (transaction journal) and the ability to encrypt files and folders.
|
Linux system
Consider the same notebook, but with Linux installed on it. It still has that same HDD that is the block device, in this case, /dev/sda. If you create partitions /dev/sda1 and /dev/sda2, then you can format them by using a file system such as EXT3. After the file systems are created, they are ready to be mounted as part of the overall file system for your Linux box.
5.1.5 Block storage and file-level storage in PureApplication System
Storage volumes in a large system like PureApplication System work much the same way as described for your notebook or a Linux system. PureApplication System might introduce new terms such as SAN, Fibre Channel, Block Storage, RAW, and LUN, but in reality the concepts are not new at all. The concepts might be unfamiliar if you are new to PureApplication System, but they are not new. They are analogous to the notebook sitting on your desk.
PureApplication System has four types of storage devices that can be used and attached to your VM: VMFS, RAW, Block, and Block Shared. This section looks at each type in more detail.
VMFS
VMFS is VMware's proprietary file system. VMware storage is defined as data stores. The file system within each data store is VMFS. On Intel based PureApplication Systems, the boot volume for each deployed VM is in a VMFS data store. Multiple ESXi hosts can share each data store, which are isolated on cloud group boundaries so that each cloud group has its own set of data stores.
PureApplication System handles storage placement for new deployments. If there is no suitable data store for a new deployment, a data store is created. The default size for new data stores is a 1.8 TB LUN on a Storwize V7000 storage system. Placement continues to use the available data stores until they are above a capacity threshold. After that threshold is reached, another new 1.8 TB LUN is created and used as a new data store.
It is possible to have circumstances where a particular data store becomes too full and certain actions start to fail because of insufficient capacity. The capacity algorithm tries to balance future growth with current density in the storage arrays.
RAW
RAW volumes are simply LUNs on a Storwize V7000 storage system that are created and available to a cloud group. RAW volumes cannot be moved between cloud groups, and they lack other advanced features that are available with Block Storage volumes.
Block
Block volumes are also RAW LUNs on a Storwize V7000 storage system. They can also be LUNs that are defined on some other attached storage device and made available through a SAN Volume Controller.
External block volume can be moved between cloud groups. For example, block volumes can be attached to a VM in cloud group A. After it is unmounted and detached, it can be moved to cloud group B.
Block volumes also can be imported, exported, cloned through IBM FlashCopy®, and grouped into consistency groups. PureApplication System allows block volumes to be replicated between two PureApplication Systems, which can be done synchronously over metropolitan distances or asynchronously otherwise. These features are often used as key components in high availability (HA) and disaster recovery (DR) scenarios.
|
Definition: FlashCopy is an IBM feature that is supported on various IBM storage devices that makes it possible to create near-instantaneous point-in-time snapshot copies of entire logical volumes or data sets.
|
Block Shared
Block Shared devices have all the same qualities as a block device, but have the unique ability to be attached to more than one VM at a time if the VMs are on different compute nodes. Block Shared devices are used by the GPFS pattern for synchronous access and data consistency.
Add-ons
Add-ons are specialized scripts that customize your VM configuration. Add-ons provide fine-tuning for hardware and operating system configurations.
At deployment time, all add-on operations are run before the custom script packages are run. The order in which add-ons are run at deployment cannot be specified in a Virtual System Pattern (VSP), unlike scripts and parts. Add-ons always run as the system is created. They are not initiated by users, and they do not run at deletion. Depending on the type of add-on, special processing is performed during deployment to add the additional hardware to the VM as needed.
Add Disk
Add Disk does exactly as its name implies: It adds a disk. The result is similar to a virtual volume. Depending on the platform and the add-on that is used, the disk might be a new VMFS volume in a data store or it might be an additional LUN.
A word of caution: The middleware team should work with the UNIX SMEs. Using an Add Disk add-on is not the same as adding a directory or a mount point. From a resource impact perspective, there is a limit to the number of volumes and LUNs that can be attached to a single VM. The SCSI bus in the VM can handle only 14 devices. If you want to separate file systems, you can accomplish that goal with partitions in a single volume.
Block Volume
Block volumes can be specified and automatically attached to a VM at deployment time. Because block volumes have a decoupled lifecycle from the deployed instance to which they are attached, this can be a powerful concept. It is possible to have a block volume that is created and attached to a temporary system. While it is attached to the temporary system, the block volume can be partitioned and formatted as needed. The block volume can be pre-populated according to the needs of future deployments. After the volume is set up as wanted, it can be detached from the temporary system and reused with other new deployments. If needed, you can make multiple FlashCopy copies of the block volume and use it for many purposes, such as testing or software installation. Another popular scenario for block volumes is backup and restore, which is described in Chapter 10, “Backup and restore” on page 275.
Block volumes can also be imported and exported. In this way, you can have pre-built file systems that are available to be attached to any system with which you want to use them.
Groups of block volumes are added to block volume groups so that they can be managed together. Block volume groups are implemented as consistency groups within the Storwize V7000 storage system.
Viewing Volumes and Volume Groups
You can view the table of the volumes that are known to the rack or to the software instance. You can see the device type, location, and VMs by using the volume, and the volume group to which the volume belongs.
To view the volumes on a PureApplication System, complete the following steps:
1. From the console, click Cloud → Storage → Volumes. A list of volumes is displayed, as shown in Figure 5-4.
Figure 5-4 List of volumes
2. Click one of the volumes to view its details, as shown in Figure 5-5. From here, you can see the volume type (VMFS, Block, or Block Shared). You can also see to which volume group the volume belongs.
Figure 5-5 Volume details
To view the volume groups on a PureApplication System, complete the following steps:
1. From the console, click Cloud → Storage → Volume Groups. A list of volume groups is displayed, as shown in Figure 5-6.
Figure 5-6 List of volume groups
2. Click one of the volume groups to view its details, as shown in Figure 5-7 on page 111.
Figure 5-7 Volume group details
5.1.6 Block storage and file-level storage in PureApplication Software
Section 5.1.5, “Block storage and file-level storage in PureApplication System” on page 107 described using block storage and file-level storage principles as they apply to PureApplication System. Because the infrastructure of PureApplication Software is fundamentally different than PureApplication System, block storage and file-level storage are handled differently.
The SAN administrator is responsible for creating LUNs and making them available to the deployment host or host group. After they are available, PureApplication Software can initiate a discovery process. The discovery step is performed by VMware, so it is easy to watch how the process flows in vCenter.
If you create VMFS data stores and make them available to PureApplication Software, then PureApplication can create VMFS volumes in the data store.
When a new VM is deployed, the boot volume for the VM is a.vmdk file that is stored in the data store. Other volumes that are created for the VM can be stored in the data store and managed by PureApplication Software. You can also create LUNs in the SAN and use them the same way that you use any other block volume.
An additional option that is available with PureApplication Software is the ability to use iSCSI to create block volumes. iSCSI can be used for block devices and for creating the data stores that you attach to your VMware hosts.
|
Definition: iSCSI is a lightweight block storage protocol that uses the data network through TCP/IP.
|
5.2 SAN Virtualization Controller for external storage
If you want to use an existing SAN by connecting it to PureApplication VMs, you can implement a SAN Virtualization Controller. The addition of a SAN Virtualization Controller allows you to access potentially the VMs at high IOPS rates. The SAN Virtualization Controller allows you to expand storage and potentially achieve higher data throughput than with the standard Storwize V7000 storage system in the rack.
5.2.1 When to use a SAN Virtualization Controller
There are two general use cases when you want to use a SAN Virtualization Controller:
1. DBaaS workloads that demand high IOPS.
2. Block devices that are shared between racks or even data centers can be facilitated through SAN Virtualization Controller.
Pros
The addition of a SAN Virtualization Controller allows you to access greater amounts of storage than what is available in the PureApplication System:
•Nearly unlimited storage for workloads
•Increased compute density
•Ultimate flexibility in options for both HA and DR scenarios
•Fast throughput and access to data for applications that demand high IOPS
Cons
Infrastructure complexity and increased cost are the drawbacks to implementing a SAN Virtualization Controller. Although the SAN Virtualization Controller appliance is relatively inexpensive, it is licensed by each TB of data that is virtualized.
•Adding a SAN Virtualization Controller and connecting it to an EMC or NetApp appliance adds to the complexity of the solution.
•Additional hardware and firmware must be tested to ensure compatibility.
•Additional vendors must be contacted for technical support.
5.3 Adding additional storage to the PureApplication System
Another option for existing clients is the ability to add native storage capacity to the Storwize V7000 storage system in the PureApplication System racks through an additional feature code and is performed in the data center. The task adds one or more additional expansion units to the Storwize V7000 storage system.
Depending on your system generation, there are different requirements:
•Gen 1 systems: There is no space for an additional enclosure. You can add storage enclosures, but they must be physically mounted in another frame. The additional frame must have PDUs for the storage array and be physically close to the PureApplication System frame. Unlike fiber cables, Direct Attach Cable (DAC) cables are not long, so proximity is vital.
•Gen 2 systems: The expansion units are powered by the PDUs in the rack and then connected to the Storwize V7000 array through a daisy-chained DAC connection.
|
Definition: Daisy chain is a wiring scheme in which multiple devices are wired together in sequence or in a ring.
|
•Gen 3 systems: If there is space in the frame, you can take advantage of the built-in PDUs.
5.4 IBM General Parallel File System
General Parallel File System (GPFS) is a high-performance, shared-disk file management system that provides reliable access to multiple nodes in a cluster environment.
GPFS allows the same file to be accessed from multiple different clients. GPFS is built to be redundant so that the file system remains active even if the host nodes become unavailable or inoperable.
PureApplication System provides the IBM Pattern for GPFS, which the system administrator can deploy as a GPFS cluster. Figure 5-8 shows an example of a GPFS cluster. The GPFS cluster provides high-performance, highly available storage to the middleware and applications that are deployed in the PureApplication System environment.
Figure 5-8 GPFS cluster example
The system administrator is responsible for the following actions:
• Deploying the GPFS Server pattern
• Attaching storage
• Defining file systems
• Deploying the GPFS shared service
5.4.1 GPFS topologies
Using the IBM Pattern for GPFS, the system administrator can deploy a number of different file server topologies. All deployments provide a GPFS Manager VM that directs and controls the underlying GPFS cluster. From the GPFS Manager VM and its application management capabilities, administrators can define file systems, add storage, add file system mirrors, manage security credentials, apply recovery actions in the case of a failure, and attach a passive replica.
The GPFS Server is the first component that is deployed when you use a GPFS shared file system topology. This component is deployed as a virtual application by using the GPFS Pattern Type. When you deploy a GPFS Server component, configure the GPFS Server nodes and attach the list of disks that are used by the GPFS shared file system. This virtual application pattern supports the following configurations:
• Primary
• Mirror
• Tiebreaker
• Passive
There are two GPFS topologies that are worth describing: Active/Active and Active/Passive.
Active/Active
The Active/Active GPFS deployment is also referred to as active-active mirroring and is considered a high availability scenario. This topology includes a GPFS Primary server along with attached GPFS Mirror and GPFS Tiebreaker servers. The tiebreaker server takes care of quorum and decides which server is primary. The tiebreaker also takes over the primary role of serving clients if either the primary or mirror server becomes unstable. The number of GPFS mirror nodes must match the number of primary nodes.
The size of the mirror and primary storage volumes must match as well. It is interesting to note that the block storage volumes are separate and not aware of each other. GPFS is responsible for the synchronization of data in the two volumes.
Active/Active mirroring is used for disaster recovery scenarios over shorter distances (less than 300 km). Figure 5-9 shows the three GPFS server configurations that are deployed to build this high availability scenario.
Figure 5-9 Active/Active GPFS deployment
Active/Passive
An Active/Passive GPFS deployment includes a GPFS Primary server configuration and a GPFS Passive server configuration. For replication, this configuration uses block storage replication, which must be set up manually. This configuration, which is shown in Figure 5-10 on page 115, is used over larger distances (less than 8000 km).
Figure 5-10 Active/Passive GPFS deployment
In the approach that is shown in Figure 5-10, the two GPFS server configurations are not aware of each other. However, the volumes are aware of each other (synchronized) through a block storage replication approach.
5.4.2 Shared service for GPFS
This shared service is deployed so that GPFS clients in the same cloud group can communicate with a deployed GPFS Server. To communicate with a GPFS Server, GPFS clients must know the client key, which contains the IP address and key that is retrieved from the Primary GPFS Manager server. The shared service for GPFS simplifies the GPFS client deployments by providing information about available GPFS Servers. As a prerequisite for this shared service, you must deploy a GPFS Server configuration by using IBM Pattern for GPFS.
5.4.3 GPFS file systems and file sets
GPFS Servers can host multiple file systems. File systems are attached to one or more block storage volumes. On the GPFS Server, file systems are mounted by running the following commands:
•/gpfs/fileSystemName1
•/gpfs/fileSystemName2
Within each file system, you define file sets that are treated as subdirectories within the file system. File sets are created in the client VM, either through a GPFS client policy or post deployment. Even though file set creation operation is run on the client side, the file set directory is created on the server VMs. A directory is created for each file system and file set in the client VM by running the following commands:
•/gpfs/fileSystemName1/fileSet1
•/gpfs/fileSystemName1/fileSet2
•/gpfs/fileSystemName2/fileSet1
•/gpfs/fileSystemName2/fileSet2
These shared file directories are linked to local file directories. This convention makes it easier for workloads to reference those directories. For example, a shared file directory might look like the following example:
/WASTranlogs links to /gpfs/fileSystemName1/fileSet1
A maximum of 14 storage volumes can be attached to all file systems for a GPFS Server. File systems must be created before a client can use them.
..................Content has been hidden....................
You can't read the all page of ebook, please click here login for view all page.