DFSMShsm
This chapter describes the changes that were made to DFSMShsm to support the cloud tier.
This chapter describes the following topics:
9.1 Cloud use overview
DFSMShsm can use storage clouds by using DFSMSdss as a data mover to migrate and recall data sets from the cloud. The use of a cloud to migrate data sets can reduce your direct access storage device and tape requirements, and provide disaster recovery (DR) capability for migrated data sets.
DFSMShsm tracks migrated data sets and their related objects in cloud storage in its Control Data Sets (CDSs). By using the Data Facility Storage Management Subsystem (DFSMS) Cloud construct to connect to the cloud (along with CDS information), DFSMShsm can manage data on cloud storage the same way it manages offline data.
Unlike tape devices, the space that is left by a deleted object is returned to available space and can be used by other objects as they are created. This feature eliminates the need for recycling containers on storage cloud, which reduces the central processing unit (CPU) resources that are related to RECYCLE processing and the window that is necessary to run DFSMShsm tasks. It also increases the availability of migrated data as RECYCLE processing holds Hierarchical Storage Manager (HSM) tapes during the recycle operation.
Also, the option of having an off-premise cloud increases the options that are available for DR. If your cloud is unaffected after a disaster, you can recover your production systems in an alternative location (including DFSMShsm CDSs) and define DFSMS Cloud construct to connect to your cloud. This ability gives you access to all data sets that are migrated to the cloud and have their related CDS records.
After a recovery situation, run an AUDIT against CDS and cloud to report and correct any mismatches between them.
9.2 Cloud container management
DFSMShsm can create cloud containers to store the objects. By default, the container name adheres to the syntax that is shown in Example 9-1.
Example 9-1 Default DFSMShsm container name
SYSZARC.<HSMplexname>.MIG.yyyyddd.<hash>
The hash value is a random hexadecimal string that ensures that there never are two containers with the same name.
By default, HSM creates a new container every 92 days. You can change the default value from 92 days to a shorter or longer period. Example 9-2 shows the PATCH command to change the default value to 10 days.
Example 9-2 PATCH command to change the container creation frequency
PATCH .MCVT.+50F X'0A'
|
Note: The default values for the period after which HSM creates a container was changed from 7 days to 92 days with APAR OA60278. You might still see the old value in your environment. As a best practice, change it to the new default or install the APAR.
|
With a shorter period, your LIST and AUDIT processing might become quicker, but many containers might impact the overall HSM performance. Be careful when considering to change it to a shorter period.
Just as DFSMShsm can automatically create containers when required, it can also delete empty and no longer used containers that are owned by DFSMShsm. You can configure DFSMShsm to perform empty container deletion as part of the Secondary Space Management tasks by using the new EMPTYCONTAINERDELETION(x) keyword on DFSMShsm SETSYS MAXSSMTASKS command.
Example 9-3 shows a sample SETSYS command to allow one Cloud Storage containers processing task. This value is the default value. To prevent DFSMShsm from deleting empty containers, set the EMPTYCONTAINERDELETION to 0.
Example 9-3 Setting container deletion task
SETSYS MAXSSMTASKS(EMPTYCONTAINERDELETION(1))
If you disable automatic deletion of containers by DFSMShsm, you must create your own process to manage empty containers.
9.3 Object management
DFSMShsm can automatically create and delete objects from cloud storage by using DFSMSdss as the data mover. For each data set, DFSMSdss is started to migrate or recall the data by using Transparent Cloud Tiering (TCT).
Storage objects that are created by DFSMShsm follow a new data set naming convention, which is similar to the naming convention that is used to Migration Level 1 (ML1) migrate data sets. The object naming convention is shown in Example 9-4.
Example 9-4 DFSMShsm object naming convention
INSTPFX.HMIG.TCCCCHH.USER1.USER2.?YDDD
The naming convention consists of the following parts:
•INSTPFX is an installation-defined prefix.
•TCCCCHH is a form of how HSM expresses the time, where CCCC is the number of hundredths of seconds since the beginning of the hour and compressed into four alphanumeric digits. HH is the hour. When there is a conflict, T can change to be from U - S (starting from T and wrapping around).
•USER1.USER2 are the first two qualifiers of the data set name that is being migrated.
•?YDDD is the Julian Date where is A - F is for decade; for example, 2000 - 2060.
How DFSMSdss handles the data depends on the request that is performed by DFSMShsm (a migration or recall process). These processes are described next.
9.3.1 Migration
When a migration process is started, DFSMShsm calls DFSMSdss to perform the data movement. HSM is responsible for passing to Data Storage Services (DSS) the data set name, along with the cloud constructs, including cloud name, account, container, and object prefix. DFSMSdss then communicates with the DS8000 passing information that is related to the tracks that should be moved to the cloud, along with cloud-related information.
The metadata is stored in the cloud directly by the host for Swift clouds, or DS8000 for S3 and IBM Cloud Object Storage clouds. DFSMSdss returns control to DFSMShsm after all data is moved to the cloud or after any failures during the process.
During the DUMP process, any data sets that are larger than 5 GB are broken up in 5 GB segments. VALIDATE processing is skipped for Virtual Storage Access Method (VSAM) data sets.
9.3.2 Recall
During a recall request, DFSMShsm sends to DFSMSdss the data set name to be restored, along with the cloud attributes. DFSMSdss issues a request to the DS8000 for the objects that should be retrieved from the cloud. Metadata is retrieved by the host for Swift clouds, and by DS8000 for S3 and IBM Cloud Object Storage clouds.
At retrieval time, object segments (for data sets larger than 5 GB) are grouped, and data set extents are reduced when possible. During this phase, no REBLOCKing function is performed.
The storage objects can be deleted or retained during the recall process, if your HSMplex is configured to support fast subsequent migration.
9.4 Fast subsequent migration
When data sets are migrated to Migration Level 2 (ML2), they are stored on tapes until the data set expires or is recalled. If a recall occurs, the data set is not physically deleted from the tape, but the CDS records are marked as invalid. With Fast Subsequent Migration, the recalled data set can be reconnected to the tape, which eliminates the need to rewrite the tape data.
Use of the storage cloud also allows you to reconnect recalled data sets to the cloud objects, which prevents a new migration, and thus reduces the network traffic to the cloud.
A new SETSYS command option (see Example 9-5) is available to include in your DFSMShsm parmlib to allow reconnection.
Example 9-5 Setting up fast subsequent migration
SETSYS CLOUDMIGRATION(RECONNECT(ALL))
9.5 Migration update and considerations
Data can be migrated to the cloud either by command, or during the automatic space management. The next topics will explain in more detail how you can manage your data for automatic and manual selection for migration.
9.5.1 Command-driven migration
A CLOUD parameter is available from the MIGRATE or HMIGRATE commands to target data sets to cloud. Example 9-6 shows a sample HSEND MIGRATE command with the CLOUD keyword.
Example 9-6 HSEND MIGRATE command cloud option
HSEND MIGRATE DSN(youdsname) CLOUD(yourcloud)
|
Note: The CLOUD parameter is mutually exclusive with MIGRATIONLEVEL1, MIGRATIONLEVEL2, and CONVERT parameters. Also, COMPACT, COMPACTPERCENT, COMPACT(ALL), CONVERSION(REBLOCKTOANY), and CONCURRENT SETSYS values are not used when migrating to the cloud.
|
To migrate a data set to the cloud, it must be SMS-managed. The types of data sets that can be migrated to the cloud, along with migration and recall restrictions, are listed in Table 9-1.
Table 9-1 Data set migration to cloud eligibility and considerations
|
Data set type
|
Can it be migrated to the cloud?
|
Comments
|
|
Non-SMS
|
N
|
Only SMS-managed data sets can be migrated to the cloud at the time of this writing.
|
|
Sequential
|
Y
|
|
|
Extended Format
|
Y
|
|
|
Extended format multi-volume
|
N
|
VSAM restrictions for HURBA=HARBA (used = allocated), and Multi-layer VSAM (volume count > stripe count) cannot be migrated.
|
|
Multi-extents sequential/partitioned
|
Y
|
Extent reduction is performed at recall time if possible.
|
|
Multi-volume sequential/partitioned
|
Y
|
|
|
Multi-stripe sequential
|
Y
|
If SMS cannot provide enough volumes to keep the stripe count, the recall fails.
|
|
VSAM
|
Y
|
VALIDATE is not performed during migration.
|
|
Multi-extent VSAM
|
Y
|
VALIDATE is not performed during migration.
|
|
Multi-volume VSAM
|
Y
|
VALIDATE is not performed during migration.
|
|
VSAM with IBM AIX® and PATHs
|
Y
|
VALIDATE is not performed during migration.
|
|
Data sets in volumes with simplex, two-site Metro Mirror, FlashCopy, Global Mirror, Metro Global Mirror (with or without HyperSwap).
|
Y
|
As of this writing, only volumes with XRC and Multi-Target Peer to Peer Remote Copy (PPRC) are not supported.
|
|
Data sets spanning more than 26 volumes
|
Y
|
An object cannot be restored to more than 26 volumes.
|
|
Multi-volume data sets spanning multiple DS8000s
|
Y
|
Cannot be restored in volumes spanning DS8000 systems.
|
The HSEND MIGRATE command that is issued from option ISPF panels to migrate a multi-extent sequential data set is shown in Example 9-7. There is no need to supply account, container, object prefix, or cloud credentials because they are handled by DFSMShsm.
Example 9-7 HSEND MIGRATE issued from ISPF panel
Menu Options View Utilities Compilers Help
DSLIST - Data Sets Matching TCT.DEMO Row 1 of 1
Command ===> Scroll ===> PAGE
Command - Enter "/" to select action Message Volume
-------------------------------------------------------------------------------
HSEND MIGRATE DSN(/) cloud(IBMCOS) CLDD2F+
***************************** End of Data Set list ****************************
After the migration process is complete, the catalog entry is updated to reflect the new location of the data set. In ISPF, to differentiate data sets that are migrated to cloud from data sets on ML1 or ML2, a new MIGRATC volume serial number is used for cloud-migrated data sets, as shown in Example 9-8.
Example 9-8 MIGRATC volume entry
Command - Enter "/" to select action Message Volume
-------------------------------------------------------------------------------
TCT.DEMO.DTA1.CP3000.LOG CLDC28
TCT.DEMO.DTA2.CP3000.LOG CLDD24
TCT.DEMO.DTA3.CACH.REPORT CLDD2B
TCT.DEMO.DTA4.XCSV CLDD29
TCT.DEMO.DTA5.JCL CLDC2A
TCT.DEMO.DTA6.TRS MIGRATC 
A new device type of x'00018000' is used to identify data sets that are migrated to the cloud. The ICF catalog entry is shown in Example 9-9 on page 97. Regardless of the migration tier, migrated data sets are always cataloged by using volser MIGRAT.
Example 9-9 Device type for data set migrated to the cloud
NONVSAM ------- TCT.DEMO.DTA6.TRS
IN-CAT --- CATALOG.MVSICF1.VCEBCU1
HISTORY
DATASET-OWNER-----(NULL) CREATION--------2018.318
RELEASE----------------2 EXPIRATION------0000.000
ACCOUNT-INFO-----------------------------------(NULL)
SMSDATA
STORAGECLASS ----TCTTEST MANAGEMENTCLASS---NOBACK
DATACLASS --------(NULL) LBACKUP ---XXXX.XXX.XXXX
VOLUMES
VOLSER------------MIGRAT DEVTYPE------X'00018000' FSEQN------------------0
ASSOCIATIONS--------(NULL)
ATTRIBUTES
*** 
The information that is necessary to recall the data set from the cloud is stored in HSM CDSs. You can issue a FIXCDS DISPLAY command to view the created CDS records. A sample FIXCDS command format to display a Migration Control Data Set (MCD) record is shown in Example 9-10.
Example 9-10 FIXCDS command to view a CDS record
HSEND FIXCDS D dsname DISPLAY
The command output for a cloud-migrated data set is shown in Example 9-11. Areas are highlighted to show recognizable eye catchers within the CDS record.
Example 9-11 Output of the FIXCDS command
F DFHSMBC,FIXCDS D TCT.DEMO.DTA6.TRS DISPLAY
MCH= 02580000 D53C0FE6 68920609 D53B23A7 A3FBB90A * N W N *
+0000 6CC3D3D6 E4C48001 00340000 0118318F 00000000 0118319F 00000000 00000000 * CLOUD *
+0020 09522468 0118319F 40006C00 009002F1 00029388 3A3D6A27 000747AD 00020000 * 1 *
+0040 C3D3C4C3 F2F30200 00000000 3030200F 09204263 0118319F 00010000 00000000 *CLDC23 *
+0060 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 * *
+0080 00000000 00000000 00000000 00000000 00000000 00000000 0000000F C4C6C8E2 * DFHS*
+00A0 D44BC8D4 C9C74BE3 F1F9F4F6 F0F94BE3 C3E34BC4 C5D4D64B C1F8F3F1 F9404040 *M.HMIG.T194609.TCT.DEMO.A8319 *
+00C0 40404040 40404040 00000000 00000000 00000000 00000000 00000000 00000000 * *
+00E0 00000000 00004040 40404040 40404040 40404040 40404040 40404040 40404040 * *
+0100 40404040 0007E3C3 E3E3C5E2 E3404040 40404040 40404040 40404040 40404040 * TCTTEST *
+0120 40404040 0006D5D6 C2C1C3D2 40404040 40404040 40404040 40404040 40404040 * NOBACK *
+0140 40404040 00000000 00000000 00880000 00000000 00000000 00000000 00000000 * *
+0160 00000000 0000C000 03E80000 00000000 00000000 03020200 00000000 00000400 * Y *
+0180 22404040 4000C400 00000000 0007E3C3 E3E3C5E2 E3000000 00000000 00000000 * D TCTTEST *
+01A0 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 * *
+01C0 00000000 00000000 0006C9C2 D4C3D6E2 40404040 40404040 40404040 40404040 * IBMCOS *
+01E0 40404040 40404040 E2E8E2E9 C1D9C34B C1D9C3D7 D3C5E7F0 4BD4C9C7 4BF2F0F1 * SYSZARC.ARCPLEX0.MIG.201*
+0200 F8F3F1F6 40404040 40404040 40404040 40404040 0000000A *8316 *
ARC0197I TYPE D, KEY TCT.DEMO.DTA6.TRS, FIXCDS DISPLAY
ARC0197I (CONT.) SUCCESSFUL
The FIXCDS command output includes the regular data set information (such as SMS constructs) and the cloud-related information (including the cloud name) and the container where the data set is stored.
9.5.2 Automatic migration
The DFSMShsm can also migrate data sets to the cloud during the automatic space management, including primary space management, interval migration, or on-demand migration.
To support automatic migration functions, new parameters were included in DFSMShsm parmlib and SMS management class constructs. These changes are described in more detail in Chapter 10, “Using automatic migration” on page 103.
9.5.3 CPU utilization considerations
Because the data movement to and from the cloud is performed directly by the DS8000, it is expected to have a variation on CPU utilization by DFSMShsm. You can create reports to estimate possible CPU savings from implementing cloud migration. The pre-implementation reporting is discussed in more detail in Chapter 11, “Operational integration and reporting considerations” on page 109.
9.6 Recall considerations
The RECALL process is automatically triggered when access to the data set is requested or the HSEND RECALL command is issued. There are no changes to the RECALL command.
During the RECALL, the Automatic Class Selection (ACS) routines are called to define the Storage Class, Management Class, and Storage Group for the data set. The data set can also be extent-reduced if possible. Example 9-12 shows the recalled data set from the HSEND MIGRATE command that was issued, where the recalled data set has a single extent.
Example 9-12 Recalled data set with consolidated extent
DSLIST - Data Sets Matching TCT.DEMO.DTA6 Row 1 of 1
Command ===> Scroll ===> CSR
Command - Enter "/" to select action Tracks %Used XT
-------------------------------------------------------------------------------
TCT.DEMO.DTA6.TRS 168840 99 3 
When data sets expire, all data and metadata objects that are related to the data set are automatically deleted from the storage cloud and the catalog entry is removed.
9.7 LIST command updates
The LIST command has updates to support the cloud. DFSMShsm also gives you the options to list the following elements:
•Cloud
•Containers
•Objects
New CLOUD, CONTAINER, and PREFIX parameters can be used within the LIST command to retrieve cloud and container content information. The LIST command can list DFSMShsm and non-DFSMShsm owned containers, which give the users the chance to list user-created containers and retrieve object information.
The output from the LIST command can be directed to a terminal or data set. The sample command that is used to list IBMREDBOOKS cloud information is shown in Example 9-13.
Example 9-13 LIST command for a specific cloud
HSEND LIST CLOUD(IBMCOS)
The command output is shown in Example 9-14 on page 99.
Example 9-14 LIST command output
CLOUD NAME CONTAINER NAME
IBMCOS SYSZARC.ARCPLEX0.MIG.2018127
IBMCOS SYSZARC.ARCPLEX0.MIG.2018316
IBMCOS SYSZARC.ARCPLEX0.MIG.2018120
IBMCOS SYSZARC.ARCPLEX0.MIG.2018134 
The output shows the cloud IBMCOS and lists four containers, such as:
SYSZARC.ARCPLEX0.MIG.2018316
DFSMShsm and user-created containers data can be displayed by using the HSEND LIST command. Add the CONTAINER(containername) keyword to your LIST command. The HSEND LIST command lists the container SYSZARC.ARCPLEX0.MIG.2018316, as shown in Example 9-15.
Example 9-15 Listing a specific container in the IBMCOS cloud
HSEND LIST CLOUD(IBMCOS) CONTAINER(SYSZARC.ARCPLEX0.MIG.2018316)
The output from the command that is shown in Example 9-15 is shown in Example 9-16.
Example 9-16 LIST the content of a specific container
CLOUD NAME CONTAINER NAME PREFIX NAME
IBMCOS SYSZARC.ARCPLEX0.MIG.2018316 DFHSM.HMIG.T194609.TCT.DEMO.A8319
DFHSM.HMIG.T474010.TCT.DEMO.A8319
Listing objects is available by using PREFIX parameter. When listing objects, the cloud and container names must also be included. The LIST command with the PREFIX name is shown in Example 9-17. This option brings all of the data and metadata objects that are stored under the specific prefix.
Example 9-17 Listing objects by using the PREFIX keyword
HSEND LIST CLOUD(IBMCOS) CONTAINER(SYSZARC.ARCPLEX0.MIG.2018316) PREFIX(DFHSM.HMIG.T194609.TCT.DEMO.A8319)
The output from the LIST command with the CLOUD, CONTAINER, and PREFIX command is shown in Example 9-18.
Example 9-18 Listing objects by PREFIX
CLOUD INFORMATION FOR DATASET: TCT.DEMO.DTA6.TRS
CLOUD NAME CONTAINER NAME OBJECT NAME
IBMCOS SYSZARC.ARCPLEX0.MIG.2018316 DFHSM.HMIG.T194609.TCT.DEMO.A8319/DTPDSNL000
00001
DFHSM.HMIG.T194609.TCT.DEMO.A8319/HDR
DFHSM.HMIG.T194609.TCT.DEMO.A8319/TCT.DEMO.D
TA6.TRS/APPMETA
DFHSM.HMIG.T194609.TCT.DEMO.A8319/TCT.DEMO.D
TA6.TRS/DTPDSHDR
DFHSM.HMIG.T194609.TCT.DEMO.A8319/TCT.DEMO.D
TA6.TRS/DTPVOLD01/NVSM/EXTENTS
DFHSM.HMIG.T194609.TCT.DEMO.A8319/TCT.DEMO.D
TA6.TRS/DTPVOLD01/NVSM/META
DFHSM.HMIG.T194609.TCT.DEMO.A8319/TCT.DEMO.D
TA6.TRS/DTPVOLD02/NVSM/EXTENTS
DFHSM.HMIG.T194609.TCT.DEMO.A8319/TCT.DEMO.D
TA6.TRS/DTPVOLD02/NVSM/META
DFHSM.HMIG.T194609.TCT.DEMO.A8319/TCT.DEMO.D
TA6.TRS/DTPVOLD03/NVSM/EXTENTS
DFHSM.HMIG.T194609.TCT.DEMO.A8319/TCT.DEMO.D
TA6.TRS/DTPVOLD03/NVSM/META
For more information about each object, see Table 3-1 on page 25.
9.8 Auditing
The tasks to audit DFSMShsm data are vital to keep CDS records error-free, and identify, report, and correct any discrepancies between the CDS records and the data sets.
Unexpected software or hardware errors during migration and recall processes might leave migrated data sets and CDS records out of sync. Regularly auditing CDS and media reduces the number of orphan or invalid data in the physical media and the CDS.
AUDIT DATASETNAMES, LEVEL, and MCDS commands can perform a cloud-migrated data sets audit. If the CDS record indicates that the data set is stored in cloud storage, DFSMShsm verifies that objects corresponding to the archive are in the expected cloud and container.
DFSMShsm lists the objects in the cloud, beginning with the prefix that is stored in the MCD record. If the expected objects are found, it moves onto the next MCD record.
In addition, a new CLOUD parameter is available from the AUDIT MEDIACONTROLS command. This option allows you to audit a cloud and validate if the objects have corresponding CDS entries. If any inconsistencies are found, the AUDIT command reports the error back to the user.
|
Note: The AUDIT command does not automatically fix any identified inconsistencies because the orphan objects can be user data that is in the wrong container.
|
Example 9-19 Auditing a specific cloud
HSEND AUDIT MEDIACONTROLS(CLOUD(IBMCOS))
The output from the command that is shown in Example 9-19 is shown in Example 9-20, with one inconsistent entry.
Example 9-20 Example of AUDIT output
AUDIT MEDIACONTROLS(CLOUD(IBMCOS)) ODS('TCTRBOOK.AUDIT')
/* ERR 210 CDD IS NOT FOUND FOR PREFIX DUMP.4XTENTS.PDS IN CONTAINER
/* DFHSM.HMIG.T015821.TCT.VSM.A7284
- END OF - ENHANCED AUDIT - LISTING -
The prefix DUMP.4XTENTS.PDS is not in the CDS as expected. The follow-up action is to investigate why it is missing and resolve the issue.
9.9 REPORT command
After you first implement a storage cloud to your z/OS systems, create reports about key metrics for analysis, such as the following ones:
•Number of data set migrations to the cloud
•Amount of data transferred
•Number of successful and failed requests
•Average times
The REPORT command provides all of the information that is necessary to efficiently monitor your data on the cloud.
You can create daily, weekly, or monthly reports and store this information for further analysis. You also can create simple programs to process the data and return reports with data growth, percentage of data sets migrated to the cloud versus standard migration, and usage trends.
A simple REPORT command to display migration statistics to the cloud is shown in Example 9-21.
Example 9-21 Report command for daily migration activity
HSEND REPORT DAILY FUNCTION(MIGRATION(TOCLOUD))
It is also possible to retrieve recall specific information by issuing the REPORT command, as shown in Example 9-22.
Example 9-22 Report command for daily recall activity
HSEND REPORT DAILY FUNCTION(RECALL(FROMCLOUD))
Migration and recall reports display the following migration-to-the-cloud information:
•Number of data sets
•Number of tracks that are read and written
•Number of bytes read and written
•Number of system requests
•Number of user requests
•Failed requests
•Average age
•Average queue time
•Average wait time
•Average process time
•Average total time
A sample output from the HSEND REPORT DAILY command is shown in Example 9-23.
Example 9-23 Report output for daily activity
DAILY STATISTICS REPORT FOR 18/11/15
STARTUPS=000, SHUTDOWNS=000, ABENDS=000, WORK ELEMENTS PROCESSED=003023
DATA SET MIGRATIONS BY VOLUME REQUEST= 0000000, DATA SET MIGRATIONS BY
EXTENT REDUCTIONS= 0000000 RECALL MOUNTS AVOIDED= 00000 RECOVER MOUNTS
DATA SET MIGRATIONS BY RECONNECTION = 000000, NUMBER OF TRACKS RECONNE
NUMBER ------READ-------- -----WRITTEN------ ---
HSM FUNCTION DATASETS TRK/BLK BYTES TRK/BLK BYTES SYS
MIGRATION
PRIMARY - CLOUD 0000382 00064520 001632042 00064520 001632042 000
**************************** Bottom of Data ****************************
SMF records are also written when data sets are migrated to the cloud. You can use SMF records to create more specific reports that are based on users, high-level qualifiers, and other information. For more information about the SMF records and how to create reports, see Chapter 11, “Operational integration and reporting considerations” on page 109.
..................Content has been hidden....................
You can't read the all page of ebook, please click here login for view all page.