If the performance of your hardware bus adapters (HBAs) is unsatisfactory or your SAN storage processors are overutilized, you can adjust your ESXi hosts' maximum queue depth value. The maximum value refers to the queue depths reported for various paths to the LUN. When you lower this value, it throttles the ESXi host's throughput and alleviates SAN contention concerns if multiple hosts overutilize the storage and fill its command queue.
To adjust the queue depth for an HBA, perform the following steps:
- Verify which HBA module is currently loaded by entering one of these commands:
- For QLogic:
# esxcli system module list | grep qla
- For Emulex:
# esxcli system module list | grep lpfc
- For Brocade:
# esxcli system module list | grep bfa
- Run one of these commands:
- For QLogic:
# esxcli system module parameters set -p ql2xmaxqdepth=64 -m qla2xxx
- For Emulex:
# esxcli system module parameters set -p lpfc0_lun_queue_depth=64 -m lpfc820
- For Brocade:
# esxcli system module parameters set -p bfa_lun_queue_depth=64 -m bfa
- In this case, the HBAs represented by ql2x and lpfc0 have their LUN queue depths set to 64. If all the Emulex cards on the host need to be updated, apply the global parameter lpfc_lun_queue_depth instead.
- Reboot your host.
- Run this command to confirm that your changes have been applied:
# esxcli system module parameters list –m driver
Here, the driver is your QLogic, Emulex, or Brocade adapter driver module, such as lpfc820, qla2xxx, or bfa.
The output appears similar to this:
Name Type Value Description
-------------------------- ---- ----- --------------------------------------
….. ql2xmaxqdepth int 64 Maximum queue depth to report for target devices.
…..
When one VM is active on an LUN, you only need to set the maximum queue depth. When multiple VMs are active on an LUN, the Disk.SchedNumReqOutstanding value is also relevant. The queue depth value, in this case, is equal to whichever value is the lowest of the two settings, namely adapter queue depth or Disk.SchedNumReqOutstanding:
In this example, you have 32. It is the sum total of all commands. And this is where you need Disk.ShedNumReqOutstanding:
But you will still have only 32 Active if you do not change the LUN queue depth as well.
The following procedures only apply to the ESXi/ESX host that the parameters are changed on. You must make the same changes to all the other ESXi/ESX hosts that have the datastore/LUN presented to them. In vSphere 5.5 and above, the parameter is set per device.
To set the VMkernel limit per device, perform the following steps:
- SSH to the ESXi host.
- Run the following command to list all the devices:
# esxcli storage core device list
- Find the device you want to change. It will start with naa.
- Run the following command with the correct naa value:
# esxcli storage core device set -d naa.xxx -O value
- The - is a capital O, not a zero. The value parameter is the number of outstanding disk requests.
- Run the following command to see the value:
# esxcli storage core device list -d naa.xxx
- The number of outstanding disk requests is shown as 'No of outstanding IOs with competing worlds:'.