Skip to main content

ESX Multipathing for iSCSI and FC

Summary

This guide expresses some changes that are required and/or will make ESX “behave” better with StorONE LUNs

Requirements

Supported Versions (if applicable)

  • S1 - 3.6..x 3.7.x 3.8.x

  • ESXI - 6.5, 6.7, 7.x, 8.x 

Prerequisites

  • ESXI Secure Shell and ESXcli access

Block Size

When creating an Application/Volume (LUN) for use with ESX if you are connecting via iSCSI or FC you must set the block size to 512.  ESXI does not support any other block size than 512 for block access.

GUI

vol-create-512b-sector.png

CLI

applications volumes create --application ESXiMPIOtest --volume ESXtest0 --capacity 5T --pools SSD-2T --n 4 --k 2 --sectorSize 512b

These are just examples demonstrating the correct sector size. Use settings appropriate for your use case.

Setting StorONE LUNs to Round Robin Multipathing via vCenter

By default ESX will set LUNs to use MRU (Most Recently Used). See VMware KB 1011340 for more information. Rather than using the default MRU policy, StorONE recommends using Round Robin (RR). To determine the current setting, you can use the CLI or the GUI.

Example

image.png

In the above example, multipathing is set to Most Recently Used. Click Edit Multipathing… to change the setting:

image.png

Setting StorONE LUNs to Round Robin Multipathing via ESX CLI

From the ESX CLI, you can display all StorONE LUNs using the following command:

Example:

 esxcli storage nmp device list |grep STORONE -B1 -A8

naa.6882e5a000000002000000000000012d
   Device Display Name: STORONE iSCSI Disk (naa.6882e5a000000002000000000000012d)
   Storage Array Type: VMW_SATP_ALUA
   Storage Array Type Device Config: {implicit_support=on; explicit_support=off; explicit_allow=on; alua_followover=on; action_OnRetryErrors=on; {TPG_id=2,TPG_state=ANO}{TPG_id=1,TPG_state=AO}}
   Path Selection Policy: VMW_PSP_MRU
   Path Selection Policy Device Config: Current Path=vmhba64:C0:T0:L10
   Path Selection Policy Device Custom Config: 
   Working Paths: vmhba64:C0:T0:L10
   Is USB: false

In the above example, the default MRU policy is set. Change it to Round Robin by running the following command:

esxcli storage nmp device set -d naa.6882e5a000000002000000000000012d --psp=VMW_PSP_RR

Example:

esxcli storage nmp device list

naa.6882e5a000000002000000000000012d
   Device Display Name: STORONE iSCSI Disk (naa.6882e5a000000002000000000000012d)
   Storage Array Type: VMW_SATP_ALUA
   Storage Array Type Device Config: {implicit_support=on; explicit_support=off; explicit_allow=on; alua_followover=on; action_OnRetryErrors=on; {TPG_id=2,TPG_state=ANO}{TPG_id=1,TPG_state=AO}}
   Path Selection Policy: VMW_PSP_RR
   Path Selection Policy Device Config: {policy=rr,iops=1000,bytes=10485760,useANO=0; lastPathIndex=1: NumIOsPending=0,numBytesPending=0}
   Path Selection Policy Device Custom Config: 
   Working Paths: vmhba64:C0:T0:L10
   Is USB: false

To change all of the StorONE LUNs at once, you can run the following script:

for x in `esxcli storage nmp device list |grep STORONE|awk '{print $7}' |cut -d '(' -f2|cut -d ')' -f1`; do esxcli storage nmp device set -d $x --psp=VMW_PSP_RR; done

Improving ESXI Round Robin Performance

The default Round Robin settings on ESX can still result in suboptimal performance.  To address this, StorONE recommends setting the max IOPS to 1. To see the iops setting for all StorONE LUNS, use the following command:

for x in `esxcfg-scsidevs -c |grep STORONE|awk '{print $1}'`; do esxcli storage nmp psp roundrobin deviceconfig get --device=$x; done

naa.6882e5a000000002000000000000012d
   Device Display Name: STORONE iSCSI Disk (naa.6882e5a000000002000000000000012d)
   Storage Array Type: VMW_SATP_ALUA
   Storage Array Type Device Config: {implicit_support=on; explicit_support=off; explicit_allow=on; alua_followover=on; action_OnRetryErrors=on; {TPG_id=2,TPG_state=ANO}{TPG_id=1,TPG_state=AO}}
   Path Selection Policy: VMW_PSP_RR
   Path Selection Policy Device Config: {policy=rr,iops=1000,bytes=10485760,useANO=0; lastPathIndex=1: NumIOsPending=0,numBytesPending=0}
   Path Selection Policy Device Custom Config: 
   Working Paths: vmhba64:C0:T0:L10
   Is USB: false

To set this value to 1 (see VMware KB 2069356 for more information), use the following command:

esxcli storage nmp psp roundrobin deviceconfig set --type=iops --iops=1 --device=naa.6882e5a000000002000000000000012d

naa.6882e5a000000002000000000000012d
   Device Display Name: STORONE iSCSI Disk (naa.6882e5a000000002000000000000012d)
   Storage Array Type: VMW_SATP_ALUA
   Storage Array Type Device Config: {implicit_support=on; explicit_support=off; explicit_allow=on; alua_followover=on; action_OnRetryErrors=on; {TPG_id=2,TPG_state=ANO}{TPG_id=1,TPG_state=AO}}
   Path Selection Policy: VMW_PSP_RR
   Path Selection Policy Device Config: {policy=iops,iops=1,bytes=10485760,useANO=0; lastPathIndex=1: NumIOsPending=0,numBytesPending=0}
   Path Selection Policy Device Custom Config: 
   Working Paths: vmhba64:C0:T0:L10
   Is USB: false

To change them all at once:

for x in `esxcfg-scsidevs -c |grep STORONE|awk '{print $1}'`; do esxcli storage nmp psp roundrobin deviceconfig set --type=iops --iops=1 --device=$x; done

ESXI Multipath Storage SATP claim rule

The commands just completed will permanently set the Multipathing policy to Round Robin, and the Round Robin IOPs to 1 for best performance for all of the LUNs currently connected to the ESXI hosts from the StorONE array.  When you add a new StorONE LUN, it will use the default settings again. To avoid performing these operations each time a new LUN is connected to our ESXI host from the StorONE array, StorONE recommends using a Storage SATP claim rule.

When a storage claim rule is implemented for a Storage array, on startup of the ESXI server the claim rule is loaded into the ESXI running configuration.  With the claim rule in place, any LUN from the StorONE array will be properly connected for Round Robin Multipathing with the improved Round Robin and IOPS settings. Add the claim rule using the following command:

esxcli storage nmp satp rule add --satp "VMW_SATP_ALUA" --vendor "STORONE" --model "S1" --psp "VMW_PSP_RR" --psp-option "iops=1" --claim-option="tpgs_on" --description "StorONE LUN Claim Rule"

You can view the new claim rule with the following esxcli commands:

esxcli storage nmp satp rule list list all host claim rules
esxcli storage nmp satp rule list | grep -i storone list the StorONE installed host claim rule

For the claim rule to take effect, the ESXI host must be rebooted.  When the rule takes effect, all existing LUNs will use the defined rule, as well as all new LUNs.

Back to top