Skip to main content

ESX Multipathing for iSCSI and FC

Summary

This guide expresses some changes that are required and/or will make ESX “behave” better with StorONE LUNs

Requirements

Supported Versions (if applicable)

  • S1 - 3.6..x 3.7.x 3.8.x

  • ESXI - 6.5, 6.7, 7.x, 8.x 

Prerequisites

  • ESXI Secure Shell and ESXcli access

Block Size

When creating an Application/Volume (LUN) for use with ESX if you are connecting via iSCSI or FC you must set the block size to 512.  ESXI does not support any other block size than 512 for block access.

GUI

image.png

CLI

applications volumes create --application ESXiMPIOtest --volume ESXtest0 --capacity 50G --pools SSD-2T --n 2 --k 2 --sectorSize 512b

(just an example of the option that needs to be added/modified to allow ESX to see the LUN)

Setting StorONE LUNs to Round Robin Multipathing via vCenter

By default ESX will set LUNs to use MRU (Most Recently Used).  See this KB for more information: https://kb.vmware.com/s/article/1011340 .  Rather than using the default MRU policy, a better option is to use RR (Round Robin).  To determine what we have existing we can use the CLI or the GUI.

Example

image.png

In the above example we can see that the Multipathing is set to Most Recently Used.  We can click on the Edit Multipathing… link to change the setting:

image.png

Setting StorONE LUNs to Round Robin Multipathing via ESX CLI

From the ESX CLI to display all of the StorONE LUNs:

Example:

 esxcli storage nmp device list |grep STORONE -B1 -A8

naa.6882e5a000000002000000000000012d
   Device Display Name: STORONE iSCSI Disk (naa.6882e5a000000002000000000000012d)
   Storage Array Type: VMW_SATP_ALUA
   Storage Array Type Device Config: {implicit_support=on; explicit_support=off; explicit_allow=on; alua_followover=on; action_OnRetryErrors=on; {TPG_id=2,TPG_state=ANO}{TPG_id=1,TPG_state=AO}}
   Path Selection Policy: VMW_PSP_MRU
   Path Selection Policy Device Config: Current Path=vmhba64:C0:T0:L10
   Path Selection Policy Device Custom Config: 
   Working Paths: vmhba64:C0:T0:L10
   Is USB: false

In the above example, we are at the default.  The change this we want to do the following:

esxcli storage nmp device set -d naa.6882e5a000000002000000000000012d --psp=VMW_PSP_RR

Example:

esxcli storage nmp device list

naa.6882e5a000000002000000000000012d
   Device Display Name: STORONE iSCSI Disk (naa.6882e5a000000002000000000000012d)
   Storage Array Type: VMW_SATP_ALUA
   Storage Array Type Device Config: {implicit_support=on; explicit_support=off; explicit_allow=on; alua_followover=on; action_OnRetryErrors=on; {TPG_id=2,TPG_state=ANO}{TPG_id=1,TPG_state=AO}}
   Path Selection Policy: VMW_PSP_RR
   Path Selection Policy Device Config: {policy=rr,iops=1000,bytes=10485760,useANO=0; lastPathIndex=1: NumIOsPending=0,numBytesPending=0}
   Path Selection Policy Device Custom Config: 
   Working Paths: vmhba64:C0:T0:L10
   Is USB: false

To change all of the StorONE LUNs at once the following script can be used:

for x in `esxcli storage nmp device list |grep STORONE|awk '{print $7}' |cut -d '(' -f2|cut -d ')' -f1`; do esxcli storage nmp device set -d $x --psp=VMW_PSP_RR; done

Improving ESXI Round Robin Performance

The default Round Robin settings on ESX can lead to some shortfalls in performance.  To correct this we can make a small modification that can usually help.   When we look at the default we see:

To look at all of the StorONE LUNS at once 

for x in `esxcfg-scsidevs -c |grep STORONE|awk '{print $1}'`; do esxcli storage nmp psp roundrobin deviceconfig get --device=$x; done

naa.6882e5a000000002000000000000012d
   Device Display Name: STORONE iSCSI Disk (naa.6882e5a000000002000000000000012d)
   Storage Array Type: VMW_SATP_ALUA
   Storage Array Type Device Config: {implicit_support=on; explicit_support=off; explicit_allow=on; alua_followover=on; action_OnRetryErrors=on; {TPG_id=2,TPG_state=ANO}{TPG_id=1,TPG_state=AO}}
   Path Selection Policy: VMW_PSP_RR
   Path Selection Policy Device Config: {policy=rr,iops=1000,bytes=10485760,useANO=0; lastPathIndex=1: NumIOsPending=0,numBytesPending=0}
   Path Selection Policy Device Custom Config: 
   Working Paths: vmhba64:C0:T0:L10
   Is USB: false

We want to lower this value to 1 ( https://kb.vmware.com/s/article/2069356?lang=en_US )

esxcli storage nmp psp roundrobin deviceconfig set --type=iops --iops=1 --device=naa.6882e5a000000002000000000000012d

naa.6882e5a000000002000000000000012d
   Device Display Name: STORONE iSCSI Disk (naa.6882e5a000000002000000000000012d)
   Storage Array Type: VMW_SATP_ALUA
   Storage Array Type Device Config: {implicit_support=on; explicit_support=off; explicit_allow=on; alua_followover=on; action_OnRetryErrors=on; {TPG_id=2,TPG_state=ANO}{TPG_id=1,TPG_state=AO}}
   Path Selection Policy: VMW_PSP_RR
   Path Selection Policy Device Config: {policy=iops,iops=1,bytes=10485760,useANO=0; lastPathIndex=1: NumIOsPending=0,numBytesPending=0}
   Path Selection Policy Device Custom Config: 
   Working Paths: vmhba64:C0:T0:L10
   Is USB: false

To change them all at once:

for x in `esxcfg-scsidevs -c |grep STORONE|awk '{print $1}'`; do esxcli storage nmp psp roundrobin deviceconfig set --type=iops --iops=1 --device=$x; done

ESXI Multipath Storage SATP claim rule

The commands just completed will permanently set the Multipathing policy to Round Robin, and the Round Robin IOPs to 1 for best performance for all of the LUNs currently connected to the ESXI hosts from the StorONE array.  However to prevent the need to perform these operations each time a new LUN is connected to our ESXI host from the StorONE array, a Storage SATP claim rule is recommended.

When a storage claim rule is implemented for a Storage array, on startup of the ESXI server the claim rule is loaded into the ESXI running configuration.  With the claim rule in place, any LUN from the StorONE array will be properly connected for Round Robin Multipathing with the IMproved Round Robin IOP setting

esxcli storage nmp satp rule add --satp "VMW_SATP_ALUA" --vendor "STORONE" --model "S1" --psp "VMW_PSP_RR" --psp-option "iops=1" --claim-option="tpgs_on" --description "StorONE LUN Claim Rule"

We can view the new claim rule with the following ESXcli command(s)

esxcli storage nmp satp rule list list all host claim rules
esxcli storage nmp satp rule list | grep -i storone list the StorONE installed host claim rule


NOTE: For the claim rule to take effect, the ESXI host will need to be rebooted.  When the rule takes effect, all existing LUNs will use the defined rule, asd well as all new LUNs.

Back to top