ESX Multipathing for iSCSI and FC

Summary

This guide expresses some changes that are required and/or will make ESX “behave” better with StorONE LUNs

Requirements

Supported Versions (if applicable)

• S1 - 3.6..x 3.7.x 3.8.x
  

• ESXI - 6.5, 6.7, 7.x, 8.x 
  

Prerequisites

• ESXI Secure Shell and ESXcli access

Block Size

When creating an Application/Volume (LUN) for use with ESX if you are connecting via iSCSI or FC you must set the block size to 512.  ESXI does not support any other block size than 512 for block access.

GUI

CLI

applications volumes create --application ESXiMPIOtest --volume ESXtest0 --capacity 50G5T --pools SSD-2T --n 24 --k 2 --sectorSize 512b

(These are just anexamples example ofdemonstrating the optioncorrect thatsector needssize. toUse besettings added/modifiedappropriate tofor allowyour ESXuse to see the LUN)case.

Setting StorONE LUNs to Round Robin Multipathing via vCenter

By default ESX will set LUNs to use MRU (Most Recently Used).  See thisVMware KB 1011340 for more information: https://kb.vmware.com/s/article/1011340 . information. Rather than using the default MRU policy, aStorONE betterrecommends optionusing isRound to use RRRobin (Round Robin)RR).  To determine whatthe wecurrent havesetting, existing weyou can use the CLI or the GUI.

Example

In the above exampleexample, we can see that the Multipathingmultipathing is set to Most Recently Used. Click We can click on the Edit Multipathing… link to change the setting:

Setting StorONE LUNs to Round Robin Multipathing via ESX CLI

From the ESX CLICLI, toyou can display all ofStorONE LUNs using the StorONEfollowing LUNs:command:

Example:

 esxcli storage nmp device list |grep STORONE -B1 -A8

naa.6882e5a000000002000000000000012d
   Device Display Name: STORONE iSCSI Disk (naa.6882e5a000000002000000000000012d)
   Storage Array Type: VMW_SATP_ALUA
   Storage Array Type Device Config: {implicit_support=on; explicit_support=off; explicit_allow=on; alua_followover=on; action_OnRetryErrors=on; {TPG_id=2,TPG_state=ANO}{TPG_id=1,TPG_state=AO}}
   Path Selection Policy: VMW_PSP_MRU
   Path Selection Policy Device Config: Current Path=vmhba64:C0:T0:L10
   Path Selection Policy Device Custom Config: 
   Working Paths: vmhba64:C0:T0:L10
   Is USB: false

In the above example, wethe aredefault atMRU policy is set. Change it to Round Robin by running the default.following  The change this we want to do the following:command:

esxcli storage nmp device set -d naa.6882e5a000000002000000000000012d --psp=VMW_PSP_RR

Example:

esxcli storage nmp device list

naa.6882e5a000000002000000000000012d
   Device Display Name: STORONE iSCSI Disk (naa.6882e5a000000002000000000000012d)
   Storage Array Type: VMW_SATP_ALUA
   Storage Array Type Device Config: {implicit_support=on; explicit_support=off; explicit_allow=on; alua_followover=on; action_OnRetryErrors=on; {TPG_id=2,TPG_state=ANO}{TPG_id=1,TPG_state=AO}}
   Path Selection Policy: VMW_PSP_RR
   Path Selection Policy Device Config: {policy=rr,iops=1000,bytes=10485760,useANO=0; lastPathIndex=1: NumIOsPending=0,numBytesPending=0}
   Path Selection Policy Device Custom Config: 
   Working Paths: vmhba64:C0:T0:L10
   Is USB: false

To change all of the StorONE LUNs at onceonce, you can run the following script can be used:script:

for x in `esxcli storage nmp device list |grep STORONE|awk '{print $7}' |cut -d '(' -f2|cut -d ')' -f1`; do esxcli storage nmp device set -d $x --psp=VMW_PSP_RR; done

Improving ESXI Round Robin Performance

The default Round Robin settings on ESX can leadstill to some shortfallsresult in suboptimal performance.  To correctaddress thisthis, weStorONE canrecommends make a small modification that can usually help.   When we look atsetting the defaultmax weIOPS see:

to

1. To looksee atthe iops setting for all ofStorONE LUNS, use the StorONEfollowing LUNS at once command:

for x in `esxcfg-scsidevs -c |grep STORONE|awk '{print $1}'`; do esxcli storage nmp psp roundrobin deviceconfig get --device=$x; done

naa.6882e5a000000002000000000000012d
   Device Display Name: STORONE iSCSI Disk (naa.6882e5a000000002000000000000012d)
   Storage Array Type: VMW_SATP_ALUA
   Storage Array Type Device Config: {implicit_support=on; explicit_support=off; explicit_allow=on; alua_followover=on; action_OnRetryErrors=on; {TPG_id=2,TPG_state=ANO}{TPG_id=1,TPG_state=AO}}
   Path Selection Policy: VMW_PSP_RR
   Path Selection Policy Device Config: {policy=rr,iops=1000,bytes=10485760,useANO=0; lastPathIndex=1: NumIOsPending=0,numBytesPending=0}
   Path Selection Policy Device Custom Config: 
   Working Paths: vmhba64:C0:T0:L10
   Is USB: false

WeTo want to lowerset this value to 1 (see https://kb.vmware.com/s/article/2069356?lang=en_USVMware KB 2069356 )for more information), use the following command:

esxcli storage nmp psp roundrobin deviceconfig set --type=iops --iops=1 --device=naa.6882e5a000000002000000000000012d

naa.6882e5a000000002000000000000012d
   Device Display Name: STORONE iSCSI Disk (naa.6882e5a000000002000000000000012d)
   Storage Array Type: VMW_SATP_ALUA
   Storage Array Type Device Config: {implicit_support=on; explicit_support=off; explicit_allow=on; alua_followover=on; action_OnRetryErrors=on; {TPG_id=2,TPG_state=ANO}{TPG_id=1,TPG_state=AO}}
   Path Selection Policy: VMW_PSP_RR
   Path Selection Policy Device Config: {policy=iops,iops=1,bytes=10485760,useANO=0; lastPathIndex=1: NumIOsPending=0,numBytesPending=0}
   Path Selection Policy Device Custom Config: 
   Working Paths: vmhba64:C0:T0:L10
   Is USB: false

To change them all at once:

for x in `esxcfg-scsidevs -c |grep STORONE|awk '{print $1}'`; do esxcli storage nmp psp roundrobin deviceconfig set --type=iops --iops=1 --device=$x; done

ESXI Multipath Storage SATP claim rule

The commands just completed will permanently set the Multipathing policy to Round Robin, and the Round Robin IOPs to 1 for best performance for all of the LUNs currently connected to the ESXI hosts from the StorONE array.  HoweverWhen toyou preventadd a new StorONE LUN, it will use the needdefault tosettings performagain. To avoid performing these operations each time a new LUN is connected to our ESXI host from the StorONE array, StorONE recommends using a Storage SATP claim rule is recommended.rule.

When a storage claim rule is implemented for a Storage array, on startup of the ESXI server the claim rule is loaded into the ESXI running configuration.  With the claim rule in place, any LUN from the StorONE array will be properly connected for Round Robin Multipathing with the IMprovedimproved Round Robin IOPand settingIOPS settings. Add the claim rule using the following command:

esxcli storage nmp satp rule add --satp "VMW_SATP_ALUA" --vendor "STORONE" --model "S1" --psp "VMW_PSP_RR" --psp-option "iops=1" --claim-option="tpgs_on" --description "StorONE LUN Claim Rule"

WeYou can view the new claim rule with the following ESXcliesxcli command(s)commands:

esxcli storage nmp satp rule list list all host claim rules

esxcli storage nmp satp rule list | grep -i storone list the StorONE installed host claim rule

NOTE: For the claim rule to take effect, the ESXI host will need tomust be rebooted.  When the rule takes effect, all existing LUNs will use the defined rule, asdas well as all new LUNs.