Skip to main content

ESX Block Multipathing for iSCSI and FC

Summary

This guide expresses some changes that are required and/or will make ESX “behave” better with StorONE LUNs

Requirements

Supported Versions (if applicable)

  • S1 - 3.6.23 -.x 3.7.4x 3.8.x

  • ESX - 6.5, 6.7, 7.x, 8.x +

Prerequisites

  • SSH and ESXcli access

Block Size

When creating an Application/Volume (LUN) for use with ESX if you are connecting via iSCSI or FC you need tomust set the block size to 512.  TheESXI ESXdoes systemnot willsupport NOTany support/useother ablock 4ksize device.than 512 for block access.

GUI

image.png

CLI

applications volumes create --application ESXiMPIOtest --volume ESXtest0 --capacity 50G --pools SSD-2T --n 2 --k 2 --sectorSize 512b

(just an example of the option that needs to be added/modified to allow ESX to see the LUN)

ESXSetting StorONE LUNs to Round Robin Multipathing via vCenter

Generally speaking, byBy default ESX will set the S1 LUNs to use MRU (Most Recently Used).  See this KB for more information: https://kb.vmware.com/s/article/1011340 .  TheRather onlythan reason it wouldn’t be set to MRU by default is ifusing the default behaviorMRU haspolicy, been modified:  https://kb.vmware.com/s/article/1036189

Aa better option is to use RR (Round Robin).  To determine what we have existing we can use the CLI or the GUI.

GUI

Example

image.png

In the above example we can see that the Multipathing is set to Most Recently Used.  We can click on the Edit Multipathing… link to change the setting:

image.png

 

Setting StorONE LUNs to Round Robin Multipathing via ESX CLI

From the ESX CLI to display all of the StorONE LUNs:


Example:

 esxcli storage nmp device list |grep STORONE -B1 -A8

naa.6882e5a000000002000000000000012d
   Device Display Name: STORONE iSCSI Disk (naa.6882e5a000000002000000000000012d)
   Storage Array Type: VMW_SATP_ALUA
   Storage Array Type Device Config: {implicit_support=on; explicit_support=off; explicit_allow=on; alua_followover=on; action_OnRetryErrors=on; {TPG_id=2,TPG_state=ANO}{TPG_id=1,TPG_state=AO}}
   Path Selection Policy: VMW_PSP_MRU
   Path Selection Policy Device Config: Current Path=vmhba64:C0:T0:L10
   Path Selection Policy Device Custom Config: 
   Working Paths: vmhba64:C0:T0:L10
   Is USB: false

In the above example, we are at the default.  The change this we want to do the following:

esxcli storage nmp device set -d naa.6882e5a000000002000000000000012d --psp=VMW_PSP_RR

 Example:

esxcli storage nmp device list

naa.6882e5a000000002000000000000012d
   Device Display Name: STORONE iSCSI Disk (naa.6882e5a000000002000000000000012d)
   Storage Array Type: VMW_SATP_ALUA
   Storage Array Type Device Config: {implicit_support=on; explicit_support=off; explicit_allow=on; alua_followover=on; action_OnRetryErrors=on; {TPG_id=2,TPG_state=ANO}{TPG_id=1,TPG_state=AO}}
   Path Selection Policy: VMW_PSP_RR
   Path Selection Policy Device Config: {policy=rr,iops=1000,bytes=10485760,useANO=0; lastPathIndex=1: NumIOsPending=0,numBytesPending=0}
   Path Selection Policy Device Custom Config: 
   Working Paths: vmhba64:C0:T0:L10
   Is USB: false

To change all of the StorONE LUNs at once the following script can be used:

for x in `esxcli storage nmp device list |grep STORONE|awk '{print $7}' |cut -d '(' -f2|cut -d ')' -f1`; do esxcli storage nmp device set -d $x --psp=VMW_PSP_RR; done

 

Improving ESXI Round Robin Performance

The default Round Robin settings on ESX can lead to some shortfalls in performance.  To correct this we can make a small modification that can usually help.   When we look at the default we see:

To look at all of the StorONE LUNS at once 

 

for x in `esxcfg-scsidevs -c |grep STORONE|awk '{print $1}'`; do esxcli storage nmp psp roundrobin deviceconfig get --device=$x; done

naa.6882e5a000000002000000000000012d
   Device Display Name: STORONE iSCSI Disk (naa.6882e5a000000002000000000000012d)
   Storage Array Type: VMW_SATP_ALUA
   Storage Array Type Device Config: {implicit_support=on; explicit_support=off; explicit_allow=on; alua_followover=on; action_OnRetryErrors=on; {TPG_id=2,TPG_state=ANO}{TPG_id=1,TPG_state=AO}}
   Path Selection Policy: VMW_PSP_RR
   Path Selection Policy Device Config: {policy=rr,iops=1000,bytes=10485760,useANO=0; lastPathIndex=1: NumIOsPending=0,numBytesPending=0}
   Path Selection Policy Device Custom Config: 
   Working Paths: vmhba64:C0:T0:L10
   Is USB: false

We want to lower this value to 1 ( https://kb.vmware.com/s/article/2069356?lang=en_US )

esxcli storage nmp psp roundrobin deviceconfig set --type=iops --iops=1 --device=naa.6882e5a000000002000000000000012d

naa.6882e5a000000002000000000000012d
   Device Display Name: STORONE iSCSI Disk (naa.6882e5a000000002000000000000012d)
   Storage Array Type: VMW_SATP_ALUA
   Storage Array Type Device Config: {implicit_support=on; explicit_support=off; explicit_allow=on; alua_followover=on; action_OnRetryErrors=on; {TPG_id=2,TPG_state=ANO}{TPG_id=1,TPG_state=AO}}
   Path Selection Policy: VMW_PSP_RR
   Path Selection Policy Device Config: {policy=iops,iops=1,bytes=10485760,useANO=0; lastPathIndex=1: NumIOsPending=0,numBytesPending=0}
   Path Selection Policy Device Custom Config: 
   Working Paths: vmhba64:C0:T0:L10
   Is USB: false

Or…To Changechange them all at once:

for x in `esxcfg-scsidevs -c |grep STORONE|awk '{print $1}'`; do esxcli storage nmp psp roundrobin deviceconfig set --type=iops --iops=1 --device=$x; done

ESXI Multipath Storage SATP claim rule

The previouscommands commandsjust completed will permanently set the Multipathing policy to Round Robin, and the Round Robin IOPs to 1 for best performance for all of the LUNs currently connected to the ESXI hosts from the StorONE array.  However to prevent the need to beperform runthese operations each time a new LULUN is addedconnected to theour ESXI host viafrom iSCSIthe orStorONE FC.array, a Storage SATP claim rule is recommended.

By implementingWhen a storage claim rule,rule whenis weimplemented maskall aLUNs new LU to the host, each host which has the claim rule installed will automatically make the performance changes we just manually made.

esxcli storage nmp satp rule add --satp "VMW_SATP_ALUA" --vendor "STORONE" --model "S1" --psp "VMW_PSP_RR" --psp-option "iops=1" --claim-option="tpgs_on" --description "StorONE LUN Claim Rule"

We can view the new claim rule with the following ESXcli command(s)

esxcli storage nmp satp rule list list all host claim rules


esxcli storage nmp satp rule list | grep -i storone list the StorONE installed host claim rule


NOTE: For the claim rule to take effect, the ESXI host will need to be rebooted.  When the rule takes effect, all existing LUNs will use the defined rule, asd well as all new LUNs.

Back to top