NVMeOF Enclosures
NVMeOF Enclosures
An NVMeOF enclosure is an external drive shelf that exposes its drives over NVMeOF instead of being directly, locally attached to a StorONE node. StorONE connects out to it as a client (an NVMeOF initiator) over one connection per node, and once connected, the drives inside the enclosure show up alongside your local drives and can be approved into pools exactly the same way —— see Drives.
This is the reverse relationship from the Network page's NVMeOF settings, which control StorONE acting as the NVMeOF target that hosts connect to for volume access. Here, StorONE is the one doing the connecting, to pull in extra drive capacity.
Listing NVMeOF enclosures
Using the web interface
Open Inventory > Resources > Drives > NVMeOF Enclosures. Use the toolbar to show advanced information per connection or per recognized drive.
Using the CLI
resources drives nvmeof list [--connectionStatus|--driveStatus]
| Option | Description |
|---|---|
--connectionStatus |
Show advanced information per connection (one row per node/enclosure connection, with its live/down status and drive count). |
--driveStatus |
Show advanced information per recognized drive (one row per drive, with the connection(s) it's reachable through). |
Examples
List enclosures with basic status:
resources drives nvmeof list
List every connection to every enclosure, to check which paths are currently live:
resources drives nvmeof list --connectionStatus
Adding an NVMeOF enclosure
Adding an enclosure registers one connection per node that should reach it —— in a multi-node system you'd normally add one connection per node, each pointing at the same enclosure over its own network path, so both nodes have access. Each connection needs the target IP address, port, and transport type (rdma or tcp).
By default, drives in an NVMeOF enclosure aren't used for volume metadata, since metadata access is latency-sensitive and a network-attached enclosure adds latency that local SSD/NVMe drives don't have. Only turn on --allowUseForMetadata if the system has no directly-attached SSD/NVMe drives to use instead.
Using the web interface
- Open Inventory > Resources > Drives > NVMeOF Enclosures.
- Click Create.
- Under Name, enter the enclosure's name, and whether to allow its drives for metadata.
- Under Connections, add one row per node/address/port/type combination.
- Confirm to add the enclosure.
Using the CLI
resources drives nvmeof add --name=<name> --nodes=<text...> --addresses=<ip...> --ports=<numbers...> --types=<text...> [--allowUseForMetadata]
| Option | Description |
|---|---|
--name=<name> |
The name of the NVMeOF enclosure. |
--nodes=<text...> |
The name of the node that connects to the NVMeOF enclosure (one entry per connection). |
--addresses=<ip...> |
The IP address of the NVMeOF enclosure connection (one entry per connection, matched by position to --nodes). |
--ports=<numbers...> |
The port of the NVMeOF enclosure connection (matched by position). |
--types=<text...> |
The connection type: rdma or tcp (matched by position). |
--allowUseForMetadata |
Allow using drives in this enclosure for metadata. Should be used only if there are no SSD/NVMe drives connected directly to the system. |
Examples
Add an enclosure with one TCP connection per node in a 2-node system:
resources drives nvmeof add --name shelf1 --nodes node1,node2 --addresses 10.10.20.11,10.10.20.12 --ports 4420,4420 --types tcp,tcp
Add an enclosure and allow its drives to hold metadata (only if there's no local SSD/NVMe available):
resources drives nvmeof add --name shelf1 --nodes node1,node2 --addresses 10.10.20.11,10.10.20.12 --ports 4420,4420 --types tcp,tcp --allowUseForMetadata
Editing an NVMeOF enclosure
Editing lets you rename the enclosure, replace its connections, and flip the metadata-eligibility setting. If you supply new connections, they replace the entire existing connection list —— you can't edit a single connection in place, so include all the connections you want to keep.
Using the web interface
- Open Inventory > Resources > Drives > NVMeOF Enclosures.
- Select the enclosure and click Edit.
- Update the name, connections, and/or metadata setting under Name / Connections.
- Confirm.
Using the CLI
resources drives nvmeof edit --name=<name> [--newname=<name>] [--nodes=<text...> --addresses=<ip...> --ports=<numbers...> --types=<text...>] [--allowUseForMetadata|--doNotAllowUseForMetadata]
| Option | Description |
|---|---|
--name=<name> |
The name of the NVMeOF enclosure to edit. |
--newname=<name> |
The new name for the NVMeOF enclosure. |
--nodes=<text...> |
The name of the node that connects to the NVMeOF enclosure (replaces all existing connections). |
--addresses=<ip...> |
The IP address of the NVMeOF enclosure connection. |
--ports=<numbers...> |
The port of the NVMeOF enclosure connection. |
--types=<text...> |
The connection type: rdma or tcp. |
--allowUseForMetadata |
Allow using drives in this enclosure for metadata. |
--doNotAllowUseForMetadata |
Do not allow using drives in this enclosure for metadata. |
Examples
Rename shelf1 to shelf1-east:
resources drives nvmeof edit --name shelf1 --newname shelf1-east
Turn off metadata eligibility for an enclosure now that local SSDs have been added:
resources drives nvmeof edit --name shelf1-east --doNotAllowUseForMetadata
Deleting an NVMeOF enclosure
You can't delete an enclosure while any of its drives are still approved and in use —— unapprove those drives first (see Drives). Deleting the enclosure disconnects it entirely; its drives will no longer be available to the system.
Using the web interface
- Open Inventory > Resources > Drives > NVMeOF Enclosures.
- Select the enclosure and click Delete.
- Confirm
—— read the confirmation message if any drives from the enclosure are still approved, since those must be unapproved first.
Using the CLI
resources drives nvmeof delete --name=<name> [--force]
| Option | Description |
|---|---|
--name=<name> |
The name of the NVMeOF enclosure to delete. |
--force |
Skip the confirmation prompt. |
Example
resources drives nvmeof delete --name shelf1-east
See also: Resources overview ·· Drives ·· Network —— enabling NVMeOF traffic for hosts connecting to StorONE, the reverse direction from this page.