NVME_TCP Discovery Client Connection Method

The NVMe-over-Fabrics specification defines the concept of a discovery controller that an NVMe host can query on a fabric network to discover NVMe subsystems. These subsystems are contained in NVMe targets that can be connected from the network.

The discovery-client is a deployable service running under systemd. Every 5 seconds (configurable), it will poll the discoveryservices in the cluster.

The discovery-client command requires several parameters that you can query for the NVMe/TCP storage server to be discovered and connected to. Note that the base NVMe specification defines the NQN (NVMe Qualified Name) format, which an NVMe endpoint (device, subsystem, etc) must follow to guarantee a unique name under the NVMe standard. In particular, the host NQN uniquely identifies the NVMe host, and can be used by the discovery controller to control what NVMe target resources are allocated to the NVMe host for a connection.

Install the discovery-client rpm.

Bash
    
 
yum install -y discovery-client-(ver).x86_64.rpm
Copy

Start the discovery-client service.

Bash
    
 
systemctl start discovery-client.service
Copy

Get the hostnqn.

Bash
    
 
cat /etc/nvme/hostnqn
Copy

Output:

Bash
    
 
root@rack03-server64 :~cat /etc/nvme/hostnqnnqn.2014-08.org.nvmexpress:uuid:4fa758a4-713c-4ef9-9511-c89d53209b73
Copy

If there is no hostnqn, you can generate one by using the following command:

Bash
    
 
nvme gen-hostnqn > /etc/nvme/hostnqn
Copy

Create a dedicated volume with the client’s NQN, specifying it using the ACL parameter.

Bash
    
 
root@rack11 -server93 :~ lbcli create volume --size="3 Tib" --name=vol_test5 **--acl="nqn.2014 -08. org.nvmexpress:uuid:4fa758a4 -713c-4ef9 -9511 - c89d53209b73"** --compression=true --replica -count =1 --project -name=defaultName UUID State Protection State NSIDSize Replicas Compression ACLRebuild Progressvol_test5 34c1236b -592a-4db9 -9218 -0301 c725d520 Creating Unknown 03.0 TiB 1 true values:"nqn .2014 -08. org.nvmexpress:uuid:4fa758a4 -713c-4ef9 -9511 - c89d53209b73"
Copy

Trigger the service to connect to the assigned NQN volumes.

Bash
    
 
discovery-client connect-all -t tcp -a <ip > -s 8009 -q <hostnqn > -p
Copy

These are only a small sample of the possible parameters. See the discovery-client help menu for a complete list and usage of the discovery-client command. We have selected these parameters as an example of the minimum required to connect to and use an NVMe/TCP volume.

After the application client connects to the Lightbits storage cluster, the volumes that this application client can access appear as local block devices. The following example shows how to use the Linux nvme command to list all block devices after the connectcommand finishes.

Sample Command

Bash
    
 
$ nvme list
Copy

Sample Output

Bash
    
 
root@rack03 -server64 :~ nvme listNode SN Model NamespaceUsage Format FW Rev---------------- -------------------- ---------------------------------------- ----------------------------------- ---------------- --------/dev/nvme0n1 adeed2240d4fe72e Lightbits LightOS 673.30 TB / 3.30 TB 4 KiB + 0 B 2.3
Copy

Last updated on

Was this page helpful?