Connecting the Client to Lightbits

AI Tools

To connect the client to Lightbits, make sure to do the following:

Ping to data_ip from the client to the target.
To use NVMe/TCP, load the NVMe host modules.

modprobe nvme_tcp; lsmod | grep nvme

The output is similar to the following example:

root@rack07-server56:~ modprobe nvme_tcp; lsmod | grep nvme nvme_tcp 36864 0 nvme_fabrics 24576 1 nvme_tcp nvme_core 118784 3 nvme_tcp ,nvme_fabrics t10_pi 16384 2 sd_mod ,nvme_core

On the target node, enter the lbcli get cluster command on the Lightbits storage server to identify the subsytem NQN.

lbcli get cluster

Sample Output

root@rack03-server72:~ lbcli get cluster UUID Subsystem NQN Current Max Replicas Supported Max Replicas MinVersionInCluster MinAllowedVersion MaxAllowedVersion d63110c9-f6f3-4ab3-ba86-1e6fd6afc66a nqn.2016-01.com.lightbitslabs:uuid:66ee2ad7-8602-49ed-9b82-0c85af8da36d 3 3 2.3.12~b793 2.3.X 2.4.X

Enter the lbcli list nodes command to identify the NIC IP address and TCP port. With the IP, port, subsystem NQN, and ACL values for the volume, you can execute the nvme connect command to connect to all of the nodes in the cluster.

lbcli list nodes

Sample Output

root@rack03-server72:~ lbcli list nodes Name UUID State NVMe endpoint Failure domains Loca server02-0 3b7a262a-ba3f-5447-9c21-e3c8ed699f0f Active 172.16.231.72:4420 [server02] None server00-0 8b980170-f941-5473-9162-36a17e866a36 Active 172.16.231.70:4420 [server00] None server01-0 cfbfdc90-43b3-5ce6-be99-1ab4171785b0 Active 172.16.231.71:4420 [server01] None

Use NVMe Connect

This command connects to a single controller in the Lightbits cluster. You will have to connect to every controller in the cluster.

nvme connect -t tcp -a $CONTROLLER_DATA_IP --ctrl-loss-tmo -1 -n $LIGHTBITS_CLUSTER_NQN -s 4420 -q $VOLUME_ACL

Sample NVMe Connect Commands

# server00-0 nvme connect -t tcp -a 172.16.231.70 --ctrl-loss-tmo -1 -n nqn.2016-01.com.lightbitslabs:uuid:66ee2ad7-8602-49ed-9b82-0c85af8da36d -s 4420 -q acl3 # server01-0 nvme connect -t tcp -a 172.16.231.71 --ctrl-loss-tmo -1 -n nqn.2016-01.com.lightbitslabs:uuid:66ee2ad7-8602-49ed-9b82-0c85af8da36d -s 4420 -q acl3 # server02-0 nvme connect -t tcp -a 172.16.231.72 --ctrl-loss-tmo -1 -n nqn.2016-01.com.lightbitslabs:uuid:66ee2ad7-8602-49ed-9b82-0c85af8da36d -s 4420 -q acl3

Use NVMe Connect-All

This command connects to all controllers using a single data IP.

nvme connect-all -t tcp -a $CONTROLLER_DATA_IP --ctrl-loss-tmo -1 -q $VOLUME_ACL

Sample NVMe Connect-All Commands

nvme connect-all -t tcp -a 172.16.231.70 --ctrl-loss-tmo -1 -q acl3

Warning

There is an issue with nvme-cli v1.9 where nvme connect-all returns a segfault. If this occurs, please update nvme-cli or use the nvme connect command.

Note

For persisting the NVMe connections across client reboots, use the Lightbits open source discovery-client tool.

Check Connection

To check which clients are connected to the Lightbits cluster, run:

lbcli list connected-hosts

Sample Output

root@rack03-server72:~ lbcli list connected-hosts Host NQN IP Address Hostname acl3 172.16.231.78

Last updated on

Was this page helpful?