Volume Live Migration (Multi-Attach)

AI Tools

The Volume Live Migration refers to dynamically binding specific volumes to different client(s). This is a fundamental feature for cloud-native applications, which can dynamically migrate the service in virtual machines or container pods located in different physical servers.

Test Purpose

The purpose of this test is to prove that this feature can work as expected with manual commands. The volume can be dynamically reassigned to different clients, without any data loss.

In a production environment, this migration is typically conducted by a cloud infrastructure; for example, Kubernetes or OpenStack, or a customer’s own orchestrator with RESTful API.

Test Steps

Allocate two client servers for this test. You can name them Client Server A and Client Server B. Use the lbcli command to create a volume and bind it to Client Server A using the client hostnqn found on the client at /etc/nvme/hostnqn as the volume ACL. For more, see Creating a Volume on the Lightbits Storage Server.

# Create volume root@lightos-server00:~ lbcli create volume --project-name=default --name=migrate-test-vol1 --replica-count=2 --size=64GB --acl=nqn.2014-08.org.nvmexpress:uuid:a878c393-29ec-494f-bba2-098628dc436c # Output Name UUID State Protection State NSID Size Replicas Compression ACL Rebuild Progress migrate-test-vol1 dfd3b46e-1cd0-4e7d-9bb7-f32ff0ab460e Creating Unknown 0 60 GiB 2 false values:"nqn.2014-08.org.nvmexpress:uuid:a878c393-29ec-494f-bba2-098628dc436c" # Check volume creation root@lightos-server00:~ lbcli get volume --project-name=default --name=migrate-test-vol1 # Output Name UUID State Protection State NSID Size Replicas Compression ACL Rebuild Progress migrate-test-vol1 dfd3b46e-1cd0-4e7d-9bb7-f32ff0ab460e Available FullyProtected 6 60 GiB 2 false values:"nqn.2014-08.org.nvmexpress:uuid:a878c393-29ec-494f-bba2-098628dc436c" None

In Client A, use the “nvme connect” command to connect the Lightbits storage cluster, and check the visibility of the newly created volume. For more, see Connecting the Client to Lightbits.

# Contoller 0 root@client-a:~ nvme connect -t tcp -a 172.16.231.70 -s 4420 -l -1 -n nqn.2016-01.com.lightbitslabs:uuid:66ee2ad7-8602-49ed-9b82-0c85af8da36d -q nqn.2014-08.org.nvmexpress:uuid:a878c393-29ec-494f-bba2-098628dc436c # Controller 1 root@client-a:~ nvme connect -t tcp -a 172.16.231.71 -s 4420 -l -1 -n nqn.2016-01.com.lightbitslabs:uuid:66ee2ad7-8602-49ed-9b82-0c85af8da36d -q nqn.2014-08.org.nvmexpress:uuid:a878c393-29ec-494f-bba2-098628dc436c # Controller 2 root@client-a:~ nvme connect -t tcp -a 172.16.231.72 -s 4420 -l -1 -n nqn.2016-01.com.lightbitslabs:uuid:66ee2ad7-8602-49ed-9b82-0c85af8da36d -q nqn.2014-08.org.nvmexpress:uuid:a878c393-29ec-494f-bba2-098628dc436c # Check volume is available root@client-a:~ nvme list Node SN Model Namespace Usage Format FW Rev ---------------- -------------------- ---------------------------------------- --------- -------------------------- ---------------- -------- /dev/nvme0n1 b4649e6a3496d720 Lightbits LightOS 6 64.00 GB / 64.00 GB 4 KiB + 0 B 2.3

Format this volume with a filesystem; for example, XFS. Mount it to /mnt, and write some content to it. Then unmount it.

# Create directory to mount to root@client-a:~ mkdir /mnt/lb_vol # Create xfs filesystem root@client-a:~ mkfs.xfs /dev/nvme0n1 Discarding blocks ... Done. meta-data=/dev/nvme0n1 isize=512 agcount=4, agsize=3906250 blks = sectsz=4096 attr=2, projid32bit=1 = crc=1 finobt=0, sparse=0 data = bsize=4096 blocks=15625000 , imaxpct=25 = sunit=0 swidth=0 blks naming=version 2 bsize=4096 ascii-ci=0 ftype=1 log=internal log bsize=4096 blocks=7629 , version=2 = sectsz=4096 sunit=1 blks , lazy-count =1 realtime=none extsz=4096 blocks=0, rtextents=0 # Mount volume root@client-a:~ mount /dev/nvme0n1 /mnt/lb_vol # Move to mounted volume directory root@client-a:~ cd /mnt/lb_vol # Write content to file root@client-a:/mnt/lb_vol echo "some contents written from Client A" > test_file1 # Check content has been written root@client-a:/mnt/lb_vol cat test_file1 some contents written from Client A # Move out of mounted volume root@client-a:/mnt/lb_vol cd ~ # Unmount volume root@client-a:~ umount /mnt/lb_vol

Use the “lbcli update volume” command to update the ACL of this volume to the hostnqn of Client B found on the client at /etc/nvme/hostnqn.

# Update volume root@lightos-server00:~ lbcli update volume --project-name=default --name=migrate-test-vol1 --acl=nqn.2014-08.org.nvmexpress:uuid:d684af36-05da-40dc-9c4a-ef10501cdcf9 # Check volume root@lightos-server00:~ lbcli get volume --project-name=default --name=migrate-test-vol1 # Output Name UUID State Protection State NSID Size Replicas Compression ACL Rebuild Progress migrate-test-vol1 dfd3b46e-1cd0-4e7d-9bb7-f32ff0ab460e Available FullyProtected 6 60 GiB 2 false values:"nqn.2014-08.org.nvmexpress:uuid:d684af36-05da-40dc-9c4a-ef10501cdcf9" None

In both Client A and B, use the “nvme” command to check the visibility of this volume. It should disappear in Client A, but appear in Client B (make sure that Client B was also connected to the Lightbits storage cluster). This volume disappears from Client A:

root@client-a:~ nvme list root@client-a:~

This volume is connected to Client B. Check whether the previously written content is still there.

# Check volume is available root@client-b:~ nvme list Node SN Model Namespace Usage Format FW Rev ---------------- -------------------- ---------------------------------------- --------- -------------------------- ---------------- -------- /dev/nvme0n1 b4649e6a3496d720 Lightbits LightOS 6 64.00 GB / 64.00 GB 4 KiB + 0 B 2.3 # Create directory to mount volume root@client-b:~ mkdir /mnt/lb_vol # Mount volume root@client-b:~ mount /dev/nvme0n1 /mnt/lb_vol # Go to volume directory root@client-b:~ cd /mnt/lb_vol # Check file exists root@client-b:/mnt/lb_vol ls test_file1 # Check contents of file root@client-b:/mnt/lb_vol cat test_file1 some contents written from Client A

Last updated on

Was this page helpful?