Volume Live Migration (Multi-Attach)
The Volume Live Migration refers to dynamically binding specific volumes to different client(s). This is a fundamental feature for cloud-native applications, which can dynamically migrate the service in virtual machines or container pods located in different physical servers.
Test Purpose
The purpose of this test is to prove that this feature can work as expected with manual commands. The volume can be dynamically reassigned to different clients, without any data loss.
In a production environment, this migration is typically conducted by a cloud infrastructure; for example, Kubernetes or OpenStack, or a customer’s own orchestrator with RESTful API.
Test Steps
- Allocate two client servers for this test. You can name them Client Server A and Client Server B. Use the lbcli command to create a volume and bind it to Client Server A using the client hostnqn found on the client at
/etc/nvme/hostnqnas the volume ACL. For more, see Creating a Volume on the Lightbits Storage Server.
# Create volumeroot@lightos-server00:~ lbcli create volume --project-name=default --name=migrate-test-vol1 --replica-count=2 --size=64GB --acl=nqn.2014-08.org.nvmexpress:uuid:a878c393-29ec-494f-bba2-098628dc436c# OutputName UUID State ProtectionState NSID Size Replicas Compression ACL Rebuild Progressmigrate-test-vol1 dfd3b46e-1cd0-4e7d-9bb7-f32ff0ab460e Creating Unknown 0 60 GiB 2 false values:"nqn.2014-08.org.nvmexpress:uuid:a878c393-29ec-494f-bba2-098628dc436c"# Check volume creationroot@lightos-server00:~ lbcli get volume --project-name=default --name=migrate-test-vol1# OutputName UUID State Protection State NSID Size Replicas Compression ACL Rebuild Progressmigrate-test-vol1 dfd3b46e-1cd0-4e7d-9bb7-f32ff0ab460e Available FullyProtected 6 60 GiB 2 false values:"nqn.2014-08.org.nvmexpress:uuid:a878c393-29ec-494f-bba2-098628dc436c" None- In Client A, use the “nvme connect” command to connect the Lightbits storage cluster, and check the visibility of the newly created volume. For more, see Connecting the Client to Lightbits.
# Contoller 0root@client-a:~ nvme connect -t tcp -a 172.16.231.70 -s 4420 -l -1 -n nqn.2016-01.com.lightbitslabs:uuid:66ee2ad7-8602-49ed-9b82-0c85af8da36d -q nqn.2014-08.org.nvmexpress:uuid:a878c393-29ec-494f-bba2-098628dc436c# Controller 1root@client-a:~ nvme connect -t tcp -a 172.16.231.71 -s 4420 -l -1 -n nqn.2016-01.com.lightbitslabs:uuid:66ee2ad7-8602-49ed-9b82-0c85af8da36d -q nqn.2014-08.org.nvmexpress:uuid:a878c393-29ec-494f-bba2-098628dc436c# Controller 2root@client-a:~ nvme connect -t tcp -a 172.16.231.72 -s 4420 -l -1 -n nqn.2016-01.com.lightbitslabs:uuid:66ee2ad7-8602-49ed-9b82-0c85af8da36d -q nqn.2014-08.org.nvmexpress:uuid:a878c393-29ec-494f-bba2-098628dc436c# Check volume is availableroot@client-a:~ nvme listNode SN Model Namespace Usage Format FW Rev---------------- -------------------- ------------------------------------------------- -------------------------- ---------------- --------/dev/nvme0n1 b4649e6a3496d720 Lightbits LightOS 6 64.00 GB / 64.00 GB 4 KiB + 0 B 2.3- Format this volume with a filesystem; for example, XFS. Mount it to /mnt, and write some content to it. Then unmount it.
# Create directory to mount toroot@client-a:~ mkdir /mnt/lb_vol# Create xfs filesystemroot@client-a:~ mkfs.xfs /dev/nvme0n1Discarding blocks ... Done.meta-data=/dev/nvme0n1 isize=512 agcount=4, agsize=3906250 blks = sectsz=4096 attr=2, projid32bit=1= crc=1 finobt=0, sparse=0 data = bsize=4096 blocks=15625000 , imaxpct=25 = sunit=0 swidth=0 blksnaming=version 2 bsize=4096 ascii-ci=0 ftype=1log=internal log bsize=4096 blocks=7629 , version=2= sectsz=4096 sunit=1 blks , lazy-count =1realtime=none extsz=4096 blocks=0, rtextents=0# Mount volumeroot@client-a:~ mount /dev/nvme0n1 /mnt/lb_vol# Move to mounted volume directoryroot@client-a:~ cd /mnt/lb_vol# Write content to fileroot@client-a:/mnt/lb_vol echo "some contents written from Client A" > test_file1# Check content has been writtenroot@client-a:/mnt/lb_vol cat test_file1some contents written from Client A# Move out of mounted volumeroot@client-a:/mnt/lb_vol cd ~# Unmount volumeroot@client-a:~ umount /mnt/lb_vol- Use the “lbcli update volume” command to update the ACL of this volume to the hostnqn of Client B found on the client at
/etc/nvme/hostnqn.
# Update volumeroot@lightos-server00:~ lbcli update volume --project-name=default --name=migrate-test-vol1 --acl=nqn.2014-08.org.nvmexpress:uuid:d684af36-05da-40dc-9c4a-ef10501cdcf9# Check volumeroot@lightos-server00:~ lbcli get volume --project-name=default --name=migrate-test-vol1# OutputName UUID State Protection State NSID Size Replicas Compression ACL Rebuild Progressmigrate-test-vol1 dfd3b46e-1cd0-4e7d-9bb7-f32ff0ab460e Available FullyProtected 6 60 GiB 2 false values:"nqn.2014-08.org.nvmexpress:uuid:d684af36-05da-40dc-9c4a-ef10501cdcf9" None- In both Client A and B, use the “nvme” command to check the visibility of this volume. It should disappear in Client A, but appear in Client B (make sure that Client B was also connected to the Lightbits storage cluster). This volume disappears from Client A:
root@client-a:~ nvme listroot@client-a:~This volume is connected to Client B. Check whether the previously written content is still there.
# Check volume is availableroot@client-b:~ nvme listNode SN Model Namespace Usage Format FW Rev---------------- -------------------- ------------------------------------------------- -------------------------- ---------------- --------/dev/nvme0n1 b4649e6a3496d720 Lightbits LightOS 6 64.00 GB / 64.00 GB 4 KiB + 0 B 2.3# Create directory to mount volumeroot@client-b:~ mkdir /mnt/lb_vol# Mount volumeroot@client-b:~ mount /dev/nvme0n1 /mnt/lb_vol# Go to volume directoryroot@client-b:~ cd /mnt/lb_vol# Check file existsroot@client-b:/mnt/lb_vol lstest_file1# Check contents of fileroot@client-b:/mnt/lb_vol cat test_file1some contents written from Client AWas this page helpful?