Capacity Scale Up
The Lightbits storage cluster supports dynamically expanding the total physical capability per requirement. This is important for reducing the TCO by delaying the purchase until needed.
The capacity expansion can support scale up and scale out. Scale up refers to adding more NVMe SSDs to storage servers, while scale out refers to adding more storage servers for both capacity and performance.
This test case will cover how to do capacity scale up by adding a new NVMe SSD into the cluster.
Test Purpose
The purpose of this test is to prove that this feature can work as expected with manual commands. After adding a new NVMe SSD, the storage cluster’s total physical capacity should increase as expected.
Test Steps
- Physically add a new NVMe SSD to one specific storage server in the cluster. Use “lbcli list nvme-devices” to check the NVMe device status of this specific storage server. Also check the current total physical capacity of the whole cluster by using “lbcli get cluster -o json”.
# List nvme devices for Lightbits serverroot@lightos-server00:~ lbcli list nvme-devices --server-uuid=375dc324-9a53-542eaf7e-4fb461f37a24Name Size NUMA ID Serial State Server UUID Node UUIDnvme11n1 7.3 TiB 1 PHLJ9503004H8P0HGN Healthy 375dc324-9a53-542e-af7e-4fb461f37a24 859bd46d-abe8-54fa-81c4-9683f8705b65nvme4n1 7.3 TiB 1 PHLJ9503008B8P0HGN Healthy 375dc324-9a53-542e-af7e-4fb461f37a24 859bd46d-abe8-54fa-81c4-9683f8705b65nvme9n1 7.3 TiB 1 PHLJ950501068P0HGN Healthy 375dc324-9a53-542e-af7e-4fb461f37a24 859bd46d-abe8-54fa-81c4-9683f8705b65nvme7n1 7.3 TiB 1 PHLJ950501G28P0HGN Healthy 375dc324-9a53-542e-af7e-4fb461f37a24 859bd46d-abe8-54fa-81c4-9683f8705b65nvme6n1 7.3 TiB 1 PHLJ950502848P0HGN Healthy 375dc324-9a53-542e-af7e-4fb461f37a24 859bd46d-abe8-54fa-81c4-9683f8705b65nvme8n1 7.3 TiB 1 PHLJ950502H18P0HGN Healthy 375dc324-9a53-542e-af7e-4fb461f37a24 859bd46d-abe8-54fa-81c4-9683f8705b65nvme5n1 7.3 TiB 1 PHLJ9513004L8P0HGN Healthy 375dc324-9a53-542e-af7e-4fb461f37a24 859bd46d-abe8-54fa-81c4-9683f8705b65nvme10n1 7.3 TiB 1 PHLJ951501A08P0HGN Healthy 375dc324-9a53-542e-af7e-4fb461f37a24 ---# Get cluster storage metricsroot@lightos-server00:~ lbcli get cluster -o json | grep -A 7 installedPhysicalStorage"installedPhysicalStorage": "214441895116800","managedPhysicalStorage": "168032827662336","effectivePhysicalStorage": "125304479671908","logicalStorage": "13300901888000","logicalUsedStorage": "417060335616","physicalUsedStorage": "415986804672","physicalUsedStorageIncludingParity ": "485317938783","freePhysicalStorage": "124888492867236",The newly-added NVMe SSD should be in “Healthy” state, but not in any storage node (instance).
- Add the new NVMe SSD to the specified storage node (instance). Enable it to be a part of the storage cluster.
root@lightos-server00:~ lbcli add nvme-device --serial-number=PHLJ951501A08P0HGN --node-uuid=859bd46d-abe8-54fa-81c4-9683f8705b65After a few minutes, use the command “lbcli list nvme-devices” to check whether the NVMe SSD was added successfully, and check the new physical capacity of the cluster.
# List nvme devices for Lightbits serverroot@lightos-server00:~ lbcli list nvme-devices --server-uuid=375dc324-9a53-542eaf7e-4fb461f37a24Name Size NUMA ID Serial State Server UUID Node UUIDnvme11n1 7.3 TiB 1 PHLJ9503004H8P0HGN Healthy 375dc324-9a53-542e-af7e-4fb461f37a24 859bd46d-abe8-54fa-81c4-9683f8705b65nvme4n1 7.3 TiB 1 PHLJ9503008B8P0HGN Healthy 375dc324-9a53-542e-af7e-4fb461f37a24 859bd46d-abe8-54fa-81c4-9683f8705b65nvme9n1 7.3 TiB 1 PHLJ950501068P0HGN Healthy 375dc324-9a53-542e-af7e-4fb461f37a24 859bd46d-abe8-54fa-81c4-9683f8705b65nvme7n1 7.3 TiB 1 PHLJ950501G28P0HGN Healthy 375dc324-9a53-542e-af7e-4fb461f37a24 859bd46d-abe8-54fa-81c4-9683f8705b65nvme6n1 7.3 TiB 1 PHLJ950502848P0HGN Healthy 375dc324-9a53-542e-af7e-4fb461f37a24 859bd46d-abe8-54fa-81c4-9683f8705b65nvme8n1 7.3 TiB 1 PHLJ950502H18P0HGN Healthy 375dc324-9a53-542e-af7e-4fb461f37a24 859bd46d-abe8-54fa-81c4-9683f8705b65nvme5n1 7.3 TiB 1 PHLJ9513004L8P0HGN Healthy 375dc324-9a53-542e-af7e-4fb461f37a24 859bd46d-abe8-54fa-81c4-9683f8705b65nvme10n1 7.3 TiB 1 PHLJ951501A08P0HGN Healthy 375dc324-9a53-542e-af7e-4fb461f37a24 859bd46d-abe8-54fa-81c4-9683f8705b65# Get cluster storage metricsroot@lightos-server00:~ lbcli get cluster -o json | grep -A 7 installedPhysicalStorage"installedPhysicalStorage": "214441895116800","managedPhysicalStorage": "176034390884352","effectivePhysicalStorage": "132265839653681","logicalStorage": "13300901888000","logicalUsedStorage": "417060958208","physicalUsedStorage": "415987427264","physicalUsedStorageIncludingParity ": "481509289495","freePhysicalStorage": "131849852226417",The newly-added NVMe SSD is now part of one specific storage node (instance), and the “managedPhysicalStorage” and “effectivePhysicalStorage” are increased.
The total capacity increase could also be monitored by using the Grafana GUI in the cluster dashboard. In this example test case, the total capacity was increased from 114TiB to 120TiB.
