Volume Shrinking and Expansion
The volume shrinking feature (available in Lightbits Version 3.0 and above) must be completed in a correct order. The layers above the block volume (such as the filesystem) must be shrunk first. If the volume is shrunk first, the filesystem could be corrupted - which could cause data loss.
To correctly shrink a volume, first shrink the filesystem (or any layers above), and shrink the Lightbits volume at the end.
Although there are many possible use cases with NVMe, we will use the following example: a simple ext4 filesystem on top of an NVMe volume. However, note that the same logical progression will apply to the rest (shrinking from the top down).
A common use case is using the XFS filesystem on top of an NVMe volume. However, as XFS does not directly support shrinking, the process is more involved: back up the data, shrink the volume, install fresh XFS on the volume, and copy the data back.
For this example, let's assume that there is a Lightbits volume named vol01, which is 2 TiB and is connected to a client as /dev/nvme0n1. This volume has a ext4 filesystem on it. Use the following steps to shrink it to 1 TiB.
During these operations, you will not have to NVMe disconnect the volume. It can be done while it is connected. However, the filesystem must be unmounted first as resize2fs does not support online shrinking (it only supports online expansion - see the next section for more details).
- Unmount the filesystem:
- From the client, shrink the filesystem to 1 TiB:
# resize2fs -p /dev/nvme0n1 1Tresize2fs 1.45.5 (07-Jan-2020)Note: Run 'e2fsck -f /dev/nvme0n1' first.The operation will not run and resize2fs will ask for a filesystem check to ensure that there are no filesystem errors before the shrinking operation is started.
Although resize2fs asks to run e2fsck with option -f (which forces a scan on the filesystem even if it is healthy), do not do that. Instead, first run a e2fsck in read-only mode using the -n flag. After that shows no errors, run it with the -f flag. This is because filesystem checks have a filesystem repairing algorithm that sometimes does more harm than good, and might leave the filesystem more corrupt and unmountable. However, if that is your only choice (i.e., the read-only result showed errors), back up your filesystem and then run e2fsck with option -f at your own risk.
Kick off a read-only filesystem check.
# e2fsck -n /dev/nvme0n1e2fsck 1.45.5 (07-Jan-2020)/dev/nvme0n1: clean, 20/134217728 files, 10806308/536870912 blocksAs the result was "clean", we can run the filesystem check with option -f.
# e2fsck -f /dev/nvme0n1e2fsck 1.45.5 (07-Jan-2020)Pass 1: Checking inodes, blocks, and sizesPass 2: Checking directory structurePass 3: Checking directory connectivityPass 4: Checking reference countsPass 5: Checking group summary information/dev/nvme0n1: 20/134217728 files (0.0% non-contiguous), 10806308/536870912 blocksNow we can successfully run the resize2fs operation again.
# resize2fs -p /dev/nvme0n1 1Tresize2fs 1.45.5 (07-Jan-2020)Resizing the filesystem on /dev/nvme0n2 to 268435456 (4k) blocks.Begin pass 2 (max = 262145)Relocating blocks XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXBegin pass 3 (max = 16384)Scanning inode table XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXThe filesystem on /dev/nvme0n2 is now 268435456 (4k) blocks long.- Collect the exact size of the new filesystem in bytes. There are multiple methods to do this. Here is one of them:
# dumpe2fs -h /dev/nvme0n1 2>&1 | awk -F: '/Block count/{count=$2} /Block size/{size=$2} END{print count*size}'1099511627776Note that 1099511627776 bytes is exactly 1 TiB.
- To calculate the exact filesystem size, do not use "df -B1", as that does not include the filesystem overhead.
- Also, do not use "nvme list" or "lsblk", as that provides the volume level size and not the filesystem size.
- The output of the resize2fs operation actually shows the final filesystem size: "the filesystem on /dev/nvme0n2 is now 268435456 (4k) blocks long". Calculate the filesystem size in bytes by multiplying the two values: 268435456 x 4096 = 1099511627776.
- From a Lightbits server, shrink the associated volume to 1 TiB. Note that for shrinking, the --force flag must be used.
The volume must be shrunk to the exact same size as the filesystem or larger. This means that it cannot be even one byte smaller than the filesystem.
- From the client, verify that the smaller size is reported at the block level via lsblk or nvme list. To get the exact byte size, use blockdev --getsize64.
# blockdev --getsize64 /dev/nvme0n11099511627776Volume Expansion
Expanding volumes is done in reverse order of shrinking. First, the volume is expanded via lbcli and then the layers above are expanded. Trying to expand in the wrong order will simply error out without doing any harm.
Although there are many possible use cases with NVMe, we will only consider the following example: a simple ext4 filesystem on top of an NVMe volume. However, the same logical progression will apply to the rest. Expand from the bottom to the top.
A common use case is using the XFS filesystem on top of an NVMe volume. XFS supports online expansion via "xfs_growfs /mount/point -D size" (meaning that it can be mounted during expansion). The only requirement is to ensure that the underlying block NVME block device has been expanded as well.
Following a similar configuration from above, assume that there is a Lightbits volume named vol01, which is 2 TiB and is connected to a client as /dev/nvme0n1. This volume has a ext4 filesystem on it. Use the following steps to expand it to 3 TiB.
- From a Lightbits server, expand the volume. Note that for expanding, the --force flag is not required.
lbcli update volume --name vol01 --project-name default --size "3 TiB"- From the client, expand the volume.
You do not need to unmount the filesystem, as resize2fs supports online expansion.
- Verify the new block level and filesystem size (see the section above for reference).
# Block Level Volume Size# blockdev --getsize64 /dev/nvme0n33298534883328# Filesystem Size# dumpe2fs -h /dev/nvme0n3 2>&1 | awk -F: '/Block count/{count=$2} /Block size/{size=$2} END{print count*size}'3298534883328