Lightbits TRIM Support

The Lightbits TRIM feature allows filesystems to reclaim unused storage space by issuing commands to the storage system (called trim/discard operations). Lightbits handles trim/discard operations issued by the filesystem on its volumes. Combined with Lightbits’ thin provisioning capabilities, this improves storage performance and capacity efficiency.

For instance, the 'fstrim' operation on an XFS filesystem mounted on a Lightbits volume can reduce storage utilization (reduce volume usage size), when invoked with the TRIM feature enabled.

Lightbits has implemented the NVMe deallocate as specified in the NVMe command set specification, to improve capacity utilization and planning. This capability works with the TRIM command, allowing the operating system to inform Lightbits about unused data blocks. For more information, refer to sections 3.3.3 and 3.3.3.2.1 of the NVMe command set specification.

When the OS deletes a file and sends a TRIM command, it is translated to an NVMe dataset management deallocate command on the client side. Lightbits receives this deallocate command and manages the information internally within its software, behaving as if the data was deallocated. This approach enables more efficient capacity utilization and garbage collection - and improves write performance and endurance.

High-Level Workflow

Configuring the TRIM Feature on the Lightbits Cluster

The TRIM feature must be enabled for 3.16 installations. For existing deployments, upgrade to 3.16 and perform the following steps to enable the TRIM feature.

Enable the feature with this command:

Viewing Reduced Storage Using TRIM

This example details the complete process of enabling TRIM, creating and deleting files, and viewing its effects on storage usage.

1. Create a Lightbits volume and set up the filesystem:

2. Check the initial storage usage:

3. Use the dd command to create a ~0.5GB file (or 0.5GB per replica/node):

4. Wait approximately one minute, and then check the storage usage:

Note the increase in used storage.

5. Delete the file:

6. Check the storage usage again:

Note that the used storage has not decreased yet.

7. Enable the TRIM feature on the Lightbits cluster:

8. Verify that the TRIM feature is set to “true”:

9. Set up and run fstrim.timer on the client machine:

10. Manually trigger a TRIM operation to see its immediate effects:

11. Wait approximately one minute, and then check the updated storage usage:

You should now see a reduction in the "used" storage, indicating that the TRIM operations have successfully reclaimed space.

Example Output:

12. To verify that TRIM is working on an ongoing basis:

Create another test file:

Check the storage usage:

Delete the file:

Wait for the next scheduled TRIM operation (or trigger it manually), and then check the storage usage again:

TRIM Operations - Best Practices

Initial Setup After Enabling TRIM

1. Enable TRIM on the Lightbits cluster:

2. On each client machine, set up fstrim.timer:

3. Verify the timer status:

4. Run an initial manual TRIM to reclaim space from previously-deleted data:

Regular Maintenance

1. Periodically check the status of fstrim.timer:

2. Monitor the storage usage after TRIM operations:

3. If the storage is not being reclaimed as expected, manually trigger TRIM:

Customizing the TRIM Schedule

If a weekly schedule is not suitable, customize the TRIM schedule as follows on every client machine.

1. Create a custom timer file:

2. Add your custom schedule. For example, to run it daily:

3. Save the file and reload the systemd daemon:

4. Restart the timer:

Avoiding Continuous TRIM

Do not use the -o discard mount option for filesystems on Linux, as it can significantly impact filesystem performance.

TRIM Troubleshooting

If TRIM is not working:

Verify that it is enabled on the Lightbits cluster:

Check the fstrim.timer status on the client:

Review the system logs for any TRIM-related errors:

If these or other issues persist, contact Lightbits Customer Success for assistance.