Release 3.20.1

New Feature (GA)

Ubuntu 24.04 LTS is now supported as the operating system for the Lightbits storage cluster, alongside the RHEL family. Clients and cluster can run on the same OS, tooling, and lifecycle processes.

LBM1-47292

New Feature (GA)

Added Okta as a supported external identity provider for federated authentication, alongside ADFS. Administrators can authenticate to lbcli and the REST API using their organization's Okta identities.

LBM1-47294

New Feature (GA)

Simplified installation: added a guided, interactive installer for streamlined cluster deployment, including a contact-request flow for users without a license key.

LBM1-45928

New Feature (GA)

Syslog log streaming, introduced in 3.19.1, is now generally available.

LBM1-47293

Enhancement

A safeguard has been added that prevents secondary nodes from accepting replications carrying an outdated placement group (PG) version, protecting data consistency during rapid path failure and recovery events.

LBM1-42779

Enhancement

Added a chart showing the number of TCP connections over time to the nodes dashboard.

LBM1-36914

Enhancement

Added a high-precision Grafana dashboard showing new per-volume metrics designed to be scraped at a high frequency (up to once per second), directly from the frontend.

LBM1-45099

Enhancement

Added a pure-iops option (--pure-iops in lbcli, pureIops in the REST API) to the create and update QoS policy operations. When enabled, each I/O operation is counted as one IOPS regardless of I/O size, for iops and iops-per-gb policy types.

LBM1-44768

Enhancement

Added GFTL health-specific events for the different health statuses (GFTLInstanceFileAccessError, GFTLSystemCapacityReadStatusError, GFTLDataIntegrityError, GFTLLFError, GFTLReplaceDiskAfterPowerUp, and GFTLUnknownStatus), improving observability of storage engine health.

LBM1-39897

Enhancement

Added new charts to the server performance dashboard showing total I/O request wait time and frequency, broken down by read/write and by service (frontend or replicator).

LBM1-44585

Enhancement

Added pagination support to the log collection scripts, enabling reliable log collection on clusters with a large number of etcd records.

LBM1-43313

Enhancement

Enabled high-frequency volume metrics (suffixed _hf) that can be scraped directly from the frontend via port 9190 (and port 9191 for the second instance, where applicable).

LBM1-45420

Enhancement

Improved node startup resiliency: startup now tolerates non-critical, previously recorded GFTL errors and continues booting, failing only on critical integrity and hardware issues (e.g., GftlDataIntegrityError, GftlStorageSchemeDataIntegrityError, GftlDiskFailureNoEC, GftlReplaceDiskAfterPowerUp, and GftlDoubleDiskFailure).

LBM1-40017

Enhancement

Improved rebuild responsiveness: a placement group (PG) rebuild now stops promptly when a read or write error occurs during the rebuild, instead of prolonging the operation.

LBM1-41399

Enhancement

Monitoring stack deploy.sh now has the --high-precision option to configure a high-precision (high frequency) scrape. For dual-instance installations, this should be accompanied with --dual-instance.

LBM1-46215

Enhancement

Monitoring-stack: Delivered a Loki-backed "Lightbits Cluster Logs" reference Grafana dashboard for viewing cluster service logs. This is filterable by cluster, node (instance), service (job), level, and free-text - with fixed datasource bindings (single loki-datasource), file-based Grafana provisioning, and the log-ingestion pipeline that feeds it (Alloy→Loki direct and rsyslog→promtail→Loki, with cluster_name labeling). Shipped as part of the monitoring-stack Docker-Compose refactor.

LBM1-39781

Enhancement

Starting with Lightbits 3.20 (fresh installation or upgrade), ports 9190 and 9191 were opened for high-frequency metrics collection. The firewall configuration follows the ipacl_explicit_allow setting; when false, the ports are opened via firewalld; when true, via iptables.

LBM1-45518

Enhancement

Faster node recovery: frontend restart before node revive.
Starting with Lightbits Block Storage Server 3.20.1, when the Lightbits frontend reports a health failure, the cluster first restarts the frontend on the affected node instance only. The shared IFM backend and the remaining services on the server keep running, so recovery completes in seconds instead of requiring a node revive or a server reboot. The number of direct restart attempts is controlled by the cluster configuration parameter MaxRestartAttempts, which now defaults to 1 (valid range: 0-16; 0 disables the feature). This default applies to both newly installed clusters and clusters upgraded from earlier releases. If the restart does not resolve the failure, recovery escalates as before:
- When the allowed restart attempts are exhausted, or a restart fails, the cluster performs an auto-revive on the affected server. This restarts the node-instance services for all node instances on that server, not only the one that reported the failure.
- On the final permitted revive attempt (AllowedNumRevives), the server is rebooted.
- IFM health failures and data-loss events skip the frontend restart and escalate directly to revive or reboot.
To disable the feature at any time:
lbcli update cluster-config --name=MaxRestartAttempts --value=0

Note: If you explicitly set MaxRestartAttempts=0 on a release earlier than 3.20.1, upgrading restores the new default of 1. Re-apply the command above after the upgrade to keep the feature disabled.

LBM1-45596

Enhancement

The Ansible cleanup_all playbook now supports containerized (Ubuntu) installations.

LBM1-39013

Enhancement

The cluster upgrade now restarts the Grafana Alloy logging agent so the upgraded binary takes effect immediately (previously, on SELinux-disabled hosts, the old Alloy process kept running until the next reboot). If a custom post-upgrade restart script is in use, add systemctl stop alloy and systemctl start alloy to it (stop last, start first), so that Alloy is restarted during the upgrade. Without this, Alloy keeps running the previous binary until the next reboot, though the upgrade itself still succeeds

LBM1-39798

Enhancement

The Kubernetes CSI plugin no longer supports the deprecated snapshot controller version 3, and the snapshot controller name no longer includes a version suffix. Verify that your cluster runs a supported snapshot controller version before upgrading.

LBM1-42955

Enhancement

The profile-generator now allocates six control cores (previously four) to accommodate the co-resident Grafana Alloy logging agent. Server profiles are regenerated accordingly.

LBM1-45017

Major

A data-integrity safeguard has been added to prevent a read from completing with incorrect data in the rare case where an SSD fails to write into the read buffer due to an IOMMU fault.

LBM1-45131

Major

Hardened the etcd-based leader election so that - in a rare combination of node failures, etcd disruptions, and network issues - two management instances can no longer be active simultaneously.

LBM1-44972

Major

Strengthened rebuild handling against a rare timing condition where a rebuild to a newly added empty node failed within a few seconds of starting; a subsequent rebuild attempt could then skip part of the data destined for that node. Rebuilds now transfer all data to the new node in this scenario.

LBM1-45601

Major

Strengthened the resiliency of node recovery and rebuild against a rare timing condition between snapshot deletion and node recovery. Snapshots are now cleaned up consistently across nodes, preventing a stale snapshot from later causing a rebuild to fail.

LBM1-44159

Minor

Added collection of Grafana Alloy logs to lb-support.

LBM1-45626

Minor

Fixed an issue where a secondary node failure could cause outstanding client writes to hang for longer than the client I/O timeout. Replications waiting on a pending connection are now canceled when the connecting peer is removed, allowing failover to complete promptly.

LBM1-45904

Minor

Fixed incorrect values (namespace ID, QoS limits, size) reported in the frontend log upon volume update completion, even though the volume updates themselves were applied correctly.

LBM1-46178

Minor

Improved offline installation robustness: installation now completes successfully on hosts that have additional repositories enabled under /etc/yum.repos.d.

LBM1-43390

Minor

Improved the resiliency of cluster encryption key (KEK) rotation, which now completes reliably under a high volume of concurrent volume and snapshot operations.

LBM1-44673

Minor

Improved the resiliency of internal management workflows (FSMs). Previously, a workflow that could not determine a valid state would exit silently, leaving its object (for example, a pending volume update) unresponsive until the management service was restarted, while the rest of the cluster continued operating normally. Workflows now pause after the maximum retry attempts and resume automatically on the next trigger, once the cluster state has settled.

LBM1-44465

Minor

Lightbits 3.19.1 introduced strict validation of REST requests to the api-service, rejecting requests containing unsupported or malformed fields that earlier releases accepted and ignored. A configurable api-service option, skipUnknownFieldValidation, is now available to accept such requests and ignore unknown fields, for clients such as VCP or other REST integrations that may send non-compliant fields. The default strict validation remains recommended. To enable this mode, for each api-service: 1. update the api-service yaml file: skipUnknownFieldValidation: false->true 2. restart the api-service.

LBM1-44819

Minor

Log streaming to rsyslog targets previously supported only the secured (TLS) mode, and streams to non-secured, plain-text rsyslog endpoints were not delivered. Log streaming to non-secured rsyslog targets is now supported.

LBM1-44762

Minor

Reduced Prometheus metric cardinality for frontend replication metrics by aggregating them per core/shard and instance (outgoing metrics labeled by peer UUID, incoming by peer IP) and removing irrelevant metrics, preventing excessive metric growth on large clusters.

LBM1-46313

Minor

The bundled Grafana Alloy log-streaming agent had anonymous usage statistics reporting enabled by default, periodically contacting an external Grafana endpoint. In air-gapped or firewalled deployments, these attempts failed and produced recurring error entries in the journal. Usage statistics reporting is now disabled; the data previously reported was non-sensitive and non-personally identifiable.

LBM1-46209

Minor

The lbcli and REST API documentation for the disable-server (evict) operation inaccurately described evicting servers hosting RF=1 (single-replica) volumes and overstated the scope of the force flag. The documentation now accurately states that force permits evicting a server whose nodes are unavailable, while still requiring the remaining nodes to be active.

LBM1-44875

Minor

The tar and gzip packages are now installed on management hosts as a requirement for lb-support, ensuring that log collection succeeds on hosts where they were absent.

LBM1-43401

Minor

The Ubuntu (containerized) deployment now installs mdadm on the host, ensuring that software RAID management is available outside the node-manager container.

LBM1-45730

Minor

Fixed an issue where lbcli fetch logs with the -x flag could fail to locate the generated log archive.

LBM1-35060

Minor

When SSD Journaling is enabled, a frontend failure caused by a non-journal issue could be misclassified as a journal device failure, leaving the node inactive in a Permanently Failed state. With journaling disabled, the only effect was a spurious, cosmetic "journal device failed" event. Failure classification is now accurate in both cases

LBM1-42963