Title
Create new category
Edit page index title
Edit category
Edit link
Lightbits Supported Events
Lightbits services will generate events for various operations, activities, statuses, errors, or warnings that can occur in a Lightbits cluster. These events can be collected via official Lightbits API, or - from Release 3.0.3 - by scraping them from the Lightbits services log files.
The format and information of events will be identical in either format they collected through (API/logs). Events in logs will be preceded by a dedicated "LBEVENT" tag, to simplify filtering them out of logs.
The default location for all logs of Lightbits services is /var/log/<service-name>.log; however, this may be customized in each cluster install by updating the service role template:
logging:
filename: /var/log/node-manager.log
Event Name | Event Cause | Event Code | Cause Code | Severity | Event Type | Reporting Service | Description |
|---|---|---|---|---|---|---|---|
Node Inactive | Local RAID failure | 201 | 102 | High | Node | Node Manager | Node state has changed to inactive due to multiple NVMe device disk failures (EC enabled)/single disk failure (no EC). Note: This event was renamed. |
Node Inactive | OS health check failed | 201 | 110 | High | Node | Node Manager | Node state has changed to inactive due to a kernel panic being detected in the OS. |
Node Inactive | Duroslight health check failed | 201 | 105 | Medium | Node | Node Manager | Node state has changed to inactive due to a failure with Duroslight. |
Node Inactive | Heartbeat to node failed | 201 | 107 | Medium | Node | Cluster Manager | Node state has changed to inactive due to a node being unreachable (network failure/server down). |
Node Inactive | Connectivity issue | 201 | 108 | Medium | Node | Cluster Manager | Node state has changed to inactive due to connectivity errors in dataplane. |
Node Inactive | FailedToEnableEncryption | 201 | 1304 | High | Node | Cluster Manager | Node state has changed to inactive due to connectivity |
Node Active | N/A | 200 | 1 | Info | Node | Cluster Manager | Node is active. |
Node in Permanent Failure State Inactive | Node Inactive | 206 | 112 | High | Node | Cluster Manager | Node entered permanent failure state (duration of node in inactive state exceeded configured time window of inactivity). |
Node Powerup | Started | 208 | 2 | Info | Node | Node Manager | Node power-up started. |
Node Powerup | Completed | 208 | 3 | Info | Node | Node Manager | Node power-up completed. |
Node Powerup | Failed | 208 | 4 | High | Node | Node Manager | Node power-up failed. |
Node High Storage Usage |
| 204 | 500 | Info | Node | Node Manager | Node entered degraded performance state due to high storage utilization (> 70%). |
Node High Storage Usage |
| 204 | 501 | Medium | Node | Node Manager | Node entered degraded performance state due to high metadata utilization. |
Node Read-Only State | Node entered read-only state | 209 | 504 | High | Node | Node Manager | Node entered read-only state. |
Node Read-Only State | Node exited read-only state | 209 | 506 | Info | Node | Node Manager | Node exited the read-only state. |
|
| 204 | 505 | High | Node | Node Manager | Node metadata is at high utilization (90%), and nearing read-only state. |
| Initiated | 205 | 2 | Medium | Node | Node Manager | Local Raid rebuild has started. |
| Completed | 205 | 3 | Info | Node | Node Manager | Local Raid rebuild has completed. |
|
| 205 | 504 | High | Node | Node Manager | Local Raid rebuild was halted (as the node entered read-only state, no further free storage to complete the rebuild). |
|
| 205 | 506 | Medium | Node | Node Manager | Local Raid rebuild resumed (continued with local Raid rebuild after it was halted). |
Node Unattached | N/A | 210 | 1 | Info | Node | Cluster Manager | Node entered Unattached state. All volume and snapshot resources were migrated off of this node. |
Server Clock Drift | Clock drift detected | 1000 | 1000 | High | Server | Node Manager | Detected a clock drift between the reporting server and the rest of the servers in the cluster. |
Server Linux VM Write Cache Configuration Error | Server Linux VM Write Cache Configuration Error | 1200 | 1200 | High | Server | Node Manager | For Internal use |
Server Upgrade | Started | 301 | 2 | Info | Server | Upgrade Manager | Upgrade of the server started. |
Server Upgrade | Finished | 301 | 3 | Info | Server | Upgrade Manager | Upgrade of the server completed. |
Server Upgrade | Failed | 301 | 4 | High | Server | Upgrade Manager | Upgrade of server has failed. Note: Additional failure-specific information will be returned here. |
Server Upgrade Skipped | Server non-upgradeable | 302 | 1100 | Medium | Server | Upgrade Manager | Skip upgrade operation for server, as the server is non-upgradeable. Note: Additional specific information to skip the cause will be returned here. |
| Failed | 700 | 4 | High | Server | Upgrade Manager | Failed to complete (upgrade manager failed to start up). |
| Started | 401 | 2 | Info | Server | Upgrade Manager | Started an upgrade of the cluster. |
| Finished | 401 | 3 | Info | Server | Upgrade Manager | Completed an upgrade of the cluster. |
| Failed | 401 | 4 | High | Server | Upgrade Manager | Upgrade of the cluster has failed. Note: Additional failure-specific information will be returned here. |
|
| 1001 | 1003 | Medium | NVMe SSD | Node Manager | An NVMe SSD device is unhealthy. |
|
| 1001 | 1002 | Medium | NVMe SSD | Node Manager | An NVMe SSD device is unhealthy. |
|
| 1001 | 1004 | Medium | NVMe SSD | Node Manager | An NVMe SSD device is unhealthy. |
|
| 1001 | 1005 | Medium | NVMe SSD | Node Manager | An NVMe SSD device is unhealthy. |
|
| 1001 | 1007 | Medium | NVMe SSD | Node Manager | An NVMe SSD device is unhealthy. |
|
| 1001 | 1006 | Medium | NVMe SSD | Node Manager | An NVMe SSD device is unhealthy. |
| N/A | 501 | 1 | High | NVMe SSD | Node Manager | An NVMe SSD device has failed. |
|
| 501 | 114 | Critical | NVMe SSD | Node Manager | An NVMe SSD used for journaling has failed. |
|
| 211 | 114 | High | Node | Node Manager | Node journaling device failed, but no failed NVMe device was detected. |
| N/A | 502 | 1 | Info | NVMe SSD | Node Manager | Added a new NVMe SSD to a node. |
|
| 503 | 901 | High | NVMe SSD | Node Manager | Failed to add a new NVMe SSD to a node. |
| N/A | 600 | 1 | Info | Volume | Cluster Manager | Volumes are in fully protected protection state (the event is issued on entering a new state; the event will return a list of volumes affected by this change of state). |
| N/A | 601 | 1 | Medium | Volume | Cluster Manager | Volumes are in degraded protection state (the event is issued on entering a new state; the event will return a list of volumes affected by this change of state). |
| N/A | 602 | 1 | High | Volume | Cluster Manager | Volumes are in read-only protection state (the event is issued on entering a new state; the event will return a list of volumes affected by this change of state). |
| N/A | 603 | 1 | High | Volume | Cluster Manager | Volumes are in unavailable protection state (the event is issued on entering a new state; the event will return a list of volumes affected by this change of state). |
|
| 402 | 1001 | High | Cluster | Cluster Manager | Cluster utilization is high. |
|
| 1100 | 1113 | Critical | Volume/NVMeSSD | Node Manager | Unrecoverable data integrity error. |
|
| 1100 | 1112 | Critical | Volume/NVMeSSD | Node Manager | Unrecoverable data integrity error. |
|
| 1100 | 1111 | Critical | Volume/NVMeSSD | Node Manager | Unrecoverable data integrity error. |
|
| 1100 | 1110 | Critical | Volume/NVMeSSD | Node Manager | Unrecoverable data integrity error. |
|
| 1101 | 1113 | High | Volume/NVMeSSD | Node Manager | Recoverable data integrity error. |
|
| 1101 | 1112 | High | Volume/NVMeSSD | Node Manager | Recoverable data integrity error. |
|
| 1101 | 1111 | High | Volume/NVMeSSD | Node Manager | Recoverable data integrity error. |
|
| 1101 | 1110 | High | Volume/NVMeSSD | Node Manager | Recoverable data integrity error. |
|
| 1100 | 1116 | Critical | Node/NVMeSSD | Node Manager | Detected data integrity error during node rebuild. |
|
| 1101 | 1115 | High | Node/NVMeSSD | Node Manager | Detected data integrity error during node rebuild, during recovery from a graceful shutdown. |
|
| 1102 | 1118 | High | Node/NVMeSSD | Node Manager | Detected data integrity error during garbage collection processing. |
| FailedToGetKEK | 1400 | 1500 | Medium | ServerEncryption | Node Manager | Failed to get the encryption key from the cluster manager. |
| FailedToInitializeTPM | 1400 | 1501 | Medium | ServerEncryption | Node Manager | Failed to initialize the TPM key. |
| FailedToReadKEK | 1400 | 1502 | Medium | ServerEncryption | Node Manager | Failed to set the encryption key in the cache. |
| FailedToWriteKEK | 1400 | 1503 | Medium | ServerEncryption | Node Manager | Failed to save the encryption key. |
| MissingDEK | 1301 | 1300 | Critical | Cluster Encryption | Node Manager | Failed to encrypt or update a volume on a node, due to a missing DEK. |
| CorruptedDEK | 1301 | 1301 | Critical | Cluster Encryption | Node Manager | Failed to encrypt or update a volume on a node, due to a corrupted DEK. |
| NotEnoughActiveNodes | 1302 | 1302 | Medium | Cluster Encryption | Cluster Manager | There are not enough servers with active nodes up to enable encryption. |
| FailedToDistributeKekToNodes | 1302 | 1303 | Medium | Cluster Encryption | Cluster Manager | Cluster Manager failed to distribute the Cluster Encryption Key to all servers with active nodes. |
| FailedToEnableEncryption | 1302 | 1304 | Medium | Cluster Encryption | Cluster Manager | Failed to enable encryption on the cluster level. |
| FailedToEnableEncryption | 1303 | 2 | Info | Cluster Encryption | Cluster Manager | Cluster encryption process was initiated. |
| FailedToEnableEncryption | 1303 | 3 | Info | Cluster Encryption | Cluster Manager | The cluster encryption process completed successfully. Your cluster is encrypted. |
| Completed | 321 | 3 | Info | Server | Cluster Manager | Server disable operation completed successfully (move server into maintenance mode). |
| Failed to Complete | 321 | 4 | Medium | Server | Cluster Manager | Server disable operation failed to complete (a more specific error cause is returned in the event description).. |
| Completed | 322 | 3 | Info | Server | Cluster Manager | Server enable operation completed successfully (move server out of maintenance mode). |
| Completed | 311 | 3 | Info | Server | Cluster Manager/Node Manager | New server created in the cluster (server added to the cluster). |
| Failed to Complete | 312 | 4 | High | Server | Cluster Manager | Failed to add a new server to the cluster. |
| Completed | 313 | 3 | Info | Server | Cluster Manager | Deleted a server from the cluster. |
| Initiated | 351 | 2 | High | Server | Cluster Manager | Started an eviction of a server's resources. |
| Completed | 351 | 3 | Info | Server | Cluster Manager | Completed an eviction of a server's resources. |
| Failed to Complete | 351 | 4 | High | Server | Cluster Manager | Failed to complete an eviction of a server's resources. |
| Initiated | 352 | 2 | Info | Server | Cluster Manager | Initiated an abortion of an ongoing server eviction. |
| Completed | 352 | 3 | Info | Server | Cluster Manager | Completed issuing an abort of an ongoing eviction. |
| Initiated | 1304 | 2 | Info | Cluster Encryption | Cluster Manager | Started the process to rotate the cluster root key. |
| Completed | 1304 | 3 | Info | Cluster Encryption | Cluster Manager | Completed the process of rotating the cluster root key. |
| Complete | 1304 | 4 | Medium | Cluster Encryption | Cluster Manager | Failed to complete cluster root key rotation. Either due to a fatal error, failing the rotation complexly, or only causing it to take longer or be stuck without failing it. |
© 2026 Lightbits Labs™