ICM Overview

Lightbits' Intelligent Cluster Management (ICM) is a standalone service that provides a single management plane across multiple Lightbits Lightbits NVMe/TCP storage clusters. It runs in Docker containers on dedicated resources outside the Lightbits clusters, and is deployed with Docker Compose.

ICM exposes a gRPC and REST API (the REST surface is served through grpc-gateway) that mirrors the Lightbits DurosAPI v2, so clients manage volumes, snapshots, and cluster-lifecycle operations across a federation of clusters through one endpoint.

ICM keeps an in-memory table of the volumes, snapshots, and cluster metadata it polls from the attached clusters, places volumes intelligently across clusters, and forwards operations to the appropriate cluster. Long-running operations - attaching and detaching clusters, and thick clones - are orchestrated as durable Temporal workflows.

ICM is the consolidation of two earlier Lightbits products - Cluster Federation (CF) and the Data Mobility Service (DMS) - into a single service. Customers migrating from DMS keep working through a backward-compatible legacy API.

Objectives

• One control plane for many clusters. Manage a fleet of Lightbits clusters through a single API and CLI endpoint instead of per-cluster tooling.

• Data mobility across clusters. Copy data between clusters with thick clones.

• Frictionless migration. DMS customers move to ICM with no client code change.

Capabilities

• Cluster federation - a transparent gRPC + REST proxy across attached clusters, with cross-cluster volume placement and a continuously-refreshed inventory of all volumes and snapshots. Attaching, detaching, and refreshing clusters run as workflows.

• Thick clones (data mobility) - block-copy a source snapshot into a new, independent volume or snapshot on another attached cluster, over NVMe/TCP through the host that ICM runs on. Each thick clone runs as a workflow and returns a workflow ID immediately; progress is observable through the workflow APIs and the Temporal UI.

• Discovery client - a Lightbits NVMe discovery client runs alongside ICM so that thick-clone destination volumes surface as local block devices on the ICM host; ICM registers every attached cluster with it automatically.

• Observability - an optional Prometheus + Grafana + node-exporter stack - plus ICM service metrics - ship with the deployment.

• Workflow inspection - list and fetch workflows through the API/CLI, or open the Temporal UI for the full event history of any operation.

Architecture

ICM is a single Go binary that serves its gRPC + REST API over TLS on port 443. Its compute is stateless: the set of attached clusters and the service key material live in a dedicated etcd registry, and workflow state lives in Temporal. A background loop polls the attached clusters on a fixed interval (30s by default) to keep the inventory current. Authentication uses JWT (RS256), both for clients calling ICM and for ICM calling the Lightbits clusters.

Deployed Services