Discovery-client - Container Deployment

You can run discovery-client as a Docker container on every node in the OpenStack cluster that talks to Lightbits storage. Controller nodes, block (cinder-volume) nodes, and compute nodes all need it. This is an alternative to the .deb / .rpm package install, and you can select whichever one fits your host management model. The two installation methods are not meant to coexist on the same host.

This guide covers a single-host install. Repeat the same steps on every node in the cluster.

Prerequisites

On the host:

• Docker engine with the compose plugin (docker compose version works).
• Root access - the container runs privileged and needs host networking.
• NVMe/TCP kernel module loaded:

xxxxxxxxxx

modprobe nvme-tcp

echo nvme-tcp | tee /etc/modules-load.d/nvme-tcp.conf

• /etc/nvme/hostid present (one-time per host). On most distributions, this is created automatically by the nvme-cli package. If the file is missing, generate it using your distribution's standard procedure before continuing.
• Network access from the host to the registry that holds the discovery-client image, and to every Lightbits discovery endpoint (default port 8009/tcp).
• No package install of discovery-client running on this host. If you previously followed the .deb / .rpm instructions, stop and disable the systemd service first, and move the existing configuration out of the way:

xxxxxxxxxx

systemctl disable --now discovery-client

mv /etc/discovery-client/discovery-client.yaml /etc/discovery-client/discovery-client.yaml.pkg-bak

• Consumer-placed files under /etc/discovery-client/discovery.d/ can stay. The container will pick them up.

Pulling the Image

The image is published to a Lightbits registry. Replace the registry, image name, and tag with the values you were given for your release:

xxxxxxxxxx

export DISCOVERY_CLIENT_IMG=docker.lightbitslabs.com/discovery-client/discovery-client:<version>

Log in to the Lightbits registry (this requires credentials provided with your release - once per host):

xxxxxxxxxx

docker login docker.lightbitslabs.com

Pull the image:

xxxxxxxxxx

docker pull "$DISCOVERY_CLIENT_IMG"

Configuring Discovery-client as a Container

discovery-client reads a single configuration file at /etc/discovery-client/discovery-client.yaml inside the container. Lightbits bind-mounts the host directory /etc/discovery-client/ onto that path, so the file lives on the host.

/etc/discovery-client/discovery-client.yaml:

xxxxxxxxxx

clientConfigDir: /etc/discovery-client/discovery.d/

internalDir: /etc/discovery-client/internal/

reconnectInterval: 5s

logPagePaginationEnabled: false

maxIOQueues: 0

nvmeHostIDPath: /etc/nvme/hostid

kato: 10

logging:

  filename: /etc/discovery-client/discovery-client.log

  maxAge: 96h

  maxSize: 100

  level: info

  reportCaller: true

debug:

  metrics: true

  enablepprof: true

  endpoint: '[::]:6060'

The values above are a reference baseline; tune them for your environment. Set logging.level: debug only when troubleshooting.

docker-compose

Save the following as $BASE_DIR/discovery-client-compose.yaml (any path is fine, but the rest of this article uses that):

xxxxxxxxxx

services:

  discovery-client:

    image: ${DISCOVERY_CLIENT_IMG}

    container_name: discovery-client

    restart: unless-stopped

    privileged: true

    network_mode: host

    volumes:

      - /dev:/dev

      - /etc/discovery-client:/etc/discovery-client

    healthcheck:

      test: ["CMD", "curl", "-f", "http://localhost:6060/metrics"]

      interval: 10s

      timeout: 5s

      retries: 3

Start:

xxxxxxxxxx

docker compose -f "$BASE_DIR/discovery-client-compose.yaml" up -d

Verify:

The container should be Up and reach healthy:

xxxxxxxxxx

docker ps --filter name=discovery-client

The metrics endpoint should respond as follows:

xxxxxxxxxx

curl -fsS http://localhost:6060/metrics | head

To move to a newer image tag, update DISCOVERY_CLIENT_IMG and re-render the compose file (or edit the image: line in place). Then:

xxxxxxxxxx

docker compose -f "$BASE_DIR/discovery-client-compose.yaml" pull

docker compose -f "$BASE_DIR/discovery-client-compose.yaml" up -d

The bind-mounted host directory is preserved across upgrades, so the internal cache and any consumer-placed discovery.d/*.conf files carry over.

Stop/Remove

xxxxxxxxxx

docker compose -f "$BASE_DIR/discovery-client-compose.yaml" down

The host directory /etc/discovery-client is left in place. Remove it manually if you are decommissioning the host.