Log Streaming (lbcli)

This article covers the same flow as Log Streaming (REST API), but uses lbcli instead of raw curl commands.

Prerequisites

A running Lightbits cluster.
lbcli installed and on your $PATH.
The system:cluster-admin JWT for the cluster.

Step 1: Setting your JWT and Server

Bash
    
xxxxxxxxxx
 
export SERVER=server00   # replace with your server hostname or IPexport JWT=eyJhbGciOiJSUzI1NiIsImtpZCI6InN5c3RlbTpyb290IiwidHlwIjoiSldUIn0...# ^ obtain from /etc/cluster-manager/system_jwt on any cluster node
Copy

All lbcli commands below pass the JWT via -J $JWT and the server via --mgmt-endpoint $SERVER. You can also set LBCLI_MGMT_ENDPOINT in your environment to avoid repeating --mgmt-endpoint on every call.

Step 2: (Optional) Creating a TLS Bundle

A TLS bundle holds the certificate material used for mTLS connections to an Rsyslog target. Skip this step if you only need a Loki target.

Bash
    
xxxxxxxxxx
 
lbcli -J $JWT create logs tls-bundles \  --name=dc-rsyslog \  --ca-pem=./scripts/certs/ca.crt \  --cert-pem=./scripts/certs/rsyslog-server.crt \  --key-pem=./scripts/certs/rsyslog-server.key
Copy

List the bundles to verify:

Bash
    
xxxxxxxxxx
 
lbcli -J $JWT list logs tls-bundles
Copy

Step 3: Creating Sources

A source tells Alloy which log stream to collect on each server. You can collect from journald (by syslog identifier) or from a file path.

Control-plane Services (journald)

Bash
    
xxxxxxxxxx
 
for svc in node-manager cluster-manager lightbox-exporter api-service \           upgrade-manager discovery-service etcd alloy; do  lbcli -J $JWT create logs sources \    --name="$svc" \    --journald-id="$svc" \    --log-format=zap-json \    --min-level=info \    --enableddone
Copy

Data-plane Services (journald)

Bash
    
xxxxxxxxxx
 
lbcli -J $JWT create logs sources \  --name=duroslight-0 \  --journald-id=duroslight-0 \  --min-level=info \  --enabled lbcli -J $JWT create logs sources \  --name=gftl \  --journald-id=gftl \  --min-level=info \  --enabled
Copy

Event Log Files (Optional)

Event logs are written to files only when emit_service_events_to_file is enabled in the Ansible config.

Bash
    
xxxxxxxxxx
 
for svc in node-manager cluster-manager api-service upgrade-manager; do  lbcli -J $JWT create logs sources \    --name="$svc-events" \    --file-paths="/var/log/$svc-events.log" \    --log-format=zap-json \    --min-level=info \    --label="job=$svc-events" \    --enableddone
Copy

List all sources to verify:

Bash
    
xxxxxxxxxx
 
lbcli -J $JWT list logs sources
Copy

Get a specific source:

Bash
    
xxxxxxxxxx
 
lbcli -J $JWT get logs sources --name=node-manager
Copy

Step 4: Creating Targets

A target is a log destination. You can have multiple targets; every collected log line is forwarded to all enabled targets.

Loki Target (No TLS)

Bash
    
xxxxxxxxxx
 
lbcli -J $JWT create logs targets \  --name=loki1 \  --type=loki \  --url=http://loki.example.com:3100/loki/api/v1/push \  --enabled
Copy

Rsyslog Target (mTLS, Requires TLS Bundle from Step 2)

Bash
    
xxxxxxxxxx
 
lbcli -J $JWT create logs targets \  --name=rsyslog1 \  --type=rsyslog \  --endpoint=rsyslog.example.com:6514 \  --rsyslog-timeout=5s \  --rsyslog-tls=dc-rsyslog \  --enabled
Copy

List the targets to verify:

Bash
    
xxxxxxxxxx
 
lbcli -J $JWT list logs targets
Copy

Get a specific target:

Bash
    
xxxxxxxxxx
 
lbcli -J $JWT get logs targets --name=loki1
Copy

Step 5: Verifying Collector Status

Once the configuration is applied, Alloy polls lightbox-exporter for the rendered pipeline config and begins shipping logs. Check that each server's Alloy agent is healthy:

Bash
    
xxxxxxxxxx
 
lbcli -J $JWT list logs collectors
Copy

When everything is healthy, each collector entry shows the following:

If PIPELINE shows DEGRADED or ERROR, check the target connectivity.

Modifying the Configuration

Sources and targets can be updated or deleted without restarting any service. Alloy picks up the new pipeline config on its next remotecfg poll (default: 1 minute).

Only the flags you explicitly pass are changed; unspecified fields keep their current values.

Bash
    
xxxxxxxxxx
 
# Raise a source's minimum log levellbcli -J $JWT update logs sources \  --name=node-manager \  --min-level=warn # Disable a target temporarilylbcli -J $JWT update logs targets \  --name=loki1 \  --enabled=false # Re-enable itlbcli -J $JWT update logs targets \  --name=loki1 \  --enabled=true # Delete a source (idempotent — safe to run even if already absent)lbcli -J $JWT delete logs sources --name=gftl # Delete a targetlbcli -J $JWT delete logs targets --name=rsyslog1 # Delete a TLS bundle (only after all targets using it are removed)lbcli -J $JWT delete logs tls-bundles --name=dc-rsyslog
Copy

Inspecting the Rendered Alloy Pipeline Config

You can inspect the pipeline config that lightbox-exporter renders for Alloy by querying the remotecfg endpoint on the server directly (no auth required — node-local only):

Bash
    
xxxxxxxxxx
 
curl -s --header "Content-Type: application/json" \    --data '{}' \    http://$SERVER:8090/observability.v1.ConfigService/GetConfig | jq -r .content
Copy

Last updated on

Was this page helpful?