This article discusses how to connect Lightbits to Kubernetes for use with persistent volumes.
Configuring Lightbits with Kubernetes
To configure Lightbits with Kubernetes:
Downloading and Installing Packages
Download the CSI plugin tarball and unpack (Note: Cross-check the latest version with Lightbits Support):
curl -1 -O 'https://dl.lightbitslabs.com/public/lightos-csi/raw/files/lb-csi-bundle-v1.22.0.25046615759.tar.gz'
sudo tar -xvf lb-csi-bundle-v1.22.0.25046615759.tar.gz
Create the Snapshot controller if not already available:
kubectl create -f k8s/snapshot-controller-4.yaml
Create the CSI plugin with the discovery-client from the extracted yaml file:
kubectl create -f k8s/lb-csi-plugin-k8s-v1.24-dc.yaml
Check the install:
kubectl get --namespace=kube-system statefulset lb-csi-controller; kubectl get --namespace=kube-system daemonsets lb-csi-node; kubectl get --namespace=kube-system pod --selector app=lb-csi-plugin -o wide
Expect to see that all pods are in a Ready state and there are no errors.
Configuring the Secret And Storage Class
Grab JWT from the Lightbits cluster and export it:
export LIGHTOS_JWT=eyJhbGc...lxQ2L7Wpe773w
Convert the JWT to base 64:
echo -n $LIGHTOS_JWT | base64 -w0 -
Copy the output and add it to the secrets file:
First, open the secrets file in examples/secrets-and-storage-class.yaml.
Next, paste in the JWT on the next line under jwt: |-, ensuring that the JWT is on one line and the pipe dash symbol stays as |-.
Finally, change the management IP addresses in the file.
The file should look like the example below:
---
# Source: lb-csi-workload-examples/charts/storageclass/templates/secret.yaml
apiVersion: v1
kind: Secret
metadata:
name: example-secret
namespace: default
labels:
helm.sh/chart: "storageclass-0.1.0"
app.kubernetes.io/instance: "RELEASE-NAME"
app.kubernetes.io/version: ""
app.kubernetes.io/managed-by: "Helm"
type: lightbitslabs.com/jwt
data:
jwt: |-
ZXlKaGJHY2lPaUpTVXpJMU5pSXNJbXRwWkNJNkluTjVjM1JsYlRweWIyOTBJaXdpZEhsd0lqb2lTbGRVSW4wLmV5SmhkV1FpT2lKTWFXZG9kRTlUS...pFLTJJSWNHSmpYWEhhNU9pQQ==
---
# Source: lb-csi-workload-examples/charts/storageclass/templates/storageclass.yaml
apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
name: example-sc
provisioner: csi.lightbitslabs.com
allowVolumeExpansion: true
parameters:
mgmt-endpoint: 10.10.0.2:443,10.10.0.3:443,10.10.0.4:443
replica-count: "3"
compression: disabled
project-name: default
mgmt-scheme: grpcs
csi.storage.k8s.io/controller-publish-secret-name: example-secret
csi.storage.k8s.io/controller-publish-secret-namespace: default
csi.storage.k8s.io/controller-expand-secret-name: example-secret
csi.storage.k8s.io/controller-expand-secret-namespace: default
csi.storage.k8s.io/node-publish-secret-name: example-secret
csi.storage.k8s.io/node-publish-secret-namespace: default
csi.storage.k8s.io/node-stage-secret-name: example-secret
csi.storage.k8s.io/node-stage-secret-namespace: default
csi.storage.k8s.io/provisioner-secret-name: example-secret
csi.storage.k8s.io/provisioner-secret-namespace: default
Create the secrets and first storage class:
kubectl create -f examples/secret-and-storage-class.yaml
Check the status:
Testing the Configuration
The examples/filesystem-workload.yaml file provided with the supplementary package contains two manifests:
PVC named example-fs-pvc, referencing example-sc StorageClass created above.
POD named example-fs-pod, binding to example-fs-pvc.
To deploy the PVC and the POD, run:
kubectl apply -f examples/filesystem-workload.yaml
persistentvolumeclaim/example-fs-pvc created
pod/example-fs-pod created
Using the following command, we will see the PV, PVC resources in Bound status and POD in READY state.
kubectl get pv,pvc,pod
NAME CAPACITY ACCESS MODES RECLAIM POLICY STATUS CLAIM STORAGECLASS REASON AGE
persistentvolume/pvc-7680be61-0694-44cf-9d1b-1f69827d0b4b 10Gi RWO Delete Bound default/example-fs-pvc example-sc 69s
NAME STATUS VOLUME CAPACITY ACCESS MODES STORAGECLASS AGE
persistentvolumeclaim/example-fs-pvc Bound pvc-7680be61-0694-44cf-9d1b-1f69827d0b4b 10Gi RWO example-sc 70s
NAME READY STATUS RESTARTS AGE
pod/example-fs-pod 1/1 Running 0 70s
If there are any errors, use the command below to view the logs:
kubectl get events --all-namespaces --sort-by='.metadata.creationTimestamp'
Verify that the Lightbits volume (PersistentVolume) is mounted to/mnt/test. There should be files in the directory such as an output of the hostname.
kubectl exec --stdin --tty example-fs-pod -- /bin/sh
ls /mnt/test
kubectl delete -f examples/filesystem-workload.yaml
persistentvolumeclaim "example-fs-pvc" deleted
pod "example-fs-pod" deleted
