Secret and StorageClass
For these workloads to work, you are required at a minimum to provide the following cluster specific parameters:
JWTmgmt-endpoints
Without modifying these parameters, the workloads will likely fail.
Storing Lightbits Authentication JWT in a Kubernetes Secret
Kubernetes and the CSI specification support passing "secrets" when invoking most of the CSI plugin operations, which the plugins can use for authentication and authorization against the corresponding storage provider. This allows a single CSI plugin deployment to serve multiple unrelated users or tenants and to authenticate with the storage provider on their behalf using their credentials, as necessary. The Lightbits CSI plugin takes advantage of this "secrets" passing functionality for authentication and authorization.
See the Lightbits documentation corresponding to the Lightbits software version you have deployed for an overview of the security model, details on how to configure the multi-tenancy mode, managing Lightbits clusters that have authentication and authorization enabled, uploading credentials to Lightbits, generating JSON Web Tokens (JWTs), etc.
In order to authenticate on Lightbits API, a Kubernetes secret containing base64 encoded JWT is needed.
Example JWT encoding:
$ export LIGHTOS_JWT=eyJhbGc...lxQ2L7Wpe773w$ echo -n $LIGHTOS_JWT | base64 -ZXlKaGJHY2lPaUpTVXpJMU5pSXNJbXRwWkNJNkluTjVjM1JsYlRweWIyOTBJaXdpZEhsd0lqb2lTbGRVSW4wLmV5SmhkV1FpT2lKTWFXZG9kRTlUSWl3aVpYaHdJam94TmpRMU5UQTNOemcyTENKcFlYUWlPakUyTVRNNU56RTNPRFlzSW1semN5STZJbk41YzNSbGMzUnpJaXdpYW5ScElqb2lhWFJ5UjNNMk1sTk1hMmxhY2xKdlNuWjNXazFhZHlJc0ltNWlaaUk2TVRZeE16azNNVGM0Tml3aWNtOXNaWE1pT2xzaWMzbHpkR1Z0T21Oc2RYTjBaWEl0WVdSdGFXNGlYU3dpYzNWaUlqb2liR2xuYUhSdmN5MWpiR2xsYm5RaWZRLlc5QXMwdTJQZnFudTIzZ3U0YXFYcTBKMXZETUJ6bkVfT3dkZkxGeEgzMUdZZVAxWHFqbUNLUWlZS3pJcXlmcTgweTdCZC02azZvZlVXbzlRZ0FDb1J6LUhRWTJjc1pYdHVHTGRpRzN3YUF3aEs3QjRIQnhROFAzSnpSeno4TzJLOVg1Z3dRY19xYnpjYTBNaUlrWTZVVjVTOWNEMTROTHNQRExwUjdvOFRMbFozbm9kSDZiRlNNVjlPeF9GRXBvTGVidzRWLUlvaURiTV9NdTFDSzZCOUJGeFpNRTV6NmJIMXlkSDZFWnRuUFlRaUVrRVdlUzFHMUJSTVNfR0hGN3Nja2NYU0c3Q1pkSFFqOHY1b0Y1YS1USHNVdXR0dmFIc1hUS3FzREFkOHRvbEphZUNUN0NWRFFHX0xUQ1hYZ3dudUI3c0ZRaHJHbHhRMkw3V3BlNzczdw==In the file examples/secret-and-storage-class.yaml, edit the Secret.data.jwt value with the base64 encoded JWT string.
Creating a StorageClass
To take advantage of dynamic provisioning, at least one StorageClass Kubernetes object must be created first. A template of the appropriate StorageClass specification looks as follows:
You will need to replace the highlighted placeholders (removing the angle brackets in the process) with the actual field values as indicated in the table below. See the Lightbits Administrator's Guide for additional information on the Lightbits management API service and volume replica counts.
| Placeholder | Description | |
|---|---|---|
<sc-name> | The name of the StorageClass you want to define. This name will be referenced from other Kubernetes object specifications (e.g.: StatefulSet, PersistentVolumeClaim), to use a volume that will be provisioned from a Lightbits storage cluster mentioned below, with the corresponding volume characteristics. | |
| <true\false> (allowVolumeExpansion) | Kubernetes PersistentVolume-s can be configured to be expandable and Lightbits supports volume expansion. If set to true, it will be possible to expand the volumes created from this StorageClass by editing the corresponding PVC Kubernetes objects. Note: CSI volumes expansion is enabled in Kubernetes v1.16 and above. CSI volume expansion in older Kubernetes versions is not supported by the Lightbits CSI plugin. | |
<lb-mgmt-address> | One of the Lightbits management API service endpoint IP addresses of the Lightbits cluster on which the volumes belonging to this StorageClass will be created. The mgmt-endpoint entry of the StorageClass spec accepts a comma-separated list of <lb-mgmt-address>:<lb-mgmt-port> pairs. For high availability, specify the management API service endpoints of all the Lightbits cluster servers, or at least the majority of the servers. | |
<lb-mgmt-port> | The port number on which the Lightbits management API service is running. Typically, this is port 443 and port 80 for encrypted and encrypted communications, respectively - but Lightbits servers can be configured to serve the management interface on other ports as well. | |
<grpc\grpcs> | The protocol to use for communication with the Lightbits management API service. Lightbits clusters with multi-tenancy support enabled can be accessed only over the TLS-protected grpcs protocol for enhanced security. Lightbits clusters with multi-tenancy support disabled can be accessed using the legacy unencrypted grpc protocol. | |
<proj-name> | The name of the Lightbits project to which the volumes from this StorageClass will belong. The JWT specified using <secret-name> below must have sufficient permissions to carry out the necessary actions in that project. | |
<num-replicas> | The desired number of replicas for volumes dynamically provisioned for this StorageClass. Valid values are: 1, 2 or 3. The number must be specified in ASCII double quotes (e.g.: "2"). | |
<compression> | Specifies whether the volumes created for this StorageClass should have compression enabled or disabled. The compression line of the StorageClass spec can be omitted altogether, in which case the Lightbits storage cluster default setting for compression will be used. However, if it is present, it must contain one of the following two values: enabled or disabled. | |
<secret-name> | The name of the Kubernetes Secret that holds the JWT to be used while making requests pertaining to this StorageClass to the Lightbits management API service. See also <secret-namespace> below. Typically the JWT used for all the different types of operations (5 in the examples below) will be the same JWT, but there is no requirement for that to be the case. | |
<secret-namespace> | The namespace in which the Secret referred to in <secret-name> above resides. | |
<qos-policy-name> | New volumes created will be attached with that qos policy. The default value is "", which means using the default qos profile. |
Kubernetes passes the values from the parameters section of the specification verbatim to the Lightbits CSI plugin, in order to inform it of the necessary provisioning actions. Here is an example of a complete StorageClass definition (also available in the examples/secret-and-storage-class.yaml file from the Supplementary Package):
You should modify the examples/secret-and-storage-class.yaml file with a list of relevant Lightbits management API service endpoints.
To create this StorageClass on the Kubernetes cluster and verify the outcome, you must run the following commands:
You can create as many StorageClass-es as you need based on a single or multiple Lightbits storage clusters - and with different replication factor and compression settings, belonging to the same or different Lightbits projects.
IPv6 Support
CSI also supports IPv6. For IPv6, the storage class yaml can be set up as in the following example:
The CSI plugin can then connect to the API over the IPv6 endpoint, and implement all of the data connections via IPv6.