Configuring Global Variables in Ansible

AI Tools

Review the configuration of the global variables in the Ansible file all.yml, located in the following location:

~/light-app/ansible/inventories/cluster_example/group_vars/all.yml

File Example

#nvme_subsystem_nqn_suffix: "some_suffix" ipacl_explicit_allow: false persistent_memory: true start_discovery_service_retries: 5 #nvme_subsystem_nqn_suffix: "some_suffix" chrony_enabled: true local_repo_base_url: https://dl.lightbitslabs.com/<YOUR_TOKEN>/lightos-3-(Minor Ver)-(Rev)-rhl-(8/9)/rpm/el/8/$basearch #optional #listen_on_all_interfaces: false #optional clusterId - if set will be injected to all nodes, else will be generated by the deployment. #clusterId: ae7bdeef-897e-4c5b-abef-20234abf21bf

If all of the machines in the cluster have persistent memory (DCPMM) installed, the persistent_memory flag must be set as follows:

persistent_memory: true

If there are machines in the cluster that do not have PMEM installed, set this flag to false. If you are planning to use Journaling instead of DCPMM, this flag should be set as false.

Note

Since the persistent_memory flag is a global property for all of the clusters, it is important to declare this flag only once under the all.yml file and not in host vars files with different values.

ipacl_explicit_allow

Provides support for restricted/non-restricted access to a cluster's data ports (nvme/tcp and discovery service) and cluster internal ports (node replication ports, etcd peer and client ports, node manager and cluster manager internal grpc ports).

For internal ports, in restricted mode (ipacl_explicit_allow = true), the Lightbits cluster will block all external traffic to these ports and allow traffic only between the IP's of the servers in the cluster. When adding a new server to the cluster, this list (on the existing server's iptables rules) should be updated either by adding the new server's IP to the list of the allow rules in iptables, or by using the create/update admin endpoint API.

For example, when adding a server with IP 172.16.123.18 to the cluster, you will need to run the create admin-endpoint API before installing the server:

$ lbcli create admin-endpoint --port=2380 --ips=172.16.123.18 $ lbcli create admin-endpoint --port=2379 --ips=172.16.123.18 $ lbcli create admin-endpoint --port=4001 --ips=172.16.123.18 $ lbcli create admin-endpoint --port=4007 --ips=172.16.123.18 $ lbcli get admin-endpoint Port IPs 2379 [172.16.123.18] 2380 [172.16.123.18] 4001 [172.16.123.18] 4007 [172.16.123.18]

For data ports, in restricted mode (ipacl_explicit_allow = true), the Lightbits cluster will maintain an allowed list of IPs per volume that allows clients to access this volume. In unrestricted mode (ipacl_explicit_allow = false), any IP address can be used to access the data ports. Any further port restrictions should be managed by your FW rules.

See IP ACL Configuration for additional information.

Note

All other ports that are required for external communication to the cluster will remain open in the iptables (22, 443, 8009, etc.).

Chrony

Chrony is the default network synchronization method for Lightbits clusters.

chrony_enabled: true

Chrony

In Chrony configurations - which are the default for Red Hat 8 based releases and onward - configure the all.yml settings.

Make sure that the Chrony service is configured on your servers before the installation.

Note

Verify that the date and time are in sync on the Lightbits storage servers and the Ansible installation host. You can use date and run it simultaneously on all servers.

Listen on All Interfaces

Configure the frontend to bind to all network interfaces, rather than a specific one.

# Default to configure the system frontend to listen on all interfaces # if true the param `ipAddr` in /etc/node-manager/duroslight-n.yaml: will be set to - '::' listen_on_all_interfaces: true
Note

This setting is applicable only in single-node deployments. With this special mode, direct NVMe connections to specific IP addresses are required over using the discovery client.

Online Repo Information

local_repo_base_url: https://dl.lightbitslabs.com/<YOUR_TOKEN>/lightos-3-(Minor Ver)-(Rev)-rhl-(8/9)/rpm/el/8/$basearch
  • This section describes from where Lightbits will be installed. In this case/example, the repo URL is provided, as this uses the online method. However, for the offline installation method, this section is different. For more, see Single-IP-Dual-NUMA Configuration.

  • The "local___repo_base_url" field must be filled in with the <TOKEN> and the remainder of the URI, to properly direct the Ansible installation to the correct Lightbits repository. Different release versions will have slightly different paths. The “local_repo_base_url” must be correct or else the installation will fail. This can be a Cloudsmith link or any HTTP/HTTPS accessible point in the local network or internet. The link must point to a valid yum repo with a "repodata" directory. The value is used by Ansible for the baseurl key in /etc/yum.repos.d/lightos.repo. The $basearch variable is replaced by the OS automatically with the architecture of the Lightbits server (e.g., x86_64).

  • In this example, there is also a “local_repo_base_url” entry that includes <YOUR_TOKEN>. This information was provided to you in the Lightbits Customer Addendum. You will need to enter this value here before proceeding.

Server Variables

Variable

Required

Description

local_repo_base_url

Yes

Mandatory unless offline installation is used. This is the same value entered for the “baseurl” you configured in the auto$ section.

auto_reboot

No

A False value means that the installation will wait for user instructions to either reboot or not after installation. A True value means that the installation will reboot in case of a kernel change without user instructions. The default value is False (recommended value).

clusterID

No

An identifier of the cluster that is used to filter the logs of a specific cluster.

Offline Installations

auto_reboot: true cluster_identifier: ae7bdeef-897e-4c5b-abef-20234abf21bf source_type: offline source_rpms_dir: "/root/install/target_rpms" source_dependencies_rpms_dir: "/root/install/deps" dest_dir: "/tmp/rpms
  • source_type: Set to "offline" for offline installations. The default is "online".

  • source_etcd_binary, source_rpms_dir, source_dependencies_rpms_dir: Direct paths on the Ansible host to the etcd binary, RPMs, and dependencies directories. Paths must be relative to where the Ansible command is run (baremetal host or inside Docker).

  • dest_dir: Where RPM packages are stored on the Lightbits node during installation.

Note

Online installations download packages from the baseurl, while offline installs have packages sent by Ansible and are manually installed with the local yum install.

Single IP Dual NUMA Configuration

Note

To use ansible_ssh_private_key_file instead of ansible_ssh_pass, see auto$.

The Single IP dual NUMA configuration (see Single-IP-Dual-NUMA Configuration), requires changes to the all.yml file. Follow the directions in that section to properly configure this setup.