Host Configuration File Examples

This section covers various configuration examples and how they affect the host configuration files. The first host configuration file will be shown server00.yml, and the host configuration files for the remaining servers will follow the same pattern. For more complex examples, two host configuration files will be shown to help better understand the configuration.

Example 1: Data Network Interface Manually Configured

Host configuration with no data interfaces (data_ifaces) provided. The playbook does not need to configure the IP on the data interface. The user configured the interfaces prior to running the playbook.

YAML
    
 
---nodes:- instanceID: 0  data_ip: 10.10.10.100  failure_domains:  - server00  ec_enabled: true  lightfieldMode: SW_LF  storageDeviceLayout:    initialDeviceCount: 6    maxDeviceCount: 12    allowCrossNumaDevices: false    deviceMatchers:#    - model =~ ".*"      - partition == false      - size >= gib(300)
Copy

Example 2: Data Network Interface Automatically Configured

Host configuration with a single data interface. The playbook configured the interface. The user did not configure interface with the data IP.

YAML
    
 
---data_ifaces:- bootproto: static  conn_name: ens1  ifname: ens1  ip4: 10.10.10.100/24nodes:- instanceID: 0  data_ip: 10.10.10.100  failure_domains:  - server00  ec_enabled: true  lightfieldMode: SW_LF  storageDeviceLayout:    initialDeviceCount: 6    maxDeviceCount: 12    allowCrossNumaDevices: false    deviceMatchers:#    - model =~ ".*"      - partition == false      - size >= gib(300)
Copy

The example above shows the format for setting IPv4 addresses using ip4: ip/subnet. IPv6 addresses can be set using ip6: ip/prefix.

Example 3: Override the Lightbits Configurations

Host configuration with Lightbits override. The provided value overrides the key listen-client-urls.

YAML
    
 
---nodes:- instanceID: 0  data_ip: 10.10.10.100  failure_domains:  - server00  ec_enabled: true  lightfieldMode: SW_LF  storageDeviceLayout:    initialDeviceCount: 6    maxDeviceCount: 12    allowCrossNumaDevices: false    deviceMatchers:#    - model =~ ".*"      - partition == false      - size >= gib(300)etcd_settings_user:  listen-client-urls: http://127.0.0.1:2379,http://10.16.173.14:2379,http://192.168.16.219:2379
Copy

Example 4: Provide Custom Datapath Configuration

Host configuration with custom datapath configuration provided.

By default, the playbook inspects the remote machine and determines the directory containing the specific configuration for Duroslight and backend services (datapath configuration). The excluding node-manager configuration uses the following logic:

Bash
    
 
<system_vendor>-<processor_count>-processor-<processor_cores>-cores
Copy

YAML
    
 
---nodes:- instanceID: 0  data_ip: 10.10.10.100  failure_domains:  - server00  ec_enabled: true  lightfieldMode: SW_LF  storageDeviceLayout:    initialDeviceCount: 6    maxDeviceCount: 12    allowCrossNumaDevices: false    deviceMatchers:#    - model =~ ".*"      - partition == false      - size >= gib(300)datapath_config: custom-datapath-config
Copy

Example 5: Use the Linux Volume Manager (LVM) Partition for etcd Data

Host configuration with custom lvm partition for etcd data.

YAML
    
 
---nodes:- instanceID: 0  data_ip: 10.10.10.100  failure_domains:  - server00  ec_enabled: true  lightfieldMode: SW_LF  storageDeviceLayout:    initialDeviceCount: 6    maxDeviceCount: 12    allowCrossNumaDevices: false    deviceMatchers:#    - model =~ ".*"      - partition == false      - size >= gib(300)use_lvm_for_etcd: trueetcd_lv_name: etcd#etcd_settings_user:etcd_lv_size: 15GiBetcd_vg_name: Alma
Copy

Example 6: Profile-Generator Overrides

Enable humans to override profile-generator output and provide for each server a custom file that will be taken by profile-generator as the system-profile.

Each host can be different so each host can specify its own override file.

YAML
    
 
nodes:- instanceID: 0  data_ip: 10.10.10.100  failure_domains:  - server00  ec_enabled: true  storageDeviceLayout:    initialDeviceCount: 6    maxDeviceCount: 12    allowCrossNumaDevices: false    deviceMatchers:    - partition == false    - size >= gib(300)profile_generator_overrides_dir: /tmp/overrides.d/server00
Copy

In case the cluster is homogeneous and we want to apply the same override to all nodes, we can provide a single setting in the groups/all.yml file or via the cmd with:

Bash
    
ansible-playbook -i ansible/inventories/cluster_example/hosts playbooks/deploy-lightos.yml -e profile_generator_overrides_dir=/tmp/overrides.d
Copy

In the above example, we specify profile_generator_overrides_dir, which is a directory on the Ansible Controller that will be copied to the target machine.

Example 7: Dual Instance Configuration

In a dual instance configuration. Lightbits is run from both NUMAs of the CPU.

Below is an example configuration for server00.yml and server01.yml.

In this configuration, both servers NUMA0 and 1 have 12 NVME drives, for a total of 24 SSD drives in the server.

Each server has a single management IP configured (not shown below) and two data IPs (one for each NUMA). Each instance has a unique data IP set for it.

server00.yml

YAML
    
 
---name: server00nodes:-   instanceID: 0    data_ip: 172.16.10.10    failure_domains:    - server00    ec_enabled: true    lightfieldMode: SW_LF    storageDeviceLayout:      initialDeviceCount: 12      maxDeviceCount: 12      allowCrossNumaDevices: false      deviceMatchers:#      - model =~ ".*"        - partition == false        - size >= gib(300)#      - name =~ "nvme0n1"-   instanceID: 1    data_ip: 172.16.20.10    failure_domains:    - server00    ec_enabled: true    lightfieldMode: SW_LF    storageDeviceLayout:      initialDeviceCount: 12      maxDeviceCount: 12      allowCrossNumaDevices: false      deviceMatchers:#      - model =~ ".*"        - partition == false        - size >= gib(300)
Copy

server01.yml

YAML
    
 
---name: server01nodes:-   instanceID: 0    data_ip: 172.16.10.11    failure_domains:    - server01    ec_enabled: true    lightfieldMode: SW_LF    storageDeviceLayout:      initialDeviceCount: 12      maxDeviceCount: 12      allowCrossNumaDevices: false      deviceMatchers:#      - model =~ ".*"        - partition == false        - size >= gib(300)#      - name =~ "nvme0n1"-   instanceID: 1    data_ip: 172.16.20.11    failure_domains:    - server01    ec_enabled: true    lightfieldMode: SW_LF    storageDeviceLayout:      initialDeviceCount: 12      maxDeviceCount: 12      allowCrossNumaDevices: false      deviceMatchers:#      - model =~ ".*"        - partition == false        - size >= gib(300)
Copy

Example 8: Single IP Dual NUMA Configuration

This example is for Single IP Dual Instance configuration. For more information, see Single-IP-Dual-NUMA Configuration.

The below is an example of a full server00.yml, with a similar physical configuration as the above example. However, instead of using two data IPs, it only uses one data IP for both of its instances.

Unlike dual instance configuration from the example above, which had a unique data IP per instance, each instance on a server has the same IP.

YAML
    
 
---name: server00nodes:-   instanceID: 0    data_ip: 10.10.10.100    failure_domains:    - server00    ec_enabled: true    lightfieldMode: SW_LF    storageDeviceLayout:      initialDeviceCount: 12      maxDeviceCount: 12      allowCrossNumaDevices: false      deviceMatchers:#      - model =~ ".*"        - partition == false        - size >= gib(300)#      - name =~ "nvme0n1"-   instanceID: 1    data_ip: 10.10.10.100    failure_domains:    - server00    ec_enabled: true    lightfieldMode: SW_LF    storageDeviceLayout:      initialDeviceCount: 12      maxDeviceCount: 12      allowCrossNumaDevices: false      deviceMatchers:#      - model =~ ".*"        - partition == false        - size >= gib(300)
Copy

Examples for Setting Journal SSD Configurations

The following are examples for setting Journal SSD configurations, as detailed in the Setting the Journal SSD Configuration __article.

Example 1

In this example, the data SSD devices are Micron XTR and the Journal is Kioxia FL6. The model number should be in the matcher so the cluster will know which devices are for data and which are for journaling:

Bash
    
 
name: server00nodes:- instanceID: 0 data_ip: 10.20.155.2 ec_enabled: false failure_domains: - server00 storageDeviceLayout:   allowCrossNumaDevices: true   deviceMatchers:   - model =~ "Micron XTR"   - partition == false   initialDeviceCount: 6   maxDeviceCount: 11 journalDeviceLayout:   deviceMatchers:   - model =~ "Kioxia FL6"   - partition == false   numDevices: 1
Copy

Example 2

The following is an example of a dual-instance node, using the size matcher so the larger devices are used for data and the smaller ones are used for journaling:

Bash
    
 
name: server00nodes:- instanceID: 0 data_ip: 10.20.155.2 ec_enabled: false failure_domains: - server00 storageDeviceLayout:   allowCrossNumaDevices: false   deviceMatchers:   - partition == false   - size > tib(1)   initialDeviceCount: 4   maxDeviceCount: 11 journalDeviceLayout:   deviceMatchers:   - partition == false   - size < gib(800)   - size > gib(400)   numDevices: 2- instanceID: 1 data_ip: 10.20.155.3 ec_enabled: false failure_domains: - server00 storageDeviceLayout:   allowCrossNumaDevices: false   deviceMatchers:   - partition == false   - size > tib(1)   initialDeviceCount: 4   maxDeviceCount: 11 journalDeviceLayout:   deviceMatchers:   - partition == false   - size < gib(800)   - size > gib(400)   numDevices: 2
Copy

Last updated on

Was this page helpful?