Troubleshooting

Ansible Role Errors

Confirm that the duroslight ports are synchronized in the Ansible default yml file, which can be overridden in inventory ymls, and the node-manager configuration Ansible default yml:

~/light-app/ansible/roles/install-lightos/defaults/main.yml

SSH Strict Key Errors When Using sshpass

If you use the sshpass utility method in your hosts file, you can receive an error related to SSH keys in the Known Hosts file, such as:

$ ansible -i ansible/inventories/cluster_example/hosts all -m ping                                                                                                                           

node02 | FAILED! => {

    "msg": "Using a SSH password instead of a key is not possible because Host Key checking is enabled and sshpass does not support this.  Please add this host's fingerprint to your known_hosts file to manage this host."

}

To avoid this error, you need to disable *_StrictHostKeyChecking *_in the /etc/ssh/ssh_config, or log into each node from your installation workstation at least once.

By default, *_StrictHostKeyChecking *_is enabled in the file /etc/ssh/ssh_config. You can disable this by un-remarking it in ssh_config and setting it to:

StrictHostKeyChecking no

Or, you can leave *StrictHostKeyChecking * enabled and log into each node from the installation workstation and “answer yes” to permanently add the host to the Known Hosts files.

The first time you SSH from one server to another the following SSH exchange occurs:

$ ssh root@192.168.16.22

The authenticity of host '192.168.16.22 (192.168.16.22)' can't be established.

ECDSA key fingerprint is SHA256:zouTZEZF2oUXfIGpnvWutrOR4/fBnd5ARqXNJj0iqD0.

ECDSA key fingerprint is

MD5:7d:0f:0a:3f:27:08:2e:66:93:ae:f5:08:c8:13:23:af.

Are you sure you want to continue connecting (yes/no)? Yes

Warning: Permanently added '192.168.16.22' (ECDSA) to the list of known hosts.

root@192.168.16.22's password: 

Last login: Wed Nov 13 19:06:13 2019 from cluster-manager

[root@node00 ~]#

So, by logging into all the servers at least once from your installation workstation before you run the Ansible playbook, there will be no issues using the sshpass method.

Free Space in Linux OS for etcd Logical Volume Manager Use

If your Linux operating system has volume groups that were created for the home, root, and swap file systems and are utilizing 100% of the storage, you must reduce one of these volume groups. The Lightbits installer requires at least 10GB of space to create an LVM for use with etcd.

For example, review the Linux OS logical volumes. The Linux Virtual Server (LVS) software is used in this example.

$ lvs

  LV   VG                Attr       LSize   Pool Origin Data%  Meta%  Move Log Cpy%Sync Convert

  home Alma_lightos-c3 -wi-ao---- <64.24g                                                    

  root Alma_lightos-c3 -wi-ao----  50.00g                                                    

  swap Alma_lightos-c3 -wi-ao----   4.00g

$ lvs

  LV   VG                Attr       LSize   Pool Origin Data%  Meta%  Move Log Cpy%Sync Convert

  home Alma_lightos-c3 -wi-ao---- <64.24g                                                    

  root Alma_lightos-c3 -wi-ao----  50.00g                                                    

  swap Alma_lightos-c3 -wi-ao----   4.00g

In this example, the LinuxOS was installed onto a 118 GB drive and the entire amount is allocated. You can resize the home LVM by 20 GB to free up some space.

To resize this file system, you need to:

1. Move any files you have in the /home file system to a safe location.
2. Unmount, resize, and recreate the file system.
3. Remount the file system.

To identify how much space is available to free up, use lsblk as follows:

$ lsblk

NAME            MAJ:MIN RM   SIZE RO TYPE MOUNTPOINT

sda               8:0    0 119.2G  0 disk 

|-sda1            8:1    0     1G  0 part /boot

|-sda2            8:2    0 118.2G  0 part 

  |-Alma-root 253:0    0    50G  0 lvm  /

  |-Alma-swap 253:1    0     4G  0 lvm  [SWAP]

  |-Alma-home 253:2    0  64.2G  0 lvm  /home

In this example, the LinuxOS is installed on device “sda” and on partition sda1 with 119.2 GB of space available. It is possible to take 20 GB away from home to free up some space and still have over 44 GB remaining.

1. Mount and record the current mount path for home. $ mount /dev/mapper/Alma_lightos--c3-home on /home type xfs (rw,relatime,attr2,inode64,noquota)
2. Unmount home and then resize it. $ umount /home $ lvresize -L -20G Alma_lightos-c3/home
3. Remake the home file system. $ mkfs.xfs -f /dev/mapper/Alma_lightos--c3-home
4. Remount home. $ mount /dev/mapper/Alma_lightos--c3-home /home