freeleaps-ops/cluster/ansible/README.md

# Freeleaps Cluster Maintain Ansible

> This folder used to Freeleaps Kubernetes Cluster maintaining.

## Pre-settings for each node

### Enable br_netfilter module for Kernel

We need enable br_netfilter module for each node's kernel.

Using command `modprobe br_netfilter` on each nodes to enable it.

### Enable HugePage on node's kernel

```bash

# Check nodes HugePage params if not 1024
sudo grep HugePages /proc/meminfo

# Add HugePage params into kernel settings
echo 1024 | sudo tee /sys/kernel/mm/hugepages/hugepages-2048kB/nr_hugepages
echo vm.nr_hugepages = 1024 | sudo tee -a /etc/sysctl.conf

```

### Change Kernel Params

You need ensure kernel params (`/etc/sysctl.conf`) like belows:

```text

net.ipv4.ip_forward=1
kernel.keys.root_maxbytes=25000000
kernel.keys.root_maxkeys=1000000
kernel.panic=10
kernel.panic_on_oops=1
vm.overcommit_memory=1
vm.panic_on_oom=0
net.ipv4.ip_local_reserved_ports=30000-32767
net.bridge.bridge-nf-call-iptables=1
net.bridge.bridge-nf-call-arptables=1
net.bridge.bridge-nf-call-ip6tables=1
vm.nr_hugepages = 1024

```

Using `sysctl -p` to make kernel params effect.

## How to use ?

### Cluster Scaling
> First execute `init-cluster.sh` to initialize the environment and execute `source venv/bin/activate` to activate the virtual environment

> Ansible version needs to be between 2.16.4 and 2.17.0
If you want to scale nodes in cluster, you can execute script `upgrade-cluster.sh` after environment initialized.

#### Tips - ANONYMOUS AUTH ENABLING BEFORE SCALE

`kubeadm` using bootstrap token to join newly added nodes into cluster, which means we need **enable anonymous authentication** for `kube-apiserver` temporally.

You need edit `/etc/kubernetes/manifests/kube-apiserver.yaml` on master-01 node and set `--anonymous-auth=true` and you must recovery it to `false` when cluster has been scaled.