62 lines
1.8 KiB
Markdown
62 lines
1.8 KiB
Markdown
# Freeleaps Cluster Maintain Ansible
|
|
|
|
> This folder used to Freeleaps Kubernetes Cluster maintaining.
|
|
|
|
## Pre-settings for each node
|
|
|
|
### Enable br_netfilter module for Kernel
|
|
|
|
We need enable br_netfilter module for each node's kernel.
|
|
|
|
Using command `modprobe br_netfilter` on each nodes to enable it.
|
|
|
|
### Enable HugePage on node's kernel
|
|
|
|
```bash
|
|
|
|
# Check nodes HugePage params if not 1024
|
|
sudo grep HugePages /proc/meminfo
|
|
|
|
# Add HugePage params into kernel settings
|
|
echo 1024 | sudo tee /sys/kernel/mm/hugepages/hugepages-2048kB/nr_hugepages
|
|
echo vm.nr_hugepages = 1024 | sudo tee -a /etc/sysctl.conf
|
|
|
|
```
|
|
|
|
### Change Kernel Params
|
|
|
|
You need ensure kernel params (`/etc/sysctl.conf`) like belows:
|
|
|
|
```text
|
|
|
|
net.ipv4.ip_forward=1
|
|
kernel.keys.root_maxbytes=25000000
|
|
kernel.keys.root_maxkeys=1000000
|
|
kernel.panic=10
|
|
kernel.panic_on_oops=1
|
|
vm.overcommit_memory=1
|
|
vm.panic_on_oom=0
|
|
net.ipv4.ip_local_reserved_ports=30000-32767
|
|
net.bridge.bridge-nf-call-iptables=1
|
|
net.bridge.bridge-nf-call-arptables=1
|
|
net.bridge.bridge-nf-call-ip6tables=1
|
|
vm.nr_hugepages = 1024
|
|
|
|
```
|
|
|
|
Using `sysctl -p` to make kernel params effect.
|
|
|
|
## How to use ?
|
|
|
|
### Cluster Scaling
|
|
> First execute `init-cluster.sh` to initialize the environment and execute `source venv/bin/activate` to activate the virtual environment
|
|
|
|
> Ansible version needs to be between 2.16.4 and 2.17.0
|
|
If you want to scale nodes in cluster, you can execute script `upgrade-cluster.sh` after environment initialized.
|
|
|
|
#### Tips - ANONYMOUS AUTH ENABLING BEFORE SCALE
|
|
|
|
`kubeadm` using bootstrap token to join newly added nodes into cluster, which means we need **enable anonymous authentication** for `kube-apiserver` temporally.
|
|
|
|
You need edit `/etc/kubernetes/manifests/kube-apiserver.yaml` on master-01 node and set `--anonymous-auth=true` and you must recovery it to `false` when cluster has been scaled.
|