How to Setup K3s Kubernetes with IPVS
Posted July 13, 2024
![How to Setup K3s Kubernetes with IPVS](/k3s-kubernetes-with-ipvs/hero.jpg)
IPVS (IP Virtual Server) provides efficient load balancing compared to iptables
. A K3s setup is created to have support for IPVS. This guide teaches you how to install K3s with IPVS and correctly configure it.
Prerequisites
To follow along with this K3s and IPVS guide ensure you have the following:
- A set of nodes (VMs or physical machines) for your K3s cluster.
- Root or sudo access on all nodes.
- Basic understanding of Kubernetes and container orchestration.
K3s and IPVS
K3s is optimized to let your environments use limited resources. Any Kubernetes resource is compatible with K3s. You just need minor twerks to get up and running. IPVS will add:
- Act as a transport layer for load balancing into the Linux kernel.
- It acts as a load-balancing IP layer, to handle network traffic.
- You get good balancing algorithms like round-robin, least connection, and destination hashing.
Configuring IPVS
Once you have updated your dependencies and the curl is ready, you will only add a funnel network to K3s. First, get the dependencies ready:
sudo apt update
sudo apt install -y curl
The first step is to Install K3s on the Master Node. However, you will need to verify kernel modules are loaded:
lsmod | grep br_netfilter
lsmod | grep overlay
If you get no output, use these commands to load the kernel modules manually:
sudo modprobe br_netfilter
sudo modprobe overlay
Rerun the command and check the results:
lsmod | grep br_netfilter
lsmod | grep overlay
Output:
br_netfilter 32768 0
bridge 421888 1 br_netfilter
overlay 212992 0
Go further and check your kernel has IPVS support:
lsmod | grep ip_vs
If again you get no results load the IPVS modules with these commands:
sudo modprobe ip_vs
sudo modprobe ip_vs_rr
sudo modprobe ip_vs_wrr
sudo modprobe ip_vs_sh
Your expected lsmod | grep ip_vs
results will be as such:
$ lsmod | grep ip_vs
ip_vs_sh 12288 0
ip_vs_wrr 12288 0
ip_vs_rr 12288 0
ip_vs 225280 6 ip_vs_rr,ip_vs_sh,ip_vs_wrr
nf_conntrack 200704 6 xt_conntrack,nf_nat,xt_nat,nf_conntrack_netlink,xt_MASQUERADE,ip_vs
nf_defrag_ipv6 24576 2 nf_conntrack,ip_vs
libcrc32c 12288 4 nf_conntrack,nf_nat,nf_tables,ip_vs
Installing K3s with IPVS
Here you will download and install K3s with the IPVS backend as the flannel network.. Use this command:
- Ensure you Uninstall K3s, you have a previos Node:
sudo /usr/local/bin/k3s-uninstall.sh
- Use these commands to get K3s ready with ip_vs:
curl -sfL https://get.k3s.io | sh -s - server --kube-proxy-arg proxy-mode=ipvs
This should install K3s and enable IPVs all at once:
If you want to set the K3s on Worker Nodes use:
curl -sfL https://get.k3s.io | K3S_URL=https://<master-node-ip>:6443 K3S_TOKEN=<node-token> sh -
Replace <master-node-ip>
with the IP address of your master node and <node-token>
with the token from your master node (/var/lib/rancher/k3s/server/node-token
).
If K3s is ready and running with IPVS, verify the kube-proxy configuration:
kubectl get nodes
## Output
mode: "ipvs"
You may get the following error:
WARN[0000] Unable to read /etc/rancher/k3s/k3s.yaml, please start the server with --write-kubeconfig-mode or --write-kubeconfig-group to modify kube config permissions
error: error loading config file "/etc/rancher/k3s/k3s.yaml": open /etc/rancher/k3s/k3s.yaml: permission denied
Reinstall K3s with this command:
curl -sfL https://get.k3s.io | sh -s - server --kube-proxy-arg proxy-mode=ipvs --write-kubeconfig-mode 644
I have created this Solving unable to read etc rancher k3s k3s yaml K3S Error in case you need more details.