Setup Multi Node Kubernetes Cluster
- Working on a containerized storage for containers which is orchestrated by Kubernetes, most of our tasks, be it development, testing and demo require us to setup and modify the nodes in Kubernetes cluster. In addition, a multi-node cluster is a must as we go beyond the initial development and testing, to explore the high availability, scale, performance and upgrade aspects.
- While minikube and minishift provide an easy way to setup kubernetes single node cluster — for multi-node cluster the fastest ways to get going are usually cloud or hosted solutions. kubeadm is the closest we can get to easily setup a cluster.
- For creating a cluster we need to run multiple OS/Nodes. Here, we have four ways , we can run the OS on the top of Laptop, Virtual Machine, AWS EC2 Service, and Server.
- Here I am using a Oracle VirtuaBox and on top of it I’ll run the RHEL8 Linux OS, one master node and two slaves to create a entire cluster(nodes varies upon the uses cases and can be added accordingly). We'll start with installing the redhat8 cli on top of virtualbox.
- Create a new virtual machine in VM.
2. Provide atleast 2gb ram to the OS.
3. Click on next and follow up the default installation settings till create.
4. Settings >Storage > Select the iso file from the location.
5. Select start and begin the installlation (Select ‘minimal install’ in software selection).
6. After installation complete. Do the following changes :
- put the hard-disk priority on top.
- Select minimum two CPU’s.
Now we are good to go.
NOTE: We need some programs to run on the OS, like “Container-Engine”, here we are going to use “docker-ce”, kubelet, and some other prerequisites . These programs are required in master node as well as slave node, so to save our time we will setup all the programs in one OS and then clone the others with the similar settings.
- Now mount the iso file or dvd to one folder.
- Setup yum to install required softwares.
- Install net-tools, vim, and httpd for future preferences.
yum install net-tools vim httpd
- Install Docker in your RHEL8 Version.
yum install docker-ce — nobest (put two hyphen before nobest).
- Disable the firewall security in redhat OS to allow the incoming ports.
systemctl stop firewalld
systemctl disable firewalld
- Make the dvd mount permanent so that after reboot no need to mount again and again.
- Install kubectl, kubeadm, kubelet using yum.
- For working of these software’s properly SELinux Security must be stopped or disabled.
- Now start the docker services to start the container engine.
systemctl start docker
systemctl enable docker
Note: We need to change the CGroup Driver to “systemd” in redhat 8 to manage everything properly.
- We can check now if docker is working properly by using “docker info” command and also check the cgroup driver is changed or not.
- We also need to disable the swap services to run the Kubernetes perfectly on Linux OS.
- Install the “iproute-tc” software on linux that manages the traffic control.
- Linux Nodes iptables to see brigged traffic correctly its configuration must be set to 1. Follow the below snap :
- Start and enable the kubelet services.
systemctl start kubelet
systemctl enable kubelet
- Our setup part is now almost complete, for creating multiple slaves and master nodes we'll clone the above OS and create the new nodes.
- Set the host name of each node so that in future we can connect or ping with the hostnames (as ip is not always same) .
- To ping the nodes with the hostname the details must be updates in the “/etc/hosts” file or in other words DNS server must be created.
- Now initialize the “Kubeadm” to the master node to start the Kubernetes in Linux Server.
- After Kubeadm init properly, to connect the services we need to apply the following commands.
- To join the worker nodes we need to run the following in each node as root.
- We need to provide one extra network layer to connect between the pods to create a multi node cluster.
Here I use “Flannel” Overlay Network Provider.
- After all the steps done successfully , we can see that our multi node cluster is created successfully.
- In my case i use windows as my client source and there i have minikube and kubectl services installed on top of virtualbox , now i will connect to my multi node cluster using the config file on windows cmd.
One Short Example to show the use-case of Multi-Node Kubernetes Cluster:
- 3 pods will be created in different slaves randomly and the ip will be provided within the range that we mentioned before.
We can verify where the pods are running by manually checking the nodes.
- Also we can Expose the Kubernetes services to the client source.
- It provides a random port number through which we can access any of the pods service by using the node/s IP address followed by the port number provided.
Thank You !
For any query feel free to contact me anytime on my LinkedIn Profile :