RPI Kubernetes

A quick overview of how to build a Raspberry Pi Cluster using Linux.

Bill of materials

• Raspberry Pi (Minimum x 2) - recommend RPi v4
• MicroSDHC Card (Minimum capacity 8GB)
• USB Charging Cable (Same number as RPi devices)
• Machine to create an RPi Disk Image

Which disk image?

The 64bit Ubuntu image is currently my preferred choice as it just works! Raspberry Buster image comes in multiple flavors.

• If you dont need the Kubernetes dashboard, use the Lite version.
• If you need the Kubernetes dashboard, use the Desktop version.

1. Download either image file to your machine

Note: the location where the file has being downloaded

Create the Disk Image

Use a utility such as Etcher or ChromeOS recovery to write the memory card.

1. Insert the memory card into your host machine and confirm it is recognised.
2. Take note of how the memory card is labelled as we will use this information later
3. Locate the downloaded file on your host.
4. Extract the file to an image file
5. Right click on the downloaded image to show the
6. Open with Disk Image Writer option.
7. In the image writer, select the memory card inserted to be used to write the image. Once the image has been written to the memory card there will be two volumes available:

8. Select the BOOT from the file manager:

• Click on the boot folder to mount the image.

• Then right click in the folder window and select the “open in terminal” option.

• In the Terminal window for the boot folder add a SSH file to the root of the boot folder

touch ssh

The above command creates a SSH file in the boot folder and ensures the SSH service is started on boot up

10. Close the open terminal window

11. Select the rootfs folder from the file manager:

• Click on the rootfs folder to mount the image.

• Then right click on the folder window and select the “open in terminal” option

• In the Terminal window for the rootfs folder. Enter the config directory

cd etc

12. Edit the hostname file (use whatever editor you like)

sudo vi hostname

13. Change the hostname from the default raspberrypi

e.g. k8s-master-01 k8s-worker-01

14. Edit the hosts file (use whatever editor you like) e.g.

sudo vi hosts

15. Amend the host file to use the updated hostname used above e.g.

16. You can now eject the rpi filesystem (i.e. boot & rootfs) mounts using the file manager

Configuring the RPi

1. Connect the devices via ethernet cables and power them on They should now be accessible via DHCP - check router

Note: Optional - reserve an IP based on the mac address of the RPi

2. To connect to the devices, use an SSH connection from a device on the same network

ssh pi@[device ip]

Note: The default password for RPI is “raspberry”

3. Update the device ( do this for each device)

sudo apt update && sudo apt upgrade

4. Edit the /boot/cmdline.txt and append the following to the end of the line

cgroup_enable=cpuset cgroup_memory=1 cgroup_enable=memory

5. Amend the swap setting to off

sudo dphys-swapfile swapoff

6. Uninstall the swapfile command

sudo dphys-swapfile uninstall -y && sudo apt purge -y dphys-swapfile

7. Reboot the device

sudo reboot

Installing Docker on the RPi

1. Install Docker

curl -sSL get.docker.com | sh

2. Add the pi user to the Docker group

sudo usermod -aG docker pi

Press ctrl-d to disconnect from the RPi

3. Update the docker daemon - edit /etc/docker/daemon.json, add the following:

{
  "exec-opts": ["native.cgroupdriver=systemd"],
  "log-driver": "json-file",
  "log-opts": {
      "max-size": "100m"
  },
  "storage-driver": "overlay2"
}

4. Reboot the device

sudo reboot

Configure Kubernetes on the RPi

1. Add the required packages

sudo apt update && sudo apt install -y apt-transport-https curl

2. Add the GPG key

curl -s https://packages.cloud.google.com/apt/doc/apt-key.gpg | sudo apt-key add -

3. Create the Kubernetes package list - /etc/apt/sources.list.d/kubernetes.list

cat <<EOF | sudo tee /etc/apt/sources.list.d/kubernetes.list
deb https://apt.kubernetes.io/ kubernetes-xenial main
EOF

4. Update the device packages

sudo apt update

5. Install the Kubernetes specific packages

sudo apt-get install -y kubelet kubeadm kubectl

6. Optional: Pin the packages

sudo apt-mark hold kubelet kubeadm kubectl

Configure a kubernetes image

7. On the master node - Pull the images

kubeadm config images pull

The following images will be pulled to the device:

• [config/images] Pulled k8s.gcr.io/kube-apiserver:v1.18.3
• [config/images] Pulled k8s.gcr.io/kube-controller-manager:v1.18.3
• [config/images] Pulled k8s.gcr.io/kube-scheduler:v1.18.3
• [config/images] Pulled k8s.gcr.io/kube-proxy:v1.18.3
• [config/images] Pulled k8s.gcr.io/pause:3.2
• [config/images] Pulled k8s.gcr.io/etcd:3.4.3-0
• [config/images] Pulled k8s.gcr.io/coredns:1.6.

Initialise Kubernetes Master

1. On the master node i.e. k8s-master-01

sudo kubeadm init --pod-network-cidr=10.17.0.0/16 --service-cidr=10.18.0.0/24 --service-dns-domain=mydomain.com

Capture the Example join command from the above command: e.g.

kubeadm join 192.168.86.183:6443 --token 9hj1ar.zko79d0pldz3x1x6 \
    --discovery-token-ca-cert-hash sha256:47cb858687610945e8b781eaf2daa361d464eb1f857ea9ca0cab927770f433e4

2. On the worker node use the join command shown on the master node

sudo kubeadm join 192.168.86.180:6443 --token 14cdv5.5m2npm147k0kqtuz \
    --discovery-token-ca-cert-hash sha256:c1640d15af659b9ed4d658b927f7358dd8e6e78a9bd0e3bc44c5caf95e159791

3. On the master node kubectl will not be working at this stage and return an error as we have a bit more configuration to do. Try running kubectl get nodes will result in the following error:

The connection to the server localhost:8080 was refused - did you specify the right host or port?

4. Make a directory for kubernetes configurations

mkdir -p $HOME/.kube

5. Copy across a configuration

sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config

6. Amend the owner permissions for the directory e.g. current USER/GROUP

sudo chown $(id -u):$(id -g) $HOME/.kube/config

7. At this point the nodes will be available but in a NotReady state. They lack a network overlay.

NAME            STATUS     ROLES    AGE     VERSION
k8s-master-01   NotReady   master   10m     v1.19.0
k8s-worker-01   NotReady   <none>   6m59s   v1.19.0

Configure a network overlay on RPI

1. On the Master Node apply a Flannel Pod network

kubectl apply -f https://raw.githubusercontent.com/coreos/flannel/master/Documentation/kube-flannel.yml

2. Confirm the Pod network is running

kubectl get pods --all-namespaces

3. Check the available nodes

kubectl get nodes

4. At this point the Cluster should be up and running as per below:

NAME            STATUS   ROLES    AGE   VERSION
k8s-master-01   Ready    master   15m   v1.19.0
k8s-worker-01   Ready    <none>   12m   v1.19.0