As a machine learning engineer, I was interested in learning more about kubernetes (k8s). While platforms like AWS, Google Cloud Platform, Azure, I wanted to try to set it up myself at home, keeping the costs fixed as well (only paying to own the hardware, and tiny power to run)

I saw a lot of guides on how to set up k8s on raspberry pis, and for my setup I followed mainly this k8s guide, with some modifications of my own, which I’ve documented below

Hardware

• 4 Raspberry pi 4s (4GB RAM each): ~60 GBP X 4
• Ethernet cables (I made them shorter with an ethernet cable cutter with modular plugs)
• TP-Link TL-SG108 Gigabit Ethernet Switch: ~20 GBP
• Raspberry pi stack with fans: ~20 GBP
• Short USB-C cables: < 10 GBP
• Micro SD cards 8GB each or more per card
• USB charger hub. Although raspberry pi 4s need at least 2.5A to run (as recommended), most USB hubs on the market didn’t seem to offer that. The one here only offered 2.4 A per slot (6 in total), but it seemed enough to power the 4 raspberry pis (plus the fans that came with the case)

Setting up the raspberry pis

• Get Raspian OS
  • The lite version worked well running headless, i.e. without a graphical user interface
  • This takes up less space, and should use less power
• Flash Rasbian OS on SD cards using etcher
• Enable SSH (to run headless!) with touch /<SD_card_path>/boot/ssh (creates an empty file in the boot directory)
• The following setup should be done one pi at a time, as the name of the pis are the same by default, so it is easier to keep track of wat settings were applied to which pi
  • Login from another computer terminal on the same home network with ssh pi@raspberrypi.local with password raspberry
  • Thanks to kaspernissen, author of the k8s guide, scripts were created to make the installation easy!
  • Copy and create a hostname_and_ip.sh file and run it this way:
    • For the master node, static IP to be assigned (e.g. 192.168.1.100), IP of the router (e.g. 192.168.1.1), run as follow: sh hostname_and_ip.sh k8s-master 192.168.1.100 192.168.1.1
    • For all other worker nodes: sh hostname_and_ip.sh k8s-worker-01 192.168.1.101 192.168.1.1 sh hostname_and_ip.sh k8s-worker-02 192.168.1.102 192.168.1.1
      • and so on…
    • This sets the hostname and static IP of each pi
  • Reboot the pi and ssh with the new assigned address, e.g. ssh pi@k8s-master.local or ssh pi@k8s-worker-01.local etc.
  • Remove the entry for raspberrypi.local in ~/.ssh/known_hosts. If not, it will complain when ssh-ing to the next pi that has a different ssh key but same default name!
  • Copy and create a install.sh (again by kaspernissen) and run it with sh install.sh
    • This installs Docker and kubeadm
    • Originally this script disables swap as well, but I found that it didn’t work for me (k8s was complaining later). So the swap should be disabled separately, as advised on this post: sudo update-rc.d dphys-swapfile remove
      • Set CONF_SWAPSIZE=0 in /etc/dphys-swapfile
  • (Optional) Disable wifi
    • Add dtoverlay=pi3-disable-wifi to /boot/config.txt to disable wifi.
  • (Optional) Slightly improve security by changing the default password of the pis to something else with sudo passwd pi
    

Installing Kubernetes

• On the master pi, run sudo kubeadm init

  • If all goes well, it should show a message like this (Note: MASTER_STATIC_IP, TOKEN, and HASH should be real details):

    Your Kubernetes control-plane has initialized successfully!
      
    To start using your cluster, you need to run the following as a regular user:
      
    mkdir -p $HOME/.kube
    sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
    sudo chown $(id -u):$(id -g) $HOME/.kube/config
      
    You should now deploy a pod network to the cluster.
    Run "kubectl apply -f [podnetwork].yaml" with one of the options listed at:
    https://kubernetes.io/docs/concepts/cluster-administration/addons/
      
    Then you can join any number of worker nodes by running the following on each as root:
      
    kubeadm join MASTER_STATIC_IP:6443 --token TOKEN \
      --discovery-token-ca-cert-hash HASH

• Follow the instructions above (running on master):

  mkdir -p $HOME/.kube
  sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
  sudo chown $(id -u):$(id -g) $HOME/.kube/config

• On other worker nodes, run sudo kubeadm join --token TOKEN MASTER_STATIC_IP:6443 --discovery-token-ca-cert-hash HASH

• Set up a weave container network:

  • Get kubectl version by running kubectl version
  • kubectl apply -f https://cloud.weave.works/k8s/net?k8s-version=VERSION (e.g. VERSION = 1.15.3)
  • This applies a yaml file from cloud.weave.works

Next steps

• The installation at this point should be more or less complete, which can be verified by running kubectl get nodes to check what is connected in the k8s cluster
• Now its playtime, which I plan to upload more articles on how I will be using my home kubernetes cluster!