/Raspberry_Bush

DevOps + Containers + Pi = Raspberry Bush

Primary LanguageShell

Last Updated: 11/04/2020 12:35pm

Built For Raspberry Pi (3B+/4B)

Software Builds:

  • k3s (Updated 9/11/2020)
    • k3s Service 1.18.8+k3s1
    • kernel 5.4.51-v71+
    • containerd 1.3.3-k3s2
  • k8s (Updated Happening In Jan 2021)
    • Raspbian Buster Lite (Kernel 5.4) (Release Date Used: 08-20-2020)
    • Custom CPU Governor Plugin 1.0 (Now Built It Script)
    • Kubernetes 1.16.15
    • Docker-CE 19.03.4
    • Flannel 0.11.0
    • WebUI 2.0.0 Beta5

Overview

This is a project to help you create a Raspberry Pi cluster and demo DevOps practices from build/release to Chaos Engineering on it. According to a few friends, it's been deemed a Raspberry Bush. Credit for the name goes to Antony Zimzores. Check out the companion blog article here.

Cost

  • Raspberry Pi 3B+ based cluster will cost around $850 for an 8 node cluster with the following specs: 28 Cores @ 1.4GHz (39.2GHz Total) / 7GB RAM / 896GB Storage (This Assumes a Master Node and 6 Worker Nodes). Network is limited to 300MB max throughput as it's bound to the same bus as the USB.

  • Raspberry Pi 4B (8GB) based cluster will cost ~$1000 for an 8 node cluster with the following specs: 28 Cores @ 1.5GHz (42GHz Total) / 56GB RAM / 896GB Storage (This Assumes a Master Node and 6 Worker Nodes). If you need a bit more power, this build will also provide the benefit of a full 1GB network.

Build Materials

There are two ways you can build this, I chose to go the Power over Ethernet (PoE) route as the switch I chose provided me with and additional two 1GB uplink ports (my chosen switch) and gave me the ability to consolidate the need for power into a single connection to the Pi. When I did the cost analysis, the additional price of the PoE Hats + PoE Switch was around $225 difference and gave me a cleaner look, less cables, 8 less power supplies, and 2 additional network ports. Feel free to use whatever switch you like as long as it doesnt exceed the INTERIOR dimensions of the case your using. Luckily I pulled spec sheets on all switches considered and found a fair compromise of what I wanted and kept things tidy and simple. I highly recommend the SanDisk Extreme Plus models as the performance is great and has the best perf/dollar ratio.

Single Node Builds

Minimum

Recommended

Cluster Builds

Pi3B+ Minimum

Recommended

Bill of Materials

Pi 3B+ Cluster

Pi 4B Cluster

Prerequisite Software

Image SD Card

  • Download the Raspbian Lite Image and and Extract the ISO
  • Flash To SD Card With Etcher
  • Note: After Flashing With Etcher Windows On First Boot Your Pi May Reboot One Extra Time To Resize Partitions (This is Normal)

Desktop Node Setup

Desktop Node will be the standalone workstation (in your cluster) from which everything is ran and learned. This will also be where we execute our DevOps and Chaos engineering consoles from as well. We'll also install all our tools on here as well. Consider it the desktop machine you would work from and where you'll install all your tools you'll learn from will reside. Thinking Ubuntu Mate Maybe??? To be continued...

Linux Desktop Node (Coming soon...)

Pretty standard stuff here...

  • Download the Raspberry Pi Desktop Image and Flash to SD with Etcher
  • Install your tools by running the command: sudo sh installtools.sh (Coming soon...)

Windows Desktop Node (Raspberry Pi 4B Only - Reccommend 8GB Version)

For the Windows Desktop node there is a niche group out there thats super passionate about Windows on RPI, here's how to get started:

  • Go to the WoRP Project Page and download the WoRP Imager
  • Sign up for the WoRP Discord channel and follow the instructions to get access to the WoRP Image
  • Run the downloaded WoRP tool and follow the instructions
  • Go get a snack, the instal will take ~35 minutes
  • Load up the SD card in your RPi and boot
  • Initial boot will take ~7 minutes and will auto-reboot (It will look like it's froze for the first few minutes, just leave it)
  • Second boot will take ~4 minutes and will take you to the OOBE screen, answer the questions and you're all set.
  • Install WOR_Control_Panel (Found In Discord Channel)

Raspbian Kubernetes Node (Master / Worker) Setup

These machines will house the Kubernetes cluster(s), both master and worker nodes, for use as a container platform to deploy our DevOps solutions and test projects.

  • First Login To the Raspberry Pi For The First Time With The Credentials: pi/raspberry
  • Change Your Root Password: passwd
  • (Optional) Configure Your Time Zone: tzselect
  • (Optional) Set Your Wifi Country: sudo iw reg set [Two Letter Country Code]
  • (Optional) Turn on SSH: sudo touch /boot/ssh && sudo service ssh start
  • Download the setup package by running the following command:
  • Run the pi config script by running the following command:
    • sudo sh setup.sh [Version (k3s/k8s)] [Node Type (Master/Worker)] [Hostname] [Desired IP] [Desired Gateway IP] [Desired DNS Server IP]
    • Example K8s Master Build: sudo sh setup.sh k8s Master kube-master 192.168.1.2 192.168.1.1 192.168.1.1
    • This process will take about 15 seconds.
  • After reboot, run the node setup script by running the following command:
    • sudo sh build_kub_node.sh
    • This process will take between 1-15 minutes (will vary based on sd card speed).

k3s Additional Container Installs (Will Customize Later)

  • Reference Install Guide
    • Helm: Package manager for Kubernetes
    • MetalLB: Load-balancer implementation for bare metal Kubernetes clusters
    • Nginx: Kubernetes Ingress Proxy
    • Cert Manager: Native Kubernetes certificate management controller.
    • Kubernetes Dashboard: A web-based Kubernetes user interface

k8s Additional Installs (Will Update In Jan 2021)

Coming Soon...

Performance Tests

Raspberry Pi 4B Build Times

  • Setup - 13 seconds
  • k3s Build - ~1 minute
  • k8s Build - 13m 42s

DevOps Tools

MORE COMING SOON.....

What's Next???

For more Pi fun, this time try Pizza Pi, write your first app to deploy and automate pizza orders from Dominos. Get started here: https://github.com/gamagori/pizzapi

Free Follow On Education

K8S Sandbox Learning Labs
Kubernetes Browser Based Training