This repository holds the code for the tutorials for creating a Raspberry Pi Cluster in Python.
Here I go through the basics of creating a simple logger to a distributed task assigned system.
Each folder relates to a new topic of creating a Raspberry Pi Cluster. If you are familiar with python the first few topics should be short topics to revise. Later we will get onto more complicated topics such as reaching a consensus of who must "lead" the cluster.
Questions comments and suggestions can be raised the specific blog post or by using issues here.
-
Logging Liveness Full tutorial - To start with we are going to make a really simple file to start logging data. This will set up a python logger and log data to it every minute.
-
Packaging common functionality Full tutorial - Here I introduce python packages and the reason why we use them. I move some of the common functionality into a package so more easily use across scripts.
-
Basic node communication between two nodes Full tutorial - Here I introduce sockets as a simple way of sending data between two concurrently running scripts.
-
Configuration files Full tutorial - Here I introduce configuration files and why we use them. I change the previous script to use configuration files.
-
Sending data with JSON Full tutorial - Here I improve I the cluster sends data around by creating a JSON based message format. The implementation is abstracted into a module to allow easy modification without changing scripts that use it.
-
Sending Slave Details to the Master Full tutorial - This tutorial focuses on modifying the original data sending functions to use a more generic format. This will allow the master to read what messages are being received and determine what action it needs to take for each message type.
-
Sending data to the Slave Full tutorial - This tutorial focuses on improving the slave by being able to request data from the master.
-
Slave Helper Functions Full tutorial - This tutorial focuses on creating a number of slave helper functions to begin the process of fully automating the slaves.
-
Multi Slave Master Full tutorial - This tutorial focuses on changing the master so that it accepts multiple slaves connecting to it.
- Socket talking - Master Broadcast
- Socket talking - Replay attacks
- Socket talking - Encryption
- Config loading - JSON
- Daemon Process -
- Master running simple webserver to offer files
- Slave /Master version/updates
- Periodic updates of the system
- Storing state of the cluster node
- Report Health metrics back (uptime, updates pending, time correctness, disk space)
- Rejoining the cluster after a master restart
- Smart slave restart
- Restarting slave once rebooted
- Keeping track of the running code revision
- Why keeping track of the time is important
- Slave git stash, pull, stash pop, resolving.