Home Server

This is the configuration for my home server running in Kubernetes.

Configuration

Network

All HTTP traffic is routed through Traefik, but TCP and UDP connections are exposed separately. Although additional entrypoints could be configured to proxy these connections, it adds no benefit other than load balancing due to the lack of host or path matching.

HTTPS traffic is secured behind Google forward authentication by default. Some exceptions are made for applications that don't work behind an additional layer of authentication (e.g., Plex and Home Assistant).

Load balancing is acheived using MetalLB for all services running inside of Kubernetes.

Filesystem

Persistent storage is achieved using NFS and Longhorn volumes.

Sealed Secrets

Sealed Secrets are stored securely alongside the rest of the server configuration in source control. They are decrypted by a controller running in the cluster.

To create new a sealed secret, follow these steps.

Base64 encode the secret value.
```
echo -n "secret" | base64 -w 0
```

Create a native Secret resource with the desired key name and encoded value

apiVersion: v1
kind: Secret
metadata:
  name: my-secret
  namespace: server
type: Opaque
data:
  MY_SECRET_KEY: c2VjcmV0

Use the kubeseal cli tool to generate a Sealed Secret. Be sure to specify the correct namespace, or else the controller won't be able to decrypt the secret.
```
kubeseal <secret.yaml >sealed-secret.yaml --controller-namespace=sealed-secrets -o yaml
```

Setup

Provision Nodes

To configure a node from scratch, first complete the following prerequisites:

Install Ubuntu 22.04
Create the user mchill

Enable ssh

sudo apt install openssh-server
sudo systemctl enable --now ssh

Add public key to ~/.ssh/authorized_keys

After following the above steps for each node, they can be provisioned with an ansible playbook.

ansible-playbook playbooks/configure-nodes.yaml -i inventory.yaml

Create/Reset Cluster

If the cluster ever needs to be completely reset, this can be done with an ansible playbook.

ansible-playbook playbooks/reset-cluster.yaml -i inventory.yaml --extra-vars "K3S_TOKEN=$K3S_TOKEN GITHUB_TOKEN=$GITHUB_TOKEN"

Edit /etc/systemd/system/k3s.service and add the required arguments to ExecStart.

ExecStart=/usr/local/bin/k3s server '--kubelet-arg=allowed-unsafe-sysctls=net.*'

Deploy Workloads

Applications are automatically deployed by CI. Manual deployment can be done with the following commands.

pushd k8s/infrastructure && ./apply.sh && popd
pushd k8s/ingresses && ./build.sh | kubectl apply --server-side --force-conflicts -f - && popd
pushd k8s/applications && ./build.sh | kubectl apply --server-side --force-conflicts -f - && popd

mchill/home