A personal project to provide security, privacy, and data-ownership for my home.

Includes:

Docker and various infrastructure concepts (data backups, parity)
Self-Hosting for Privacy, such as: Adblocking, Passwords, Family Photos, Finances, Documents
Self-Hosting for Entertainment, such as: Movies, TV Shows, Music, Board and Video Games
Self-Hosting for Home, such as: Automation, Grocery Shopping
Self-Hosting for Friends, such as: Event Scheduling, Idea Curation, Expense Management
Practical Security: limit exposure surfaces, promote self-healing and proactive measures
Practical Data Backup: easily export the most important files. Ideally, directly to the cloud and detached storage. Deterministic data is not prioritized.

Technology

Infrastructure	Media	Other
Caddy	Jellyfin	Homepage
CrowdSec	Jellyseerr	Tandoor Recipes
DuckDNS	Prowlaar	Gloomhaven Secretariat
Dozzle	Sonarr	Pinry
Scrutiny	Radarr	Rallly
Diun	qBitTorrent	ChangeDetection.io
Endlessh	Gluetun	Paperless-ngx
Uptime Kuma	Podgrab
Pihole
Unbound
Speedtest Tracker

Notes

Watchtower is intentionally avoided based off advice from the Selfhosted.show podcast. The idea is to have full control over the versions of containers (rather than automated updates) to improve reliability.

Setup and Operation

Starting services

Explanation

Services are grouped into similar purposes via "Profiles". There are two primary goals:

Enable one docker-compose file to be useful in a variety of situations. A server with less resources can easily run a limited version without "fluff". Stacks can easily be started/stopped/restarted, which can help make testing or issue resolution faster
Mitigate docker timeouts. As more services are added, it's more likely a monolithic docker-compose would fail to run successfully.
Less primary: I tried to get multiple docker-compose files to work with a shared collection of environment files (eg. ports.env). Unfortunately, Docker really doesn't like that. Symlinks or scripts are an option, but prohibitively complex. Profiles appear to achieve the best combination of (1) "modularity" and (2) easy of env maintenance

Start each service

docker-compose --profile **stack** up -d Alternatively, customize COMPOSE_PROFILES= in the .env file for a more "static" approach

Profile	Services	Note
`external`	docker-socket-proxy, crowdsec, endlessh, caddy, duckdns	Makes connecting to a publicly facing set of services possible, securely
`admin`	docker-socket-proxy, uptime-kuma, homepage	Local system management and status. Non-local access to Uptime Kuma requires `external`
`network`	pihole, unbound, speedtest-tracker	Tools to support your home internet
`monitor`	docker-socket-proxy, dozzle, diun, scrutiny	Monitoring system health
`downloads`	wireguard, qbittorrent, podgrab	Allow for secure file transfers, without additional overhead from library management
`media-request`	jellyseerr, sonarr, radarr, prowlarr, wireguard, qbittorrent, podgrab	Full stack for end user media requests and file transfer. Non-local access to Jellyseerr requires `external`
`recipes`	tandoor_recipes, postres	Home recipes. Non-local access to tandoor requires `external`
`gloomhaven`	gloomhaven-secretary, ghs-server	Board games! Non-local access to client and server requires `external`
`lifestyle`	pinry, changedetection.io, selenium/chrome	Tools for the family
`paperless`	paperless-ngx, postgress, redis	Important Document
`calendar`	rallly, postgres	Help scheduling events with people
`ripping`	automatic-ripping machine	local-only, not required all the time

Beginning

WIP

Become familiar with the Project Structure below. This is your "map" of where to expect different things to be. In turn, this will help you configure your setup correctly
SAMBA: After copying smb.conf, customize it as needed: interfaces, share names, users, passwords, etc.

Project Structure

Recommendations via multiple docker files, Where to Put Docker Compose, TRaSH Guides

File System

├── /opt
│  └── docker
│     └── homelab-docker (this repo)
|        ├── dockerfiles (for custom builds)
│        |  └── builder-*.sh (for building files to upload)
│        |  └── *.dockerfile (for adhoc builds)
|        ├── staticconfig (service-specific configuration)
│        |  └── * (for each service)
│        |     └── *
│        ├── .env
│        └── docker-compose.yml
├── /srv
│  ├── docker (for container's configurations)
│  ├── cache
│  └── logs
└── /mnt/storage
   ├── db
   ├── staticfiles
   │  ├── icons
   │  ├── paperless
   │  ├── tandoor_media
   │  └── wallpaper
   ├── downloads
   │  ├── audiobooks
   │  ├── movies
   │  ├── music
   │  ├── paperless (shared r/w)
   │  ├── podcasts
   │  └── tv
   └── media
      ├── audiobooks
      ├── music
      ├── pictures
      ├── podcasts
      ├── movies
      └── tv

Directories may be created with the following cmds

Please review this script before running it. It is a work in progress and may not run as expected. cd scripts chmod +x start.sh
./start.sh

Recursively own the /data directory

sudo chown -R $USER:$USER /data sudo chmod -R a=,a+rX,u+w,g+w /data

Docker Compose (and needed files)

/opt explanation: reserved for the installation of add-on application software packages

This github repo represents this folder. It's safely committed to public repos and shouldn't contain anything sensitive.

Example files:

./docker-compose.yml
./.env
./dockerfiles/custom-build-for-caddy.dockerfile
./staticonfig/caddy/Caddyfile

Persistent Data and Configuration

/srv

How you configure the apps and their current states. This is separated from the Docker Compose (ie. "setup") as these become specific to how you use the services - not how they're installed/maintained.

Media Staging (such as magnets)

/mnt/storage/downloads

Media Storage (such as Podcasts, Movies, TV Shows, Audiobooks):

/mnt/storage/media/movies /mnt/storage/media/podcasts

This creates a clear distinction between the files many services could use or want and the files those services just need to access. Separation presumably allows for alternate backup or hosting mechanisms, as well. It's an attempt to achieve Least Privilege.

Nesting the media adjacent to downloads is suggested via servarr.com as a way to allow atomic file moves as opposed to a more intensive/longer copy+paste action. Explained here.

Container Roles and Access to Files

Let's recognize four kinds of Media Server roles containers/apps:

A Curator whose role is to enable the discovery and selection of Media by an End User. It then delivers a Work Order to the Acquirer. It also monitors the Media and ensures it is accurately described and and of desired quality.Ex. Sonarr, Radarr, Lidarr, Readarr, Ombi
An Indexer whose role is to interpret how a Work Order should be completed. It figures out where the Media should come from while negotiating with each supplier. Ex. Jackett, Prowlaar
An Acquirer whose role is to accept and execute a Work Order to retrieve a specific Media. It then delivers that Media to the Provider. Ex. qBitTorrent, Transmission
A Provider whose role is to provide Media (movies, podcasts, etc) to end users. It is unconcerned with how the Media came to exist and isn't responsible for its quality or description. Ex. Emby, Plex, Jellyfin

Port Reservations

Ports are controlled through variables (ie. .env) to provide a central "fact check".

Optionals

Caddy and DuckDns.org

If you do not use DuckDns.org and/or use another provider which needs a Caddy DNS module, alter dockerfiles/caddy.dockerfile appropriately

shaunjanssens/homelab-docker