🌔 lunison

Secure, nightly file sync.

USB Hard Drive Encryption
VPN Setup
Unison Sync
Systemctl Scheduling

These are instructions for how I set up a Raspberry Pi with an external hard drive to securely sync files with another system. I also run a separate script that makes nightly web requests for files over a VPN, so I included my settings for the VPN and for setting up the schedule.

USB Hard Drive Encryption

Mostly inspired by USB Hard Drive Encryption on a Raspberry Pi.

Use lsblk to identify the HDD partition. NOTE: this output is after the drive had been configured as an encrypted device.

$ lsblk
NAME        MAJ:MIN RM   SIZE RO TYPE  MOUNTPOINT
sda           8:0    0 931.5G  0 disk
└─sda1        8:1    0 931.5G  0 part
  └─vault   254:0    0 931.5G  0 crypt /media/vault
mmcblk0     179:0    0  29.8G  0 disk
├─mmcblk0p1 179:1    0   2.2G  0 part
├─mmcblk0p2 179:2    0     1K  0 part
├─mmcblk0p5 179:5    0    32M  0 part
├─mmcblk0p6 179:6    0   256M  0 part  /boot
└─mmcblk0p7 179:7    0  27.3G  0 part  /

Format the /dev/<PART> partition with mkfs.ext4, e.g.

sudo mkfs.ext4 /dev/sda1

Install cryptsetup and start the dm-crypt modules. This lets us avoid rebooting later.

sudo apt-get install cryptsetup
sudo modprobe dm-crypt sha256 aes

Use cryptsetup to setup the encrypted partition. The flags:

c: the cipher (aes); (cat /proc/crypto for available ciphers)
s: the keysize in bits (256 bits)
h: the hash used to create the encryption key from the passphrase

sudo cryptsetup --verify-passphrase luksFormat /dev/sda1 -c aes -s 256 -h sha256

Open the encrypted partition, giving it a <NAME> e.g. 'vault'

sudo cryptsetup luksOpen /dev/sda1 vault

Format the encrypted partition.

sudo mkfs -t ext4 -m 1 /dev/mapper/vault

Create a mount point, mount the encrypted partition, and make it available to the non-root user.

sudo mkdir /media/vault
sudo mount /dev/mapper/vault /media/vault/
sudo chown pi:pi /media/vault/

Create a keyfile from urandom.

if: input file (urandom)
of: output file (/root/key/file)
bs: read and write number of bytes (1024 bytes)
count: copy only this many input blocks (4) This creates a 4KB keyfile:

sudo dd if=/dev/urandom of=/root/key/file bs=1024 count=4

Make the keyfile read-only to root

sudo chmod 0400 /root/key/file

Add the keyfile to the encrypted partition

sudo cryptsetup luksAddKey /dev/sda1 /root/key/file

Add an entry to the /etc/crypttab file with the following format

<target name>    <source device>    <key file>    <options>

For example:

vault    /dev/sda1    /root/key/file    luks

Mount the device in fstab -- add an entry to /etc/fstab:

/dev/mapper/vault   /media/vault ext4   defaults,rw       0       0

Once these steps are completed, the USB hard drive should auto-mount on system startup. It will be an encrypted drive, so if it was connected to another system (without the keyfile or password) the contents would not be accessible.

Sources

VPN Setup

Mostly inspired by NordVPN on the Raspberry Pi. This assumes a NordVPN account already. Additional instructions found at the NordVPN site

sudo apt-get update
sudo apt-get upgrade

Install the OpenVPN package, if it's not already installed, and navigate to its directory

sudo apt-get install openvpn
cd /etc/openvpn

Download the OpenVPN config files here:

sudo wget https://downloads.nordcdn.com/configs/archives/servers/ovpn.zip

Extract the files.

sudo unzip ovpn.zip

Go to NordVPN Tools to determine the best server for your requirements.
Start OpenVPN with the config for the selected server.

sudo openvpn [file name]

OpenVPN will ask you for your NordVPN credentials which can be found in the Nord Account dashboard.

The VPN will now be active. You can test with

curl -ks https://ipv4.ipleak.net/json/

Now, we want to setup the VPN to connect on boot.

Store the username and password in an auth.txt file

sudo vim auth.txt

Store the username and password on separate lines,

username
password

Copy the specified server config and simplify the name, e.g.

sudo cp au151.nordvpn.com.tcp443.ovpn au151.conf

Edit the file to instruct it to use the auth.txt file.

sudo vim au151.conf

Replace

auth-user-pass

with

auth-user-pass auth.txt

Reboot the Pi and reconnect via SSH

sudo reboot

The VPN should still be active, check again with

curl -ks https://ipv4.ipleak.net/json/

Now we want to ensure that DNS does not leak.

Edit dhcpcd.conf to set the static DNS to Cloudflare's 1.1.1.1.

sudo vim /etc/dhcpcd.conf

Replace

#static domain_name_servers=192.168.0.1

with

static domain_name_servers=1.1.1.1

and reboot

sudo reboot

We can confirm that the DNS leak is closed,

time curl -ks https://www.dnsleaktest.com/ | grep flag

Now the Raspberry Pi has an encrypted external drive, and an encrypted connection to the web. Next we want to setup a simple file sync.

Unison Sync

We'll use Unison as a syncing utility. From ArchWiki:

Unison is a bidirectional file synchronization tool that runs on Unix-like operating systems (including Linux, macOS, and Solaris) and Windows. It allows two replicas of a collection of files and directories to be stored on different hosts (or different disks on the same host), modified separately, and then brought up to date by propagating the changes in each replica to the other.

The trickiest part of this setup is that both systems (for us, the Raspberry Pi and a macOS) have to use almost identical unison versions. This means that either the apt and brew versions must be aligned, or, more likely, that we'll need to manually install one side to match the other. Some hints can be found here.

I preferred to use brew on the macOS system, and then manually match that unison version with the Raspberry Pi.

Install unison on macOS via brew

brew update && brew upgrade && brew cleanup && brew install unison

Check the unison version to determine which to match

unison -version
unison version 2.51.3 (ocaml 4.10.0)

This will tell you which unison and ocaml versions you'll need on your other system.

On the Raspberry Pi, install the OCaml package manager opam

sudo apt-get install opam m4

Set the OCaml version to match the macOS unison

opam init -y --compiler=4.10.0
eval `opam config env`

Download the source for the unison version to match macOS and build

wget http://www.cis.upenn.edu/\~bcpierce/unison/download/releases/unison-2.51.3/unison-2.51.3.tar.gz
tar -xvzf unison-2.51.3.tar.gz
cd src
make UISTYLE=text

Copy the utility to the path /usr/local/bin

sudo cp ./unison /usr/local/bin

Now we need to setup the necessary keys.

On the macOS, generate an ssh key, without a passphrase.

ssh-keygen -t ed25519

Copy the pubkey to the Raspberry Pi

ssh-copy-id -i ~/.ssh/id_ed25519.pub pi@raspberrypi

cat .ssh/authorized_keys
ssh-ed25519 <pubkey> <macOS>

On both systems, setup a test directory at ~/UnisonTest/. Test the connection with a quick -testserver flag. On macOS,

unison ~/UnisonTest ssh://pi@raspberrypi//home/pi/UnisonTest -testserver
Unison 2.51.3 (ocaml 4.10.0): Contacting server...
Connected [//MBP//Users/username/UnisonTest -> //raspberrypi//home/pi/UnisonTest]

On macOS, create a test file and test the sync

echo "testing unison" > ~/UnisonTest/test.txt

unison ~/UnisonTest ssh://pi@raspberrypi//home/pi/UnisonTest
Unison 2.51.3 (ocaml 4.10.0): Contacting server...
Connected [//MBP//Users/username/UnisonTest -> //raspberrypi//home/pi/UnisonTest]
Looking for changes
Warning: No archive files were found for these roots, whose canonical names are:
	/Users/username/UnisonTest
	//raspberrypi//home/pi/UnisonTest
This can happen either
because this is the first time you have synchronized these roots,
or because you have upgraded Unison to a new version with a different
archive format.
...

Check that the file was synced. On the Raspberry Pi,

pi@raspberrypi:~ $ ls UnisonTest/
test.txt

See the default.prf file in this repo to see some custom configuration options. This will let you just run unison without specifying any parameters. See also this Unison tutorial for additional config file details.

Systemctl Scheduling

I wanted a script that I could run nightly, regardless of whether my laptop was on and connected. So I setup the Raspberry Pi to run the script. It makes web requests so I wanted it protected with the VPN. The script sometimes generates output files so I wanted to sync those back to my macOS system when I wanted, so I setup unison. Lastly, I wanted the script to run nightly, at least when the Raspberry Pi was on and connected, so I setup systemctl to run my nightly.py script. Systemctl is a more modern scheduler than cron, and fits my use-case well.

I've included my files in the repo and below.

The first step is to create a service.

sudo vim /lib/systemd/system/nightly.service

Add the following content

[Unit]
Description=Run all nightly tasks
# Prevent the script from triggering before the network is available
After=network.target

[Service]
Type=simple
ExecStart=/home/pi/scripts/nightly.py
User=pi

[Install]
# Allow the script to run without needing a user to be logged in
WantedBy=multi-user.target

Next, create a timer.

sudo vim /lib/systemd/system/nightly.timer

Add the following content

[Unit]
Description=Schedule for nightly scraping
RefuseManualStart=no        # Allow manual starts
RefuseManualStop=no         # Allow manual stops

[Timer]
# Execute job if it missed a run due to machine being off
Persistent=true
# Run 5 min (300 seconds) after boot for the first time
OnBootSec=300
# Run every day at midnight
OnCalendar=daily
# File describing job to execute
Unit=nightly.service

[Install]
WantedBy=timers.target

To interface with systemctl:

Enable Service

systemctl enable nightly.service
systemctl enable nightly.timer

Schedule Service

systemctl start nightly.timer

Check Status

systemctl status nightly
systemctl list-unit-files

Manually Start and Stop Service

systemctl start nightly.service
systemctl stop nightly.service

gstro/lunison