Sink server daemon container for zrepl push jobs. This is intended for environments which have a kernel that support ZFS but are unable to deploy zrepl sink job type such as TrueNAS Scale.
This is experimental proof of concept for testing. You should already have working knowledge of zrepl
, its configuration and why you would want a sink server.
-
Create a dataset container to hold sink job datasets from remote hosts
- Adjust the ZFS pool name and dataset name for your needs,
zroot/zrepl_sink_data
is used here:
zfs create zroot/zrepl_sink_data -o mountpoint=none -o canmount=off -o readonly=on
- Adjust the ZFS pool name and dataset name for your needs,
-
Create
config
directory forzrepl.yaml
and certificates.-
This directory will be mapped into the container
-
Do store this within the sink dataset as it will remain unmounted
mkdir ./config
-
-
Create TLS Certificates for
zrepl sink
daemon container- zrepl transport documents different method to support inbound connections and client identification, this example assumes TLS certificates
ca.crt
- certificate authority certificatesink-srv.crt
- Sink server daemon certificatesink-srv.key
- Sink server daemon private key
$ ls -l ./config .rw------- rich rich 1.2 KB Thu Aug 3 14:39:09 2023 ca.crt .rw-r--r-- rich rich 4.6 KB Thu Aug 3 14:39:21 2023 sink-srv.crt .rw------- rich rich 1.7 KB Thu Aug 3 14:39:31 2023 sink-srv.key
-
Customize
zrepl.yml
configuration file (see example sink config file)- The
jobs
section defined thesink
job for the daemon:
jobs: - type: sink name: zrepl_sink_server
-
The
root_fs
defined the ZFS pool and dataset the daemon will use (dataset created in Step1
)- ZFS filesystems are received to
$root_fs/$client_identity/$source_path
- ZFS filesystems are received to
root_fs: "zroot/zrepl_sink_data"
- Define how connections will be served, this will listen for
tls
connections on port8448
:
serve: type: tls listen: ":8448" listen_freebind: true
- Define the full pathname for certificates used inside the Sink Server daemon container (paths are inside the container):
ca: /config/ca.crt cert: /config/sink-srv.crt key: /config/sink-srv.key
- Define names of clients allowed to connect to the Sink Server daemon (adjust the names to match the
CN
values in the certificates you generated):
client_cns: - "dldsk01" - "k3s01" - "k3s02" - "k3s03" - "k3s04" - "k3s05" - "k3s06"
- Review property overrides, below prevents the Sink Server daemon host from trying to mount or allow modifications of replicated datasets:
recv: properties: # Force mountpoint to be inherited from Sink container (set to none) inherit: - "mountpoint" override: { # These two need to be disabled to support ZVOL replication # "canmount": "off", # "mountpoint": "none" "readonly": "on", "openzfs.systemd:ignore": "on" }
- Review properties assigned to placeholder datasets.
zrepl
will maintain the hierarchy of your filesystem datasets even if you do not replicate all of them. Datasets not replicated will have a placeholder created for them:
placeholder: encryption: inherit
- The
The following environment variables can be set within the container:
Variable | Description | Default Value |
---|---|---|
CONFIG |
Full pathname inside container to zrepl.yml |
/config/zrepl.yml |
docker run -d --privileged -p 8448:8448 \
-v ./config:/config \
-v /etc/timezone:/etc/timezone:ro \
--name zrepl_sink quay.io/reefland/zrepl_sink:latest
- Container runs as
root
and requires--privileged
to access the underlying hosts/dev/zfs
device to issuezfs
commands - The internal port number
8448
is defined in thezrepl.yml
file, external port8448
will be used for inbound connection from clients (adjust as needed)
$ docker logs zrepl_sink
* Default Config File Set: /config/zrepl.yml
* Config location verified.
* root_fs value for sink pool: zroot/sink
NAME USED AVAIL REFER MOUNTPOINT
zroot/sink 247M 515G 320K none
Attempting zrepl config check...
Attempting to start zrepl daemon...
2023-08-06T15:25:38Z [INFO]: zrepl version=v0.6.0 go=go1.19.2 GOOS=linux GOARCH=amd64 Compiler=gc
2023-08-06T15:25:38Z [INFO]: starting daemon
2023-08-06T15:25:38Z [INFO][_control][job][Uv38$Uv38]: starting job
2023-08-06T15:25:38Z [INFO][zrepl_sink_server][job][Uv38$Uv38]: starting job
See zrepl
project documentation on monitoring for details.
In the global:
section of the zrepl.yml
file add:
monitoring:
- type: prometheus
listen: ":9811"
listen_freebind: true
- Add the port forwarding to the
docker run
command:-p 9811:9811
- Add the container IP address to the Prometheus Scape jobs (as well as all zrepl clients)
My work in progress Grafana dashboard for Zrepl Sink Server:
- See Configure Clients for Push Replication example to Sink Server daemon.