wulfy23/rpi4

Docker host?

Opened this issue · 4 comments

molly78 commented

Anyone tried running docker host on wulfy23’s build?

I’d love to get wireguard-easy as well as speedtest-tracker on it, instead of a seperate pi as they’re so hard to find

thanks!

wulfy23 commented

gday, i'm no docker master but afair docker stuff is kinda nuts and bolts in openwrt... in that most application level stuff can be done if you are adept at the lower levels of docker but networking layers might need some hand modding for optimal isolation

fancy (non pi 64 specific) containers or ones that do anythings else with rpi hardware often have expectations from the host OS being 'pi foundation (pty ltd)' ( vc stuff, python gpio stuff maybe etc.)

build wise I would not object to some tweaks/automation to streamline docker network layer isolation / simplification but due to complixity and variation I don't think we can do too much without diverging from generic / helpfull to all...

either way, if you do get something running, the persistent partition features might come in handy / be an option if you are not pointing container storage to an external mountpoint

edit: having said the above, I believe I saw some recipe for this wg-easy thing somewhere and it's of interest to me and likely other users of the build so if i've time over the next few weeks I'll likely have a crack for a few hours and see what can be done...

  • ❤️1
molly78 commented

Thanks for the reply!

One pi can do so much :-)

molly78 commented

gday, i'm no docker master but afair docker stuff is kinda nuts and bolts in openwrt... in that most application level stuff can be done if you are adept at the lower levels of docker but networking layers might need some hand modding for optimal isolation

fancy (non pi 64 specific) containers or ones that do anythings else with rpi hardware often have expectations from the host OS being 'pi foundation (pty ltd)' ( vc stuff, python gpio stuff maybe etc.)

build wise I would not object to some tweaks/automation to streamline docker network layer isolation / simplification but due to complixity and variation I don't think we can do too much without diverging from generic / helpfull to all...

either way, if you do get something running, the persistent partition features might come in handy / be an option if you are not pointing container storage to an external mountpoint

edit: having said the above, I believe I saw some recipe for this wg-easy thing somewhere and it's of interest to me and likely other users of the build so if i've time over the next few weeks I'll likely have a crack for a few hours and see what can be done...

Any progress wulfy23?

it would be awesome to free up the second pi in many of my setups :-)

especially as so many dockers just idle, much like openwrt

  • 👍1
wulfy23 commented

some quick and dirty notes on a quick and dirty spin-up (no actual wg test) on 23.05 be aware steps totally disregard docker network obfuscation / nat layers so use only for testing or just don't use... for now was just checking docker service / build constraints etc.

#FYI... i just verified this (wg-easy) installs / starts etc. on 23.05.2...

#like I said i'm no docker guru but i'll share what stood out and some shortcuts around the networking to get you started

CAN SKIP THIS FOR QUICK TEST IF YOU HAVE ~400MB space

  1. wg-easy + install took ~ 350MB and just fit inside the small (not expanded partition)

A PERSISTENT ADDITIONAL DRIVE OR PARTITION IS NEEDED
For one on your sdcard ("/z" persistent 'drive' )
-change ROOTFSEXPAND=1 to ROOTFSEXPAND_DATAPART=1
in /root/wrt.ini (if not already done before last upgrade)
-upgrade ( sysupgrade -R /tmp/XYZ.img.gz)

step x.x)
then change /etc/config/dockerd data_root to your persistent drive and restart it);

	option data_root '/opt/docker/'
        option data_root '/z/docker/'

then create / migrate /opt/docker to /z

/etc/init.d/dockerd stop; sleep 20
cp -arf /opt/docker /z
/etc/init.d/dockerd start

(above steps need fine tuning)

i did not do this but would be the first thing I do...

  1. Installing docker (manually) needs --force-overwrite due to libwebsockets...so packaging conflicts so;
opkg install luci-app-dockerman **--force-overwrite**
opkg install dockerd
opkg install docker-compose
  1. Disable iptables functions for docker
    then change /etc/config/dockerd;
	#option iptables '1'
	option iptables '0'
  1. Restart dockerd
/etc/init.d/dockerd restart
  1. For this quick and dirty test we'll need WEBUI on LAN and WG ports on all(0.0.0.0 aka WAN)?
    so for the compose yaml change at least;
version: "3.8"
volumes:
  etc_wireguard:

services:
  wg-easy:
    environment:
      # ⚠️ Required:
      # Change this to your host's public address didnt use for my test
      #####################################################
      - WG_HOST=**YOURPUBLICIPHERE**
      #####################################################

      # Optional:
      # - PASSWORD=foobar123
      # - WG_PORT=51820
      # - WG_DEFAULT_ADDRESS=10.8.0.x
      # - WG_DEFAULT_DNS=1.1.1.1
      # - WG_MTU=1420
      #You'll need below didnt use for my test
      # - WG_ALLOWED_IPS=192.168.15.0/24, 10.0.1.0/24
      # - WG_PERSISTENT_KEEPALIVE=25
      # - WG_PRE_UP=echo "Pre Up" > /etc/wireguard/pre-up.txt
      # - WG_POST_UP=echo "Post Up" > /etc/wireguard/post-up.txt
      # - WG_PRE_DOWN=echo "Pre Down" > /etc/wireguard/pre-down.txt
      # - WG_POST_DOWN=echo "Post Down" > /etc/wireguard/post-down.txt

    image: ghcr.io/wg-easy/wg-easy
    container_name: wg-easy
    volumes:
      - etc_wireguard:/etc/wireguard
    ports:
	#DEFAULT - "51820:51820/udp"
        #DEFAULT - "51821:51821/tcp"
       ############################################
	- "51820:51820/udp"
      - "**YOURLANIPHERE**:51821:51821/tcp"
       ############################################
    restart: unless-stopped
    cap_add:
      - NET_ADMIN
      - SYS_MODULE
    sysctls:
      - net.ipv4.ip_forward=1
      - net.ipv4.conf.all.src_valid_mark=1

5.1 start it

docker-compose -f above.yaml create
docker-compose -f above.yaml up

(or however you need to do it)

5.2 You should now be able to get to WEBUI at YOURROUTERIP:51821

  1. Lastly you probably need to punch a hole in your wan zone for udp 51820?

Anyway, just a quick and dirty test... but kinda need an expert
to advise on whether this nft/ipt translation of bridging
is workable or this way is way too risky etc.

From my point of view all i'd need is on package restore
add that --force-overwrite or similar... and maybe setup
some of that dockerd config once known...
(edit: tested and restore worked already for me but would
want to see that many times before manual intervention)

If you can't get into the webui I did add some stuff to firewall
and restarted it but not sure if was needed in this raw setup...
(edit: nope not needed for this blunt setup)

No pressure to use / test... mostly notes for myself... it is kinda tricky with a "multihomed' service... as compared to a purely lan / private segment resource

edit: ref: https://forum.openwrt.org/t/docker-containers-bypass-openwrt-firewall/122530/5

edit2: these lower network parameters seem suited to this topology (dont need the pihole section)
https://github.com/wg-easy/wg-easy/wiki/Using-WireGuard-Easy-with-Pi-Hole

edit3: dang! just tested an upgrade with;
-docker files stored in the persistent partition
and
-docker automatically being re-installed on upgrade
(sysupgrade -R abc.img.gz)

and the service came up! so while I wouldn't say it's reliable but at least the steps / automatic restore and persistent docker images on upgrade worked for me in a single test... i'll update the actual notes on what I did and do some more reliability tests on request if needed.