Bridged Networking or User mode? Replace ebtables & iptables with iptables-nft to allow bridged networking.
sickcodes opened this issue · 14 comments
Here are the two types of networking
# usermode
-netdev user,id=net0,hostfwd=tcp::10022-:22,
-device e1000-82545em,netdev=net0,id=net0,mac=52:54:00:09:49:17
# bridged
-netdev tap,id=net0,ifname=tap0,script=no,downscript=no
-device vmxnet3,netdev=net0,id=net0,mac=52:54:00:AB:F8:B7
user mode
The image has the same IP as your Docker IP ip n to see neighbors
QEMU cannot be reached unless you add ports in the QEMU args during "docker run"
Can't expose Docker ports on a running container (AFAIK), you have to start a new image and move the installation around.
You also have to edit the Launch.sh or add in arguments
bridged mode
The image will run on the container's own internal network, like 10.0.2.15 or something or 192.168.122.11x
After that, you can ssh OR docker exec into the Docker and then ssh into the QEMU image.
which is better?
Realistically, they both appear exactly the same.
Bridge mode would let people forward all connections to the bridge IP.
# usermode
# you have to add
docker run ... -p 50922:10022 -e EXTRA='-netdev user,hostfwd=tcp::10022-:22'
# bridge-mode
docker run ... -p 50922:10022 -ip p.ubl.ic.ip
If anyone wants to test it out:
sudo tee -a /etc/sysctl.conf <<< 'net.ipv4.ip_forward=1'
sudo sysctl -p
sudo pacman -S iptables-nft
sudo systemctl enable --now libvirtd.service
sudo systemctl enable --now virtlogd.service
sudo ip tuntap add dev tap0 mode tap
sudo ip link set tap0 up promisc on
sudo virsh net-autostart default
sudo virsh net-start default
sudo ip link set dev virbr0 up
sudo ip link set dev tap0 master virbr0
# use the networking from above in the QEMU args
The actual bug is ebtalbes + dnsmasq doesn't work virsh networking.
You can test it out:
sudo pacman -S libvirtd
virsh net-info --domain default
# works
Bugged:
sudo pacman -S ebtables dnsmasq
virsh net-info --domain default
# fails
Fix:
sudo pacman -S iptables-nft
virsh net-info --domain default
# works
It's a strange bug, I though related to polkit, however, I found it incompatible with ebtables.
Possibly just with Arch, that the default network doesn't get installed.
Easy fix: replace iptables and ebtables with iptables-nft. If not, virsh hangs, sudo virsh hangs.
On the other hand, QEMU and Docker networking is really, really easy to debug. If bridged network is added, you'd have to start the bridge every time, I also think you need --net host
I was having issues with other machines communicating with a program running inside the macos docker, this might have something to do with it. If I get some free time ill try some of this stuff.
I was having issues with other machines communicating with a program running inside the macos docker, this might have something to do with it. If I get some free time ill try some of this stuff.
If you use the two -net command strings above in the xml file while using virt-manager you can switch between bridged and userMode networking. Haven't done bridged in the dockerfile but it wouldn't be much more than whats already here.
In summary:
Use virt-manager without docker
Load the xml file from OSX-KVM
Try the networking commands above
Otherwise you can just add more ports to the QEMU command using the -e EXTRA env variable and also forwarding it again as a normal -p 111:2222 to the docker command.
e1000-82545em is preventing iMessage from working on Catalina and above
However, a fix is available here: https://github.com/chris1111/AppleIntelE1000e
Couldn't get this working. Trying to have a port shared so that the host can access a server running on the mac on port 8080.
-e EXTRA='-netdev user,hostfwd=tcp::8080-:8080'
This produces: qemu-system-x86_64: Parameter 'id' is missing
-e EXTRA='-netdev user,id=net0,hostfwd=tcp::8080-:8080'
This produces: Duplicate ID 'net0' for netdev
-p 8080:8080 -e EXTRA='-netdev user,id=net1,hostfwd=tcp::8080-:8080'
(qemu) qemu-system-x86_64: warning: netdev net1 has no peer
let's the container run... but can't access the port on the host :/
I switched master to vmxnet3 in #160 as it did not affect iMessage and it massively increases speed.
@sickcodes Just unsure how to get port forwarding working. I suppose worst case scenario I can just forward via an ssh tunnel.
Maybe we can add another ${ADDITIONAL_PORTS} at the end of the line that has 5900 in it too for any other networking arguments for that interface
At the end of this line: https://github.com/sickcodes/Docker-OSX/blob/master/Dockerfile#L228
I'll add it in shortly
@sickcodes just tested it out, it works! Thanks for your really speedy response!
@mikob No problem!
Here is a worked example for anyone else reading, as written here: #162 (comment)
On the host
docker run -it \
--device /dev/kvm \
-p 50922:10022 \
-e ADDITIONAL_PORTS='hostfwd=tcp::10023-:80,' \
-p 10023:10023 \
sickcodes/docker-osx:autoInside the container:
/bin/bash -c "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/HEAD/install.sh)"
brew install nginx
sudo sed -i -e 's/8080/80/' /usr/local/etc/nginx/nginx.conf
# sudo nginx -s stop
sudo nginxnginx should appear on the host at port 10023
P.S. you don't need to install nginx for a test server, you can just do eg. python3 -m http.server 8080 to spin up a static test server that serves the CWD and it ships with python by default, which ships with osx by default.
I am unable to run:
sudo tee -a /etc/sysctl.conf <<< 'net.ipv4.ip_forward=1'
sudo sysctl -p
sudo pacman -S iptables-nft
sudo systemctl enable --now libvirtd.service
sudo systemctl enable --now virtlogd.service
sudo ip tuntap add dev tap0 mode tap
sudo ip link set tap0 up promisc on
sudo virsh net-autostart default
sudo virsh net-start default
sudo ip link set dev virbr0 up
sudo ip link set dev tap0 master virbr0
# use the networking from above in the QEMU args
without getting several errors. However, I believe this is the solution to the problems I'm having. Any help would be appreciated.
Output:
Package (1) Old Version New Version Net Change
core/iptables-nft 1:1.8.9-1 1:1.8.9-1 0.00 MiB
Total Installed Size: 5.64 MiB
Net Upgrade Size: 0.00 MiB
:: Proceed with installation? [Y/n] Y
checking keyring...
checking package integrity...
loading package files...
checking for file conflicts...
:: Processing package changes...
reinstalling iptables-nft...
:: Running post-transaction hooks...
(1/2) Reloading system manager configuration...
Skipped: Current root is not booted.
(2/2) Arming ConditionNeedsUpdate...
System has not been booted with systemd as init system (PID 1). Can't operate.
Failed to connect to bus: Host is down
System has not been booted with systemd as init system (PID 1). Can't operate.
Failed to connect to bus: Host is down
Authorization not available. Check if polkit service is running or see debug message for more information.
error: failed to connect to the hypervisor
error: Operation not supported: Cannot use direct socket mode if no URI is set
Authorization not available. Check if polkit service is running or see debug message for more information.
error: failed to connect to the hypervisor
error: Operation not supported: Cannot use direct socket mode if no URI is set
Cannot find device "virbr0"
Error: argument "virbr0" is wrong: Device does not exist
Here is my docker compose:
services:
macos:
container_name: 'MacOS'
privileged: true
network_mode: br0
devices:
- /dev/kvm
- /dev/snd
- /dev/null
environment:
- RAM=6
- NETWORKING=vmxnet3
- 'USERNAME=user'
- 'PASSWORD=pass'
- 'DISPLAY=${DISPLAY:-:0.0}'
image: 'docker-osx-vnc:latest'
#network_mode: host