ben-grande/qusal

qvm-port-forward QUBESRPC error

n1ete opened this issue · 4 comments

n1ete commented

Hey there, thanks for the inspiration and contribution!
It helped me to understand saltstack better and getting my fresh 4.2 up and ready in no time!
However i stumbled into some trouble when using the qvm-port-forward command.
Using qvm-port-forward -a add -q sys-syncthing -n tcp -p 22000
throws me the following error:

info sys-sycthing: adding input rule daddr 10.137.0.56
Unknown command "QUBESRPC qubes.VMShell dom0" [while processing "QUBESRPC qubes.VMShell dom0"]

Is there just a policy missing and how would look such a policy that allows to use the port forwarding command from dom0?
How do i debug this or where do i have to look if the nessecary rules got created and how to remove this if the command isnt working but in case has created some rules....?
I couldnt find anything related to port forwarding in the "sys-net" qube

ben-grande commented

Hey there, thanks for the inspiration and contribution! It helped me to understand saltstack better and getting my fresh 4.2 up and ready in no time!

I am glad I could be helpful. Please note the warning in the main readme, this project is testing only, so be careful, especially about data loss and future breaking changes I might do. If you are comfortable with that, than enjoy the project. PS: I think sys-syncthing is done, but as you have just found, port forwarding is broke.

However i stumbled into some trouble when using the qvm-port-forward command. Using qvm-port-forward -a add -q sys-syncthing -n tcp -p 22000 throws me the following error:

info sys-sycthing: adding input rule daddr 10.137.0.56
Unknown command "QUBESRPC qubes.VMShell dom0" [while processing "QUBESRPC qubes.VMShell dom0"]

Is there just a policy missing and how would look such a policy that allows to use the port forwarding command from dom0? How do i debug this or where do i have to look if the nessecary rules got created and how to remove this if the command isnt working but in case has created some rules....? I couldnt find anything related to port forwarding in the "sys-net" qube

I don't think you are missing any policies, dom0 is running qubes.VMShell and the policy does not need to allow dom0 to do that.

About debugging this, I have never encountered this issue before... To debug, you can run the script with xtrace:

sh -x /usr/local/bin/qvm-port-forward ARGS

to see where it is failing.

Example:

sh -x /usr/local/bin/qvm-port-forward qvm-port-forward -a add -q sys-syncthing -n tcp -p 22000 2>&1 | tee /tmp/qpf.log

Take a look at /tmp/qpf.log. You don't have to share it all if you don't want to, you might be able to find the problem, but if you do share the logs, please attach it as a file instead of pasting a large comment.

There are no rules in sys-net because the script didn't finish, it aborts on the first error. Which I believe is safer instead of having hanging around an invalid configuration. Although it does not delete rules without you saying it to do so.

This is what you should have seen instead:

[user@dom0 ~]
$ qvm-port-forward -a add -q sys-syncthing -n tcp -p 22000
info: sys-syncthing: adding input rule daddr 10.137.0.79
info: sys-pihole: adding forward rule dev eth0 saddr 192.168.2.0/24 daddr 10.137.0.79
info: disp-sys-net: adding forward rule dev wls7 saddr 192.168.2.0/24 daddr 10.137.0.104

For further debugging, please answer the following questions:

  1. Are you using a terminal different than Xterm of Xfce-terminal in Dom0 or the netvms fro sys-syncthing? (sys-firewall, sys-net).
  2. What is your sys-firewall or sys-net templates based on? Are they Debian? Fedora? Which version? Upgraded in-place or clean-install? Sys-firewall and sys-net is just an example, if you use other names for the netvms of sys-syncthing, it equally applies.
  3. Was your R4.2 installed in-place from R4.1 or clean-install?
  4. Add any relevant information about your netvm chain, from sys-syncthing to sys-net, such as qvm-prefs output for the netvm qubes.
n1ete commented

I am totally aware that this is not something you would describe as "production ready". But i do have backups of several other qubes i depend on in case something breaks.
To your questions:
1.
Nope just xfce4-terminal and xterm.
2. and 3.
Templates are all Debian-12-minimal based an were build with your salststates from a clean and fresh 4.2 install.
4.
The Syncthing qube-routing-chain is sys-syncthing -> disp-sys-mirage-firewall -> sys-net
I will try the debugging script and will check the qvm-prefs output soon (tm) and report back.
Should this work with the sys-mirage-firewall?
Thanks in advance for your time and help!

ben-grande commented

Should this work with the sys-mirage-firewall?

It doesn't work because Mirage Firewall does not have a proper shell, it basically only can be a netvm and use the Qubes Firewall, but custom Firewall via rewriting with Nftables does not work.

See:

Maybe comment there asking for the feature, but I find it difficult to implement on the mirage side especially the persistentence of rules, where they don't have a proper file system. A better implementation would be if this was done on the Qubes API side, but this is also a feature request for Qubes...

I dont know of a perfect solution of how I can block starting the script if a netvm in the chain is not supported, such as Mirage. Grepping for the word Mirage is not a good solution because many other unsupported options can exist, such as a Unix distro OS integrated virtualized with HVM that was installed from an ISO, such as OpenBSD, might also not work.

A good enough solution for me, would be to fail early if the following command fails:

qvm-run -p disp-sys-mirage-firewall echo test

But this is not perfect, in case the VM has a shell but firewall is not managed by Qubes.

I will implement this as a fail-early mechanism with an error message that a certain netvm in chain doesn't support port forwarding.

n1ete commented

Switching the Firewall qube solved the issue! Thanks for your help and time!