noble switch from iptables to nftables

[ramonskie]

as in noble it is easier to create proper firewall rules in nftables
see cloudfoundry/bosh-linux-stemcell-builder@e1245f4

i'm also wondering why the bosh-agent feels responsible for setting up iptables for monit.
https://github.com/cloudfoundry/bosh-agent/blob/main/platform/net/firewall_provider_linux.go#L77-L101
as this was already done for example in the jammy stemcell https://github.com/cloudfoundry/bosh-linux-stemcell-builder/blob/ubuntu-jammy/master/stemcell_builder/stages/bosh_monit/assets/restrict-monit-api-access

i think the agent should not be responsible for this.

for setting up the nats rules we can use nftables
in https://github.com/cloudfoundry/bosh-agent/blob/main/platform/net/firewall_provider_linux.go#L115-L130
example" nft add rule inet filter output skuid 0 ip daddr 10.0.0.1 tcp dport 4222 log prefix "Matched skuid director nats rule:" accept

@nouseforaname investigated https://pkg.go.dev/github.com/google/nftables@v0.2.0
but this seems way to complicated for what we are actually trying to achieve
for example adding the rule above would look something like

			conn.AddRule(&nftables.Rule{
				Table:    table,
				Chain:    chain,
				Position: 0,
				Handle:   0,
				Flags:    0,
				Exprs: []expr.Any{
					&expr.Meta{Key: 10, SourceRegister: false, Register: 1},
					&expr.Cmp{Op: 0, Register: 1, Data: []byte{0, 0, 0, 0}},
					&expr.Meta{Key: 15, SourceRegister: false, Register: 1},
					// skuid 0
					&expr.Cmp{Op: 0, Register: 1, Data: []byte{2}},
					&expr.Payload{OperationType: 0, DestRegister: 1, SourceRegister: 0, Base: 1, Offset: 16, Len: 4, CsumType: 0, CsumOffset: 0, CsumFlags: 0},
					// ip daddr 10.0.0.1
					&expr.Cmp{Op: 0, Register: 1, Data: []byte{10, 0, 0, 1}},
					&expr.Meta{Key: 16, SourceRegister: false, Register: 1},
					// dport
					&expr.Cmp{Op: 0, Register: 1, Data: []byte{6}},
					&expr.Payload{OperationType: 0, DestRegister: 1, SourceRegister: 0, Base: 2, Offset: 2, Len: 2, CsumType: 0, CsumOffset: 0, CsumFlags: 0},
					// port 5000 is caluclated
					// 5000 / 256 = 19 * 256 = 4864
					// 500 - 4864 = 136
					&expr.Cmp{Op: 0, Register: 1, Data: []byte{19, 136}},
					// log prefix "Mathced skuid director nats rule:"
					&expr.Log{Level: 4, Flags: 0, Key: 36, Snaplen: 0, Group: 0, QThreshold: 0, Data: []byte("Mathced skuid director nats rule:")},
					// accept
					&expr.Verdict{
						Kind: expr.VerdictAccept,
					},
				},

more investigation is probably needed
but do not that the iptable rules do not work anymore in noble as we are using cgroupv2
so action is needed

[nouseforaname]

i'm also wondering why the bosh-agent feels responsible for setting up iptables for monit. https://github.com/cloudfoundry/bosh-agent/blob/main/platform/net/firewall_provider_linux.go#L77-L101

it doesn't it sets up a rule for itself to be able to access monit. But with nftables that wouldn't even be needed anymore, since the statement we ended up using is not relying on cgroups, but rather the uid that is associated with the request socket. Going this way also removes the need to have the monit access helper because being root is enough to not be blocked.

As far as the nftables package goes, it was promising until trying to add a rule. The readability of the Expression structs is horrible. While it works, it seems that having this done via ex/exec is more readable ( even though checking for existing rules is going to end up being a bit if stdout parsing.)

As far as testing goes, it would also be easier to test expected os/exec commands with binmock ( or another thing )..

While there is a way to setup a TestDialer with the package, checking expectations involves matching bytearrays 💥:
https://github.com/google/nftables/blob/main/nftables_test.go#L234-L260

I think all of this could be solved by a unit that is baked into the stemcell that waits for /var/vcap/bosh/agent_settings.json and then sets up nftable rules via a script ( after parsing out mbus IP:PORT))

[ramonskie]

will be address in #332