DelRule fails

[greenpau]

It appears that deleting a rule previosly fetched via GetRule() fails.

            forwardChainRules, err := nb.conn.GetRule(addr.table, forwardChain)

...
            for _, r := range forwardChainRules {
                nb.conn.DelRule(r)
...

The error is netlink receive: no such file or directory not to have occurred.

At the same time, the deleting by constructing a rule from scractch works:

                nb.conn.DelRule(&nftables.Rule{
                    Table:  &nftables.Table{Name: addr.table.Name, Family: addr.table.Family},
                    Chain:  &nftables.Chain{Name: forwardChain.Name, Type: forwardChain.Type},
                    Handle: forwardChainJumpRule.Handle,
                })

                if err := nb.conn.Flush(); err != nil {
                    return fmt.Errorf(
                        "error deleting jump rule to %s chain found in chain %s in %s table: %s",
                        chainName, forwardChain.Name, addr.table.Name, err,
                    )
                }

[stapelberg]

Can you share a minimum program that reproduces the issue?

Are you flushing in between the two operations?

[greenpau]

Are you flushing in between the two operations?

@stapelberg , yes.

Can you share a minimum program that reproduces the issue?

@stapelberg , this would take some time. Will post later.

[greenpau]

@stapelberg , here is the example.

Clone the directory at this commit: greenpau/cni-plugins@8980b21

Next, run:

make dep
make test

The output is as follows, i.e. error deleting jump rule to cnins-3-4026541879-dummy0 chain found in chain FORWARD in filter table: Receive: netlink receive: address family not supported by protocol

=== RUN   TestPlugin
    TestPlugin: firewall_test.go:121: Container Namespace Path: /var/run/netns/cnitest-950d1727-62e7-10d5-9409-f7c92c6213e9
    TestPlugin: firewall_test.go:122: Debug Namespace:
        sudo ip netns exec cnitest-4199ebfe-8b9b-93b1-6f92-8449c23e9495 nft --debug=netlink list ruleset
=== RUN   TestPlugin/configures_nftables_for_a_single_dual-stack_interface
    TestPlugin/configures_nftables_for_a_single_dual-stack_interface: firewall_test.go:142: &{{0.4.0 test firewall map[] {} {[]  [] []} map[] 0xc00007ea50} filter FORWARD}
    TestPlugin/configures_nftables_for_a_single_dual-stack_interface: firewall_test.go:143: &{0.4.0 [{Name:dummy0 Mac: Sandbox:}] [{Version:4 Interface:0xc000019608 Address:{IP:192.168.200.10 Mask:ffffff00} Gateway:<nil>} {Version:6 Interface:0xc000019618 Address:{IP:2001:db8:1:2::1 Mask:ffffffffffffffff0000000000000000} Gateway:<nil>}] [] {[]  [] []}}
    TestPlugin/configures_nftables_for_a_single_dual-stack_interface: firewall_test.go:182: cni-nftables-firewall.Del() error: error deleting jump rule to cnins-3-4026541879-dummy0 chain found in chain FORWARD in filter table: Receive: netlink receive: address family not supported by protocol
--- FAIL: TestPlugin (0.02s)
    --- FAIL: TestPlugin/configures_nftables_for_a_single_dual-stack_interface (0.01s)
=== RUN   TestSupportedVersion
    TestSupportedVersion: version_test.go:16: [0.4.0]
--- PASS: TestSupportedVersion (0.00s)
FAIL
coverage: 78.2% of statements
make: *** [test] Error 1

Currently, I reference the rules without constructing them from scratch. If you uncomment, then the first delete would succeed, while second would fail.

https://github.com/greenpau/cni-plugins/blob/8980b21eef23fa423bad8fcda30ac996ac72b888/pkg/firewall/firewall.go#L144-L179

[greenpau]

@stapelberg , also for the easy of use and the inspection of tables and chains, I output ip netns commands:

=== RUN   TestPlugin
    TestPlugin: plugin_test.go:121: Container Namespace Path: /var/run/netns/cnitest-b81650f0-9592-1080-a7ac-f6f9896b2b4d
    TestPlugin: plugin_test.go:122: Debug Namespace:
        sudo ip netns exec cnitest-b181b09a-469a-14e5-b363-ec5c2166f8c6 nft --debug=netlink list ruleset

[greenpau]

Apparently, with nftables v0.8, the deletion of a chain is a two step process. First, flush, then delete. Tested it with nft it works. Now, I will try it with this library.

 sudo ip netns exec cnitest-4e286b84-d12c-1d82-40f2-eba8d85d4d42 nft --debug=netlink flush chain filter cni0d00f873743fd948b10cc81ebd2f
sudo ip netns exec cnitest-4e286b84-d12c-1d82-40f2-eba8d85d4d42 nft --debug=netlink delete chain filter cni0d00f873743fd948b10cc81ebd2f

[greenpau]

Found the issue. It is not necessarily a bug ... rather, it is usage issue.

Rule: To delete a non-empty chain, one should first FlushChain and then DelChain:

                p.conn.FlushChain(&nftables.Chain{
                    Name: addrChain.Name,
                    Table: &nftables.Table{
                        Name:   addrChain.Table.Name,
                        Family: addrChain.Table.Family,
                    },
                })

                p.conn.DelChain(&nftables.Chain{
                    Name: addrChain.Name,
                    Table: &nftables.Table{
                        Name:   addrChain.Table.Name,
                        Family: addrChain.Table.Family,
                    },
                })

                if err := p.conn.Flush(); err != nil {
                    return fmt.Errorf(
                        "error deleting %s chain in %s table: %s",
                        chainName, addr.table.Name, err,
                    )
                }

[stapelberg]

Thanks for sharing the solution!