containers/netavark

`nftables error: got invalid json: data did not match any variant of untagged enum NfObject`

Closed this issue · 7 comments

g00g1 commented
podman info 

admin@flopster ~ $ sudo podman info
host:
  arch: amd64
  buildahVersion: 1.35.0
  cgroupControllers:
  - cpuset
  - cpu
  - io
  - memory
  - hugetlb
  - pids
  - misc
  cgroupManager: cgroupfs
  cgroupVersion: v2
  conmon:
    package: app-containers/conmon-2.1.8
    path: /usr/libexec/podman/conmon
    version: 'conmon version 2.1.8, commit: 00e08f4a9ca5420de733bf542b930ad58e1a7e7d'
  cpuUtilization:
    idlePercent: 92.28
    systemPercent: 0.82
    userPercent: 6.9
  cpus: 12
  databaseBackend: sqlite
  distribution:
    distribution: gentoo
    version: "2.14"
  eventLogger: file
  freeLocks: 2043
  hostname: flopster.at.encryp.ch
  idMappings:
    gidmap: null
    uidmap: null
  kernel: 6.6.16
  linkmode: dynamic
  logDriver: k8s-file
  memFree: 44018987008
  memTotal: 67366854656
  networkBackend: netavark
  networkBackendInfo:
    backend: netavark
    dns:
      package: app-containers/aardvark-dns-1.10.0
      path: /usr/libexec/podman/aardvark-dns
      version: aardvark-dns 1.10.0
    package: app-containers/netavark-1.10.3
    path: /usr/libexec/podman/netavark
    version: netavark 1.10.3
  ociRuntime:
    name: crun
    package: app-containers/crun-1.8.4
    path: /usr/bin/crun
    version: |-
      crun version 1.8.4
      commit: 5a8fa99a5e41facba2eda4af12fa26313918805b
      rundir: /run/crun
      spec: 1.0.0
      +SELINUX +APPARMOR +CAP +SECCOMP +EBPF +YAJL
  os: linux
  pasta:
    executable: ""
    package: ""
    version: ""
  remoteSocket:
    exists: true
    path: /run/podman/podman.sock
  security:
    apparmorEnabled: false
    capabilities: CAP_CHOWN,CAP_DAC_OVERRIDE,CAP_FOWNER,CAP_FSETID,CAP_KILL,CAP_NET_BIND_SERVICE,CAP_SETFCAP,CAP_SETGID,CAP_SETPCAP,CAP_SETUID,CAP_SYS_CHROOT
    rootless: false
    seccompEnabled: true
    seccompProfilePath: /usr/share/containers/seccomp.json
    selinuxEnabled: false
  serviceIsRemote: false
  slirp4netns:
    executable: ""
    package: ""
    version: ""
  swapFree: 0
  swapTotal: 0
  uptime: 2h 3m 27.00s (Approximately 0.08 days)
  variant: ""
plugins:
  authorization: null
  log:
  - k8s-file
  - none
  - passthrough
  network:
  - bridge
  - macvlan
  - ipvlan
  volume:
  - local
registries: {}
store:
  configFile: /etc/containers/storage.conf
  containerStore:
    number: 5
    paused: 0
    running: 0
    stopped: 5
  graphDriverName: overlay
  graphOptions:
    overlay.mountopt: nodev
  graphRoot: /var/lib/containers/storage
  graphRootAllocated: 502787899392
  graphRootUsed: 210496536576
  graphStatus:
    Backing Filesystem: extfs
    Native Overlay Diff: "true"
    Supports d_type: "true"
    Supports shifting: "true"
    Supports volatile: "true"
    Using metacopy: "false"
  imageCopyTmpDir: /var/tmp
  imageStore:
    number: 1
  runRoot: /run/containers/storage
  transientStore: false
  volumePath: /var/lib/containers/storage/volumes
version:
  APIVersion: 5.0.0-rc5
  Built: 1709918602
  BuiltTime: Fri Mar  8 19:23:22 2024
  GitCommit: ""
  GoVersion: go1.21.7
  Os: linux
  OsArch: linux/amd64
  Version: 5.0.0-rc5


admin@flopster ~ $ sudo podman run -dt -p 127.0.0.255:8080:80/tcp docker.io/library/httpd 
Error: netavark: nftables error: got invalid json: data did not match any variant of untagged enum NfObject at line 1 column 4050

Please guide me on what information to provide.

Thanks!

mheon commented

Can you provide the version of your kernel and nftables packages? Likely a schema change in the JSON output of nft.

g00g1 commented 
admin@flopster ~ $ uname -a
Linux flopster.at.encryp.ch 6.6.16 #1 SMP PREEMPT_DYNAMIC Mon Feb 26 23:01:05 EET 2024 x86_64 AMD Ryzen 5 3600 6-Core Processor AuthenticAMD GNU/Linux
admin@flopster ~ $ nft -v
nftables v1.0.9 (Old Doc Yak #3)
Luap99 commented

Did this worked before or did you just switched to the nftables backend?

Also please provide the output of nft --json list ruleset

g00g1 commented

This is fresh setup in a nftables-only environment.

nft.json

  • 👍1
Luap99 commented

Thanks, I doubt there was some schema change.
We use https://github.com/namib-project/nftables-rs to parse the output, I will try to feed it in there and see which rule exactly blows up.

Luap99 commented

It fails on these two parts.

{
                        "match": {
                            "op": "==",
                            "left": {
                                "&": [
                                    {
                                        "payload": {
                                            "base": "th",
                                            "offset": 64,
                                            "len": 8
                                        }
                                    },
                                    248
                                ]
                            },
                            "right": 64
                        }
                    },

which is part of your OpenVPN reflection rule and

                    {
                        "synproxy": {
                            "mss": 1460,
                            "wscale": 7,
                            "flags": [
                                "timestamp",
                                "sack-perm"
                            ]
                        }
                    }

which is part of the filter-2 table input chain.

While these are valid bugs that should be fixed in the json parser that raises the question why do we even parse this in the first place. Netavark should only care about rules in our netavark table, there is no need to read all rules for us, that just makes things slower and more prone to such bugs when parsing "unknown" rules.

  • 👍1
Luap99 commented

FYI I submitted some fixes to the nftables json parser last week, most importantly nftables-rs/nftables-rs#17 which allows us to only dump the content only for our netavark table so it should be impossible to fail with that once we implement this here.

  • ❤️1