toreanderson/clatd

Could not find a global IPv6 address on enp2s0 from which to derive a CLAT IPv6 address

Lucaslah opened this issue · 29 comments

Lucaslah commented

Hi, I have been unable to get this work, when I start clatd I get this error:

Starting clatd v1.5 by Tore Anderson <tore@fud.no>
Performing DNS64-based PLAT prefix discovery (cf. RFC 7050)
Using PLAT (NAT64) prefix: 64:ff9b::/96
Device facing the PLAT: enp2s0
Attempting to derive a CLAT IPv6 address from an IPv6 address on 'enp2s0'
<error> Could not find a global IPv6 address on enp2s0 from which to derive a CLAT IPv6 address (try setting 'clat-v6-addr')

enp2s0 interface info

2: enp2s0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel state UP group default qlen 1000
    link/ether 74:86:e2:24:82:7a brd ff:ff:ff:ff:ff:ff
    inet6 [removed]::a5/128 scope global dynamic noprefixroute 
       valid_lft 5254sec preferred_lft 2554sec
    inet6 fe80::5a93:449d:1c08:bd8d/64 scope link noprefixroute 
       valid_lft forever preferred_lft forever

How would I get this working? do I need to set clat-v6-addr to something? if so what?

timriker commented

Read the LIMITATIONS in the README. You need to be using slaac so that the clat app can generate it's own ip address. It looks like you are either usijng dhcpv6 or a single static address. This won't work from what I'm reading.

toreanderson commented

@timriker is correct. Actually, the LIMITATIONS section is a bit misleading and should be updated, as clatd will (since 81f2c61) try to auto-generate a clat-v6-addr for any prefixes /120 or larger - SLAAC or no SLAAC.

Anyway, @Lucaslah: you need to set clat-v6-addr to an IPv6 address that is routed to your host by the surrounding network. It might be that the link does have a prefix routed to it, and if so you might be able to set the address to some adjacent address to the one you already have, such as [removed]::a6. Try it and see?

Lucaslah commented

@timriker is correct. Actually, the LIMITATIONS section is a bit misleading and should be updated, as clatd will (since 81f2c61) try to auto-generate a clat-v6-addr for any prefixes /120 or larger - SLAAC or no SLAAC.

Anyway, @Lucaslah: you need to set clat-v6-addr to an IPv6 address that is routed to your host by the surrounding network. It might be that the link does have a prefix routed to it, and if so you might be able to set the address to some adjacent address to the one you already have, such as [removed]::a6. Try it and see?

Setting the clat-v6-addr allows clatd to start but still cannot access the IPv4 internet

Starting clatd v1.5 by Tore Anderson <tore@fud.no>
Performing DNS64-based PLAT prefix discovery (cf. RFC 7050)
Using PLAT (NAT64) prefix: 64:ff9b::/96
Device facing the PLAT: enp2s0
Using CLAT IPv4 address: 192.0.0.1
Using CLAT IPv6 address: [removed]::a6
Enabling Proxy-ND for [removed]::a6on enp2s0
Creating and configuring up CLAT device 'clat'
Created persistent tun device clat
Adding IPv4 default route via the CLAT
Starting up TAYGA, using config file '/tmp/wD0rfl6_YT'

~/Desktop > ping 1.1.1.1 -I clat
PING 1.1.1.1 (1.1.1.1) from 192.0.0.1 clat: 56(84) bytes of data.
^C
--- 1.1.1.1 ping statistics ---
22 packets transmitted, 0 received, 100% packet loss, time 21486ms

The IP address is allocated via DHCPv6, I don't want to use SLAAC, is there any other way to make this work? (other than allocating a /120 or larger to every client, if that's the only way would prefix delegation work?)

timriker commented

Side note: Android does not support DHCPv6 specifically because it only gives one IP and the xlat code wants another IP. Android will only use SLAAC.
https://issuetracker.google.com/issues/36949085

toreanderson commented

@Lucaslah I got an idea. Could you try the following?

nft "add table ip6 clatd"
nft "add chain ip6 clatd napt66 { type nat hook postrouting priority srcnat; }"
nft "add rule ip6 clatd napt66 iifname clat masquerade"
clatd clat-v6-addr=fd00::c1a7 # or another random address, doesn't really matter
Lucaslah commented

Sure, Ill try it sometime today.

Lucaslah commented

Side note: Android does not support DHCPv6 specifically because it only gives one IP and the xlat code wants another IP. Android will only use SLAAC. https://issuetracker.google.com/issues/36949085

Oh I was unaware of that, just enabled SLAAC on vlans that have mobile devices on them, I just don't want SLAAC on some particular networks.

Lucaslah commented

@Lucaslah I got an idea. Could you try the following?

nft "add table ip6 clatd"
nft "add chain ip6 clatd napt66 { type nat hook postrouting priority srcnat; }"
nft "add rule ip6 clatd napt66 iifname clat masquerade"
clatd clat-v6-addr=fd00::c1a7 # or another random address, doesn't really matter

This worked! thanks so much for your help.

Lucaslah commented

Only one issue, not sure if this is related to clatd or not but on ubuntu building from source this worked fine, however on fedora 36 installing with dnf install clatd clatd started but is still unable to reach the internet on the clat interface.

Edit: still getting the same issue on fedora when installing from source.

toreanderson commented

Great to hear that it worked! You can make this happen automatically by using script-up and script-down, for example by creating an /etc/clatd.conf file containing the following:

clat-v6-addr=fd00::c1a7
proxynd-enable=false
script-up=nft 'add table ip6 clatd; add chain ip6 clatd napt66 { type nat hook postrouting priority srcnat; }; add rule ip6 clatd napt66 iifname clat masquerade'
script-down=nft 'delete table ip6 clatd'

I will consider implementing native support for this in clatd in the future.

  • 👍1
Lucaslah commented

@toreanderson Alright thanks again, any ideas about it not working on fedora?

toreanderson commented

No. Please send a debug log (clatd -d -d [whatever other options you normally use]) from when reproducing the issue.

Lucaslah commented 
eadconf('/etc/clatd.conf')
Configuration successfully read, dumping it:
  clat-dev=clat
  clat-v4-addr=192.0.0.1
  clat-v6-addr=fd00::c1a7
  cmd-ip=ip
  cmd-ip6tables=ip6tables
  cmd-tayga=tayga
  debug=2
  dns64-servers=[removed]
  forwarding-enable=1
  ip6tables-enable=<undefined>
  plat-dev=<undefined>
  plat-prefix=<undefined>
  proxynd-enable=false
  quiet=0
  script-down=nft 'delete table ip6 clatd'
  script-up=nft 'add table ip6 clatd; add chain ip6 clatd napt66 { type nat hook postrouting priority srcnat; }; add rule ip6 clatd napt66 iifname clat masquerade'
  tayga-conffile=<undefined>
  tayga-v4-addr=192.0.0.2
  v4-conncheck-delay=10
  v4-conncheck-enable=false
  v4-defaultroute-advmss=0
  v4-defaultroute-enable=1
  v4-defaultroute-metric=2048
  v4-defaultroute-mtu=1260
  v4-defaultroute-replace=0
Starting clatd v1.6 by Tore Anderson <tore@fud.no>
Performing DNS64-based PLAT prefix discovery (cf. RFC 7050)
cfg(dns64-servers)
Looking up 'ipv4only.arpa' using DNS64 server 2001:4860:4860::6464
check_wka(): Testing to see if 64:ff9b:0:0:0:0:c000:aa was DNS64-synthesised
Looking for Well-Known Addresses at prefix length /96
Looking for WKA ::c000:aa
Found it!
Inferred PLAT prefix 64:ff9b::/96 from AAAA record 64:ff9b:0:0:0:0:c000:aa
check_wka(): Testing to see if 64:ff9b:0:0:0:0:c000:ab was DNS64-synthesised
Looking for Well-Known Addresses at prefix length /96
Looking for WKA ::c000:aa
Didn't find it
Looking for WKA ::c000:ab
Found it!
Inferred PLAT prefix 64:ff9b::/96 from AAAA record 64:ff9b:0:0:0:0:c000:ab
Using PLAT (NAT64) prefix: 64:ff9b::/96
get_plat_dev(): finding which network dev faces the PLAT
cfg(plat-prefix)
cfg(cmd-ip)
get_plat_dev(): Found PLAT-facing device: enp2s0
Device facing the PLAT: enp2s0
Using CLAT IPv4 address: 192.0.0.1
Using CLAT IPv6 address: fd00::c1a7
cfgint(v4-defaultroute-mtu)
cfgbool(v4-conncheck-enable)
Skipping IPv4 connectivity check at user request
cfg(tayga-conffile)
Using temporary conffile for TAYGA: /tmp/cnX1Xan0se
cfg(clat-dev)
cfg(plat-prefix)
cfg(tayga-v4-addr)
cfg(clat-v4-addr)
cfg(clat-v6-addr)
cfgbool(forwarding-enable)
Reading sysctl /proc/sys/net/ipv6/conf/all/forwarding
/proc/sys/net/ipv6/conf/all/forwarding is set to '1'
cfgbool(ip6tables-enable)
cfgbool(proxynd-enable)
cfg(clat-dev)
Creating and configuring up CLAT device 'clat'
cfg(cmd-tayga)
cfg(tayga-conffile)
cfgint(debug)
cmd(tayga --config /tmp/cnX1Xan0se --mktun -d)
Created persistent tun device clat
cfg(cmd-ip)
cfg(clat-dev)
cmd(ip link set up dev clat)
cfg(cmd-ip)
cfg(clat-v4-addr)
cfg(clat-dev)
cmd(ip -4 address add 192.0.0.1 dev clat)
cfg(cmd-ip)
cfg(clat-v6-addr)
cfg(clat-dev)
cmd(ip -6 route add fd00::c1a7 dev clat)
cfgbool(v4-defaultroute-replace)
cfgbool(v4-defaultroute-enable)
cfg(clat-dev)
cfgint(v4-defaultroute-metric)
cfgint(v4-defaultroute-metric)
cfgint(v4-defaultroute-mtu)
cfgint(v4-defaultroute-mtu)
cfgint(v4-defaultroute-advmss)
cfgint(v4-defaultroute-advmss)
Adding IPv4 default route via the CLAT
cfg(cmd-ip)
cmd(ip -4 route add default dev clat metric 2048 mtu 1260 advmss 1220)
Script env: clat-dev=clat
Script env: clat-v4-addr=192.0.0.1
Script env: clat-v6-addr=fd00::c1a7
Script env: cmd-ip=ip
Script env: cmd-ip6tables=ip6tables
Script env: cmd-tayga=tayga
Script env: debug=2
Script env: dns64-servers=[removed]
Script env: forwarding-enable=1
Script env: ip6tables-enable=
Script env: plat-dev=enp2s0
Script env: plat-prefix=64:ff9b::/96
Script env: proxynd-enable=false
Script env: quiet=
Script env: script-down=nft 'delete table ip6 clatd'
Script env: script-up=nft 'add table ip6 clatd; add chain ip6 clatd napt66 { type nat hook postrouting priority srcnat; }; add rule ip6 clatd napt66 iifname clat masquerade'
Script env: tayga-conffile=/tmp/cnX1Xan0se
Script env: tayga-v4-addr=192.0.0.2
Script env: v4-conncheck-delay=10
Script env: v4-conncheck-enable=false
Script env: v4-defaultroute-advmss=1220
Script env: v4-defaultroute-enable=1
Script env: v4-defaultroute-metric=2048
Script env: v4-defaultroute-mtu=1260
Script env: v4-defaultroute-replace=
cfg(script-up)
cfg(script-up)
Running custom startup script: nft 'add table ip6 clatd; add chain ip6 clatd napt66 { type nat hook postrouting priority srcnat; }; add rule ip6 clatd napt66 iifname clat masquerade'
cfg(script-up)
cmd(nft 'add table ip6 clatd; add chain ip6 clatd napt66 { type nat hook postrouting priority srcnat; }; add rule ip6 clatd napt66 iifname clat masquerade')
Starting up TAYGA, using config file '/tmp/cnX1Xan0se'
cfg(cmd-tayga)
cfg(tayga-conffile)
cfgint(debug)
cmd(tayga --config /tmp/cnX1Xan0se --nodetach -d)
starting TAYGA 0.9.2
Using tun device clat with MTU 1500
TAYGA's IPv4 address: 192.0.0.2
TAYGA's IPv6 address: 64:ff9b::c000:2
NAT64 prefix: 64:ff9b::/96
Note: traffic between IPv6 hosts and private IPv4 addresses (i.e. to/from 64:ff9b::10.0.0.0/104, 64:ff9b::192.168.0.0/112, etc) will be dropped.  Use a translation prefix within your organization's IPv6 address space instead of 64:ff9b::/96 if you need your IPv6 hosts to communicate with private IPv4 addresses.
toreanderson commented

This looks pretty normal. The next steps are:

  1. Verify that traffic is going into the clat device as IPv4, passing through the TAYGA translation engine, and coming back out as IPv6. This you can do by running tcpdump -ni clat port 80 in one shell while sending some IPv4 traffic from another (e.g. nc -zv4 github.com 80

  2. Verify that the IPv6 traffic that came out of the clat interface is forwarded out the enp2s0 towards the NAT64 after undergoing NAPT66. Verify that by running another tcpdump -ni enp2s0 port 80 in a third shell while sending the IPv4 traffic.

  3. Verify that the responses come back in the reverse direction. The output from the two tcpdump processes should tell me all of that, so just send me the output from all three commands during a single test.

Lucaslah commented

Here are the outputs, looks like the traffic is not forwarded to ens160 (interface is different because I'm testing in a VM now)

[user@fedora ~]$ sudo tcpdump -ni clat port 80
dropped privs to tcpdump
tcpdump: verbose output suppressed, use -v[v]... for full protocol decode
listening on clat, link-type RAW (Raw IP), snapshot length 262144 bytes
16:38:02.008982 IP 192.0.0.1.58356 > 140.82.113.3.http: Flags [S], seq 569728093, win 64660, options [mss 1220,sackOK,TS val 105218179 ecr 0,nop,wscale 7], length 0
16:38:02.009033 IP6 fd00::c1a7.58356 > 64:ff9b::8c52:7103.http: Flags [S], seq 569728093, win 64660, options [mss 1220,sackOK,TS val 105218179 ecr 0,nop,wscale 7], length 0
16:38:03.045348 IP 192.0.0.1.58356 > 140.82.113.3.http: Flags [S], seq 569728093, win 64660, options [mss 1220,sackOK,TS val 105219215 ecr 0,nop,wscale 7], length 0
16:38:03.045512 IP6 fd00::c1a7.58356 > 64:ff9b::8c52:7103.http: Flags [S], seq 569728093, win 64660, options [mss 1220,sackOK,TS val 105219215 ecr 0,nop,wscale 7], length 0
16:38:05.093428 IP 192.0.0.1.58356 > 140.82.113.3.http: Flags [S], seq 569728093, win 64660, options [mss 1220,sackOK,TS val 105221263 ecr 0,nop,wscale 7], length 0
16:38:05.093710 IP6 fd00::c1a7.58356 > 64:ff9b::8c52:7103.http: Flags [S], seq 569728093, win 64660, options [mss 1220,sackOK,TS val 105221263 ecr 0,nop,wscale 7], length 0
16:38:09.125317 IP 192.0.0.1.58356 > 140.82.113.3.http: Flags [S], seq 569728093, win 64660, options [mss 1220,sackOK,TS val 105225295 ecr 0,nop,wscale 7], length 0
16:38:09.125589 IP6 fd00::c1a7.58356 > 64:ff9b::8c52:7103.http: Flags [S], seq 569728093, win 64660, options [mss 1220,sackOK,TS val 105225295 ecr 0,nop,wscale 7], length 0

[user@fedora ~]$ sudo tcpdump -ni ens160 port 80
dropped privs to tcpdump
tcpdump: verbose output suppressed, use -v[v]... for full protocol decode
listening on ens160, link-type EN10MB (Ethernet), snapshot length 262144 bytes

[user@fedora ~]$ nc -zv4 github.com 80
Ncat: Version 7.93 ( https://nmap.org/ncat )
Ncat: TIMEOUT.
toreanderson commented

Interesting. Can you (while clatd is running), verify the state of the IPv6 forwarding sysctls with grep . /proc/sys/net/ipv6/conf/*/forwarding? If enabled (they should be), do you have some kind of firewall running that would block this traffic? Check nft list ruleset and/or ip6tables-save.

Lucaslah commented

Output of grep . /proc/sys/net/ipv6/conf/*/forwarding

/proc/sys/net/ipv6/conf/all/forwarding:1
/proc/sys/net/ipv6/conf/clat/forwarding:1
/proc/sys/net/ipv6/conf/default/forwarding:1
/proc/sys/net/ipv6/conf/ens160/forwarding:1
/proc/sys/net/ipv6/conf/lo/forwarding:1

Output of nft list ruleset

[user@fedora ~]$ sudo nft list ruleset
[sudo] password for user: 
table inet firewalld {
	ct helper helper-netbios-ns-udp {
		type "netbios-ns" protocol udp
		l3proto ip
	}

	chain mangle_PREROUTING {
		type filter hook prerouting priority mangle + 10; policy accept;
		jump mangle_PREROUTING_ZONES
	}

	chain mangle_PREROUTING_POLICIES_pre {
		jump mangle_PRE_policy_allow-host-ipv6
	}

	chain mangle_PREROUTING_ZONES {
		iifname "ens160" goto mangle_PRE_FedoraWorkstation
		goto mangle_PRE_FedoraWorkstation
	}

	chain mangle_PREROUTING_POLICIES_post {
	}

	chain nat_PREROUTING {
		type nat hook prerouting priority dstnat + 10; policy accept;
		jump nat_PREROUTING_ZONES
	}

	chain nat_PREROUTING_POLICIES_pre {
		jump nat_PRE_policy_allow-host-ipv6
	}

	chain nat_PREROUTING_ZONES {
		iifname "ens160" goto nat_PRE_FedoraWorkstation
		goto nat_PRE_FedoraWorkstation
	}

	chain nat_PREROUTING_POLICIES_post {
	}

	chain nat_POSTROUTING {
		type nat hook postrouting priority srcnat + 10; policy accept;
		jump nat_POSTROUTING_ZONES
	}

	chain nat_POSTROUTING_POLICIES_pre {
	}

	chain nat_POSTROUTING_ZONES {
		oifname "ens160" goto nat_POST_FedoraWorkstation
		goto nat_POST_FedoraWorkstation
	}

	chain nat_POSTROUTING_POLICIES_post {
	}

	chain nat_OUTPUT {
		type nat hook output priority -90; policy accept;
		jump nat_OUTPUT_POLICIES_pre
		jump nat_OUTPUT_POLICIES_post
	}

	chain nat_OUTPUT_POLICIES_pre {
	}

	chain nat_OUTPUT_POLICIES_post {
	}

	chain filter_PREROUTING {
		type filter hook prerouting priority filter + 10; policy accept;
		icmpv6 type { nd-router-advert, nd-neighbor-solicit } accept
		meta nfproto ipv6 fib saddr . mark . iif oif missing drop
	}

	chain filter_INPUT {
		type filter hook input priority filter + 10; policy accept;
		ct state { established, related } accept
		ct status dnat accept
		iifname "lo" accept
		ct state invalid drop
		jump filter_INPUT_ZONES
		reject with icmpx admin-prohibited
	}

	chain filter_FORWARD {
		type filter hook forward priority filter + 10; policy accept;
		ct state { established, related } accept
		ct status dnat accept
		iifname "lo" accept
		ct state invalid drop
		ip6 daddr { ::/96, ::ffff:0.0.0.0/96, 2002::/24, 2002:a00::/24, 2002:7f00::/24, 2002:a9fe::/32, 2002:ac10::/28, 2002:c0a8::/32, 2002:e000::/19 } reject with icmpv6 addr-unreachable
		jump filter_FORWARD_ZONES
		reject with icmpx admin-prohibited
	}

	chain filter_OUTPUT {
		type filter hook output priority filter + 10; policy accept;
		ct state { established, related } accept
		oifname "lo" accept
		ip6 daddr { ::/96, ::ffff:0.0.0.0/96, 2002::/24, 2002:a00::/24, 2002:7f00::/24, 2002:a9fe::/32, 2002:ac10::/28, 2002:c0a8::/32, 2002:e000::/19 } reject with icmpv6 addr-unreachable
		jump filter_OUTPUT_POLICIES_pre
		jump filter_OUTPUT_POLICIES_post
	}

	chain filter_INPUT_POLICIES_pre {
		jump filter_IN_policy_allow-host-ipv6
	}

	chain filter_INPUT_ZONES {
		iifname "ens160" goto filter_IN_FedoraWorkstation
		goto filter_IN_FedoraWorkstation
	}

	chain filter_INPUT_POLICIES_post {
	}

	chain filter_FORWARD_POLICIES_pre {
	}

	chain filter_FORWARD_ZONES {
		iifname "ens160" goto filter_FWD_FedoraWorkstation
		goto filter_FWD_FedoraWorkstation
	}

	chain filter_FORWARD_POLICIES_post {
	}

	chain filter_OUTPUT_POLICIES_pre {
	}

	chain filter_OUTPUT_POLICIES_post {
	}

	chain filter_IN_FedoraWorkstation {
		jump filter_INPUT_POLICIES_pre
		jump filter_IN_FedoraWorkstation_pre
		jump filter_IN_FedoraWorkstation_log
		jump filter_IN_FedoraWorkstation_deny
		jump filter_IN_FedoraWorkstation_allow
		jump filter_IN_FedoraWorkstation_post
		jump filter_INPUT_POLICIES_post
		meta l4proto { icmp, ipv6-icmp } accept
		reject with icmpx admin-prohibited
	}

	chain filter_IN_FedoraWorkstation_pre {
	}

	chain filter_IN_FedoraWorkstation_log {
	}

	chain filter_IN_FedoraWorkstation_deny {
	}

	chain filter_IN_FedoraWorkstation_allow {
		ip6 daddr fe80::/64 udp dport 546 ct state { new, untracked } accept
		tcp dport 22 ct state { new, untracked } accept
		udp dport 137 ct helper set "helper-netbios-ns-udp"
		udp dport 137 ct state { new, untracked } accept
		udp dport 138 ct state { new, untracked } accept
		ip daddr 224.0.0.251 udp dport 5353 ct state { new, untracked } accept
		ip6 daddr ff02::fb udp dport 5353 ct state { new, untracked } accept
		udp dport 1025-65535 ct state { new, untracked } accept
		tcp dport 1025-65535 ct state { new, untracked } accept
	}

	chain filter_IN_FedoraWorkstation_post {
	}

	chain nat_POST_FedoraWorkstation {
		jump nat_POSTROUTING_POLICIES_pre
		jump nat_POST_FedoraWorkstation_pre
		jump nat_POST_FedoraWorkstation_log
		jump nat_POST_FedoraWorkstation_deny
		jump nat_POST_FedoraWorkstation_allow
		jump nat_POST_FedoraWorkstation_post
		jump nat_POSTROUTING_POLICIES_post
	}

	chain nat_POST_FedoraWorkstation_pre {
	}

	chain nat_POST_FedoraWorkstation_log {
	}

	chain nat_POST_FedoraWorkstation_deny {
	}

	chain nat_POST_FedoraWorkstation_allow {
	}

	chain nat_POST_FedoraWorkstation_post {
	}

	chain filter_FWD_FedoraWorkstation {
		jump filter_FORWARD_POLICIES_pre
		jump filter_FWD_FedoraWorkstation_pre
		jump filter_FWD_FedoraWorkstation_log
		jump filter_FWD_FedoraWorkstation_deny
		jump filter_FWD_FedoraWorkstation_allow
		jump filter_FWD_FedoraWorkstation_post
		jump filter_FORWARD_POLICIES_post
		reject with icmpx admin-prohibited
	}

	chain filter_FWD_FedoraWorkstation_pre {
	}

	chain filter_FWD_FedoraWorkstation_log {
	}

	chain filter_FWD_FedoraWorkstation_deny {
	}

	chain filter_FWD_FedoraWorkstation_allow {
	}

	chain filter_FWD_FedoraWorkstation_post {
	}

	chain nat_PRE_FedoraWorkstation {
		jump nat_PREROUTING_POLICIES_pre
		jump nat_PRE_FedoraWorkstation_pre
		jump nat_PRE_FedoraWorkstation_log
		jump nat_PRE_FedoraWorkstation_deny
		jump nat_PRE_FedoraWorkstation_allow
		jump nat_PRE_FedoraWorkstation_post
		jump nat_PREROUTING_POLICIES_post
	}

	chain nat_PRE_FedoraWorkstation_pre {
	}

	chain nat_PRE_FedoraWorkstation_log {
	}

	chain nat_PRE_FedoraWorkstation_deny {
	}

	chain nat_PRE_FedoraWorkstation_allow {
	}

	chain nat_PRE_FedoraWorkstation_post {
	}

	chain mangle_PRE_FedoraWorkstation {
		jump mangle_PREROUTING_POLICIES_pre
		jump mangle_PRE_FedoraWorkstation_pre
		jump mangle_PRE_FedoraWorkstation_log
		jump mangle_PRE_FedoraWorkstation_deny
		jump mangle_PRE_FedoraWorkstation_allow
		jump mangle_PRE_FedoraWorkstation_post
		jump mangle_PREROUTING_POLICIES_post
	}

	chain mangle_PRE_FedoraWorkstation_pre {
	}

	chain mangle_PRE_FedoraWorkstation_log {
	}

	chain mangle_PRE_FedoraWorkstation_deny {
	}

	chain mangle_PRE_FedoraWorkstation_allow {
	}

	chain mangle_PRE_FedoraWorkstation_post {
	}

	chain filter_IN_policy_allow-host-ipv6 {
		jump filter_IN_policy_allow-host-ipv6_pre
		jump filter_IN_policy_allow-host-ipv6_log
		jump filter_IN_policy_allow-host-ipv6_deny
		jump filter_IN_policy_allow-host-ipv6_allow
		jump filter_IN_policy_allow-host-ipv6_post
	}

	chain filter_IN_policy_allow-host-ipv6_pre {
	}

	chain filter_IN_policy_allow-host-ipv6_log {
	}

	chain filter_IN_policy_allow-host-ipv6_deny {
	}

	chain filter_IN_policy_allow-host-ipv6_allow {
		icmpv6 type nd-neighbor-advert accept
		icmpv6 type nd-neighbor-solicit accept
		icmpv6 type nd-router-advert accept
		icmpv6 type nd-redirect accept
	}

	chain filter_IN_policy_allow-host-ipv6_post {
	}

	chain nat_PRE_policy_allow-host-ipv6 {
		jump nat_PRE_policy_allow-host-ipv6_pre
		jump nat_PRE_policy_allow-host-ipv6_log
		jump nat_PRE_policy_allow-host-ipv6_deny
		jump nat_PRE_policy_allow-host-ipv6_allow
		jump nat_PRE_policy_allow-host-ipv6_post
	}

	chain nat_PRE_policy_allow-host-ipv6_pre {
	}

	chain nat_PRE_policy_allow-host-ipv6_log {
	}

	chain nat_PRE_policy_allow-host-ipv6_deny {
	}

	chain nat_PRE_policy_allow-host-ipv6_allow {
	}

	chain nat_PRE_policy_allow-host-ipv6_post {
	}

	chain mangle_PRE_policy_allow-host-ipv6 {
		jump mangle_PRE_policy_allow-host-ipv6_pre
		jump mangle_PRE_policy_allow-host-ipv6_log
		jump mangle_PRE_policy_allow-host-ipv6_deny
		jump mangle_PRE_policy_allow-host-ipv6_allow
		jump mangle_PRE_policy_allow-host-ipv6_post
	}

	chain mangle_PRE_policy_allow-host-ipv6_pre {
	}

	chain mangle_PRE_policy_allow-host-ipv6_log {
	}

	chain mangle_PRE_policy_allow-host-ipv6_deny {
	}

	chain mangle_PRE_policy_allow-host-ipv6_allow {
	}

	chain mangle_PRE_policy_allow-host-ipv6_post {
	}
}
table ip6 clatd {
	chain napt66 {
		type nat hook postrouting priority srcnat; policy accept;
		iifname "clat" masquerade
	}
}
toreanderson commented

Looks to me that filter_FORWARD rejects all packets. Does it work if you stop firewalld? No output from ip6tables-save? (clatd should have added the necessary rules there)

Lucaslah commented

Yeah ip6tables-save returns nothing, Ill try stopping firewalld

Lucaslah commented

As soon as I stopped firewalld it started working.

toreanderson commented

Could you try with ip6tables-enable=true in the configuration file? I think what is happening is that you did not have the ip6table_filter kernel module loaded, so clatd skips adding the rules. This is based on an assumption that with no ip6table_filter, all forwarded traffic is permitted by default. However, that's not necessarily true since nft became a thing.

ip6tables-enable=true should force clatd to add the rules no matter what. (Note that you running ip6table-save might have loaded the ip6table_filter module, so you might want to test after a reboot to make sure it works in all cases.)

Lucaslah commented

After adding ip6tables-enable=true and rebooting, I still cannot access the internet on the clat interface unless I disable firewalld

user@fedora ~]$ sudo ip6tables-save
[sudo] password for user: 
# Generated by ip6tables-save v1.8.7 on Tue Mar 14 09:57:53 2023
*filter
:INPUT ACCEPT [0:0]
:FORWARD ACCEPT [0:0]
:OUTPUT ACCEPT [0:0]
-A FORWARD -i ens160 -o clat -j ACCEPT
-A FORWARD -i clat -o ens160 -j ACCEPT
COMMIT
# Completed on Tue Mar 14 09:57:53 2023
toreanderson commented

Ok, I guess that ip6tables and nft aren't really compatible with each other. You can try getting around that by setting ip6tables-enable=false (to not bother with ip6tables at all) and instead try the following when clatd (and firewalld) is running:

nft 'add chain ip6 clatd forwarding { type filter hook forward priority filter; }; add rule ip6 clatd forwarding iifname "clatd" accept; add rule ip6 clatd forwarding oifname "clatd" accept'

If that works, you can add these commands to your clatd.conf so that it reads:

script-up=nft 'add table ip6 clatd; add chain ip6 clatd napt66 { type nat hook postrouting priority srcnat; }; add rule ip6 clatd napt66 iifname clat masquerade; add chain ip6 clatd forwarding { type filter hook forward priority filter; }; add rule ip6 clatd forwarding iifname "clat" accept; add rule ip6 clatd forwarding oifname "clat" accept'

Edit: it might be that you need to increase the priority in order to short-circuit firewalld. If it doesn't work, try again with priority 120 and see if that is better. Let me know how it goes!

Lucaslah commented

Set ip6tables-enable=false in /etc/clatd.conf and then started firewalld and clatd, and then ran nft 'add chain ip6 clatd forwarding { type filter hook forward priority filter; }; add rule ip6 clatd forwarding iifname "clatd" accept; add rule ip6 clatd forwarding oifname "clatd" accept' still cannot access internet on the clat interface.

Where would I add priority 120 to?

toreanderson commented

Argh, I made a mistake, sorry. It should have been iifname "clat" and oifname "clat" (not clat*d*). That is:

nft 'add chain ip6 clatd forwarding { type filter hook forward priority filter; }; add rule ip6 clatd forwarding iifname "clat" accept; add rule ip6 clatd forwarding oifname "clat" accept'

If that doesn't work, delete what you have with nft 'delete table ip6 clatd' and try again with an increased priority:

nft 'add chain ip6 clatd forwarding { type filter hook forward priority 120; }; add rule ip6 clatd forwarding iifname "clat" accept; add rule ip6 clatd forwarding oifname "clat" accept'
Lucaslah commented

Unfortunately I tried with both and still cannot get it to work.

toreanderson commented

It is possible that it can't work. I am not sure, but it might be that the reject with icmpx admin-prohibited rule in firewalld's chain filter_FORWARD cannot be overridden by other nft tables. If so you would need to reconfigure firewalld to allow that traffic. If you find out how, I would be interested in knowing about it.

thpryrchn commented

Great to hear that it worked! You can make this happen automatically by using script-up and script-down, for example by creating an /etc/clatd.conf file containing the following:

clat-v6-addr=fd00::c1a7
proxynd-enable=false
script-up=nft 'add table ip6 clatd; add chain ip6 clatd napt66 { type nat hook postrouting priority srcnat; }; add rule ip6 clatd napt66 iifname clat masquerade'
script-down=nft 'delete table ip6 clatd'

I will consider implementing native support for this in clatd in the future.

This did not work for me... I don't have any firewall installed. Im in Debian 12..

But I did find something that does!

clat-v6-addr=fd00::c1a7
proxynd-enable=false
script-up=nft "add table ip6 clatd" ; nft "add chain ip6 clatd POSTROUTING { type nat hook postrouting priority srcnat; }" ; nft "add rule ip6 clatd POSTROUTING masquerade"
script-down=nft 'delete table ip6 clatd'
toreanderson commented

This did not work for me... I don't have any firewall installed. Im in Debian 12..

But I did find something that does!

clat-v6-addr=fd00::c1a7
proxynd-enable=false
script-up=nft "add table ip6 clatd" ; nft "add chain ip6 clatd POSTROUTING { type nat hook postrouting priority srcnat; }" ; nft "add rule ip6 clatd POSTROUTING masquerade"
script-down=nft 'delete table ip6 clatd'

Interesting. The only actual difference between the two, as far as I can tell, is that you do not match on iifname clat. The other differences, namely that you call your chain something else than I did (POSTROUTING vs napt66), and that you inject the rules using multiple invocations of nft instead of a single one, should not make any difference.

Have you renamed the CLAT device by setting clat-dev to something else than its default value clat? That would explain why it didn't work for you.