unable to cluster over vlan sub-interface
Closed this issue · 1 comments
baycarbone commented
I'm trying to setup a microcloud cluster by picking a vlan sub-interface for the cluster internal traffic, but although network connectivity is fine (nodes can ping each other for example), the microcloud snap is not able to discover the other nodes.
When taking traces on the initiating node and the other nodes, I can see the MDNS requests arriving on the other nodes, but no reply is sent back.
Here is the microcloud init output:
ubuntu@micro-node10:~$ sudo microcloud init
Waiting for LXD to start...
Select an address for MicroCloud's internal traffic:
Space to select; enter to confirm; type to filter results.
Up/down to move; right to select all; left to select none.
+--------------+---------+
| ADDRESS | IFACE |
+--------------+---------+
[ ] | 172.27.41.34 | enp0s25 |
[ ] | 10.102.0.4 | ext.102 |
[ ] | 10.104.0.4 | ext.104 |
> [x] | 10.103.0.4 | ext.103 |
[ ] | 10.101.0.4 | ext.101 |
+--------------+---------+
Here is the network config on 2 nodes:
ubuntu@micro-node10:~$ ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
valid_lft forever preferred_lft forever
inet6 ::1/128 scope host
valid_lft forever preferred_lft forever
2: enp0s25: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel state UP group default qlen 1000
link/ether b8:ae:ed:7b:f3:42 brd ff:ff:ff:ff:ff:ff
inet 172.27.41.34/23 brd 172.27.41.255 scope global enp0s25
valid_lft forever preferred_lft forever
inet6 fe80::baae:edff:fe7b:f342/64 scope link
valid_lft forever preferred_lft forever
3: ext: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel state UP group default qlen 1000
link/ether 8c:ae:4c:df:53:48 brd ff:ff:ff:ff:ff:ff
inet6 fe80::8eae:4cff:fedf:5348/64 scope link
valid_lft forever preferred_lft forever
4: ext.100@ext: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
link/ether 8c:ae:4c:df:53:48 brd ff:ff:ff:ff:ff:ff
inet6 fe80::8eae:4cff:fedf:5348/64 scope link
valid_lft forever preferred_lft forever
5: ext.102@ext: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
link/ether 8c:ae:4c:df:53:48 brd ff:ff:ff:ff:ff:ff
inet 10.102.0.4/24 brd 10.102.0.255 scope global ext.102
valid_lft forever preferred_lft forever
inet6 fe80::8eae:4cff:fedf:5348/64 scope link
valid_lft forever preferred_lft forever
6: ext.104@ext: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
link/ether 8c:ae:4c:df:53:48 brd ff:ff:ff:ff:ff:ff
inet 10.104.0.4/24 brd 10.104.0.255 scope global ext.104
valid_lft forever preferred_lft forever
inet6 fe80::8eae:4cff:fedf:5348/64 scope link
valid_lft forever preferred_lft forever
7: ext.103@ext: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
link/ether 8c:ae:4c:df:53:48 brd ff:ff:ff:ff:ff:ff
inet 10.103.0.4/24 brd 10.103.0.255 scope global ext.103
valid_lft forever preferred_lft forever
inet6 fe80::8eae:4cff:fedf:5348/64 scope link
valid_lft forever preferred_lft forever
8: ext.101@ext: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
link/ether 8c:ae:4c:df:53:48 brd ff:ff:ff:ff:ff:ff
inet 10.101.0.4/24 brd 10.101.0.255 scope global ext.101
valid_lft forever preferred_lft forever
inet6 fe80::8eae:4cff:fedf:5348/64 scope link
valid_lft forever preferred_lft forever
ubuntu@micro-node01:~$ ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
valid_lft forever preferred_lft forever
inet6 ::1/128 scope host
valid_lft forever preferred_lft forever
2: enp0s25: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel state UP group default qlen 1000
link/ether b8:ae:ed:7b:f3:2a brd ff:ff:ff:ff:ff:ff
inet 172.27.41.0/23 brd 172.27.41.255 scope global enp0s25
valid_lft forever preferred_lft forever
inet6 fe80::baae:edff:fe7b:f32a/64 scope link
valid_lft forever preferred_lft forever
3: ext: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel state UP group default qlen 1000
link/ether 8c:ae:4c:fd:a6:3b brd ff:ff:ff:ff:ff:ff
inet6 fe80::8eae:4cff:fefd:a63b/64 scope link
valid_lft forever preferred_lft forever
4: ext.101@ext: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
link/ether 8c:ae:4c:fd:a6:3b brd ff:ff:ff:ff:ff:ff
inet 10.101.0.6/24 brd 10.101.0.255 scope global ext.101
valid_lft forever preferred_lft forever
inet6 fe80::8eae:4cff:fefd:a63b/64 scope link
valid_lft forever preferred_lft forever
5: ext.100@ext: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
link/ether 8c:ae:4c:fd:a6:3b brd ff:ff:ff:ff:ff:ff
inet6 fe80::8eae:4cff:fefd:a63b/64 scope link
valid_lft forever preferred_lft forever
6: ext.104@ext: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
link/ether 8c:ae:4c:fd:a6:3b brd ff:ff:ff:ff:ff:ff
inet 10.104.0.6/24 brd 10.104.0.255 scope global ext.104
valid_lft forever preferred_lft forever
inet6 fe80::8eae:4cff:fefd:a63b/64 scope link
valid_lft forever preferred_lft forever
7: ext.102@ext: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
link/ether 8c:ae:4c:fd:a6:3b brd ff:ff:ff:ff:ff:ff
inet 10.102.0.6/24 brd 10.102.0.255 scope global ext.102
valid_lft forever preferred_lft forever
inet6 fe80::8eae:4cff:fefd:a63b/64 scope link
valid_lft forever preferred_lft forever
8: ext.103@ext: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
link/ether 8c:ae:4c:fd:a6:3b brd ff:ff:ff:ff:ff:ff
inet 10.103.0.6/24 brd 10.103.0.255 scope global ext.103
valid_lft forever preferred_lft forever
inet6 fe80::8eae:4cff:fefd:a63b/64 scope link
valid_lft forever preferred_lft forever
I've also attached 2 pcap files, where node10 is the one initiating the clustering.
traces.zip
When doing the same using the physical interfaces i.e. ext in the example above, it works fine.
tomponline commented
Should be fixed by #213