multiple containers on same host, on same network as host, accessible from outside host
Closed this issue · 3 comments
like others, I am trying to use containers to implement multiple instances of smaller applications.
the application has multiple parts and assumes that it can use the callers IP address to open a socket on a different port back to the callers system. so the builtin Ip address won't work..
I just don't understand what the host side of this network connection is.
the host has 192.168.2.33 as address fro dhcp server. the containers will be in that same address space 192.168.2.x.
if I use pipework ?? container dhcp, I get eventually and ip address but no default route.
technically it should be '192.168.2.33' I think, as that is the only way to get out of the container anywhere..
but there is no default route.
if I do it manually (no dhcp), it sets up what I tell it, 192.168.2.51@192.168.2.33, with default route thru 33, but it doesn't work.
I assume the host side is not setup properly..
as I might have 20 containers on this system, I don't want to consume 40 IP addresses. only need 1 for each container.
eth0 is the .33 address. it is not a bridge.
if I specify pipework eth0 nothing changes on the host side, but nothing works
if I specify pipework br0 I get a new interface on the host, but nothing works.
ubuntu 14.04, docker 1.6.2, latest pipework loaded today
commands and results
sudo pipework eth0 $(docker run -d --name web larsks/simpleweb) 192.168.2.51/24@192.168.2.33
sam@buildserver:$ docker exec web ip addr$ docker exec web ip route
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
valid_lft forever preferred_lft forever
inet6 ::1/128 scope host
valid_lft forever preferred_lft forever
75: eth0: <BROADCAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default
link/ether 02:42:ac:11:00:13 brd ff:ff:ff:ff:ff:ff
inet 172.17.0.19/16 scope global eth0
valid_lft forever preferred_lft forever
inet6 fe80::42:acff:fe11:13/64 scope link
valid_lft forever preferred_lft forever
77: eth1@if2: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UNKNOWN group default
link/ether 46:14:58:21:74:77 brd ff:ff:ff:ff:ff:ff
inet 192.168.2.51/24 scope global eth1
valid_lft forever preferred_lft forever
inet6 fe80::4414:58ff:fe21:7477/64 scope link
valid_lft forever preferred_lft forever
sam@buildserver:
default via 192.168.2.33 dev eth1
172.17.0.0/16 dev eth0 proto kernel scope link src 172.17.0.19
192.168.2.0/24 dev eth1 proto kernel scope link src 192.168.2.51
host info after pipework command
sam@buildserver:~$ sudo ifconfig
docker0 Link encap:Ethernet HWaddr 56:84:7a:fe:97:99
inet addr:172.17.42.1 Bcast:0.0.0.0 Mask:255.255.0.0
inet6 addr: fe80::5484:7aff:fefe:9799/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:544 errors:0 dropped:0 overruns:0 frame:0
TX packets:362 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:0
RX bytes:47188 (47.1 KB) TX bytes:34921 (34.9 KB)
eth0 Link encap:Ethernet HWaddr 1c:c1: de:50:8e:cb
inet addr:192.168.2.33 Bcast:192.168.2.255 Mask:255.255.255.0
inet6 addr: fe80::1ec1:deff:fe50:8ecb/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:152966 errors:0 dropped:2575 overruns:0 frame:0
TX packets:59295 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:121662946 (121.6 MB) TX bytes:4266361 (4.2 MB)
sam@buildserver:~$ sudo ip route
default via 192.168.2.1 dev eth0 proto static
172.17.0.0/16 dev docker0 proto kernel scope link src 172.17.42.1
192.168.2.0/24 dev eth0 proto kernel scope link src 192.168.2.33 metric 1
sam@buildserver:~$ sudo ip addr
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
valid_lft forever preferred_lft forever
inet6 ::1/128 scope host
valid_lft forever preferred_lft forever
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP group default qlen 1000
link/ether 1c:c1: de:50:8e:cb brd ff:ff:ff:ff:ff:ff
inet 192.168.2.33/24 brd 192.168.2.255 scope global eth0
valid_lft forever preferred_lft forever
inet6 fe80::1ec1:deff:fe50:8ecb/64 scope link
valid_lft forever preferred_lft forever
with dhcp
sudo pipework eth0 $(docker run -d --name web larsks/simpleweb) dhcp
sam@buildserver:$ docker exec web ip routesam@buildserver:$ docker exec web ip addr
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
valid_lft forever preferred_lft forever
inet6 ::1/128 scope host
valid_lft forever preferred_lft forever
78: eth0: <BROADCAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default
link/ether 02:42:ac:11:00:14 brd ff:ff:ff:ff:ff:ff
inet 172.17.0.20/16 scope global eth0
valid_lft forever preferred_lft forever
inet6 fe80::42:acff:fe11:14/64 scope link
valid_lft forever preferred_lft forever
80: eth1@if2: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UNKNOWN group default
link/ether 2e:30:c9:73:ff:86 brd ff:ff:ff:ff:ff:ff
inet 192.168.2.108/24 brd 192.168.2.255 scope global eth1
valid_lft forever preferred_lft forever
inet6 fe80::2c30:c9ff:fe73:ff86/64 scope link
valid_lft forever preferred_lft forever
default via 172.17.42.1 dev eth0
172.17.0.0/16 dev eth0 proto kernel scope link src 172.17.0.20
192.168.2.0/24 dev eth1 proto kernel scope link src 192.168.2.108
sam@buildserver:~$ docker exec web ip addr
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
valid_lft forever preferred_lft forever
inet6 ::1/128 scope host
valid_lft forever preferred_lft forever
78: eth0: <BROADCAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default
link/ether 02:42:ac:11:00:14 brd ff:ff:ff:ff:ff:ff
inet 172.17.0.20/16 scope global eth0
valid_lft forever preferred_lft forever
inet6 fe80::42:acff:fe11:14/64 scope link
valid_lft forever preferred_lft forever
80: eth1@if2: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UNKNOWN group default
link/ether 2e:30:c9:73:ff:86 brd ff:ff:ff:ff:ff:ff
inet 192.168.2.108/24 brd 192.168.2.255 scope global eth1
valid_lft forever preferred_lft forever
inet6 fe80::2c30:c9ff:fe73:ff86/64 scope link
valid_lft forever preferred_lft forever
using udhcpc on the host now gets the default route from dhcp added to the container..
the route gets the dhcp server supplied route.. 192.168.2.1, and the container can ping that address, and anything else on the network,
and I can get to the container on the lan from other stations, but not the local host.. so this means it must be a macvlan..
also, had to use --net=none on starting the container, otherwise the normal network setup using docker0 is still created and that default route in the container is left intact.
using the code from the doc for the macvlan reconfig on the host solves my issues.. will close this
how can I detect if the default route device is a macvlan based device?
I want to be able to setup the macvlan if needed before starting containers.
ip link show
results in
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN mode DEFAULT group default
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP mode DEFAULT group default qlen 1000
link/ether 1c:c1: de:50:8e:cb brd ff:ff:ff:ff:ff:ff
3: wlan0: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc mq state DOWN mode DORMANT group default qlen 1000
link/ether 70:f1:a1:a7:00:82 brd ff:ff:ff:ff:ff:ff
146: eth0m@eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UNKNOWN mode DEFAULT group default
link/ether 9e:ad:55:89:91:ff brd ff:ff:ff:ff:ff:ff
ip route
results in
default via 192.168.2.1 dev eth0m
192.168.2.0/24 dev eth0m proto kernel scope link src 192.168.2.33
so, it looks like if the interface starts with the default route device (eth0m) and contains an @
then it might be a macvlan?