Support publishing container ports to the same "host" port on different VIC container networks
alatdev opened this issue · 5 comments
Hi,
I'm currently trying out vmware vic v1.3.1.
So I installed one vic-appliance and one VCH with a container-network to directly talk to the containers without NATing.
I've tried to scale out a container to have a second instance running, but in Admiral i get the error "No compute hosts found with unexposed ports".
I thought - since I have a container-network, each container is getting his own IP, and in the template for the container I've selected Network Mode "None" to have no bridged network - it would be possible.
For infos see attachments.
docker_inspect.txt
docker_info.txt
@alatdev looking through your docker inspect, it looks like you have published container port 8080 to host port 8080. You are right that each container gets its own IP address and can map the same port, however, you can publish only one of these instances to 8080 on the host machine. Possible solutions for your case:
- in the
Networktab of theEnter additional detailsscreen for container provisioning, leave the host port field blank and only fill 8080 in the container port field. This will publish container port 8080 to a random host port for each container instance. You will be able to scale up a given container and also provision multiple instances of the same template. You will also be able to access each of the provisioned instances on the host IP address by using the published ports (these can be checked in the UI or via the docker CLI) - if you do not need to be able to access the containers on the IP address of the host machine and can reach them on their internal addresses (i.e. you have access to the container network they are connected to), you can just not publish any ports in the Network tab. User-created networks (at least in standard docker) are bridges and all ports of all containers are accessible on that network.
By the way, setting the network mode is actually supposed to completely toggle off the networking for the container. It doesn't make a lot of sense to connect networks to such a container. This probably the reason why your container is not in none networking mode (according to your docker inspect). Note that it will be in none mode on standard docker host but will still be connected to the specified network.
Please leave a reply if you need further assistance.
@shadjiiski since Container Networks are special to VMware VIC, i thought it would be possible that each container with a Container Network can have the same port mapping - and that the usual network restrictions of standard Docker aren't valid.
@alatdev have you confirmed that this is the case with the docker CLI against a VCH? The only environment that I have handy right now is a bit broken, but on it I get:
- for normal containers that use the bridge network
$ docker --tls run -itd -p 8080:8080 alpine
16d8c7af2f87692e52ac21a2b493ae29dfd84651cc1c6649b4881578be73424e
$ docker --tls run -itd -p 8080:8080 alpine
4db76cf4929b5d7ab0a216cc04323c8905a18e3d5a0973fccb21586f617d1f83
C:\Program Files\Docker Toolbox\docker.exe: Error response from daemon: Server error from portlayer: error mapping ports: port 8080 is not available.
- for containers connected to a container network (
some-networkis a port group that was made available as a container network during the creation of the VCH)
$ docker --tls run -itd --network some-network -p 8081:8081 alpine
48385eaa6fc7743679bd0bb72efa3e6e5c1a58955da5c95f38288b8f6eb3e32b
C:\Program Files\Docker Toolbox\docker.exe: Error response from daemon: Server error from portlayer: unable to wait for process launch status: Post https://x.x.x.x/sdk: context deadline exceeded.
$ docker --tls run -itd --network some-network -p 8082 alpine
496be1d114bedf7b2ce3cf65fafafb35f8002949ab29de51c73f2f5c1da7e935
C:\Program Files\Docker Toolbox\docker.exe: Error response from daemon: Server error from portlayer: unable to wait for process launch status: Post https://x.x.x.x/sdk: context deadline exceeded.
From this, I would conclude that it is not possible to map the same port on the default bridge network and that it is actually not possible to publish ports at all while being connected to a container network only.
@shadjiiski I've tested your commands in my environment:
- containers using bridge network
docker run -itd -p 1234:80 hello-world
5cbbfb495209fc5cab69b36019e2c45a5fd473cb5492613c84489ec47a19e35f
docker run -itd -p 1234:80 hello-world
146a5676db2ed16d89bfe45d02c52412b28ab41914bb710973cb50be56ca2274
docker: Error response from daemon: Server error from portlayer: error mapping ports: port 1234 is not available.
- container using container-network 10.10.x.x made during creation of the VCH
docker run -itd --network 10.10.x.x -p 1234:80 hello-world
47e2152cfffbf426f26fcf55a2aae504f957ded1ae961155e0c507bd24b3546c
docker run -itd --network 10.10.x.x -p 1234:80 hello-world
ed8705a901ed334e9d7642d0608def26acaa52d8ef6868303f49fd262acfc450
docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
ed8705a901ed hello-world "/bin/sh -c /run.sh" About a minute ago Up Less than a second 10.10.2.231:1234->80/tcp keen_wiles
47e2152cfffb hello-world "/bin/sh -c /run.sh" 2 minutes ago Up 39 seconds 10.10.2.230:1234->80/tcp boring_bhaskara
As you can see in docker ps, each container got his own IP. So it look like this is an Admiral-Bug, per CLI it work's.
@alatdev, thanks for your investigation and also thanks for bringing our attention to this gap in functionality. Originally, the networking features in Admiral were developed with plain old Docker in mind because VIC networking was not available back then. It seems that we have missed to cover this specific case for container networks in VIC, so I am marking the issue as an enhancement. Until this is resolved, please use the workarounds provided in one of my previous comments or alternatively specify distinct "host" ports for your containers.