Provide a version without EXPOSEd ports
Closed this issue · 11 comments
See moby/moby#3465 - Due to Dockerfile limitations it's not possible to remove exposed ports when extending a base image.
It would be nice to have a Docker image specifically designed for building more complex ones, without the default port 80 exposed, e.g. removing this line https://github.com/nginxinc/docker-nginx/blob/97b65112180e0c7764465aa47a974fc7af3c99ae/mainline/stretch/Dockerfile#L95
I don't think this sole change warrants another variant of the image for all the possible targets...
@thresheek wrote:
I don't think this sole change warrants another variant of the image for all the possible targets...
As a relative newbie to Docker and Nginx, I'm probably coming from a position of ignorance. So please forgive me. However, I'm now discovering that I have the same issue as @dluc, and I believe he and I are representative of other's that aren't posting here. There's a documented performance hit with Docker 'bridge' networks vs 'host' networks, and combining the Dockerfile limitation referenced above with this image exposing port 80, this means we can only run 1 instance of Nginx per physical/virtual host in this configuration.
I realize that this issue has an impact however it's addressed (creating a new variant, or just removing/commenting 'EXPOSE 80' entirely). But the spirit of the request, however it's addressed, adds functionality overall.
So I get that it will have a downstream impact either way.
-
New Variant with 'EXPOSE 80' Removed
This creates another build to manage, adding additional cycles and overhead to maintaining Nginx overall. And you're right, it does seem like a very minor change. -
Remove/Comment 'EXPOSE 80' from the Dockerfile
I assume that this will probably impact a metric 'butt-ton' of users that use this image. Although, and this is where my newbie-ish-ness definitely comes into play. Of this 'butt-ton' group, there's some percentage (which I assume to also be quite large), that won't be affected at all and won't even realize that they may have been affected, since they are using one of several other mechanisms to expose the port:- 'docker run -p' to expose the port manually.
[At least, this worked when I tested it with the 'helloworld' container on the Docker docs 'Get Started' with '#EXPOSE 80' and 'docker run -p 4000:80 friendlyhello'] - 'docker-compose up & docker-compose.yml' to expose the port manually.
[This worked when I tested it with the Dockerfile for 'nginxinc/docker-nginx:stable-alpine' with '#EXPOSE 80' as my internal Nginx reverse proxy. Admittedly, port 80 was still exposed, but I was using port 443 which was never exposed in the Dockerfile, only the docker-compose.yml file.] - 'docker stack deploy -c & docker-compose.yml' to expose the port manually.
[I'm assuming this will work the same as #1, but in fairness, I haven't tested it.]
- 'docker run -p' to expose the port manually.
And let's say "hey, newbie, it simply doesn't work like in your 1 - 3 examples, and many many people will in fact be impacted." I'd point out that those people have a remedy (the ability to expose the port) that other people (represented by @dluc and I) don't have. And to add to that, deprecating 'EXPOSE 80' can be communicated and scheduled such that the first group can accommodate the change, and everyone ends up with more configuration options.
To me, as a self admitted Docker/Nginx beginner, the removal of 'EXPOSE 80' seems like a valid request, even if creating a new variant is not the preferred way to go.
Also as I'm learning, all input, even critical, is welcome. Thank you for your time.
EDIT: I forgot to mention that as there seems to be a general industry push to HTTPS and HSTS domains, it seems that the inclusion of 'EXPOSE 80' over 'EXPOSE 443' seems, to a beginner point of view, rather arbitrary. Yes, defaulting to exposing port 80 gives functionality out of the gate. However, to open the door to that functionality, another door has to be closed to certain configurations. If the underlying components (Docker) are architected to allow a one way flow for exposing ports, it seems that leaving this entirely up to the users would be the preferred way to go. Otherwise, the only option is for users that require port 80 not be exposed is to prefer downloading and modifying the Dockerfile here rather than just pulling the image and going on about their merry way.
@RussellNS
The "EXPOSE" directive does not actually publish the port. That's up to you (with -p on cli, docker-compose, docker stack or any other orchestration platform)
See https://docs.docker.com/engine/reference/builder/#expose
So it is possible to run multiple such containers on the same host machine. You can publish their container ports if you want to different host ports.
There is no need to remove this line from the base image.
Thank you for your reply! What you said is absolutely true for multiple containers running on the Docker 'bridge' network. However, for containers running on the Docker 'host' network, the container takes on the IP of the host machine, and uses all ports exposed in the Dockerfile (published ports are ignored).
-
See: https://docs.docker.com/network/host/
Notable quote: "If you use the host network mode for a container, that container’s network stack is not isolated from the Docker host (the container shares the host’s networking namespace), and the container does not get its own IP-address allocated. For instance, if you run a container which binds to port 80 and you use host networking, the container’s application is available on port 80 on the host’s IP address.
Note: Given that the container does not have its own IP-address when using host mode networking, port-mapping does not take effect, and the -p, --publish, -P, and --publish-all option are ignored, producing a warning instead:
WARNING: Published ports are discarded when using host network mode" -
See also: https://success.docker.com/article/networking
Notable quote: "Typically with other networking drivers, each container is placed in its own network namespace (or sandbox) to provide complete network isolation from each other. With the host driver containers are all in the same host network namespace and use the network interfaces and IP stack of the host. All containers in the host network are able to communicate with each other on the host interfaces. From a networking standpoint this is equivalent to multiple processes running on a host without containers. Because they are using the same host interfaces, no two containers are able to bind to the same TCP port. This may cause port contention if multiple containers are being scheduled on the same host."
This means that the Nginx image currently cannot run multiple containers on the same host on the 'host' network. I do run a personal Nginx reverse proxy and multiple websites (portainer and whatnot). This is why I'm just now discovering this issue.
And after testing it with the actual, current Nginx Dockerfile published here by commenting '#EXPOSE 80', I found that the resulting image/container still works perfectly well. Publishing port 80 on a Docker 'Bridge' network still works perfectly well. Also, any port(s) used inside the container (80, 443, whatever) end up being exposed automatigically on Docker 'host' networks.
So, and again, I'm new, so I'm not challenging, but it seems like (to a newbie) the exact opposite of your statement is more accurate: There's no reason not to remove this line from the base image.
Hm, ok. Thanks for your explanation. But I have a few questions:
- Why do you want to run a nginx container without exposing its port? Where is the usecase for a non-accessible webserver?
- Any reason why using the host-net? (especially if "port conflicts" can occur in your env)
In my opinion, it is still good to declare this port by default (at least for documentation). I think that almost all user will try to access the server this way and need the port. There are always special cases where the "official" base image is not suitable for (and that's not the purpose of those images).
The main task of a webserver is to serve content to someone outside, which makes it necessary to be acessible. And with this in mind, I think it is legitimate to make this the default behavior.
You're hitting on something that I'm (very recently) learning myself.
- Why do you want to run a nginx container without exposing its port? Where is the usecase for a non-accessible webserver?
Even with 'EXPOSE 80' commented out, the webserver is still accessible via port 80 (or port 443, or any other unexposed, user published port). Here, try it:
- Create a directory named 'nginx_test'.
- Create a 'Dockerfile' that contains the Stable Alpine version of Nginx Dockerfile: https://github.com/nginxinc/docker-nginx/blob/master/stable/alpine/Dockerfile.
- Comment the 'EXPOSE 80' line ('#EXPOSE 80').
- Build this Docker image that has no ports exposed ('docker build --tag=nginx_test .').
- Run a container from this image ('docker run --rm -dp 4000:80 nginx_test')
- Verify that the container is running and the port is published->exposed ('docker container ls').
- Open a web browser, and navigate to that container ('http://docker_host_ip:4000').
Voila! It runs just fine. I've also tested this using Docker Compose.
- Stop your 'nginx_test' container.
- Delete the 'nginx_test' image.
- Create a 'docker-compose.yml' file that contains:
version: '3.7'
services:
web:
build:
context: .
dockerfile: ./Dockerfile
container_name: nginx_test
ports:
- 4000:80
- Run this service ('docker-compose up -d')
- Verify that the container is running and the port is published->exposed ('docker container ls').
- Open a web browser, and navigate to that container ('http://docker_host_ip:4000').
Voila again. It runs fine.
So my answer to this question is, you're right in that there is no use case for a non-accessible webserver. But that's not the case here. And there's definitely use cases whereby user's would gain from the flexibility of being able to assign all their own ports, and only their own ports. Because of the forward nature of exposing ports in Dockerfile (there's simply no way to unexpose a port), the current Nginx Dockerfile doesn't allow. But discussing use cases is a great segue into...
- Any reason why using the host-net? (especially if "port conflicts" can occur in your env)
The Docker 'host' network is not the only use case. It just happens to by my use case. @dluc has one where he'd like to build more complex Docker images based on this one. He didn't go into that much, but it's still a valid use case.
My use case is performance. There's a performance hit when using Docker 'bridge' vs 'host' networks. See: https://www.google.com/search?q=docker+host+bridge+performance. I looked for a link from 'docker.com', but I couldn't find one. So I'll link just the first result: https://jtway.co/docker-network-performance-b95bce32b4b9. This gentleman used 'iperf3' and found that there's a 20% hit in network throughput when using Docker 'bridge' networks over Docker 'host' networks.
I deal with files that are upwards of 30-ish GB, and there's a MikroTik 10Gb switch/router that's less expensive than high end wireless routers for the home. My storage is not optimally configured, so I'm only getting about 400MB/s throughput. On Docker 'bridge' networks, this means I get about 320MB/s. I know, oh boo-hoo me. However, I'm not the only user that needs to optimize for network performance. Nginx has a huge corporate presence, and there are plenty of corporate use cases that need the same optimization (and self admittedly, many more that don't).
In my opinion, it is still good to declare this port by default (at least for documentation). I think that almost all user will try to access the server this way and need the port. There are always special cases where the "official" base image is not suitable for (and that's not the purpose of those images).
I completely agree with the desire to declare/document this port. And I think that commenting '#EXPOSE 80' would be better than removing it. For that matter, adding an additional port '#EXPOSE 80 443' would, in my opinion be even better still, for the same reasons.
As for "official" base images not being suitable for all purposes, yeah, that's part of my journey of discovery. As I said, I'm new to Docker and Nginx. So I've just grabbed the Dockerfile for the stable Alpine version, commented the port myself, and have built off that. I'm not stopped, in that sense. However, I have to now maintain my own Dockerfile based on the updates of the Nginx team so that other people can presumably have functionality that they aren't being prevented from either way. I get that this "primary" group is large, and my "secondary" group is smaller. I also get that there may be edge cases within this "primary" group that may be impacted if this line in the base image were commented. However, those edge case people, I believe, represent a smaller group than the "secondary" group. As well, they still have a pretty easy remedy of:
Dockerfile
FROM: nginx:stable-alpine
EXPOSE 80
Boom, done. Their pulls are always up to date, and guarantee that port 80 is exposed. The group I'm in don't have that option. We have to maintain a Dockerfile in parallel with the Nginx team here.
The main task of a webserver is to serve content to someone outside, which makes it necessary to be acessible. And with this in mind, I think it is legitimate to make this the default behavior.
I also agree that the main task of a web server is to serve content to someone outside. However, with this line commented, it is still accessible. And that's why I feel that it's a legitimate request to comment the line.
I hope I'm not being too "wordy" or belaboring a point. And I do welcome more responses (thank you @code-chris), especially critical ones. I have an issue that parallels @dluc, and found his after searching. I'd post a separate issue, but I'm new, and I don't know if it's frowned upon, to champion someone elses input as your own. So I'd like @thresheek to reconsider, and re-open this issue.
Now that I'm re-reading the full original post, @dluc's original post title is clearly asking for a new variant, and I guess I'm advocating for a change in the base image. Would opening a new issue invite more discussion, and be the better way to go? Am I doing this wrong to champion for @dluc's issue to be reopened rather than just creating a new issue myself?
EXPOSE doesn't have an effect on --net=host or in how the container can be run. A port listed in EXPOSE does not mean that it will be listening there. See also https://github.com/docker-library/docker/ where we have two ports and only use one of them by default.
net=host removes all docker networking from the container and gives it direct access to the host network devices. The nginx daemon will listen on whatever port you put in the config (and the provided default is 80). Unless you are changing the listen in the nginx config, you cannot run two of them with net=host since they will be using the same port on the same network devices.
$ docker run -d --name ngn1 --net=host nginx
0cc388f8c5a9a38fbe10d760ca4b14da24f9f64766732c2aca7d23ed31938e9b
$ $ curl localhost
...
<h1>Welcome to nginx!</h1>
<p>If you see this page, the nginx web server is successfully installed and
working. Further configuration is required.</p>
...
$ docker run -it --rm --name ngn2 --net=host nginx
2019/08/09 20:51:12 [emerg] 1#1: bind() to 0.0.0.0:80 failed (98: Address already in use)
nginx: [emerg] bind() to 0.0.0.0:80 failed (98: Address already in use)
2019/08/09 20:51:12 [emerg] 1#1: bind() to 0.0.0.0:80 failed (98: Address already in use)
nginx: [emerg] bind() to 0.0.0.0:80 failed (98: Address already in use)
2019/08/09 20:51:12 [emerg] 1#1: bind() to 0.0.0.0:80 failed (98: Address already in use)
nginx: [emerg] bind() to 0.0.0.0:80 failed (98: Address already in use)
2019/08/09 20:51:12 [emerg] 1#1: bind() to 0.0.0.0:80 failed (98: Address already in use)
nginx: [emerg] bind() to 0.0.0.0:80 failed (98: Address already in use)
2019/08/09 20:51:12 [emerg] 1#1: bind() to 0.0.0.0:80 failed (98: Address already in use)
nginx: [emerg] bind() to 0.0.0.0:80 failed (98: Address already in use)
2019/08/09 20:51:12 [emerg] 1#1: still could not bind()
nginx: [emerg] still could not bind()
$ # this is nginx failing to bind the already in-use port and not docker's "expose"@yosifkit
Wow, I can't believe I didn't think to test this before posting. Sure enough, I re-ran the tests I had posted here but with the Dockerfile using 'EXPOSE 80' uncommented, and in my own words...
"Voila. It runs just fine."
Thank you for that insight! That pretty much enables me entirely, and completely invalidates the reason for me asking for any form of change.
Again, thank you to all who contributed!
@yosifkit some web hosting services scan (nginx) docker images definitions, looking for the port where the web content is available, and automatically open and route web requests to the first port available, which is port 80 in all nginx images.
It's a usability feature that makes very easy to deploy dockerized web content and apps, without having to teach users about TCP ports, firewalls, etc. Users in this space most of the times don't know about root privileges or don't care at all.
On the other hand, advanced users transfer all their content using ports above 1024, to avoid the need for root privileges within the docker image, and have to deal with those hosting services which try to use port 80 by default.
The fact that neither moby (moby/moby#3465) nor nginx recognize this problem leads to insecure setups and waste of effort to design and maintain workarounds.
@dluc This is a similar case, the VSCode Docker will auto scan available ports in image and open it. So if I has multiple nginx image instance, even I EXPOSE difference port, these ones still has EXPOSE 80. So will cause conflicts for VSCode Docker.
Just wondering if anyone already under maintenance non EXPOSE version of nginx image and can provide?