network_mode fails with a large number of ports
cholick opened this issue · 4 comments
cholick commented
Description
In version 4.34.0, host networking support for mac went GA. This feature starts failing with a large number of ports (maxing out at 18 consistently using the example in this issue). It fails a bit inconsistently. In testing with our actual project, once we pass a certain threshold, all ports stop responding from the host (but the services still respond testing from inside the container). When creating a minimal test case to file an issue, more often what I see is that a subset of the ports stop responding from the host. I was able to reproduce this same behavior on a colleague's computer.
Reproduce
- Run compose file specified below. The threshold where things start to have problems seems to be around ~20
docker compose --file compose-bug.yaml up
- See that not all containers are responding. You could do this manually, but this is a the script I used:
import http.client for i in range(1, 21): port = 8000 + i conn = http.client.HTTPConnection("localhost", port) try: conn.request("GET", "/") response = conn.getresponse() print(f"Response from localhost:{port}: {response.status}") except Exception as e: print(f"Error connecting to localhost:{port}: {e}") finally: conn.close()
Here's some sample output:
Response from localhost:8001: 200
Response from localhost:8002: 200
Response from localhost:8003: 200
Error connecting to localhost:8004: [Errno 61] Connection refused
Response from localhost:8005: 200
Response from localhost:8006: 200
Response from localhost:8007: 200
Response from localhost:8008: 200
Response from localhost:8009: 200
Response from localhost:8010: 200
Response from localhost:8011: 200
Response from localhost:8012: 200
Response from localhost:8013: 200
Error connecting to localhost:8014: [Errno 61] Connection refused
Response from localhost:8015: 200
Response from localhost:8016: 200
Response from localhost:8017: 200
Response from localhost:8018: 200
Response from localhost:8019: 200
Response from localhost:8020: 200
network_mode didn't work for 4 and 14 here, but other runs would result in different failures. They do work from the inside:
root@docker-desktop:/# curl -I localhost:8004
HTTP/1.0 200 OK
...
Compose file:
services:
"1":
image: python:latest
network_mode: host
command: ["/usr/local/bin/python3", "-m", "http.server", "8001"]
"2":
image: python:latest
network_mode: host
command: ["/usr/local/bin/python3", "-m", "http.server", "8002"]
"3":
image: python:latest
network_mode: host
command: ["/usr/local/bin/python3", "-m", "http.server", "8003"]
"4":
image: python:latest
network_mode: host
command: ["/usr/local/bin/python3", "-m", "http.server", "8004"]
"5":
image: python:latest
network_mode: host
command: ["/usr/local/bin/python3", "-m", "http.server", "8005"]
"6":
image: python:latest
network_mode: host
command: ["/usr/local/bin/python3", "-m", "http.server", "8006"]
"7":
image: python:latest
network_mode: host
command: ["/usr/local/bin/python3", "-m", "http.server", "8007"]
"8":
image: python:latest
network_mode: host
command: ["/usr/local/bin/python3", "-m", "http.server", "8008"]
"9":
image: python:latest
network_mode: host
command: ["/usr/local/bin/python3", "-m", "http.server", "8009"]
"10":
image: python:latest
network_mode: host
command: ["/usr/local/bin/python3", "-m", "http.server", "8010"]
"11":
image: python:latest
network_mode: host
command: ["/usr/local/bin/python3", "-m", "http.server", "8011"]
"12":
image: python:latest
network_mode: host
command: ["/usr/local/bin/python3", "-m", "http.server", "8012"]
"13":
image: python:latest
network_mode: host
command: ["/usr/local/bin/python3", "-m", "http.server", "8013"]
"14":
image: python:latest
network_mode: host
command: ["/usr/local/bin/python3", "-m", "http.server", "8014"]
"15":
image: python:latest
network_mode: host
command: ["/usr/local/bin/python3", "-m", "http.server", "8015"]
"16":
image: python:latest
network_mode: host
command: ["/usr/local/bin/python3", "-m", "http.server", "8016"]
"17":
image: python:latest
network_mode: host
command: ["/usr/local/bin/python3", "-m", "http.server", "8017"]
"18":
image: python:latest
network_mode: host
command: ["/usr/local/bin/python3", "-m", "http.server", "8018"]
"19":
image: python:latest
network_mode: host
command: ["/usr/local/bin/python3", "-m", "http.server", "8019"]
"20":
image: python:latest
network_mode: host
command: ["/usr/local/bin/python3", "-m", "http.server", "8020"]
Expected behavior
network_mode works for all ports when a large number are used, not just a subset.
docker version
cholick@cholick-work-mbpro ~/workspace/github-scalar-tuesday/src $ docker version
Client:
Version: 27.2.0
API version: 1.47
Go version: go1.21.13
Git commit: 3ab4256
Built: Tue Aug 27 14:14:45 2024
OS/Arch: darwin/amd64
Context: desktop-linux
Server: Docker Desktop 4.34.2 (167172)
Engine:
Version: 27.2.0
API version: 1.47 (minimum version 1.24)
Go version: go1.21.13
Git commit: 3ab5c7d
Built: Tue Aug 27 14:15:15 2024
OS/Arch: linux/amd64
Experimental: false
containerd:
Version: 1.7.20
GitCommit: 8fc6bcff51318944179630522a095cc9dbf9f353
runc:
Version: 1.1.13
GitCommit: v1.1.13-0-g58aa920
docker-init:
Version: 0.19.0
GitCommit: de40ad0
docker info
Client:
Version: 27.2.0
Context: desktop-linux
Debug Mode: false
Plugins:
buildx: Docker Buildx (Docker Inc.)
Version: v0.16.2-desktop.1
Path: /Users/cholick/.docker/cli-plugins/docker-buildx
compose: Docker Compose (Docker Inc.)
Version: v2.29.2-desktop.2
Path: /Users/cholick/.docker/cli-plugins/docker-compose
debug: Get a shell into any image or container (Docker Inc.)
Version: 0.0.34
Path: /Users/cholick/.docker/cli-plugins/docker-debug
desktop: Docker Desktop commands (Alpha) (Docker Inc.)
Version: v0.0.15
Path: /Users/cholick/.docker/cli-plugins/docker-desktop
dev: Docker Dev Environments (Docker Inc.)
Version: v0.1.2
Path: /Users/cholick/.docker/cli-plugins/docker-dev
extension: Manages Docker extensions (Docker Inc.)
Version: v0.2.25
Path: /Users/cholick/.docker/cli-plugins/docker-extension
feedback: Provide feedback, right in your terminal! (Docker Inc.)
Version: v1.0.5
Path: /Users/cholick/.docker/cli-plugins/docker-feedback
init: Creates Docker-related starter files for your project (Docker Inc.)
Version: v1.3.0
Path: /Users/cholick/.docker/cli-plugins/docker-init
sbom: View the packaged-based Software Bill Of Materials (SBOM) for an image (Anchore Inc.)
Version: 0.6.0
Path: /Users/cholick/.docker/cli-plugins/docker-sbom
scout: Docker Scout (Docker Inc.)
Version: v1.13.0
Path: /Users/cholick/.docker/cli-plugins/docker-scout
Server:
Containers: 55
Running: 20
Paused: 0
Stopped: 35
Images: 27
Server Version: 27.2.0
Storage Driver: overlayfs
driver-type: io.containerd.snapshotter.v1
Logging Driver: json-file
Cgroup Driver: cgroupfs
Cgroup Version: 2
Plugins:
Volume: local
Network: bridge host ipvlan macvlan null overlay
Log: awslogs fluentd gcplogs gelf journald json-file local splunk syslog
Swarm: inactive
Runtimes: io.containerd.runc.v2 runc
Default Runtime: runc
Init Binary: docker-init
containerd version: 8fc6bcff51318944179630522a095cc9dbf9f353
runc version: v1.1.13-0-g58aa920
init version: de40ad0
Security Options:
seccomp
Profile: unconfined
cgroupns
Kernel Version: 6.10.4-linuxkit
Operating System: Docker Desktop
OSType: linux
Architecture: x86_64
CPUs: 14
Total Memory: 54.92GiB
Name: docker-desktop
ID: c17df932-c0a7-48a8-ab99-ffae416396f3
Docker Root Dir: /var/lib/docker
Debug Mode: false
HTTP Proxy: http.docker.internal:3128
HTTPS Proxy: http.docker.internal:3128
No Proxy: hubproxy.docker.internal
Labels:
com.docker.desktop.address=unix:///Users/cholick/Library/Containers/com.docker.docker/Data/docker-cli.sock
Experimental: false
Insecure Registries:
hubproxy.docker.internal:5555
127.0.0.0/8
Live Restore Enabled: false
WARNING: daemon is not using the default seccomp profile
Diagnostics ID
Skipped, this repros in colleagues computers as well, not specific to my install
Additional Info
No response