SwarmSpawner does not create the c.SwarmSpawner.network_name on the worker node

[xNok]

Bug description

In manager to setup jupyterhub. If no worker has joined the swarm them everything is fine the container is spawned and works properly. As soon as I add a worker then the process fail.

I am using portainer on the side, I can see that portainer successfully create its agent on the worker and monitors it. Thanks to the portainer agent I can see that the container (jupyterlab) is successfully created and running on the worker, but jupyterhub seems not able to communicate with the newly created container so it destroys it after a couple of retries.

I looked at the worker node and it seems that the network defined in the jupyterhub_config.py is not created on the worker node so jupyterhub is not able to communicate with the jupytherlab container.

[I 2020-11-03 02:07:57.761 JupyterHub dockerspawner:1009] Starting service jupyter-qwe (id: 6p4q27d)
[I 2020-11-03 02:07:58.375 JupyterHub log:181] 302 GET /hub/spawn/qwe -> /hub/spawn-pending/qwe (qwe@10.0.0.2) 1019.53ms
[I 2020-11-03 02:07:58.502 JupyterHub pages:398] qwe is pending spawn
[I 2020-11-03 02:07:58.508 JupyterHub log:181] 200 GET /hub/spawn-pending/qwe (qwe@10.0.0.2) 19.76ms
[E 2020-11-03 02:08:02.625 JupyterHub user:690] Unhandled error starting qwe's server: Service jupyter-qwe not found
[W 2020-11-03 02:08:02.648 JupyterHub swarmspawner:128] Service jupyter-qwe not found

Your personal set up

• OS: lunix debian
• Version: latest ->
• Configuration:

version: '3'

services:
  # Configuration for Hub+Proxy
  jupyterhub:
    build: jupyterhub                # Build the container from this folder.
    image: jupyterhub_img
    # This is necessary to prevent the singleton hub from using its service number as its hostname
    hostname: jupyterhub
    volumes:                         # Give access to Docker socket.
      - /var/run/docker.sock:/var/run/docker.sock
      - jupyterhub_data:/srv/jupyterhub
    environment:                     # Env variables passed to the Hub process.
      DOCKER_JUPYTER_IMAGE: jupyterlab_img
      DOCKER_NETWORK_NAME: jupyterhub_jupyterhub_network
    networks:
      - jupyterhub_network
    deploy:
      replicas: 1
      placement:
        constraints:
          - node.role == manager

  # Configuration for the single-user servers
  jupyterlab:
    build: jupyterlab
    image: jupyterlab_img
    command: echo
    deploy:
      replicas: 0

volumes:
  jupyterhub_data:

networks:
  jupyterhub_network:
    driver: overlay

import os, sys

# We start with the configuration of the spawner:
# we use the class DockerSpawner to spawn single-user servers in a separate Docker container.
#  We use here the environment variables that we have set in docker-compose.yml:
c.JupyterHub.hub_ip = '0.0.0.0'
c.JupyterHub.spawner_class = 'dockerspawner.SwarmSpawner'
c.DockerSpawner.image = os.environ['DOCKER_JUPYTER_IMAGE']

# Network configurations
# https://jupyterhub-dockerspawner.readthedocs.io/en/latest/spawner-types.html#swarmspawner
network_name = os.environ['DOCKER_NETWORK_NAME']
c.SwarmSpawner.network_name = network_name

# Optionally, we may want to stop the single-user servers after a certain amount of idle time. Following this example, we register a JupyterHub service like this:
c.JupyterHub.services = [
    {
        'name': 'cull-idle',
        'admin': True,
        'command': [sys.executable, 'cull_idle_servers.py', '--timeout=3600'],
    }
]

# Redirect to JupyterLab, instead of the plain Jupyter notebook
c.Spawner.default_url = '/lab'

#The quickest way to get a JupyterHub server running with a working authentication, is to delegate to an authentication service such as Github.
# from oauthenticator.github import GitHubOAuthenticator
# c.JupyterHub.authenticator_class = GitHubOAuthenticator

# for local testing
c.JupyterHub.authenticator_class = 'dummyauthenticator.DummyAuthenticator'
c.DummyAuthenticator.password = "your strong password"

# user data persistence
# see https://github.com/jupyterhub/dockerspawner#data-persistence-and-dockerspawner
# Explicitly set notebook directory because we'll be mounting a host volume to
# it.  Most jupyter/docker-stacks *-notebook images run the Notebook server as
# user `jovyan`, and set the notebook directory to `/home/jovyan/work`.
# We follow the same convention.
# notebook_dir = os.environ.get('DOCKER_NOTEBOOK_DIR') or '/home/jovyan/work'
# c.SwarmSpawner.notebook_dir = notebook_dir

# Mount the real user's Docker volume on the host to the notebook user's
# notebook directory in the container
# c.SwarmSpawner.volumes = { 'jupyterhub-user-{username}': notebook_dir }

[welcome]

Thank you for opening your first issue in this project! Engagement like this is essential for open source projects! 🤗

If you haven't done so already, check out Jupyter's Code of Conduct. Also, please try to follow the issue template as it helps other other community members to contribute more effectively.

You can meet the other Jovyans by joining our Discourse forum. There is also an intro thread there where you can stop by and say Hi! 👋

Welcome to the Jupyter community! 🎉

[minrk]

Looks like a typo here:

DOCKER_NETWORK_NAME: jupyterhub_jupyterhub_network

where your network name that you create below is just jupyterhub_network. These two should match.