Add `placement` as a regular config option and not only for user_options
isuftin opened this issue · 7 comments
As with other options we can choose for configuring the swarm deployment, I feel as though placement should be available as a configuration option not only to be set through user_options.
An example of what I mean...
As a configuration level option, it currently does not work:
jupyterhub_config.py
[...]
c.JupyterHub.spawner_class = 'cassinyspawner.SwarmSpawner'
c.SwarmSpawner.placement = ["node.role == worker"]
[...]
Launch logs:
jupyterhub_jupyterhub.1.m9kw855bogr9@manager | Adding user testuser
jupyterhub_jupyterhub.1.m9kw855bogr9@manager | [I 2017-08-11 15:38:18.479 JupyterHub app:724] Loading cookie_secret from /srv/jupyterhub/jupyterhub_cookie_secret
jupyterhub_jupyterhub.1.m9kw855bogr9@manager | [I 2017-08-11 15:38:18.545 JupyterHub app:892] Not using whitelist. Any authenticated user will be allowed.
jupyterhub_jupyterhub.1.m9kw855bogr9@manager | [W 2017-08-11 15:38:18.560 JupyterHub configurable:168] Config option `placement` not recognized by `SwarmSpawner`.
jupyterhub_jupyterhub.1.m9kw855bogr9@manager | [I 2017-08-11 15:38:18.636 JupyterHub swarmspawner:229] Docker service 'jupyter-90613d4f8204b906ead85556f075fd71-1' is gone
jupyterhub_jupyterhub.1.m9kw855bogr9@manager | [W 2017-08-11 15:38:18.636 JupyterHub swarmspawner:194] Docker service not found
jupyterhub_jupyterhub.1.m9kw855bogr9@manager | [W 2017-08-11 15:38:18.640 JupyterHub configurable:168] Config option `placement` not recognized by `SwarmSpawner`.
jupyterhub_jupyterhub.1.m9kw855bogr9@manager | [I 2017-08-11 15:38:18.644 JupyterHub swarmspawner:229] Docker service 'jupyter-2f4c77557cd6c786d40e488583d77e79-1' is gone
jupyterhub_jupyterhub.1.m9kw855bogr9@manager | [W 2017-08-11 15:38:18.645 JupyterHub swarmspawner:194] Docker service not found
jupyterhub_jupyterhub.1.m9kw855bogr9@manager | [I 2017-08-11 15:38:18.652 JupyterHub app:1453] Hub API listening on http://0.0.0.0:8081/hub/
jupyterhub_jupyterhub.1.m9kw855bogr9@manager | [I 2017-08-11 15:38:18.655 JupyterHub app:1176] Starting proxy @ http://0.0.0.0:443/
jupyterhub_jupyterhub.1.m9kw855bogr9@manager | 15:38:18.859 - info: [ConfigProxy] Proxying https://0.0.0.0:443 to http://127.0.0.1:8081
jupyterhub_jupyterhub.1.m9kw855bogr9@manager | 15:38:18.864 - info: [ConfigProxy] Proxy API at http://127.0.0.1:444/api/routes
jupyterhub_jupyterhub.1.m9kw855bogr9@manager | [I 2017-08-11 15:38:18.964 JupyterHub app:1485] JupyterHub is now running at http://127.0.0.1:443/
But it does work if I add it under user_options and set to use user_options to true
jupyterhub_config.py
[...]
c.JupyterHub.spawner_class = 'cassinyspawner.SwarmSpawner'
# c.SwarmSpawner.placement = ["node.role == worker"]
c.SwarmSpawner.use_user_options = True
user_options = {
'placement' : ["node.role == worker"]
}
[...]
Launch logs:
jupyterhub_jupyterhub.1.879snisqard1@manager | Adding user testuser
jupyterhub_jupyterhub.1.879snisqard1@manager | [I 2017-08-11 15:47:19.836 JupyterHub app:724] Loading cookie_secret from /srv/jupyterhub/jupyterhub_cookie_secret
jupyterhub_jupyterhub.1.879snisqard1@manager | [I 2017-08-11 15:47:19.893 JupyterHub app:892] Not using whitelist. Any authenticated user will be allowed.
jupyterhub_jupyterhub.1.879snisqard1@manager | [I 2017-08-11 15:47:19.973 JupyterHub swarmspawner:229] Docker service 'jupyter-90613d4f8204b906ead85556f075fd71-1' is gone
jupyterhub_jupyterhub.1.879snisqard1@manager | [W 2017-08-11 15:47:19.973 JupyterHub swarmspawner:194] Docker service not found
jupyterhub_jupyterhub.1.879snisqard1@manager | [W 2017-08-11 15:47:19.975 JupyterHub app:1097] isuftin not running.
jupyterhub_jupyterhub.1.879snisqard1@manager | [I 2017-08-11 15:47:19.983 JupyterHub swarmspawner:229] Docker service 'jupyter-2f4c77557cd6c786d40e488583d77e79-1' is gone
jupyterhub_jupyterhub.1.879snisqard1@manager | [W 2017-08-11 15:47:19.984 JupyterHub swarmspawner:194] Docker service not found
jupyterhub_jupyterhub.1.879snisqard1@manager | [I 2017-08-11 15:47:19.996 JupyterHub app:1453] Hub API listening on http://0.0.0.0:8081/hub/
jupyterhub_jupyterhub.1.879snisqard1@manager | [I 2017-08-11 15:47:19.999 JupyterHub app:1176] Starting proxy @ http://0.0.0.0:443/
jupyterhub_jupyterhub.1.879snisqard1@manager | 15:47:20.208 - info: [ConfigProxy] Proxying https://0.0.0.0:443 to http://127.0.0.1:8081
jupyterhub_jupyterhub.1.879snisqard1@manager | 15:47:20.213 - info: [ConfigProxy] Proxy API at http://127.0.0.1:444/api/routes
Hi @isuftin , how can you say that it works?
Because you miss this line in the logs Config option "placement" not recognized by "SwarmSpawner"?
@barrachri The line you refer to is when the configuration uses the placement as a top-level configuration option instead of adding it into the user_options dictionary. When used as a top-level configuration option is when DockerSpawner complains. My feature request is to have it added as a top-level configuration option.
Unless you're saying that even when used in the user_options dictionary, it doesn't work?
Sorry, your statement was slightly difficult to read.
Hi @isuftin sorry, my message was a bit unclear.
But you got my point:
Unless you're saying that even when used in the user_options dictionary, it doesn't work?
DockerSpawner complains because placement is not recognised as valid option, in your second example the option is not just used at all, it's just a dict inside the config.
I think it's easy to implement this feature, would like to submit a PR?
@barrachri I can try to submit a PR once I have a little time to do so. I hope to do so this week.
The PR is in and I've tested it as working. With the following set in jupyterhub_config.py:
c.SwarmSpawner.placement = ["node.role == worker"]
I can inspect the launched notebook process via docker inspect <service id> and I can see:
"Spec": {
"Name": "jupyter-5d9c68c6c50ed3d02a2fcf54f63993b6-1",
"Labels": {},
"TaskTemplate": {
"ContainerSpec": {
[...]
"Placement": {
"Constraints": [
"node.role == worker"
]
}
[...]
$ docker node inspect worker1 | grep worker
"Role": "worker",
"Hostname": "worker1",
$ docker service ls
ID NAME MODE REPLICAS IMAGE PORTS
5ub2qq4lt1ho registry_registry replicated 1/1 registry:2.6.2 *:5000->5000/tcp
kvm3thjufitn jupyterhub_jupyterhub replicated 1/1 localhost:5000/dlcc/jupyterhub:latest *:443->443/tcp
u4waxtqq1t3r jupyter-5d9c68c6c50ed3d02a2fcf54f63993b6-1 replicated 1/1 localhost:5000/dlcc/jupyter-singleuser:latest
$ docker service ps u4
ID NAME IMAGE NODE DESIRED STATE CURRENT STATE ERROR PORTS
wysn1oijgmdp jupyter-5d9c68c6c50ed3d02a2fcf54f63993b6-1.1 localhost:5000/dlcc/jupyter-singleuser:latest worker1 Running Running 9 minutes ago
Working as expected!
Thanks!
merged.