Feature: docker swarm automatically download image when not available on a node
Closed this issue · 5 comments
Hello,
When using docker swarm, docker swarm itself will download the image when an image is not available on a node. While the Shinyproxy current behaviour is that an image on a public repo(no auth) will succeed but when on a private repo(need auth) will fail.
I will create a Pull Request for this feature. Please review and see if it can be accepted. The code need be changed on both containerproxy and shinyproxy.
Pull request
TL;DR
in containerproxy source code, it creates a service without authentication, and it is the root cause.
String serviceId = dockerClient.createService(serviceSpecBuilder.build()).id();
to fix that, it needs to pass authentication info like this
serviceId = dockerClient.createService(serviceSpecBuilder.build(), registryAuth).id();
To better function for shinyproxy, I am using three new container settings(container-auth-domain, container-auth-user and container-auth-password) in application.xml to give users more flexibility and fewer steps compare to use docker default configuration ~/.docker/config.json. (It will hit a bug if use docker default configuration which jersey-common version is not the same as the docker-client's dependency. org.glassfish.jersey.internal.util.Base64 package is removed from version 2.30 which the end output from shinyproxy jar)
For testing
if you want to skip java building phase, the jar file can be downloaded from here
the application.xml can be something like this.
proxy:
title: ShinyProxy Test
port: 8080
container-wait-time: 60000
authentication: simple
users:
- name: admin
password: admin
groups: shiny-admin
- name: jeff
password: password
container-backend: docker-swarm
docker:
internal-networking: true
specs:
- id: 01_hello
display-name: Hello Application
description: Application which demonstrates the basics of a Shiny app
container-cmd: ["R", "-e", "shinyproxy::run_01_hello()"]
container-image: openanalytics/shinyproxy-demo
container-network: shiny_shiny-net
- id: 02_hello_private_repo
display-name: Hello Application from private repo
description: Application which demonstrates using private repo
container-cmd: ["R", "-e", "shinyproxy::run_01_hello()"]
container-image: ziyunxiao/shinyproxy-demo
container-auth-domain: docker.io
container-auth-user: ziyunxiao
container-auth-password: yourpassword
container-network: shiny_shiny-net
- id: 06_tabsets
container-cmd: ["R", "-e", "shinyproxy::run_06_tabsets()"]
container-image: openanalytics/shinyproxy-demo
container-network: shiny_shiny-net
- id: shiny_admin_dashboard
display-name: Shiny Admin
description: This application used for admin only. It includes only Keycloak Admin info for now.
container-cmd: ["./start.sh"]
container-image: yourowndomain.com/yourownimage:lastest
container-auth-domain: yourowndomain
container-auth-user: admin
container-auth-password: yourpassword
container-network: shiny_shiny-net
logging:
level:
root: INFO
server:
#useForwardHeaders: true #2.3.1
forward-headers-strategy: native
#useForwardHeaders: false
servlet.session.timeout: 36000
And to deploy shiny on docker swarm, the docker-compose.yml can be something like this
version: "3.8"
services:
shiny:
image: yournamespace/shinyproxy:2.5.0
volumes:
- /var/run/docker.sock:/var/run/docker.sock
- /youpath/local_test/application_docker.yml:/opt/shinyproxy/application.yml
ports:
- 8080:8080
networks:
- shiny-net
deploy:
mode: replicated
replicas: 1
restart_policy:
condition: on-failure
placement:
constraints:
- node.role==manager
- node.hostname==yournodename
networks:
shiny-net:
driver: overlay
For test deploy shiny docker stack deploy -c docker-compose.yml shiny
Thanks,
Robin
@LEDfan could you please take a look if this feature/update can be merged into future releases? I need this feature but don't want to maintain a copy of the code.
Hi @ziyunxiao
Thank you for your contribution! I understand the use-case and see the value. However, it will take some time for me to get though your PR and merge it. For example, I have to think about the bigger picture of this feature. When using plain Docker, there is also the issue that ShinyProxy doesn't automatically pull docker images. Maybe we should fix these two issues at the same time?
For plain Docker it can be fixed by adding pull image logic, there is a way to get it to work.
In file DockerEngineBackend.java around line 90, add logic to pull the image if the image does not exist.
ContainerConfig containerConfig = ContainerConfig.builder()
.hostConfig(hostConfigBuilder.build())
.image(spec.getImage())
.labels(labels)
.exposedPorts(portBindings.keySet())
.cmd(spec.getCmd())
.env(buildEnv(spec, proxy))
.build();
List<Image> images = dockerClient.listImages(com.spotify.docker.client.DockerClient.ListImagesParam.byName(spec.getImage()));
if (images.size() == 0){
dockerClient.pull(spec.getImage());
}
ContainerCreation containerCreation = dockerClient.createContainer(containerConfig);
While this requires the user(admin) to do docker login
first on the private repo. It differently requires either document or a better logic to handle this.
Regards,
For me, this is also a key topic.
Could it be merged in order to fetch Docker images from private Docker Hub repos?
This is now part of ShinyProxy 3.0.0!