How to enable HistomicsTK web UI

Question

How to enable HistomicsTK web UI

Closed this issue 5 years ago · 20 comments

I have run the docker-compose.yml in devops/dsa and got HistomicUI up and running. BTW I had trouble with Mongo refusing connections, during this part. Maybe put in a note to delete all the contents of devops/dsa/db and devops/dsa/logs if one runs into this issue when starting out.

I pulled dsarchive/histomicstk:latest, connected it to teh dsa_default network, and tried to add it in the slicer_cli_web plugin settings. Nothing in the interface changes, and there is no HistomicsTK endpoint I can reach, nor a link to it via large_image files.

I'de imagine HistomicsTK is served off a girder_worker, as per the diagrams I have seen? I am confused as to how I deploy a web app that includes the annotation tools of HistomicsTK from the command line, preferably from a single call to docker-compose.

Can you set me straight?

Answer 1 · 2020-05-28T23:27:18.000Z

Bette ryet.. http://<<SERVERNAMEHERE>>:8080/histomicstk so http://localhost:8080/histomicstk The service is already running... it's just at a different endpoint. We are in the process of making our documentation better.... but since "we" wrote most of this, we take for granted many things that are fundamentally not very intuitive..

…

On Thu, May 28, 2020 at 5:34 PM Alex Braun ***@***.***> wrote: I have run the docker-compose.yml in devops/dsa and got HistomicUI up and running. BTW I had trouble with Mongo refusing connections, during this part. Maybe put in a note to delete all the contents of devops/dsa/db and devops/dsa/logs if one runs into this issue when starting out. I pulled dsarchive/histomicstk:latest and tried to add it in the slicer cli web plugin settings. Nothing in the interface changes, and there is no HistomicsTK endpoint I can reach, nor a link to it via large_image files. I'de imagine HistomicsTK is served off a girder_worker, as per the diagrams I have seen? I am confused as to how I deploy a web app that includes the annotation tools of HistomicsTK from the command line, preferably from a single call to docker-compose. Can you set me straight? — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#115>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAFODTQSYQOB6FSUFU4MEETRT3KEVANCNFSM4NNOPXOQ> .

-- David A Gutman, M.D. Ph.D. Associate Professor of Neurology Emory University School of Medicine

Answer 2 · 2020-05-29T00:22:45.000Z

That endpoint does not work

Answer 3 · 2020-05-29T00:29:03.000Z

I have found that if I add it as a pip package in the dsa_girder dockerfile, create a new user, upload a file, convert it to a large image and then click open in HistomicsUI in the collections modal (while cacheing is disabled) I can get to a HistomicsTK-like interface. But it has no analyses functionality, such as color deconvolution.

Answer 4 · 2020-05-29T12:48:30.000Z

You can install different algorithms via the slicer_cli_web plugin settings page. Specifically, you can install HistomicsTK algorithms like so:

specifying the name of the docker image (dsarchive/histomicstk:latest) and clicking import.

After this finishes, the next time you got to the HistomicsUI interface and load an image, you'll see the Analyses menu which will have the HistomicsTK algorithms.

Answer 5 · 2020-05-29T19:00:51.000Z

That isn't working for me.

Started to Load Docker images
Failed to create the Docker Client
Error while fetching server API version: ('Connection aborted.', PermissionError(13, 'Permission denied'))
Error with job 
 Could not create the docker client (image name: None )

In error log:

Traceback (most recent call last):
  File "/usr/local/lib/python3.7/site-packages/slicer_cli_web/image_job.py", line 150, in jobPullAndLoad
    docker_client = docker.from_env(version='auto')
  File "/usr/local/lib/python3.7/site-packages/docker/client.py", line 85, in from_env
    timeout=timeout, version=version, **kwargs_from_env(**kwargs)
  File "/usr/local/lib/python3.7/site-packages/docker/client.py", line 40, in __init__
    self.api = APIClient(*args, **kwargs)
  File "/usr/local/lib/python3.7/site-packages/docker/api/client.py", line 190, in __init__
    self._version = self._retrieve_server_version()
  File "/usr/local/lib/python3.7/site-packages/docker/api/client.py", line 215, in _retrieve_server_version
    'Error while fetching server API version: {0}'.format(e)
docker.errors.DockerException: Error while fetching server API version: ('Connection aborted.', PermissionError(13, 'Permission denied'))

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.7/site-packages/slicer_cli_web/image_job.py", line 158, in jobPullAndLoad
    raise DockerImageError('Could not create the docker client')
slicer_cli_web.models.exceptions.DockerImageError: Could not create the docker client (image name: None )
Additional info:
  Request URL: GET /
  Query string: 
  Remote IP: 127.0.0.1
  Request UID: [none]

Answer 6 · 2020-05-29T19:09:19.000Z

I ran compose with:
DSA_PORT=8080 CURRENT_UID=1000:1000 docker-compose up

Answer 7 · 2020-05-29T19:35:12.000Z

Does it work for you if you use the ansible/deploy_docker.py script rather than docker-compose?

We create docker containers to run algorithms, and it sounds like the user inside the docker container doesn't have the right permissions to access the docker daemon and create a container. The deploy_docker.py program does some things to try to alleviate these issues -- for instance, the user id within the main dsa container might be in a group that isn't the docker group outside of the container.

Answer 8 · 2020-05-29T19:36:38.000Z

Also this inside worker.log:

[2020-05-29 19:16:23,629: ERROR/MainProcess] consumer: Cannot connect to amqp://guest:**@rabbitmq:5672//: [Errno 111] Connection refused.
Trying again in 4.00 seconds... (2/100)

Answer 9 · 2020-05-29T19:54:39.000Z

HistomicsTK loads but the menu item doesn't show up

Answer 10 · 2020-05-29T20:15:01.000Z

Did you install the HistomicsTK algorithms via the slicer_cli_web plugin settings page?

Answer 11 · 2020-05-29T20:26:17.000Z

yes

Traceback (most recent call last):
  File "/opt/slicer_cli_web/slicer_cli_web/image_job.py", line 309, in getCliData
    cli_desc = getDockerOutput(name, '%s --%s' % (key, desc_type), client)
  File "/opt/slicer_cli_web/slicer_cli_web/image_job.py", line 294, in getDockerOutput
    'Attempt to docker run %s %s failed' % (imgName, command), imgName)
slicer_cli_web.models.exceptions.DockerImageError: Attempt to docker run dsarchive/histomicstk:latest NucleiClassification --xml failed (image name: dsarchive/histomicstk:latest )
Additional info:
  Request URL: GET /
  Query string: 
  Remote IP: 127.0.0.1
  Request UID: [none]
[2020-05-29 20:25:30,336] ERROR: Error getting dsarchive/histomicstk:latest cli data from image
Traceback (most recent call last):
  File "/opt/slicer_cli_web/slicer_cli_web/image_job.py", line 309, in getCliData
    cli_desc = getDockerOutput(name, '%s --%s' % (key, desc_type), client)
  File "/opt/slicer_cli_web/slicer_cli_web/image_job.py", line 294, in getDockerOutput
    'Attempt to docker run %s %s failed' % (imgName, command), imgName)
slicer_cli_web.models.exceptions.DockerImageError: Attempt to docker run dsarchive/histomicstk:latest NucleiClassification --xml failed (image name: dsarchive/histomicstk:latest )
Additional info:
  Request URL: GET /
  Query string: 
  Remote IP: 127.0.0.1
  Request UID: [none]

Answer 12 · 2020-05-29T20:27:37.000Z

dsa doesn't like the image

Answer 13 · 2020-06-01T18:27:56.000Z

Not sure what else I should try.

Answer 14 · 2020-06-01T23:47:46.000Z

I run this:

master ▶ docker run dsarchive/histomicstk:latest NucleiClassification --xml                                                                                                       
Traceback (most recent call last):
  File "NucleiClassification/NucleiClassification.py", line 7, in <module>
    from sklearn.externals import joblib
ImportError: cannot import name 'joblib'

Looks like the container is missing the joblib module

Ran:
docker run dsarchive/histomicstk:latest ComputeNucleiFeatures --xml
ans it worked fine.

Looks like your NucleiClassification module in the latest dsarchive/histomicstk docker container needs fixing.

Answer 15 · 2020-06-01T23:58:43.000Z

I submitted an issue on the histomicstk repo

Answer 16 · 2020-06-02T16:37:16.000Z

Thanks. scikit-learn "unvendored" a dependency in the last month, causing this problem. HistomicsTK has been updated to handle it, and new a docker image automatically was published. Please confirm if this is now working for you.

Answer 17 · 2020-06-02T19:23:42.000Z

Confirmed, it works now.

Answer 18 · 2020-10-07T11:38:41.000Z

Hi! Just yesterday I deployed DSA via docker-compose and faced the same issue trying to install HistomicsTK.

Started to Load Docker images
Failed to create the Docker Client
Error while fetching server API version: ('Connection aborted.', PermissionError(13, 'Permission denied'))
Error with job 
 Could not create the docker client (image name: None )

Any suggestions?

Answer 19 · 2020-10-07T17:28:56.000Z

It sounds like a docker configuration issue. Can you use docker on your machine? Do docker --version and docker ps -a both work?

Answer 20 · 2020-10-07T17:30:18.000Z

I will do it definitely.
For now I successfully solved it deploying with ansible.