update to singularity 2.4
Closed this issue · 11 comments
That would be great, but since docker2singularity mostly serves legacy systems (Singularity 2.2 and older) it would be good to default to creation of backward compatible images (which probably means using .img instead of squashfs).
Any chance that the feature to export as squashfs could be developed and then be callable as an optional flag/option to the docker run command? We're still on Singularity 2.2 but are interested in upgrading to Singularity 2.4 and squashfs, while allowing/supporting builds on Windows via docker2singularity.
Something like
docker run \
-v /var/run/docker.sock:/var/run/docker.sock \
-v D:\host\path\where\to\output\singularity\image:/output \
--privileged -t --rm \
singularityware/docker2singularity \
-m "/shared_fs /custom_mountpoint2" \
--file-format=squashfs \
ubuntu:14.04
where --file-format=img is the default
I'll try to have a look at this PR tomorrow.
However, I am a bit confused about your use case. If you upgrade your systems to 2.4 wouldn't your users be able to run and convert containers directly from Docker Hub directly on your system without the need for elevated privileges? I thought this is what 2.4 enabled. This would mean your users would not need to use docker2singularity and transfer images.
hey @gnguy - I addressed this issue, it's currently a PR to this repository #27, and it will produce a sandbox (folder) and then a final squashfs image. If you would like to test it would be great! @chrisfilo has a lot of responsibility so he hasn't been able to review yet. Might you help?
@chrisfilo I would guess that it's easier to run Docker on Windows than Vagrant with Singularity, but just a guess. It may be they want to test locally before moving the entire cluster to 2.4.
@chrisfilo Thank you for your response! Currently, my organization is storing our Docker images using an on-prem VMWare storage system rather than DockerHub. Reading through the documentation on singularity build, it seemed like building off of docker images was primarily reliant on them being hosted on DockerHub -- is that correct? In addition, it'd be nice for people to run Docker on Windows and do the conversion that way.
@vsoch Thanks for the PR, I'd love to test it out, should have a chance tomorrow to try it.
Not sure how does your setup work, but AFAIK Singularity can pull images from private docker registries - you might want to try this out.
Hi @chrisfilo!
We have a use case where it's still useful to use docker to generate up to date singularity images: we often run on HPC systems which have no internet connection (due to security concerns). As such, we need to create singularity containers on a different system and then transfer the image files.
I could just require people to install singularity on their local systems, but in my experience people are more likely to have docker installed already so it would be great to have this option.
If you're curious for context - I'm writing a helper tool for the http://nf-co.re/ genomics research community.
Anyway, it looks like PR #27 is making great progress, so I'll keep my eye on that :)
Phil
Indeed looks like a very cool project! I think #27 needs update to the latest singularity version and hopefully the bugs will go away.
@chrisfilo let's close the v2.4 PR (it was to update master, and lives on the branch v2.4 if anyone needs) and I'll create a new branch for 2.5 that will have the updated version. I do think a lot of the bugs (that were still present in 2.4) are hopefully resolved. Stay tuned!
This was done, never closed. We have a (less buggy) version than 2.4 :)