jodyphelan/NTM-Profiler

Consider publishing a custom container for `ntm-profiler`

Closed this issue · 3 comments

abhi18av commented

Hi dear @jodyphelan ,

Thanks for your efforts on this tool - we're looking forward to accommodating this functionality! 🎉

At the moment, I am trying to integrate ntm-profiler into the MAGMA pipeline and came across a couple of issues with the associated biocontainer i.e. quay.io/biocontainers/ntm-profiler:0.3.0--pyhdfd78af_0, which are blockers for us. Therefore, would it be possible for you to publish containers directly on Github which can perhaps side-step these issues and provide an "official" version of container https://docs.github.com/packages/working-with-a-github-packages-registry/working-with-the-container-registry

The issues are mentioned below:

  1. The container does not have zip installed, therefore I cannot do ntm-profiler update_db --commit 002fec58b78cd5bfbefd1106a1ac26777147ec6b

  2. The container does not have apt-get setup, so we cannot install the zip command, leading to point-1.

sh-5.0# ntm-profiler update_db --commit 002fec58b78cd5bfbefd1106a1ac26777147ec6b
Traceback (most recent call last):
  File "/usr/local/bin/ntm-profiler", line 390, in <module>
    args.func(args)
  File "/usr/local/bin/ntm-profiler", line 202, in main_update_db
    pp.run_cmd("zip -F species/sourmash-split.zip --out species/sourmash.sbt.zip")
  File "/usr/local/lib/python3.10/site-packages/pathogenprofiler/utils.py", line 385, in run_cmd
    raise ValueError("Cant find programs: %s\n" % (", ".join(missing)))
ValueError: Cant find programs: zip

Cleaning up after failed run
[16:20:46] ERROR                                                                                        ntm-profiler:55
                                                                                                                       
################################# ERROR #######################################                    
                                                                                                                       
                    This run has failed. Please check all arguments and make sure all input files                      
                    exist. If no solution is found, please open up an issue at                                         
                    https://github.com/jodyphelan/NTM-Profiler/issues/new and paste or attach the                      
                    contents of the error log (5033cc43-ff6e-475d-9dd3-980a00cb503c.errlog.txt)                        
                                                                                                                       
                   

###############################################################################

CC @TimHHH @vrennie @mdediegofuertes

jodyphelan commented

Hi @abhi18av

In the next release of ntm-profiler this dependancy will be removed. Thanks for letting me know about the container registry will definitely look at this.

  • 🚀1
abhi18av commented

Hi @jodyphelan ,

I tested with the latest biocontainer associated with 0.4.0 but the container fails - any ideas?

Happy to connect and help create "official" Github repo based containers for both tb-profiler and ntm-profiler which might offer an easier pathway for users of containers?

+  p$ docker pull quay.io/biocontainers/ntm-profiler:0.4.0--pyhdfd78af_0
0.4.0--pyhdfd78af_0: Pulling from biocontainers/ntm-profiler
ca7680d1025d: Already exists
bd9ddc54bea9: Already exists
2903feb65aa7: Extracting [==================================================>]   1.01GB/1.01GB
failed to register layer: write /usr/local/lib/python3.10/site-packages/pysam/libcvcf.cpython-310-x86_64-linux-gnu.so: no space left on device
abhi18av commented

Hi @jodyphelan ,

Never mind this comment #35 (comment) , I have been able to build a custom container directly from the new upstream biocontainer TORCH-Consortium/MAGMA@907ac5f

The problem was with my docker cache and once I did docker system prune -a and retried the process, everything worked.

I'm closing this issue as the existing biocontainer ntm-profiler v0.4.0 works as expected ✅