libpath to put the custom lib folder in the front

Question

libpath to put the custom lib folder in the front

crazyhottommy opened this issue 2 years ago · 11 comments

Describe the bug

Hi, I am using the docker container and want to use my custom R library on my host

sudo docker run \
  	-v /home/R/host-site-library:/usr/local/lib/R/host-site-library \
  	-p 8787:8787 \
  	bioconductor/bioconductor_docker:RELEASE_3_15

> .libPaths()
[1] "/home/rstudio/R/x86_64-pc-linux-gnu-library/4.2" "/usr/local/lib/R/host-site-library"             
[3] "/usr/local/lib/R/site-library"                   "/usr/local/lib/R/library"

but the "/home/rstudio/R/x86_64-pc-linux-gnu-library/4.2" is before "/usr/local/lib/R/host-site-library", so I can not install the new packages to the host.

Thanks!

Answer 1 · 2022-12-15T22:37:21.000Z

Saw your tweet. These are two distinct issues. You are using -v correctly with Docker to bring a directory into the container. That however does not magically tell R to resort its library path and what you see there are the standard three directories implemented since Fritz Leisch and Kurt Hornik asked me / convinced me to set it up that way (plus the local user path below $HOME).

To change .libPaths(), you have to set it. Either via R_LIBS_{USER,SITE} or by calling .libPaths() in your local or default profile. If you want to do this 'permanently' you need to alter the files in /etc/R/ on the container, or else call .libPaths() for your session.

Answer 2 · 2022-12-15T22:45:55.000Z

Thanks for your answer.

 sudo docker run \
    -v /home/R/host-site-library:/usr/local/lib/R/host-site-library \
    -e R_LIBS_USER=/home/R/host-site-library \
  	-p 8787:8787 \
  	bioconductor/bioconductor_docker:RELEASE_3_15

This seems to do the trick to remove the "/home/rstudio/R/x86_64-pc-linux-gnu-library/4.2"

> .libPaths()
[1] "/usr/local/lib/R/host-site-library" "/usr/local/lib/R/site-library"     
[3] "/usr/local/lib/R/library"

but the path is not writable

> install.packages("Seurat")
Installing package into ‘/usr/local/lib/R/host-site-library’
(as ‘lib’ is unspecified)
Warning in install.packages :
  'lib = "/usr/local/lib/R/host-site-library"' is not writable

Answer 3 · 2022-12-15T22:48:35.000Z

Ah, yes, very good -- didn't think of the -e option to set an environment variable which R would pick up in its startup. That is indeed your preferred choice, and a good trick.

Which variable (between R_LIBS_USER and R_LIBS_SITE) differs a little between releases as a R changed a little there. Glad to know you're sorted so feel free to close this -- and maybe amplify the solution you have there in a tweet at a PSA.

Answer 4 · 2022-12-15T22:49:26.000Z

Ah, yes, very good -- didn't think of the -e option to set an environment variable which R would pick up in its startup. That is indeed your preferred choice, and a good trick.

Which variable (between R_LIBS_USER and R_LIBS_SITE) differs a little between releases as a R changed a little there. Glad to know you're sorted so feel free to close this -- and maybe amplify the solution you have there in a tweet at a PSA.

ha, I updated reply. the path is not writable

Answer 5 · 2022-12-15T22:54:18.000Z

That seems like something you can sort out with usual Docker options. There is also -u 1000:1000 to set user 1000 in the container (instead of root) which is what I am. With a bit of chmod fiddling you should be able to share as 'you' on both sides and read/write.

Answer 6 · 2022-12-15T23:06:14.000Z

thanks for the pointer. I saw this too #10
inside the container, I tried

sudo chmod u+w /usr/local/lib/R/host-site-library
[sudo] password for rstudio:

and by default, log in user is rstudio https://www.bioconductor.org/help/docker/#mounting
should I log in as root?

I am not an expert on docker. Thanks for your help!

Answer 7 · 2022-12-15T23:07:36.000Z

So that we're on the same page: you have a folder on the outside ie your host system that you own and that you want to use in position one in the container, for read / and write?

Answer 8 · 2022-12-15T23:15:25.000Z

So this works me:

I am in my directory /usr/local/lib/R-devel/lib/R/library which I use for an on-the-system, outside-the-$PATH r-devel installation. I have about 200 packages here to be able to do RD CMD check with that r-devel.
I am launching the Rocker image drd (with cronjob-driven weekly rebuilds) which may have a more current r-devel.
Test 1 is to use -v and -e. That works:

edd@rob:/usr/local/lib/R-devel/lib/R/library$ docker run -v $PWD:/opt/ -e R_LIBS_USER=/opt rocker/drd Rscript -e '.libPaths()'
[1] "/opt"                          "/usr/local/lib/R/site-library"
[3] "/usr/lib/R/site-library"       "/usr/lib/R/library"           
edd@rob:/usr/local/lib/R-devel/lib/R/library$

Test 2 is to use the container to install something. I use the fairly light-weight spdl library of mine, just update:

edd@rob:/usr/local/lib/R-devel/lib/R/library$ docker run -v $PWD:/opt/ -e R_LIBS_USER=/opt rocker/drd Rscript -e 'install.packages("spdl")'
Installing package into ‘/opt’                                                                           
(as ‘lib’ is unspecified)                                                                                
also installing the dependency ‘RcppSpdlog’    
[... rest omitted ...]

That worked. On the outside I have the modified directory that was created (and useable) in the container:

edd@rob:/usr/local/lib/R-devel/lib/R/library$ ls -ltr | tail -2
drwxrwxr-x 10 root root  4096 Dec 15 17:11 RcppSpdlog
drwxrwxr-x  6 root root  4096 Dec 15 17:11 spdl
edd@rob:/usr/local/lib/R-devel/lib/R/library$

Docker defaults to root so they were written by root.

Answer 9 · 2022-12-15T23:32:40.000Z

Thanks for your answer.

Test1 works on my side too.

Test 2 (installing package) does not work for me, as "/usr/local/lib/R/host-site-library"' is not writable

inside the container:

rstudio@f5a92942e075:/usr/local/lib/R$ ls -l
total 64
drwxr-xr-x  3 root root   4096 Jun 23 12:32 bin
-rw-r--r--  1 root root  18011 Jun 23 12:32 COPYING
drwxr-xr-x  1 root root   4096 Jun 23 12:32 doc
drwxr-xr-x  1 root root   4096 Dec 15 23:26 etc
drwxr-xr-x  2 1006  1007  4096 Dec 15 23:09 host-site-library
drwxr-xr-x  3 root root   4096 Jun 23 12:32 include
drwxr-xr-x  2 root root   4096 Jun 23 12:32 lib
drwxr-xr-x  1 root root   4096 Aug 11 16:32 library
drwxr-xr-x  2 root root   4096 Jun 23 12:32 modules
drwxr-xr-x 11 root root   4096 Jun 23 12:32 share
drwxrwsr-x  1 root staff  4096 Aug 11 16:33 site-library
-rw-r--r--  1 root root     46 Jun 23 12:32 SVN-REVISION

host-site-library folder is owned by 1006.

Answer 10 · 2022-12-16T00:00:35.000Z

So who is 1006 with group 1007? Is that you on the outside? On my Ubuntu system here I am 1000:1000 because it is effectively single-user:

edd@rob:~$ id
uid=1000(edd) gid=1000(edd) groups=1000(edd),4(adm),....
edd@rob:~$

You now have a local deployment problem as we addressed any issues you had with docker / the BioC container.

Answer 11 · 2022-12-16T01:54:00.000Z

appreciate your patience.
I forget to mention that I am working on a google cloud VM

 $ id
uid=1006(tommytang) gid=1007(tommytang) groups=1007(tommytang),4(adm),30(dip),44(video),46(plugdev),1000(google-sudoers)

after

chmod a+w -R  /home/R/host-site-library

on the host

Now, it works. Thanks again