`reticulate::py_config()` does not find python library when it is already installed

Question

`reticulate::py_config()` does not find python library when it is already installed

Opened this issue 3 years ago · 12 comments

I can see the r-miniconda and r-reticulate belw, but when I try and check the py_config() it doesn't detect them. I'm not sure how to fix this?

reticulate::conda_list()
#>           name
#> 1  r-miniconda
#> 2 r-reticulate
#>                                                              python
#> 1                   /Users/njtierney/Library/r-miniconda/bin/python
#> 2 /Users/njtierney/Library/r-miniconda/envs/r-reticulate/bin/python
reticulate::py_config()
#> Error: Python shared library not found, Python bindings not loaded.
#> Use reticulate::install_miniconda() if you'd like to install a Miniconda Python environment.
reticulate::install_miniconda()
#> Error: Miniconda is already installed at '/Users/njtierney/Library/r-miniconda'

^{Created on 2021-10-04 by the reprex package (v2.0.1)}

Session info

sessioninfo::session_info()
#> ─ Session info ───────────────────────────────────────────────────────────────
#>  setting  value                       
#>  version  R version 4.1.1 (2021-08-10)
#>  os       macOS Big Sur 10.16         
#>  system   x86_64, darwin17.0          
#>  ui       X11                         
#>  language (EN)                        
#>  collate  en_AU.UTF-8                 
#>  ctype    en_AU.UTF-8                 
#>  tz       Australia/Perth             
#>  date     2021-10-04                  
#> 
#> ─ Packages ───────────────────────────────────────────────────────────────────
#>  package     * version date       lib source        
#>  backports     1.2.1   2020-12-09 [1] CRAN (R 4.1.0)
#>  cli           3.0.1   2021-07-17 [1] CRAN (R 4.1.0)
#>  crayon        1.4.1   2021-02-08 [1] CRAN (R 4.1.0)
#>  digest        0.6.28  2021-09-23 [1] CRAN (R 4.1.0)
#>  ellipsis      0.3.2   2021-04-29 [1] CRAN (R 4.1.0)
#>  evaluate      0.14    2019-05-28 [1] CRAN (R 4.1.0)
#>  fansi         0.5.0   2021-05-25 [1] CRAN (R 4.1.0)
#>  fastmap       1.1.0   2021-01-25 [1] CRAN (R 4.1.0)
#>  fs            1.5.0   2020-07-31 [1] CRAN (R 4.1.0)
#>  glue          1.4.2   2020-08-27 [1] CRAN (R 4.1.0)
#>  highr         0.9     2021-04-16 [1] CRAN (R 4.1.0)
#>  htmltools     0.5.2   2021-08-25 [1] CRAN (R 4.1.0)
#>  jsonlite      1.7.2   2020-12-09 [1] CRAN (R 4.1.0)
#>  knitr         1.36    2021-09-29 [1] CRAN (R 4.1.0)
#>  lattice       0.20-44 2021-05-02 [1] CRAN (R 4.1.1)
#>  lifecycle     1.0.1   2021-09-24 [1] CRAN (R 4.1.0)
#>  magrittr      2.0.1   2020-11-17 [1] CRAN (R 4.1.0)
#>  Matrix        1.3-4   2021-06-01 [1] CRAN (R 4.1.1)
#>  pillar        1.6.3   2021-09-26 [1] CRAN (R 4.1.0)
#>  pkgconfig     2.0.3   2019-09-22 [1] CRAN (R 4.1.0)
#>  png           0.1-7   2013-12-03 [1] CRAN (R 4.1.0)
#>  purrr         0.3.4   2020-04-17 [1] CRAN (R 4.1.0)
#>  R.cache       0.15.0  2021-04-30 [1] CRAN (R 4.1.0)
#>  R.methodsS3   1.8.1   2020-08-26 [1] CRAN (R 4.1.0)
#>  R.oo          1.24.0  2020-08-26 [1] CRAN (R 4.1.0)
#>  R.utils       2.11.0  2021-09-26 [1] CRAN (R 4.1.0)
#>  Rcpp          1.0.7   2021-07-07 [1] CRAN (R 4.1.0)
#>  reprex        2.0.1   2021-08-05 [1] CRAN (R 4.1.0)
#>  reticulate    1.22    2021-09-17 [1] CRAN (R 4.1.0)
#>  rlang         0.4.11  2021-04-30 [1] CRAN (R 4.1.0)
#>  rmarkdown     2.11    2021-09-14 [1] CRAN (R 4.1.0)
#>  rstudioapi    0.13    2020-11-12 [1] CRAN (R 4.1.0)
#>  sessioninfo   1.1.1   2018-11-05 [1] CRAN (R 4.1.0)
#>  stringi       1.7.4   2021-08-25 [1] CRAN (R 4.1.0)
#>  stringr       1.4.0   2019-02-10 [1] CRAN (R 4.1.0)
#>  styler        1.6.2   2021-09-23 [1] CRAN (R 4.1.0)
#>  tibble        3.1.5   2021-09-30 [1] CRAN (R 4.1.0)
#>  utf8          1.2.2   2021-07-24 [1] CRAN (R 4.1.0)
#>  vctrs         0.3.8   2021-04-29 [1] CRAN (R 4.1.0)
#>  withr         2.4.2   2021-04-18 [1] CRAN (R 4.1.0)
#>  xfun          0.26    2021-09-14 [1] CRAN (R 4.1.0)
#>  yaml          2.2.1   2020-02-01 [1] CRAN (R 4.1.0)
#> 
#> [1] /Library/Frameworks/R.framework/Versions/4.1/Resources/library

Answer 1 · 2021-10-04T16:56:58.000Z

What happens if you try use_conda("r-reticulate", required = TRUE)? Or use_python("/path/to/miniconda/env/bin/python", required = TRUE)? (And then py_config() to force Python to be initialized.)

Answer 2 · 2021-10-04T23:09:57.000Z

Hi @kevinushey ,

Do you mean, use_condaenv ? I couldn't find, use_conda

This is what I get:

# reticulate::conda_list()
# reticulate::py_config()
# reticulate::install_miniconda()

library(reticulate)
use_condaenv("r-reticulate", required = TRUE)
system2("which","python", stdout = TRUE)
#> [1] "/usr/bin/python"
use_python("/usr/bin/python", required = TRUE)
py_config()
#> Error: Python shared library not found, Python bindings not loaded.
#> Use reticulate::install_miniconda() if you'd like to install a Miniconda Python environment.

^{Created on 2021-10-05 by the reprex package (v2.0.1)}

Session info

sessioninfo::session_info()
#> ─ Session info ───────────────────────────────────────────────────────────────
#>  setting  value                       
#>  version  R version 4.1.1 (2021-08-10)
#>  os       macOS Big Sur 10.16         
#>  system   x86_64, darwin17.0          
#>  ui       X11                         
#>  language (EN)                        
#>  collate  en_AU.UTF-8                 
#>  ctype    en_AU.UTF-8                 
#>  tz       Australia/Perth             
#>  date     2021-10-05                  
#> 
#> ─ Packages ───────────────────────────────────────────────────────────────────
#>  package     * version date       lib source        
#>  backports     1.2.1   2020-12-09 [1] CRAN (R 4.1.0)
#>  cli           3.0.1   2021-07-17 [1] CRAN (R 4.1.0)
#>  crayon        1.4.1   2021-02-08 [1] CRAN (R 4.1.0)
#>  digest        0.6.28  2021-09-23 [1] CRAN (R 4.1.0)
#>  ellipsis      0.3.2   2021-04-29 [1] CRAN (R 4.1.0)
#>  evaluate      0.14    2019-05-28 [1] CRAN (R 4.1.0)
#>  fansi         0.5.0   2021-05-25 [1] CRAN (R 4.1.0)
#>  fastmap       1.1.0   2021-01-25 [1] CRAN (R 4.1.0)
#>  fs            1.5.0   2020-07-31 [1] CRAN (R 4.1.0)
#>  glue          1.4.2   2020-08-27 [1] CRAN (R 4.1.0)
#>  highr         0.9     2021-04-16 [1] CRAN (R 4.1.0)
#>  htmltools     0.5.2   2021-08-25 [1] CRAN (R 4.1.0)
#>  jsonlite      1.7.2   2020-12-09 [1] CRAN (R 4.1.0)
#>  knitr         1.36    2021-09-29 [1] CRAN (R 4.1.0)
#>  lattice       0.20-44 2021-05-02 [1] CRAN (R 4.1.1)
#>  lifecycle     1.0.1   2021-09-24 [1] CRAN (R 4.1.0)
#>  magrittr      2.0.1   2020-11-17 [1] CRAN (R 4.1.0)
#>  Matrix        1.3-4   2021-06-01 [1] CRAN (R 4.1.1)
#>  pillar        1.6.3   2021-09-26 [1] CRAN (R 4.1.0)
#>  pkgconfig     2.0.3   2019-09-22 [1] CRAN (R 4.1.0)
#>  png           0.1-7   2013-12-03 [1] CRAN (R 4.1.0)
#>  purrr         0.3.4   2020-04-17 [1] CRAN (R 4.1.0)
#>  R.cache       0.15.0  2021-04-30 [1] CRAN (R 4.1.0)
#>  R.methodsS3   1.8.1   2020-08-26 [1] CRAN (R 4.1.0)
#>  R.oo          1.24.0  2020-08-26 [1] CRAN (R 4.1.0)
#>  R.utils       2.11.0  2021-09-26 [1] CRAN (R 4.1.0)
#>  Rcpp          1.0.7   2021-07-07 [1] CRAN (R 4.1.0)
#>  reprex        2.0.1   2021-08-05 [1] CRAN (R 4.1.0)
#>  reticulate  * 1.22    2021-09-17 [1] CRAN (R 4.1.0)
#>  rlang         0.4.11  2021-04-30 [1] CRAN (R 4.1.0)
#>  rmarkdown     2.11    2021-09-14 [1] CRAN (R 4.1.0)
#>  rstudioapi    0.13    2020-11-12 [1] CRAN (R 4.1.0)
#>  sessioninfo   1.1.1   2018-11-05 [1] CRAN (R 4.1.0)
#>  stringi       1.7.4   2021-08-25 [1] CRAN (R 4.1.0)
#>  stringr       1.4.0   2019-02-10 [1] CRAN (R 4.1.0)
#>  styler        1.6.2   2021-09-23 [1] CRAN (R 4.1.0)
#>  tibble        3.1.5   2021-09-30 [1] CRAN (R 4.1.0)
#>  utf8          1.2.2   2021-07-24 [1] CRAN (R 4.1.0)
#>  vctrs         0.3.8   2021-04-29 [1] CRAN (R 4.1.0)
#>  withr         2.4.2   2021-04-18 [1] CRAN (R 4.1.0)
#>  xfun          0.26    2021-09-14 [1] CRAN (R 4.1.0)
#>  yaml          2.2.1   2020-02-01 [1] CRAN (R 4.1.0)
#> 
#> [1] /Library/Frameworks/R.framework/Versions/4.1/Resources/library

Answer 3 · 2021-10-05T02:07:40.000Z

@njtierney are you using the latest stable release of RStudio? If so, I think we are experiencing the same issue. When I run reticulate::py_config(), reticulate::py_available(), etc. from an R session in my terminal, I get the expected behavior. However, running this commands from within RStudio displays the same Error: Python shared library not found, Python bindings not loaded. you are experiencing. I suspect there is a configuration issue with the latest version of RStudio, but I can't for the life of me figure out what that is...

EDIT
reticulate::conda_list() displays local conda environments (even from with RStudio), however Global Options -> Python (select) -> Conda Environments displays no available environments.

Answer 4 · 2021-10-05T04:08:48.000Z

Hi Daniel, Yes - I'm using Ghost Orchid: ``` $mode [1] "desktop" $version [1] ‘2021.9.0.351’ $long_version [1] "2021.09.0+351" $release_name [1] "Ghost Orchid" ``` That would track with my experience actually, I'll try and use a previous version of RStudio and see how I go ...

…

On Tue, 5 Oct 2021 at 10:07, Daniel Albohn ***@***.***> wrote: @njtierney <https://github.com/njtierney> are you using the latest stable release of RStudio? If so, I think we are experiencing the same issue. When I run reticulate::py_config(), reticulate::conda_list(), reticulate::use_condaenv(), etc. from an R session in my terminal, I get the expected behavior. However, running this commands from within RStudio displays the same Error: Python shared library not found, Python bindings not loaded. you are experiencing. I suspect there is a configuration issue with the latest version of RStudio, but I can't for the life of me figure out what that is... — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#1062 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ABRQDJMENHNR5XUCPX4KJBTUFJMXRANCNFSM5FITR4CQ> . Triage notifications on the go with GitHub Mobile for iOS <https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675> or Android <https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub>.

Answer 5 · 2021-10-06T01:27:40.000Z

I can confirm that reticulate works as expected with conda in RStudio Desktop 1.3.1093 but not in Rstudio Desktop 2021.09.0.

At this point, I am not sure if this issue should be transferred to the rstudio/rstudio repo or not.

Answer 6 · 2021-10-06T01:42:46.000Z

I think we should cross post this to rstudio/rstudio at least, this is a substantial bug

Answer 7 · 2021-10-07T09:12:50.000Z

Hello,

I am experiencing similar issues in VSCode (version 1.60.2) so probably not only related to RStudio:

r$> reticulate::conda_list()
       name                                                python
1 Anaconda3            C:\\Users\\Myself\\Anaconda3\\python.exe
2       rtf C:\\Users\\Myself\\Anaconda3\\envs\\rtf\\python.exe
r$> reticulate::py_config()
Error: could not find a Python environment for C:/Users/Myself/Anaconda3/python.exe

Trying to set the anaconda environment using use_condaenv or setting python path with use_python doesn't help.

session info

r$> sessioninfo::session_info()
- Session info ----------------------------------------------------------------------------------------------------------------------------------------------------------------------
 setting  value
 version  R version 4.1.1 (2021-08-10)
 os       Windows 10 x64
 system   x86_64, mingw32
 ui       RTerm
 language (EN)
 collate  English_Germany.1252
 ctype    English_Germany.1252
 tz       Europe/Berlin
 date     2021-10-07

- Packages --------------------------------------------------------------------------------------------------------------------------------------------------------------------------
 package     * version   date       lib source
 cli           3.0.1     2021-07-17 [1] CRAN (R 4.1.1)
 jsonlite      1.7.2     2020-12-09 [1] CRAN (R 4.1.1)
 lattice       0.20-45   2021-09-22 [1] CRAN (R 4.1.1)
 Matrix        1.3-4     2021-06-01 [1] CRAN (R 4.1.1)
 png           0.1-7     2013-12-03 [1] CRAN (R 4.1.1)
 rappdirs      0.3.3     2021-01-31 [1] CRAN (R 4.1.1)
 Rcpp          1.0.7     2021-07-07 [1] CRAN (R 4.1.1)
 reticulate    1.22-9000 2021-10-07 [1] Github (rstudio/reticulate@d98c60e)
 rlang         0.4.11    2021-04-30 [1] CRAN (R 4.1.1)
 sessioninfo   1.1.1     2018-11-05 [1] CRAN (R 4.1.1)
 withr         2.4.2     2021-04-18 [1] CRAN (R 4.1.1)

[1] C:/Users/Gendarme/Documents/R/R-4.1.1/library

Answer 8 · 2021-10-21T17:41:59.000Z

Also experiencing the issue as described by @njtierney, and on my end downgrading to RStudio Desktop 1.3.1093 like @d-bohn said did seem to unblock me from the issue. Unfortunately now stuck on a different topic around package install, but different topic, thank you for this thread, was helpful. Really hoping one day we get a re-design of reticulate, super useful but very buggy and hard to manage.

Answer 9 · 2021-10-21T17:45:15.000Z

You might need to use Sys.unsetenv("RETICULATE_PYTHON") to unset an erroneously-set version of Python. See rstudio/rstudio#9952 for more details. Sorry for the trouble :-/

Answer 10 · 2021-10-21T19:45:10.000Z

Thank you @kevinushey, I will keep playing around with this and try to get to the bottom of my issues. Really appreciate all your work, you are super awesome and sorry to sound negative there. I will come back to this comment and edit it if I find anything useful, but it seems like the specific issue discussed here got solved on my end by downgrading RStudio.

session info

> sessioninfo::session_info()
─ Session info ───────────────────────────────────────────────────────────────────────────────────
 setting  value                       
 version  R version 4.0.3 (2020-10-10)
 os       macOS Big Sur 10.16         
 system   x86_64, darwin17.0          
 ui       RStudio                     
 language (EN)                        
 collate  en_US.UTF-8                 
 ctype    en_US.UTF-8                 
 tz       America/New_York            
 date     2021-10-21                  

─ Packages ───────────────────────────────────────────────────────────────────────────────────────
 package     * version     date       lib source                      
 cli           3.0.1       2021-07-17 [1] CRAN (R 4.0.2)              
 digest        0.6.27      2020-10-24 [1] CRAN (R 4.0.2)              
 evaluate      0.14        2019-05-28 [1] CRAN (R 4.0.1)              
 fastmap       1.1.0       2021-01-25 [1] CRAN (R 4.0.2)              
 here          1.0.1       2020-12-13 [1] CRAN (R 4.0.2)              
 htmltools     0.5.2       2021-08-25 [1] CRAN (R 4.0.2)              
 jsonlite      1.7.2       2020-12-09 [1] CRAN (R 4.0.2)              
 knitr         1.33        2021-04-24 [1] CRAN (R 4.0.2)              
 lattice       0.20-41     2020-04-02 [1] CRAN (R 4.0.3)              
 Matrix        1.2-18      2019-11-27 [1] CRAN (R 4.0.3)              
 png           0.1-7       2013-12-03 [1] CRAN (R 4.0.2)              
 rappdirs      0.3.3       2021-01-31 [1] CRAN (R 4.0.2)              
 Rcpp          1.0.7       2021-07-07 [1] CRAN (R 4.0.2)              
 reticulate  * 1.22        2021-09-17 [1] CRAN (R 4.0.2)              
 rlang         0.4.11.9001 2021-09-19 [1] Github (r-lib/rlang@6d1fa42)
 rmarkdown     2.11        2021-09-14 [1] CRAN (R 4.0.3)              
 rprojroot     2.0.2       2020-11-15 [1] CRAN (R 4.0.2)              
 rsconnect     0.8.16      2019-12-13 [1] CRAN (R 4.0.2)              
 sessioninfo   1.1.1       2018-11-05 [1] CRAN (R 4.0.2)              
 withr         2.4.2       2021-04-18 [1] CRAN (R 4.0.2)              
 xfun          0.24        2021-06-15 [1] CRAN (R 4.0.2)              
 yaml          2.2.1       2020-02-01 [1] CRAN (R 4.0.2)              

[1] /Library/Frameworks/R.framework/Versions/4.0/Resources/library

Answer 11 · 2021-10-21T21:18:56.000Z

No worries at all, I appreciate your taking the time to report the issue and provide the information necessary to help us troubleshoot. I can definitely understand as a user how frustrating it can be when software updates break your existing workflows!

Answer 12 · 2021-11-02T00:45:29.000Z

Just wanted to flag that you can download older versions of RStudio here: https://www.rstudio.com/products/rstudio/older-versions/