Journal name and connection errors

Question

Journal name and connection errors

ernestocurty opened this issue 2 years ago · 1 comments

Hi Ana,

I just installed Adjutant with R version 4.1.3 (2022-03-10) and updated all dependencies (see list below).

The shiny app is working correctly however I notice two possible bugs:

 1. The shiny app it is not retrieving the journal name.
 2. I get a connection error for large number of results. A query string that returns 635 results works fine. Another one that returns 
     around 7500 gives me the error. I am using the NCBI API key.

I have limited R knowledge but if you point me the direction (on the code) or some references I can try to help troubleshoot this.
Maybe I could slow down the queries per second as an workaround?

error message:
Could not get PubMed results
Adjutant could not retrieve PubMed results. It's likely that there wasa problem with your Internet connection (including your firewall settings). Also NCBI limits the number of requests from a single IP, it can help to get an NCBI API key (see Adjutant's online repo for details).
Ok

** More stringent query string (632 hits, working fine)
("decellularized extracellular matrix"[All Fields]) OR ("dECM"[All Fields]) AND (2008:2022[pdat])

** Less stringent query string (7533 hits, raises error) :
(("decellularized"[All Fields]) OR ("decellularization"[All Fields]) OR ("Acellular"[All Fields])) AND (("extracellular matrix"[All Fields]) OR ("matrix"[All Fields]) OR ("dECM"[All Fields]) OR ("Scaffold"[All Fields])) AND (2008:2022[pdat])

List of updated dependencies (all updated for the suggested version)
4: cli (3.2.0 -> 3.4.1 ) [CRAN]
5: vctrs (0.3.8 -> 0.4.2 ) [CRAN]
6: pillar (1.7.0 -> 1.8.1 ) [CRAN]
7: magrittr (2.0.2 -> 2.0.3 ) [CRAN]
8: viridisLite (0.4.0 -> 0.4.1 ) [CRAN]
9: RColorBrewer (1.1-2 -> 1.1-3 ) [CRAN]
10: lifecycle (1.0.1 -> 1.0.3 ) [CRAN]
11: farver (2.1.0 -> 2.1.1 ) [CRAN]
12: tibble (3.1.6 -> 3.1.8 ) [CRAN]
13: scales (1.1.1 -> 1.2.1 ) [CRAN]
14: rlang (1.0.2 -> 1.0.6 ) [CRAN]
15: isoband (0.2.5 -> 0.2.6 ) [CRAN]
16: gtable (0.3.0 -> 0.3.1 ) [CRAN]
17: digest (0.6.29 -> 0.6.30) [CRAN]
18: htmltools (0.5.2 -> 0.5.3 ) [CRAN]
19: Rcpp (1.0.8.3 -> 1.0.9 ) [CRAN]
20: sass (0.4.1 -> 0.4.2 ) [CRAN]
21: bslib (0.3.1 -> 0.4.0 ) [CRAN]
22: commonmark (1.8.0 -> 1.8.1 ) [CRAN]
23: crayon (1.5.0 -> 1.5.2 ) [CRAN]
24: fontawesome (0.2.2 -> 0.3.0 ) [CRAN]
25: jsonlite (1.8.0 -> 1.8.2 ) [CRAN]
26: httpuv (1.6.5 -> 1.6.6 ) [CRAN]
27: sys (3.4 -> 3.4.1 ) [CRAN]
28: openssl (2.0.0 -> 2.0.4 ) [CRAN]
29: curl (4.3.2 -> 4.3.3 ) [CRAN]
30: stringi (1.7.6 -> 1.7.8 ) [CRAN]
31: tidyselect (1.1.2 -> 1.2.0 ) [CRAN]
32: generics (0.1.2 -> 0.1.3 ) [CRAN]
33: yaml (2.3.5 -> 2.3.6 ) [CRAN]
34: stringr (1.4.0 -> 1.4.1 ) [CRAN]
35: purrr (0.3.4 -> 0.3.5 ) [CRAN]
36: ggplot2 (3.3.5 -> 3.3.6 ) [CRAN]
37: shiny (1.7.1 -> 1.7.2 ) [CRAN]
38: httr (1.4.2 -> 1.4.4 ) [CRAN]
39: cpp11 (0.4.2 -> 0.4.3 ) [CRAN]
40: dplyr (1.0.8 -> 1.0.10) [CRAN]
41: tidyr (1.2.0 -> 1.2.1 ) [CRAN]
42: Rtsne (0.15 -> 0.16 ) [CRAN]
43: DT (0.21 -> 0.25 ) [CRAN]
44: plyr (1.8.6 -> 1.8.7 ) [CRAN]

Answer 1 · 2022-10-20T18:32:57.000Z

I dug a little in the <process_pubmed.R> file and Identified the source of the Journal Title problem.
The RISmed EUtilsGet function is failing to fetch the correct data for @title field (among other fields).

pubResults@Title
[1] NA NA NA NA NA NA NA NA NA NA NA NA NA NA NA NA NA NA NA NA NA NA NA NA NA NA NA NA NA NA NA NA NA NA NA NA NA NA NA NA NA
[42] NA NA NA NA NA NA NA NA NA NA NA NA NA NA NA NA NA NA NA NA NA NA NA NA NA NA NA NA NA NA NA NA NA NA NA NA NA NA NA NA NA
[83] NA NA NA NA NA NA NA NA NA NA NA NA NA NA NA NA NA NA

The field retrieval error was pointed in at least 3 bug reports on the GitHub page for the RISmed library (linked below). But so far no solution:
skoval/RISmed#34
skoval/RISmed#28
skoval/RISmed#27

A quick and dirt workaround is to replace the @title by @ISOAbbreviation on line #194.