Searches with submittedDate ranges give varying results

Question

Searches with submittedDate ranges give varying results

Opened this issue 10 years ago · 3 comments

The tests of sorted results are giving sporadic errors, in which the arxiv_search seems to retrieve fewer than the expected number of results.

I put some example results here. That's for three successive runs of test(), with no other changes. The first two gave errors (but not exactly the same errors), while the last run was clean.

Note that the error message

 Lengths (2, 1) differ (string compare on first 1)

means that the expected result had length 2 but the code was giving a result with length 1.

Answer 1 · 2014-09-09T15:14:17.000Z

I added some better R code with the example results with the problem, and also posted XML responses for a search that sometimes gives a single record and sometimes gives two records.

Here's the search that showed the problem:

http://export.arxiv.org/api/query?search_query=ti:deconvolution+AND+submittedDate:[199001010000+TO+201409062400]&max_results=2

I'd initially thought this issue occurred when I used sortBy or sortOrder, but rather it seems to happen when I use a range for submittedDate (hence the change in the title of this issue). For example, the following does not show any problems.

http://export.arxiv.org/api/query?search_query=ti:deconvolution&max_results=2

Answer 2 · 2014-09-09T17:29:36.000Z

I posted a question on this to the arxiv-api discussion list.

Answer 3 · 2014-09-10T02:18:04.000Z

I think that this problem is the same as this question at the arxiv-api discussion list. There's a bug in the case of repeated identical queries that has to do with the caching of search results.