Getting "429 Too Many Requests" error from crossref

Question

Getting "429 Too Many Requests" error from crossref

vjainhighwirepress opened this issue 3 years ago · 7 comments

vjainhighwirepress commented 3 years ago

Hi Crossref Support Team,

Please note that we are getting the "429 Too Many Requests" error from crossref while we are making requests for any ISSN. Below are some sample URLs we are trying to hit but getting the "You have sent too many requests in a given amount of time." response.

https://api.crossref.org/works?filter=issn%3A2470-9468%2Ctype%3Ajournal-article%2Cfrom-pub-date%3A2020-06-01&rows=1000&cursor=%2A
https://api.crossref.org/works?cursor=%2A&filter=issn%3A0008-5472%2Ctype%3Ajournal-article%2Cfrom-pub-date%3A2020-06-01&rows=1000
https://api.crossref.org/works?cursor=%2A&filter=issn%3A0028-3878%2Ctype%3Ajournal-article%2Cfrom-pub-date%3A2020-06-01&rows=1000
https://api.crossref.org/works?cursor=%2A&filter=issn%3A1067-151X%2Ctype%3Ajournal-article%2Cfrom-pub-date%3A2020-06-01&rows=1000
https://api.crossref.org/works?cursor=%2A&filter=issn%3A2154-1663%2Ctype%3Ajournal-article%2Cfrom-pub-date%3A2020-06-01&rows=1000

Is there any ongoing issue?

Thanks
Vineet

Answer 1 · 2021-07-01T10:54:18.000Z

@ppolischuk can this be looked into urgently?

Answer 2 · 2021-07-01T17:07:12.000Z

Hi Vineet,

I don't see anything wrong with those requests, and they return a valid response for me.

It looks like all of the requests with the highwire vizor-developer email address listed for the mailto:, all coming from the same IP, are returning a 429 response, regardless of whether your request is for an ISSN or anything else.

Do you have some kind of script or other process that is exceeding our rate limit? That would explain the 429 http response code.

Answer 3 · 2021-07-01T18:05:29.000Z

Hi @ppolischuk,

We stopped all the processes running but still, we are observing the same error. Please can you tell by when can start making requests to Crossref to avoid rate limit violation?

Answer 4 · 2021-07-01T18:14:56.000Z

Are you using only the public pool? Have you tried with an email specified? Can you provide the ip address you're using to do the queries? (You can use a service like https://whatismyipaddress.com/ to get your ip). Can you do a query with curl -v and provide the result?

Answer 5 · 2021-07-01T18:53:26.000Z

@myalter @ppolischuk Yes, we use public pools only.

Please find below the output of curl command

and IP address of the machine is 54.200.35.56

Answer 6 · 2021-07-02T10:00:02.000Z

@ppolischuk @myalter please can you tell by when can we start submitting the requests?

Answer 7 · 2021-07-05T11:01:06.000Z

@ppolischuk @myalter please close this case we are able to submit the requests now. Thanks.