Status code was not 200. Returned status code:401 ERROR AND SUGGESTION FOR FIX
philipwlewis opened this issue · 8 comments
Hi,
From this afternoon I was getting a 401 error code on the requests. I looked into the URLs and compared to PyTrends package which was still working and noticed the userConfig userType set to USER_TYPE_UNSPECIFIED
I added this into the payload creation and this seemed to fix the error
payload2$userConfig$userType <- "USER_TYPE_UNSPECIFIED"
Thanks for writing the nice package - hope this will help!
Still works here without it:
> library(gtrendsR)
> res <- gtrends("covid19")
> str(res)
List of 7
$ interest_over_time :'data.frame': 260 obs. of 7 variables:
..$ date : POSIXct[1:260], format: "2017-05-28" "2017-06-04" "2017-06-11" "2017-06-18" ...
..$ hits : int [1:260] 0 0 0 0 0 0 0 0 0 0 ...
..$ keyword : chr [1:260] "covid19" "covid19" "covid19" "covid19" ...
..$ geo : chr [1:260] "world" "world" "world" "world" ...
..$ time : chr [1:260] "today+5-y" "today+5-y" "today+5-y" "today+5-y" ...
..$ gprop : chr [1:260] "web" "web" "web" "web" ...
..$ category: int [1:260] 0 0 0 0 0 0 0 0 0 0 ...
$ interest_by_country:'data.frame': 250 obs. of 5 variables:
..$ location: chr [1:250] "Kiribati" "Uganda" "Solomon Islands" "Togo" ...
..$ hits : chr [1:250] "" "100" "" "" ...
..$ keyword : chr [1:250] "covid19" "covid19" "covid19" "covid19" ...
..$ geo : chr [1:250] "world" "world" "world" "world" ...
..$ gprop : chr [1:250] "web" "web" "web" "web" ...
$ interest_by_region : NULL
$ interest_by_dma :'data.frame': 306 obs. of 5 variables:
..$ location: chr [1:306] "Alpena MI" "Honolulu HI" "Juneau AK" "Boston MA-Manchester NH" ...
..$ hits : int [1:306] 100 60 58 55 54 54 53 53 53 52 ...
..$ keyword : chr [1:306] "covid19" "covid19" "covid19" "covid19" ...
..$ geo : chr [1:306] "world" "world" "world" "world" ...
..$ gprop : chr [1:306] "web" "web" "web" "web" ...
$ interest_by_city :'data.frame': 200 obs. of 5 variables:
..$ location: chr [1:200] "Kampala" "Bankura" "Agartala" "Bhiwani" ...
..$ hits : int [1:200] 100 NA NA NA NA NA NA NA NA NA ...
..$ keyword : chr [1:200] "covid19" "covid19" "covid19" "covid19" ...
..$ geo : chr [1:200] "world" "world" "world" "world" ...
..$ gprop : chr [1:200] "web" "web" "web" "web" ...
$ related_topics :'data.frame': 20 obs. of 5 variables:
..$ subject : chr [1:20] "100" "12" "6" "5" ...
..$ related_topics: chr [1:20] "top" "top" "top" "top" ...
..$ value : chr [1:20] "Coronavirus disease 2019" "India" "Vaccine" "COVID-19 vaccine" ...
..$ keyword : chr [1:20] "covid19" "covid19" "covid19" "covid19" ...
..$ category : int [1:20] 0 0 0 0 0 0 0 0 0 0 ...
..- attr(*, "reshapeLong")=List of 4
.. ..$ varying:List of 1
.. .. ..$ value: chr "top"
.. .. ..- attr(*, "v.names")= chr "value"
.. .. ..- attr(*, "times")= chr "top"
.. ..$ v.names: chr "value"
.. ..$ idvar : chr "id"
.. ..$ timevar: chr "related_topics"
$ related_queries :'data.frame': 50 obs. of 5 variables:
..$ subject : chr [1:50] "100" "71" "27" "22" ...
..$ related_queries: chr [1:50] "top" "top" "top" "top" ...
..$ value : chr [1:50] "covid" "covid 19" "covid19 cases" "covid19 india.org" ...
..$ keyword : chr [1:50] "covid19" "covid19" "covid19" "covid19" ...
..$ category : int [1:50] 0 0 0 0 0 0 0 0 0 0 ...
..- attr(*, "reshapeLong")=List of 4
.. ..$ varying:List of 1
.. .. ..$ value: chr "top"
.. .. ..- attr(*, "v.names")= chr "value"
.. .. ..- attr(*, "times")= chr "top"
.. ..$ v.names: chr "value"
.. ..$ idvar : chr "id"
.. ..$ timevar: chr "related_queries"
- attr(*, "class")= chr [1:2] "gtrends" "list"
> and
> plot(res)
> 
Could it be something at your end?
I am also getting this error. Where comes from USER_TYPE_UNSPECIFIED?
I looked at the url requests being made from the pytrends package that was still working for me https://pypi.org/project/pytrends/ and comparing the requests saw that it had the user type included unlike the gtrendsR request. I can't see where that is specified in the source code, but possibly it is picking up something that google sends back at some point? I am more of a hacker than being familiar with the API etc its odd that it worked for eddelbuettel (I teach a class and make use of the package for an assignment and a few students reported the issue). Let me know if I need to do anything on GitHub regarding the issue opening - this is my first time raising anything, Phil
We could triage, if we wanted to, over curl versions. It's a little strange that it bites in some cases but not in others. And I don't think the quick suggested fix -- already committed too !! -- will do any harm,
It's a little strange that it bites in some cases but not in others
And I can not see what could cause this. I will try to investigate more.
@philipwlewis I fixed the issue in a separate branch. Maybe you and your students can give it a try.
I was also getting this error so I tested the new branch version and the error went away.
Seems like the backend now tests for it. Odd.
Anyway, kudos to the pytrends team for the hint, to @philipwlewis for alerting us, and to @PMassicotte for a prompt update. Might be worth a routine quick update at CRAN too...
I will prepare a submission today.