`post_tweet()` with `rtweet_bot()` no longer works
Closed this issue · 15 comments
Previously I was using code like the following:
auth_as(rtweet_bot(
api_key = Sys.getenv("TWITTER_API_KEY"),
api_secret = Sys.getenv("TWITTER_API_SECRET"),
access_token = Sys.getenv("TWITTER_ACCESS_TOKEN"),
access_secret = Sys.getenv("TWITTER_ACCESS_SECRET")
))
post_tweet("whatever")My bot's been offline for about 12 months for unrelated reasons. I'm working on it again and the same code which previously worked is now failing to post tweets.
Now I get this behaviour:
x <- post_tweet("whatever")
#> Your tweet has been posted!
x
#> Your message has not been posted! FALSETweet posted from
#> Text:Needless to say, having checked my bot's account, tweets are no longer being posted.
I've tried renewing my keys/tokens/secrets to no avail. I've also tried scouring the twitter API documentation for some insight, but my skill with APIs is unfortunately the limiting factor here.
Any help greatly appreciated.
Session info
─ Session info ─────────────────────────────────────────────────────────────────────────────────────────────────────────────
setting value
version R version 4.2.1 (2022-06-23)
os macOS Ventura 13.2
system x86_64, darwin17.0
ui RStudio
language (EN)
collate en_US.UTF-8
ctype en_US.UTF-8
tz Africa/Nairobi
date 2023-11-20
rstudio 2023.06.2+561 Mountain Hydrangea (desktop)
pandoc 2.19.2 @ /usr/local/bin/pandoc
─ Packages ─────────────────────────────────────────────────────────────────────────────────────────────────────────────────
! package * version date (UTC) lib source
P askpass 1.2.0 2023-09-03 [?] CRAN (R 4.2.0)
P brio 1.1.3 2021-11-30 [?] CRAN (R 4.2.0)
P cachem 1.0.8 2023-05-01 [?] CRAN (R 4.2.0)
P callr 3.7.3 2022-11-02 [?] CRAN (R 4.2.0)
P cli 3.6.1 2023-03-23 [?] CRAN (R 4.2.0)
P colorspace 2.1-0 2023-01-23 [?] CRAN (R 4.2.0)
P crayon 1.5.2 2022-09-29 [?] CRAN (R 4.2.0)
P curl 5.1.0 2023-10-02 [?] CRAN (R 4.2.0)
P desc 1.4.2 2022-09-08 [?] CRAN (R 4.2.0)
P devtools 2.4.5 2022-10-11 [?] CRAN (R 4.2.0)
P digest 0.6.33 2023-07-07 [?] CRAN (R 4.2.0)
P dplyr * 1.1.3 2023-09-03 [?] CRAN (R 4.2.0)
P ellipsis 0.3.2 2021-04-29 [?] CRAN (R 4.2.0)
P fansi 1.0.5 2023-10-08 [?] CRAN (R 4.2.0)
P fastmap 1.1.1 2023-02-24 [?] CRAN (R 4.2.0)
P forcats * 1.0.0 2023-01-29 [?] CRAN (R 4.2.0)
P fs 1.6.3 2023-07-20 [?] CRAN (R 4.2.0)
P generics 0.1.3 2022-07-05 [?] CRAN (R 4.2.0)
P ggplot2 * 3.4.4 2023-10-12 [?] CRAN (R 4.2.0)
P glue 1.6.2 2022-02-24 [?] CRAN (R 4.2.0)
P gtable 0.3.4 2023-08-21 [?] CRAN (R 4.2.0)
P hms 1.1.3 2023-03-21 [?] CRAN (R 4.2.0)
P htmltools 0.5.7 2023-11-03 [?] CRAN (R 4.2.1)
P htmlwidgets 1.6.2 2023-03-17 [?] CRAN (R 4.2.0)
P httpuv 1.6.12 2023-10-23 [?] CRAN (R 4.2.1)
P httr 1.4.7 2023-08-15 [?] CRAN (R 4.2.0)
P httr2 * 1.0.0 2023-11-14 [?] CRAN (R 4.2.1)
P jose 1.2.0 2021-11-06 [?] CRAN (R 4.2.0)
P jsonlite 1.8.7 2023-06-29 [?] CRAN (R 4.2.0)
P later 1.3.1 2023-05-02 [?] CRAN (R 4.2.0)
P lifecycle 1.0.4 2023-11-07 [?] CRAN (R 4.2.1)
P lubridate * 1.9.3 2023-09-27 [?] CRAN (R 4.2.0)
P magrittr 2.0.3 2022-03-30 [?] CRAN (R 4.2.0)
P memoise 2.0.1 2021-11-26 [?] CRAN (R 4.2.0)
P mime 0.12 2021-09-28 [?] CRAN (R 4.2.0)
P miniUI 0.1.1.1 2018-05-18 [?] CRAN (R 4.2.0)
P munsell 0.5.0 2018-06-12 [?] CRAN (R 4.2.0)
P openssl 2.1.1 2023-09-25 [?] CRAN (R 4.2.0)
P pillar 1.9.0 2023-03-22 [?] CRAN (R 4.2.0)
P pkgbuild 1.4.2 2023-06-26 [?] CRAN (R 4.2.0)
P pkgconfig 2.0.3 2019-09-22 [?] CRAN (R 4.2.0)
P pkgload 1.3.3 2023-09-22 [?] CRAN (R 4.2.0)
P prettyunits 1.2.0 2023-09-24 [?] CRAN (R 4.2.0)
P processx 3.8.2 2023-06-30 [?] CRAN (R 4.2.0)
P profvis 0.3.8 2023-05-02 [?] CRAN (R 4.2.0)
P promises 1.2.1 2023-08-10 [?] CRAN (R 4.2.0)
P ps 1.7.5 2023-04-18 [?] CRAN (R 4.2.0)
P purrr * 1.0.2 2023-08-10 [?] CRAN (R 4.2.0)
P R6 2.5.1 2021-08-19 [?] CRAN (R 4.2.0)
P rappdirs 0.3.3 2021-01-31 [?] CRAN (R 4.2.0)
P Rcpp 1.0.11 2023-07-06 [?] CRAN (R 4.2.0)
P readr * 2.1.4 2023-02-10 [?] CRAN (R 4.2.0)
P remotes 2.4.2.1 2023-07-18 [?] CRAN (R 4.2.0)
P renv 1.0.3 2023-09-19 [?] CRAN (R 4.2.0)
P rlang 1.1.2 2023-11-04 [?] CRAN (R 4.2.1)
P rprojroot 2.0.4 2023-11-05 [?] CRAN (R 4.2.1)
P rstudioapi 0.15.0 2023-07-07 [?] CRAN (R 4.2.0)
P rtweet 1.2.1 2023-10-16 [?] CRAN (R 4.2.0)
P scales 1.2.1 2022-08-20 [?] CRAN (R 4.2.0)
P sessioninfo 1.2.2 2021-12-06 [?] CRAN (R 4.2.0)
P shiny 1.7.4 2022-12-15 [?] CRAN (R 4.2.0)
P stringdist 0.9.10 2022-11-07 [?] CRAN (R 4.2.0)
P stringi 1.7.12 2023-01-11 [?] CRAN (R 4.2.0)
P stringr * 1.5.0 2022-12-02 [?] CRAN (R 4.2.0)
P testthat * 3.2.0 2023-10-06 [?] CRAN (R 4.2.0)
P tibble * 3.2.1 2023-03-20 [?] CRAN (R 4.2.0)
P tidyr * 1.3.0 2023-01-24 [?] CRAN (R 4.2.0)
P tidyselect 1.2.0 2022-10-10 [?] CRAN (R 4.2.0)
P tidyverse * 2.0.0 2023-02-22 [?] CRAN (R 4.2.0)
R tidyverse-dev-bot * 0.0.0.9000 <NA> [?] <NA>
P timechange 0.2.0 2023-01-11 [?] CRAN (R 4.2.0)
P tzdb 0.4.0 2023-05-12 [?] CRAN (R 4.2.0)
P urlchecker 1.0.1 2021-11-30 [?] CRAN (R 4.2.0)
P usethis 2.2.2.9000 2023-11-11 [?] Github (r-lib/usethis@52b45dc)
P utf8 1.2.4 2023-10-22 [?] CRAN (R 4.2.1)
P V8 4.4.0 2023-10-09 [?] CRAN (R 4.2.0)
P vctrs 0.6.4 2023-10-12 [?] CRAN (R 4.2.0)
P withr 2.5.2 2023-10-30 [?] CRAN (R 4.2.1)
P xtable 1.8-4 2019-04-21 [?] CRAN (R 4.2.0)
────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
You need to use the function using the API v2, tweet_post(). I didn't deprecate the function because apparently some (random?) users can still work with some API v1 endpoints. Also the API v2 don't allow the "old" rtweet_bot to work (for the same reasons I haven't deprecated (but I am reconsidering this as I need to make a new release soon)
To use the new API v2 you'll need to authenticate via the new OAuth2 mechanism, as described in the vignette. There might be some problems with the recent update on httr2. Please open a new issue with the details too (as nicely done here). Thanks
Thanks for the (very!) quick response. Previously I've been using what I believe is a 3-legged oauth flow, because this bot posts using GitHub actions and needs a persistent authorisation which doesn't rely on a web browser. My understanding is that there's no solution that fits this use-case with the OAuth2 mechanism. Am I mistaken?
Very grateful for your help!
AFAIK no, I think there might be a way but I am not sure it works or how and it is not implemented on the released rtweet.
However, you can authenticate locally and move your credential to a GHA (encrypting the file with a password or removing the user token and adding it on the GHA or some similar methods).
After a bit of pain, I've now got my bot back up and working again. I strongly suspect I've not strictly followed best practice, but the bot seems to be working and my credentials are all secure.
For anyone facing the same predicament, here's what I needed to do:
-
Update my app in line with the latest rtweet documentation. In particular, this meant changing the type of my app from "Web App, Automated App or Bot" to "Native App", and updating the callback URI to http://127.0.0.1:1410/.
-
As directed above, ditch the
rtweet_bot()withpost_tweet()approach in favour ofrtweet_client()/rtweet_oauth2()withtweet_post(). As an aside, I find the naming ofpost_tweet()vstweet_post()quite distressing - it felt like a real gotcha when I realised there were two functions, neither of which reference the other in their documentation. I'd suggest updating the docs to explain the difference, and perhaps renamingtweet_post()topost_tweet2()so intellisense can tell folks there's a new approach. -
This posed a challenge in that
rtweet_oauth2()returns a token which expires within 2 hours, after which time a user must manually confirm in the browser that they're happy for rtweet to continue to use their account. My project is an automated bot running hourly using GitHub actions, so this wasn't an option. Here's the workaround I used:-
After some digging, I found that
rtweet_oauth2()returns an object with a refresh_token which can be used to keep the authorisation active without needing to confirm in a browser. -
The issue is that, this refresh token can only be used once, extending authorisation for 2 more hours. When it's used, the request returns a new refresh token which can be used in the same way, and at the same time invalidates the original token. httr2 errors in such cases to prevent this sort of usage, so I had to use an internal function
httr2:::token_refresh()to bypass the error (NB httr2 1.0 merely warns in such cases, but at the time of writing, rtweet doesn't seem to work with versions later than 0.2.2). -
The refresh token is obviously sensitive, so to avoid exposing it in my source code, I used
httr2::secret_read_rds()andhttr2::secret_write_rds()to keep it saved in its own file, which could then be continuously updated as and when the refresh token is used and regenerated (my bot runs on GitHub actions and saves changes to files by simply committing them to the repo). This approach allowed me add an encryption key as a secret environmental variable to encrypt/decrypt the refresh token in my GA workflow.
-
-
I also ran into an issue in that, unlike
post_tweet(),tweet_post()doesn't make it obvious how to tweet in reply to another tweet. Reading the twitter API docs I found I could do this using, e.g.tweet_post("text", reply = list(in_reply_to_tweet_id = 12345)).
Anyone interested in seeing the full approach can check out the source code at wurli/tidyverse-dev-bot. In particular, see the source for tidyverse_dev_bot_twitter_token().
After a bit of pain, I've now got my bot back up and working again. I strongly suspect I've not strictly followed best practice, but the bot seems to be working and my credentials are all secure.
For anyone facing the same predicament, here's what I needed to do:
1. Update my app in line with the latest [rtweet documentation](https://docs.ropensci.org/rtweet/articles/auth.html#oauth2-protocol). In particular, this meant changing the type of my app from "_Web App, Automated App or Bot_" to "_Native App_", and updating the callback URI to [http://127.0.0.1:1410](http://127.0.0.1:1410/).
I'm glad the documentation helped!
2. As directed above, ditch the `rtweet_bot()` with `post_tweet()` approach in favour of `rtweet_client()`/`rtweet_oauth2()` with `tweet_post()`. As an aside, I find the naming of `post_tweet()` vs `tweet_post()` quite distressing - it felt like a real gotcha when I realised there were two functions, neither of which reference the other in their documentation. I'd suggest updating the docs to explain the difference, and perhaps renaming `tweet_post()` to `post_tweet2()` so intellisense can tell folks there's a new approach.
Yes, I will ditch post_tweet() but for some users it might still work.
I'll update the docs and in the warning I'll provide the new replacement function.
3. This posed a challenge in that `rtweet_oauth2()` returns a token which expires within 2 hours, after which time a user must manually confirm in the browser that they're happy for rtweet to continue to use their account. My project is an automated bot running hourly using GitHub actions, so this wasn't an option. Here's the workaround I used:
This workaround exists only because before httr2 1.0.0 there was no way for httr to handle this scenario. I had to roll my own solution. Which stayed mostly on Github until I released the 1.2.1 on the 2023-10-16 after a prompt from CRAN.
* After some digging, I found that `rtweet_oauth2()` returns an object with a _refresh_token_ which can be used to keep the authorisation active without needing to confirm in a browser. * The issue is that, this refresh token can only be used once, extending authorisation for 2 more hours. When it's used, the request returns a new refresh token which can be used in the same way, and at the same time invalidates the original token. httr2 errors in such cases to prevent this sort of usage, so I had to use an internal function `httr2:::token_refresh()` to bypass the error (NB httr2 1.0 [merely warns](https://httr2.r-lib.org/reference/req_oauth_refresh.html?q=oauth_flow_refre) in such cases, but at the time of writing, rtweet doesn't seem to work with versions later than 0.2.2).
I made compulsory to use 0.2.3 because rtweet depends on a fix on that version in order to manage the token.
* The refresh token is obviously sensitive, so to avoid exposing it in my source code, I used `httr2::secret_read_rds()` and `httr2::secret_write_rds()` to keep it saved in its own file, which could then be continuously updated as and when the refresh token is used and regenerated (my bot runs on GitHub actions and saves changes to files by simply committing them to the repo). This approach allowed me add an encryption key as a secret environmental variable to encrypt/decrypt the refresh token in my GA workflow. 4. I also ran into an issue in that, unlike `post_tweet()`, `tweet_post()` doesn't make it obvious how to tweet in reply to another tweet. Reading the [twitter API docs](https://developer.twitter.com/en/docs/twitter-api/tweets/manage-tweets/api-reference/post-tweets) I found I could do this using, e.g. `tweet_post("text", reply = list(in_reply_to_tweet_id = 12345))`.
Yes, in the functions using the API v2 all documentations link to the API docs so users can make an informed decision. If people use rtweet I might write more example or ease some steps (like #778) but at this moment, my motivation is low.
Anyone interested in seeing the full approach can check out the source code at wurli/tidyverse-dev-bot. In particular, see the source for
tidyverse_dev_bot_twitter_token().
Thanks for adding the link.
As a final note as now httr2 1.0.0 has been released, I might update again rtweet to make it easier to work with the refresh token and deprecate all the old functions (there is also an error in CRAN related to old token in rtweet < 1.0.0).
Hi all. I just tried the changes recommended above but it did not authenticate so I had to abort it and I got the following error. Perhaps they have changed something in the API again?
client_id <- 'acbdefg'
client_secret <- 'ghijklm'
client <- rtweet::rtweet_client(client_id = client_id,
app = 'myapp',
client_secret = client_secret)
user_oauth2 <- rtweet::rtweet_oauth2(client = client)
Warning messages:
1: Thehost_nameargument ofoauth_flow_auth_code()is deprecated as
of httr2 1.0.0.
ℹ Please use theredirect_uriargument instead.
ℹ The deprecated feature was likely used in the httr2 package.
Please report the issue at https://github.com/r-lib/httr2/issues.
This warning is displayed once every 8 hours.
Calllifecycle::last_lifecycle_warnings()to see where this warning
was generated.
@rafapereirabr This is a warning, not an error, and comes from changes in httr2 1.0.0. Do you have the error message?
you're right, @llrs. The function only throws a warning message. The error occurs on the Twitter page. See image below. That's why I asked if perhaps they changed somehting about their api again.
Could you share the output of auth_sitrep(), the version of rtweet you are using and the specific code (and output) used until you get to this screen? Thanks
Hi @llrs . When I run auth_sitrep(), there is no output. I'm using {rtweet} v1.2.1, and here is the code I'm using:
library(rtweet)
###### 0. Authenticate Tweeter API --------------------------------
client_id <- 'blablabla'
client_secret <- 'blablabla'
client <- rtweet::rtweet_client(client_id = client_id,
app = 'cadacantodobrasil',
client_secret = client_secret)
# Authenticate
user_oauth2 <- rtweet::rtweet_oauth2(client = client)
auth_as(user_oauth2)
# auth_save(user_oauth2, "oauth2_authentication")
auth_sitrep()
Ok, you don't seem to have any credential stored not from rtweet < 1.0 or from rtweet > 1.0. The rtweet_oauth2 will be deprecated in the next version, because I won't maintain a custom system to handle authorization for the API and will be directly handled by httr2.
You will need to first save the client client_save(client) just once! and use it in all the other sessions with client_as("cadacantodobrasil"). If the requests to the API v2 (aka tweet_post fail, then there might still be a problem of rtweet: please open a new issue with all the information.
Hi @llrs. thanks so much for the work you've put updating the package! Sorry for my slow response here. I've now updated to the new stable version of rtweet on CRAN v2.0.0 but I still had no success. Here's a bit of context before sharing my code below.
Context: I'm using rtweet to automate on a bot that tweets a satellite image of a Brazilian census tract every 30 minutes. This is the bot. The bot is automated using github actions and all packges are managed using {renv}. The bot used to work well but it was broken after Twitter changed the authentication process, and now I'm trying to put it live back again. As a start, I'm not trying to automate the tweets yet. I'm simply trying to manually authentucate and post a tweet from within R, but I still have no success.
Attempt 1
First I tried this from a fresh R session:
library(renv)
library(rtweet)
# set up your client app
client <- rtweet::rtweet_client(app = "cadacantodobrasil")
#> I manually insert client id and secret on the pop-up window
# save client (JUST ONCE)
#client_save(client)
#> Saving client to 'C:\Users\user\AppData\Roaming/R/config/R/rtweet/clients/.rds'
client_as(client)
#> Using client cadacantodobrasil
# this generates no output
auth_sitrep()
# post tweet
tweet_post(paste0("test ", Sys.time()))
Error in
httr2::req_perform():
! HTTP 403 Forbidden.
Runrlang::last_trace()to see where the error occurred.
Attempt 2
Next, I tried this from a fresh R session:
library(renv)
library(rtweet)
client_as('cadacantodobrasil')
Error in
load_client():
! Can't find saved client with name 'cadacantodobrasil'
Runrlang::last_trace()to see where the error occurred.
Attempt 3
Finally, I tried this from a fresh R session:
app <- rtweet_app(bearer_token = my_bearer_token )
auth_as(app)
# post tweet
tweet_post(paste0("test ", Sys.time()))
Error in
httr2::req_perform():
! HTTP 403 Forbidden.
Runrlang::last_trace()to see where the error occurred.
Thanks for your guidance here!
and here's my
> sessionInfo()
R version 4.3.1 (2023-06-16 ucrt)
Platform: x86_64-w64-mingw32/x64 (64-bit)
Running under: Windows 11 x64 (build 22621)
Matrix products: default
locale:
[1] LC_COLLATE=English_United States.utf8
[2] LC_CTYPE=English_United States.utf8
[3] LC_MONETARY=English_United States.utf8
[4] LC_NUMERIC=C
[5] LC_TIME=English_United States.utf8
time zone: America/Sao_Paulo
tzcode source: internal
attached base packages:
[1] stats graphics grDevices datasets utils methods base
other attached packages:
[1] rtweet_2.0.0 renv_1.0.5
loaded via a namespace (and not attached):
[1] later_1.3.2 R6_2.5.1 httpuv_1.6.15 magrittr_2.0.3
[5] rappdirs_0.3.3 glue_1.7.0 lifecycle_1.0.4 promises_1.2.1
[9] cli_3.6.2 askpass_1.2.0 openssl_2.1.1 withr_3.0.0
[13] compiler_4.3.1 tools_4.3.1 curl_5.2.1 Rcpp_1.0.12
[17] httr2_1.0.1 jsonlite_1.8.8 rlang_1.1.3
@rafapereirabr Please open a separate issue with the detailed information: I'll need to know the client_list() after each step with the client. It is weird that saving the client via client_save results in a file with no name. There might be a problem there.
Note that this repository will be archived soon to signal that there won't be further updates as I stepped down of maintaining it.
I have emailed CRAN about orphaning the package but unfortunately they haven't marked it on CRAN.
If anyone is willing to maintain it feel free to comment.
Thanks @llrs , I will open a separate issue. I'm sorry to hear you'll be stepping down as maintainer of {rtweet}, it's great package. Nonetheless, I understand priorities changes, and maitaining a package can be very demanding, specially with a very popular package. I hope the R community and rOpenSci will find someone to continue your work.