Disconnect from twitter at same time since 1 of december
StarvingDeveloper opened this issue · 28 comments
December 1 I found in railway logs twitter got disconnected, manually restarted, but on December 2 at excatly same time twitter got disconnected again:
[ INFO/2022-12-01 16:03:15,093] tweepy.asynchronous.streaming: Stream disconnected
[ INFO/2022-12-02 16:03:27,766] tweepy.asynchronous.streaming: Stream disconnected
Looks like something need to be changed.
Having same issue.
@New-dev0 any chances you could look in this issue?
@naujokas2021 you got disconnected today again or not?
December 1 I found in railway logs twitter got disconnected, manually restarted, but on December 2....
Is there a possibility that, Railway interrupt app every night?
Also it's not from twitter side, there is a different error for that.
Does the app/tweets get stucked after that message?
Does the app/tweets get stucked after that message?
Otherwise I would not make new issue - I noticed I am not getting new notifications and went check logs. Restarted once. Next day saw disconnect at exactly same time and saw 429 error:
[ERROR/2022-12-03 07:38:22,381] TgTwitterStreamer: Stream Encountered HTTP Error: 429
[ INFO/2022-12-03 07:38:22,381] TgTwitterStreamer: Refer https://developer.twitter.com/ja/docs/basics/response-codes to know about error code.
[ERROR/2022-12-03 07:38:22,381] tweepy.asynchronous.streaming: HTTP error response text: {"title":"ConnectionException","detail":"This stream is currently at the maximum allowed connection limit.","connection_issue":"TooManyConnections","type":"[https://api.twitter.com/2/problems/streaming-connection"](https://api.twitter.com/2/problems/streaming-connection%22);}
So I just increased sleep time from 10 to 60 seconds as temporary solution and got no problem after that. But not sure its because I made changes in code, or because these 2 disconnections was just casual.
@naujokas2021 you got disconnected today again or not?
I moved out from railways to VPS. No more issues
I moved out from railways to VPS. No more issues
Must be was just 2 casual disconnections. Lets hope we can forget about that problem.
. Next day saw disconnect at exactly same time and saw 429 error:
[ERROR/2022-12-03 07:38:22,381] TgTwitterStreamer: Stream Encountered HTTP Error: 429
Are you using TRACK_WORDS ? or tracking lot of number of tweets?
Or using same credentials in multiple app?
429 is a rate limit error from twitter, due to above possible reasons.
On Sunday got disconnected again
[ERROR/2022-12-04 22:22:12,138] tweepy.asynchronous.streaming: Stream encountered an exception
Traceback (most recent call last):
File "/usr/local/lib/python3.11/site-packages/tweepy/asynchronous/streaming.py", line 84, in _connect
await self.on_data(line)
File "/usr/local/lib/python3.11/site-packages/tweepy/asynchronous/streaming.py", line 938, in on_data
await self.on_errors(errors)
File "/TgTwitterStreamer/tstreamer.py", line 243, in on_errors
LOGGER.info(f"{error['resource_id']}: {error['detail']}")
~~~~~^^^^^^^^^^^^^^^
KeyError: 'resource_id'
[ INFO/2022-12-04 22:22:12,181] tweepy.asynchronous.streaming: Stream disconnected
It's on twitter side since things get changed after Musk take over or what?
@naujokas2021 you got problems on your VPS? Mybe that just railway problems?
It's on twitter side since things get changed after Musk take over or what? @naujokas2021 you got problems on your VPS? Mybe that just railway problems?
No issues since I moved out from railways (2 days already and counting). I do believe there is something from railway side as it was not a first time when Streamer stopped working.
I moved from railway to docker container on my rpi, but today got disconnection anyway:
[ERROR/2022-12-06 16:19:48,083] TgTwitterStreamer: [{'title': 'operational-disconnect', 'disconnect_type': 'OperationalDisconnect', 'detail': 'This stream has been disconnected for operational reasons.', 'type': 'https://api.twitter.com/2/problems/operational-disconnect'}]
[ERROR/2022-12-06 16:19:48,114] TgTwitterStreamer: list index out of range
Traceback (most recent call last):
File "/usr/local/lib/python3.11/site-packages/tweepy/asynchronous/streaming.py", line 84, in _connect
await self.on_data(line)
File "/usr/local/lib/python3.11/site-packages/tweepy/asynchronous/streaming.py", line 946, in on_data
await self.on_response(
File "/TgTwitterStreamer/tstreamer.py", line 88, in on_response
user = users[0]
~~~~~^^^
IndexError: list index out of range
[WARNING/2022-12-06 16:19:48,202] TgTwitterStreamer: <<---|| Stream Disconnected. ||--->>
Seems like Musk fight with bots when he try allow only real people reach twitter.
@New-dev0 any thoughts? Maybe for beginning send notification to TG when stream get disconnected?
Try fetching recent changes. 2a34a72
Bot will try to reconnect on getting disconnect. (default 20 retries)
set MAX_RECONNECT
to higher value, if you think there is a high possibility of disconnect.
Link to a Reference: https://twittercommunity.com/t/peroidically-stream-disconnection/178354
Try fetching recent changes. 2a34a72 Bot will try to reconnect on getting disconnect. (default 20 retries) set
MAX_RECONNECT
to higher value, if you think there is a high possibility of disconnect.
New updated fetched 1 day ago as it was released. Will try 2a34a72
Reconnect to twitter is working fine but after streamer reconnects, tweets dont get sent to Telegram BOT API.
Try fetching recent changes. 2a34a72 Bot will try to reconnect on getting disconnect. (default 20 retries) set
MAX_RECONNECT
to higher value, if you think there is a high possibility of disconnect.New updated fetched 1 day ago as it was released. Will try 2a34a72 Reconnect to twitter is working fine but after streamer reconnects, tweets dont get sent to Telegram BOT API.
@New-dev0
So, tried new update, same issue. Reconnect is working, but no new tweets are sent via bot after reconnecting. Any ideas how to solve it?
@naujokas2021 try checking your twitter developer portal, whether you find something suspicious ?
@naujokas2021 try checking your twitter developer portal, whether you find something suspicious ?
Nothing suspicious. I do believe after Twitter Stream connection reconnect there is an issue with Telegram Bot API communication.
Now solution is to run TgStreamer as service in VPS and add restart every 1 hour. As restart solves the problem instantly.
@New-dev0 I suggest send message to telegram every time stream get connected or disconnected. If there no problem then such message wont annoy anyone. And if someone have problem, he will know your script need attention.
any new news of disconnect ?
I didnt experienced any disconnect message in last 2 days of stream at https://t.me/futurecodeschat
Last few days I was disconnected on daily basic, so I restart container every 6 hours for now, since you was silent. I can disable that and check if I get disconnected again if you want to know.
Ok, 2 days without disconnecting. So it must be something on twitter side.
@naujokas2021 are you still facing disconnect issue or are tweets not getting send ?
I heard Musk on monday launched something against bots and this morning found disconnect again:
[ INFO/2022-12-12 07:37:49,801] TgTwitterStreamer: <<<---||| Stream Connected |||--->>>
[ERROR/2022-12-12 16:02:44,894] tweepy.asynchronous.streaming: Stream encountered an exception
Traceback (most recent call last):
File "/usr/local/lib/python3.11/site-packages/tweepy/asynchronous/streaming.py", line 84, in _connect
await self.on_data(line)
File "/usr/local/lib/python3.11/site-packages/tweepy/asynchronous/streaming.py", line 938, in on_data
await self.on_errors(errors)
File "/TgTwitterStreamer/tstreamer.py", line 243, in on_errors
LOGGER.info(f"{error['resource_id']}: {error['detail']}")
~~~~~^^^^^^^^^^^^^^^
KeyError: 'resource_id'
[ INFO/2022-12-12 16:02:45,062] tweepy.asynchronous.streaming: Stream disconnected
@StarvingDeveloper, it seems that, you are not on latest commit of main
My bad. Just redeployed. Will inform if any news.
Seems stable for now. Tnx for code update and useful tool.