If providing a large-scale service doesn't work, can we just provide an RSS service?
mycfctech opened this issue · 15 comments
If providing a large-scale service doesn't work, can we just provide an RSS service?
RSS traffic is heavier than you might expect. @animegrafmays do you think this is feasible? It would be sad to reduce Nitter to only be able to view single tweets (without comments) and the main "Tweets" tab on a profile + RSS, but I suppose it's better than nothing.
in 24 hours we've had 653,736 total requests (not including media) with 40,174 of those being strictly rss or roughly 6% of total requests were rss. could work if some limits were imposed. the majority of issues we face are people who's rss reader is configured to check every 10 seconds or so. once a minute per account would be tolerable and manageable i think
RSS is cached for 15 minutes by default, so shouldn't matter really. The problem is once you switch to just RSS, the scrapers will follow. nitter.net made about 8 million API requests daily for comparison (~100 per second). Back when RSS was still enabled, 50-100k feeds would be cached at any given moment. At the low end, 50k feeds being updated once every 15 minutes would mean 4.8 million API reqs daily, or 55 per second.
people who's rss reader is configured to check every 10 seconds or so
Ha. That's being pretty generous. I've seen readers (even the more "mature" ones) make up to 20-30 different requests at the exact same millisecond.
It would be sad to reduce Nitter to only be able to view single tweets (without comments) and the main "Tweets" tab on a profile + RSS, but I suppose it's better than nothing.
For what it's worth (if anything), me personally that's why I used Nitter in the first place. I could not care about seeing replies (both from the account itself and comments in reply to the tweet). Just show me the tweet.
I vote for this solution. Much better than using actual twitter. Please implement.
The RSS service was invaluable, if nitter can be more sustainably continued by limiting it to that use case, I'm all for it.
I, too, would like to see the RSS service continue at the very least. I'll be honest, I found Nitter long ago because I was searching for a way to convert Twitter posts to an RSS feed for my browser's Live Bookmarks and my phone's RSS Aggregator App. As great as it was that Nitter allowed us to browse Twitter without ads and bloat, I found the RSS component to be beyond invaluable. Keeping the RSS function, even without the rest of Nitter, would definitely provide a search that doesn't seem to exist anywhere else out there.
Somehow Google serves the tweets so makes sense
In terms of making RSS feeds even better I'm using the following two commits on my private instance:
The first one in particular makes RSS much, much better by including quoted tweets in the feed.
To make the feed perfect it would be great to have a link to videos included, which seems to be more practical than the code necessary for embedded videos to work:
Firstly, a huge thank you to @zedeus and all the contributors to this incredible project. It's individuals like you who ensure we still have an open and free corner on the internet, away from the clutches of corporations and capitalism.
Regarding the RSS service, it would be fantastic to have it available separately, perhaps provided in Nitter and as another repository. This way, those who only want RSS can easily access it.
I fully agree, however, for better readability, (rss)tweets should include the cited content, not just links to it.
If it's just RSS, see https://github.com/DIYgod/RSSHub
Looking at the issues tracker for RSSHub is seems to me that their Twitter RSS is broken?
Edit: It works but IMHO isn't very good. There is no image caching for starters. Nitter is much better.
Apparently rss-bridge does work correctly, but it requires api, so whatever.