Checking Rate limits before hitting the limits, for `Clients`
ccppoo opened this issue · 6 comments
- This is not about commands.Cooldown
While making app using TwitchIO,
I was aware of API Rate limits, requesting for Stream or User info for example.
So I tried to use Class TwitchHTTP :: bucket (RateBucket) because it updates every time when calling API endpoints
from response headers Ratelimit-Reset Ratelimit-Remaining
I tested with APIs but couldn't see remaining drop under 799 even I tested with
from twitchio import Client
class TwitchApp(Client):
...
async def event_ready(self):
self.test_routine.start()
@routines.routine(seconds=3)
async def test_routine(
self,
):
# generating random twitch user_id
import random
a = [random.randint(100000000, 200000000) for _ in range(100) ]
# not using cache
_ = await self.fetch_users(ids=a, force=True)
print(datetime.datetime.now().replace(microsecond=0))
print(f"{self._http.bucket.tokens=}") # 1
# self._http.bucket.reset_time == 60
# self._http.bucket.limit== 800 (HTTPLIMIT)and at twitchio/http.py
if utilize_bucket:
reset = resp.headers.get("Ratelimit-Reset")
remaining = resp.headers.get("Ratelimit-Remaining")
# Added this line ↓↓↓↓
print(f"{reset=} {remaining=}")
self.bucket.update(reset=reset, remaining=remaining)at
and result was
reset='1660143625' remaining='799'
2022-08-11 00:00:24
self._http.bucket.tokens=1
self._http.bucket.reset_time=60
self._http.bucket.limit=800
=-= =-= =-= =-= =-=
reset='1660143628' remaining='799'
2022-08-11 00:00:27
self._http.bucket.tokens=1
self._http.bucket.reset_time=60
self._http.bucket.limit=800
=-= =-= =-= =-= =-=
reset='1660143631' remaining='799'
2022-08-11 00:00:30
self._http.bucket.tokens=1
self._http.bucket.reset_time=60
self._http.bucket.limit=800
....remaining='799' was same after I run this sample code over 3 minutes
I expected remaining='798', remaining='797', remaining='796', ...
I used token generated from my Twitch account (not Application made from developer's console)
I was expecting remaining decremented everytime test_routine is called
I googled before writing this issue, to understand how Twitch API limits works,
Question : Rate Limiting of twitch API - Twtich Dev Forum
This shows, even it's not a bot(Application) I do have 800 API limits (for my token)
I'm confused with that answer and test result I got from above
Conclusion
- Is there a way to check API
remainingbefore hitting the limit? (not for bot commands but http)
Hello! Thanks for the issue. If this is a general help question, for a faster response consider joining the official Discord Server
Else if you have an issue with the library please wait for someone to help you here.
Is there a publicly accessible way? No. You've hit the right part, however I'm not sure what your worry is. This is only making enough hits to use up one token before twitchs leaky bucket method replenishes it. The library makes use of its own bucket to sleep before hitting ratelimits, and then updates itself to be true to twitch after the request
@IAmTomahawkx
estimated amount of API requests called was about 10K+ after launching my App
(getting data, starting from empty DB)
My interest is estimating and make that warm up(calling APIs for data) as fast as possible not hitting API limits
so I was finding a way to check number of API calls are left in a minute
Hello, using a modified version of your code, and by putting a print in to get raw bucket data in the http code:
class TwitchApp(twitchio.Client):
async def event_ready(self):
print("hi")
for i in range(500):
#await asyncio.sleep(0.2)
await self.test_routine()
async def test_routine(
self,
):
# generating random twitch user_id
import random
a = [random.randint(100000000, 200000000)]
# not using cache
_ = await self.fetch_users(ids=a, force=True)
print(datetime.datetime.now().replace(microsecond=0))
print(f"{self._http.bucket.tokens=}")I was able to get the tokens to increment as they are designed:
https://beta.mystb.in/BritishFatalArctic
I'm seeing no abnormal behaviour here, the lib is behaving as intended, you simply aren't going fast enough with your test code to see it.
The ratelimits are abstracted away from you, if you're interested in seeing them, you'll have to access the tokens as you've been doing
@IAmTomahawkx
Oh I didn't expected that fast requests lol
Yes, I tried to get rate limits under abstract methods
I appreciate for the test code you made,
I'll find another way to get direct access to tokens for someone like me
I'll make another issue or PR for this
feel free to close this issue
The ratelimits are intended to not be accessed by users, as exposing it as a public api forces us to apply semver guarantees to it, meaning we can't make changes as-needed/wanted