Move KP throttle outside fetcher lookup
maximusunc opened this issue · 0 comments
maximusunc commented
When we have 75 multi-queries going, each query has its own throttle, but they aren't aware of each other and we end up with 75x our rate limit. Not to mention we have 4 separate uvicorn workers, but that might be inevitable.