vutran1710/PyrateLimiter

3.x performance drop

Closed this issue · 5 comments

dreo commented

After migrating to 3.x, I've noticed that PyrateLimiter does not scale lineary with request counts anymore (unlike 2.x).

Consider this snipper:

hourly_rate = Rate(500000000, Duration.HOUR)
daily_rate = Rate(1000000000, Duration.DAY)
monthly_rate = Rate(10000000000, Duration.WEEK)

limiter = Limiter(InMemoryBucket([hourly_rate, daily_rate, monthly_rate]))
print(f'Limiter - acquire: {timeit.timeit(stmt=lambda: limiter.try_acquire("id"), number=XXX)}')

Results for 3x:

number=1000: 0.024325600010342896
number=10000: 0.8553861999535002
number=100000: 75.72125629999209
number=1000000: does not finish in 10 minutes

Results for 2x:

number=1000000: 0.1329616999719292

vutran1710 commented

Thats not possible :D

hwalinga commented

@vutran1710 Yes, there really is a gigantic performance drop introduced with 3.0. But with the latest 3.3 that became from unbearable to just 30% compared with 2.0. It seems your latest PR !158 has fixed it. It is still not up to 2.0 levels, but it is probably acceptable.

vutran1710 commented

@vutran1710 Yes, there really is a gigantic performance drop introduced with 3.0. But with the latest 3.3 that became from unbearable to just 30% compared with 2.0. It seems your latest PR !158 has fixed it. It is still not up to 2.0 levels, but it is probably acceptable.

Thats so strange. Can you share with me some snippet so i can try it myself?

hwalinga commented

For 2:

import timeit

from pyrate_limiter import Duration, Limiter, RequestRate

hourly_rate = RequestRate(500000000, Duration.HOUR)
daily_rate = RequestRate(1000000000, Duration.DAY)
monthly_rate = RequestRate(10000000000, Duration.MONTH)

limiter = Limiter(hourly_rate, daily_rate, monthly_rate)

numbers = [100, 1_000, 10_000, 100_000, 1_000_000]

for XXX in numbers:
    timed = timeit.timeit(stmt=lambda: limiter.try_acquire("id"), number=XXX)
    print(f'limiter.try_acquire {XXX}: {timed}')

For 3:

import timeit

from pyrate_limiter import Duration, InMemoryBucket, Limiter, Rate

hourly_rate = Rate(500000000, Duration.HOUR)
daily_rate = Rate(1000000000, Duration.DAY)
monthly_rate = Rate(10000000000, Duration.WEEK)

limiter = Limiter(InMemoryBucket([hourly_rate, daily_rate, monthly_rate]))


numbers = [100, 1_000, 10_000, 100_000, 1_000_000]

for XXX in numbers:
    timed = timeit.timeit(stmt=lambda: limiter.try_acquire("id"), number=XXX)
    print(f'limiter.try_acquire {XXX}: {timed}')

Results:

2.10.0:

limiter.try_acquire 100: 0.0008874609999338645
limiter.try_acquire 1000: 0.008765025000002424
limiter.try_acquire 10000: 0.09226327299995774
limiter.try_acquire 100000: 0.8978742110000439
limiter.try_acquire 1000000: 8.994104164999953

3.2.1

limiter.try_acquire 100: 0.0023891220000678004
limiter.try_acquire 1000: 0.03898568099998556
limiter.try_acquire 10000: 1.3275317239999822
limiter.try_acquire 100000: 285.53246885500005
^C

3.3.0

limiter.try_acquire 100: 0.0012090879999959725
limiter.try_acquire 1000: 0.012968415000045752
limiter.try_acquire 10000: 0.12044174899995141
limiter.try_acquire 100000: 1.212711079000087
limiter.try_acquire 1000000: 12.330737776000092

(Debian 12, Python 3.12.1)

vutran1710 commented

Please upgrade to the latest version (v3.4)

I think you will be pleased.

And thanks a lot for helping out with the snippets and all.

  • 👍1