Activating specific users with Redis AOF causes quadratic writes
nate00 opened this issue · 2 comments
nate00 commented
When Redis's AOF persistence is enabled, activating x specific users causes quadratic writes to disk, since each additional user causes the entire set of activated users to be written.
Considering only the IDs that get written to the AOF, and assuming 8 bytes per ID, we have:
| Number of users added | Bytes written to AOF |
|---|---|
| x | 4 * x ** 2 + 4 * x bytes |
| 1 | 8 bytes |
| 1_000 | 3.8 megabytes |
| 10_000 | 381 megabytes |
| 100_000 | 37.3 gigabytes |
(For some background, I wanted to enable a feature for tens of thousands of users, selected based on a costly-to-compute criterion.)
nate00 commented
I'd be happy to submit a PR that allows bulk activations or adds a warning to the README about activating many specific users.