withoutboats/waitmap

optimize how wakers are stored

withoutboats opened this issue · 0 comments

wakers are currently stored as a smallvec of options, with space for 1 waker in the inline repr. there are some essential trade offs here:

  • the larger the inline representation, the larger it increases the repr of values
  • all wakers will be woken at once, keeping them in as close together is good
  • wakers that cancel their interest should not be spuriously woken
  • additional heap allocations have overhead

in particular, right now the implementation never reuses space from cancelled wakers (they are set to None, but their slot is never reused). one option would be to use a proper slab implementation with a smallvec optimization for storing 2 wakers inline (since a slab is 5 words IIRC), and always reuse slots. the downside of this would be for maps which have values of less than 5 words in size, this would increase the size of their values, which is probably a lot of cases.

another option is to just be smarter when all the wakers have lost interest - currently we leave a smallvec of Nones, when instead we could either leave an empty smallvec or even remove the value from the map entirely.