Fix running `.add()` right after `.remove()`

[ehmicky]

The .remove() method takes 2 ticks to fully resolve. This creates a race condition the same stream is removed then re-added in the same tick. .remove() ends up resolving right after .add(), which cancels it.