Improve pooling resources.
Closed this issue · 7 comments
Just a few thoughts about pooling resources. In general, there are several resources to manage:
- Connection to channels
- Connection to twitch IRC servers as client
I suggest to use crate like bb8 as pool manager. It's may simplify code and help with connection GC. Also, might worth to pool even main client connections, to allow scale bigger than 1k connections per client. Welcome to discuss.
I was briefly looking at bb8 when trying to implement a PubSub client, but I couldn't figure out how to handle duplicate subscriptions. The same problem applies to IRC. bb8's ManageConnection::connect is basically fn (&self) -> Connection, but how would you ensure no duplicate channels are joined? One approach would be some shared data, but what would you do when you detect a duplicate?
Maybe I didn't understand the crate correctly. If so, please provide a basic description of the use (e.g. how one would join a channel - via the connection or via the manager).
I was briefly looking at bb8 when trying to implement a PubSub client, but I couldn't figure out how to handle duplicate subscriptions. The same problem applies to IRC. bb8's
ManageConnection::connectis basicallyfn (&self) -> Connection, but how would you ensure no duplicate channels are joined? One approach would be some shared data, but what would you do when you detect a duplicate?Maybe I didn't understand the crate correctly. If so, please provide a basic description of the use (e.g. how one would join a channel - via the connection or via the manager).
Good point! Well, I suggest to use bb8 especially as a connection broker, allocate/free. If we assume that client connections are also pooled, we have to use some scheduler entity, which will know about every channel, which client and network connection handle that channel.
Since #206 will be implemented, mentioned scheduler will be able to track channels as deep as needed.
One approach would be some shared data, but what would you do when you detect a duplicate?
Hard to answer simply. In some cases multiple network connections to the same channel is useful, because if channels will be banned, there is not a single connection still listening, but multiple. According to maintainer of this crate, @RAnders00 , twitch API allows to use IRC chat even if channel banned, it's just unavailable to join to IRC. Even if we want to prevent duplicates, that's also possible though using hashmap or something.
bb8 and the likes are sensible for generally stateless connections, such a HTTP client connections, Redis connections, connections to SQL servers, etc. They are fundamentally not suitable for the kind of work that happens in this library, which is the management of stateful connections. The connection pool is custom-built with great care to ensure channels are always reconnected, even in weird unlikely chains of failures and the likes.
For this reason, I'm closing this issue. bb8 and such are not suitable.
If all you wanted to do was to send messages, but not to listen to them, you could make a client using bb8 and the likes.
If all you wanted to do was to send messages, but not to listen to them, you could make a client using bb8 and the likes.
You could also use the Send Chat Message endpoint.