Realtime micro-services
Elytum opened this issue · 7 comments
Hi !
Feathers-distributed is quite useful, but I hit a wall when it comes to events and channels.
In a schema where users access the micro-services through a gateway, and therefore connection only exist gateway-side, how should feathers-distributed be used to propagate the events accordingly ?
Example: A gateway with a user service, a store service and an order service.
When an order is added to a store, users with read access to it should receive every order events related to the store.
Thanks !
Maybe I do not get you correctly but it seems to me that feathers-distributed already dispatches required real-time events for your use case under the hood by default. You can easily see this in action by running our example. In this example, there is a gateway app + two todos service apps. A simple HTML page connects using WebSockets to the gateway then requests the todos stored in the todos services through the gateway. If you then perform a create operation on one of the todos service using eg Postman and a POST REST request you will see the created event poping in the WebSocket from the gateway to the client.
Hi ! Thanks for taking the time to answer.
I did manage to make it work that way, but every event propagation condition (Permissions etc) using channels had to be implemented on the gateway itself, which defeats a bit the goal of microservices. Was I doing it wrong or is it something feathers-distributed doesn't handle ?
I don't think it is specifically related to feathers-distributed. Whatever the option ("standard" app or "distributed" app) you need to manage event propagation depending on your constraints (eg permissions) there is no magic here. feathers-distributed is basically a backend tool and does not deal with the client-side, this is done using the usual "Feathers way" (ie channels). So I think you have two options:
- filter events at the service level (eg your order service) so that the gateway will not receive it if you don't need it
- filter events at the gateway level using channels as usual if your gateway needs to be aware of all events but not your client
Take care that in the gateway scenario only the gateway manages the channels as the client is not directly connected to services, so you cannot do that at the service level.
If you have a generic permissions system, handling the dispatching should not be a big deal, eg have a look to our Akt'n'Map app. More information on this in our docs.
Ok, so the best way to reproduce the channel system accross multiple micro-services is to use an external pub/sub server which itself handles channels and is consumed on the gateway to propagate the appropriate data to the appropriate connection ?
Not sure to understand what you mean, I don't think you need an external pub/sub server as Feathers events/channels is already one. Events are pub/sub in the backend side, while channels are pub/sub in the frontend side. So your gateway will act as the pub/sub server filtering events from the backend (and all microservices thanks to feathers-distributed) to dispatch it to the right channels on the frontend.
It's probably me explaining/getting it wrong, sorry.
Considering a case with 10+ micro-services, I try to separate the micro-services as much as possible.
Events filtering in the gateway will probably be my strategy for now, but for code maintability, I try (which in retrospective might not be a good idea) to affect the datas to the correct users in the emitting service (Other stores should not be able to see orders not related to them, and different roles might give access to only parts of orders).
As a side benefit, adding a whole new service with filtering being independant of gateway would avoid a rolling update (kubernetes) of gateway, which would have had to be updated if the filtering code was handled in it.
Also, considering the possible quantity of events, sending them to all gateways instances when only one or two should propagate maybe is overkill, isn't it ?
It really depends on your use case. Of course if you have thousands of events per second it might be a good idea to filter upfront. However, checking against permissions could require knowing all connected users and only the gateway can do that in this case as users are not directly connected to the order service. Imagine there are administrators seeing all orders in real-time and users only seeing their own orders. So when you create an order you should dispatch it to all administrators, as the order service doesn't know if there is any administrator currently connected (only the gateway knows) it has to dispatch the event to the gateway, no easy means to filter it. In some others cases you might probably find an appropriate way to filter upfront.
The argument about the separate deployment of gateway/filtering is probably the best one so that if the permission system changes you avoid updating the gateway. But IMHO it does not solve the problem of scaling with a large number of events and collaborative permissions.