New changes in EI 2.2.1 queuing up events in Event-handler threads causes events lost if EI crashes or restarts.
Opened this issue · 0 comments
Description
A potential critical issue can occur in the new EI 2.2.1 when number of SubscriptionHandler threads has been changes from unlimit to limit number of aggregation process.
This causes events process is stuck in waiting state in EventHandler threads if SusbcriptionHandler Queue is full and QueueCapacity is reached.
If EI crashes or restarts events/aggrations will be lost.
If QueueCapacity is reached, then event and aggregations might be lost as well, depending on if threads is blocking or not blocking when trying to add aggregationobject to the SubscriptionHandlerThread queue.
Most of the events might have been aggregated and some not, but no SubscriptionHandler thread will never be created for those Events aggregations and no matching with Subscriptions will never be made.
Better to remove those two ThreadHandlers and use only one Threadhandler and ThreadPool, and let Events to be handled in AMQP queue instead of the Java application memory.
Or revert the 2.2.1 version changes.
Motivation
Avoid loss of Events/Aggregation and keep the persistent of events in the AMQP queue.
Exemplification
Benefits
Persistence of un-processed events and aggregations.
Memory consumption by EI will probably be reduced.
Other resource consumption will probably be reduced also, due to less threads and queue threads in memory.
Load on MongoDb will might be decreased.
Processing number of events from AMQP queue will probably be the same.