Orange-OpenSource/fiware-cepheus

Add guaranteed delivery to Cepheus Broker

marcc-orange opened this issue · 5 comments

marcc-orange commented

When the Cepheus Broker does not manage to forward updates to a remote broker, it just ignores the failed updates (just logging an error).

The broker could support a guaranteed delivery mode where not delivered updates would be stored locally (in memory or persisted on disk) and send back to the remote broker when the connection is recovered.

Optionally, the broker could smartly only keep the latest update for a given context entity to limit local storage and traffic when historic data is not relevant.

sabrine2020 commented

Hello,
yes that is the features that we need in Cepheus.

Best reagrds,
Sabrine

sabrine2020 commented

Hello,
any updates on this feature? and how can we help?

Thanks & best regards,
Sabrine

marcc-orange commented

Hi @sabrine2020,
Sorry, if my previous comments could sound misleading. Orange currently has no plan to implement this feature any time soon, I just created this issue to track any progress or question around this subject from other potential contributors.

As mentioned on SO in the original discussion, any contribution to the project is welcomed. Feel free to fork the repo, possibly contribute with a PR that we could merge if feature can be toggled by configuration.

pooja1pathak commented

Hi @marcc-orange , I have some queries regarding this issue:

  1. For storing updates locally, which memory should we use like tmp or any other?
  2. To support a guaranteed delivery, it should send update or check connection for n number of times, till the remote broker's connection is recovered. After what interval we need to send updates to remote broker or check if connection is recovered or not?
  3. What if Cepheus broker instance is restarted, do we need to start sending these files again?
  4. How long these files should exist in memory?

I want to contribute on this issue. Please guide me through my queries.

marcc-orange commented

Hi @pooja1pathak, I did not work on any of these guaranteed delivery scenarios. I suppose this could be implemented using persisted queues. I can't speak about the original demand but I suppose for a production grade implementation, something like Kafka could be used.