dotnet/orleans

using PersistentState with Redis provider could cause an error if data is loss.

scalalang2 opened this issue · 1 comments

scalalang2 commented

Motivation.

Redis uses asynchronous replication, which can lead to data loss in certain cases.

  • Leader A accepts a SET request from user A.
    • It returns OK if the request is successful.
    • User A acknowledges that everything is OK.
  • If Leader A suddenly goes down, replication may fail.
  • A newly elected leader might not be aware of the request from user A.

Grains activated with RedisPersistentState cannot serve requests in this case.
It'll keep emit InconsistentStateException until the grain is deactivated.

We want our services to be available even if Redis loses small portion of data.

Suggestion.

Provides a configurable way to allow inconsistent state.

bool AllowInconsistentState { get; set; } = false; // This is not allowed by default.
scalalang2 commented

I'll close this issue.

https://www.vldb.org/pvldb/vol17/p3720-eldeeb.pdf

Recall from §2.1.3 that multiple instances of the same grain might be active simultaneously in some failure cases.
It is therefore necessary to do a write to transaction storage to perform an ETag check and
ensure that the grain version read by the transaction is indeed the latest and not a stale version.