Restarting byodc deployments occasionally leaves cashboxes locked
gregzip opened this issue · 2 comments
gregzip commented
Describe the bug
When we restart the byodc deployment for whatever reason, some cashboxes stay locked in Redis, leading to locking timeouts until we manually clear the lock.
We're not talking about anything forceful, rather a regular:
kubectl -n our-byodc-namespace rollout restart deployment byodc
To Reproduce
Restart the deployment while pods are processing requests.
Expected behavior
Locks are properly released. Requests to new pods work properly.
Screenshots
N/A
STDOUT/STDERR
fail: Microsoft.AspNetCore.Server.Kestrel[13]
Connection id \\\"0HMPKUK46AJIA\\\", Request id \\\"0HMPKUK46AJIA:00000007\\\": An unhandled exception was thrown by the application.
System.TimeoutException: Failed to acquire lock after 00:01:00. Timeout
at fiskaltrust.SignatureCloud.DE.Services.Locking.RedisLockService.ProcessLockiningAsync[T](String lockName, Func`1 executeMethod) in /src/src/fiskaltrust.SignatureCloud.DE/Services/Locking/RedisLockService.cs:line 36
at [fiskaltrust.SignatureCloud.DE.Controllers.SignController.Post](http://fiskaltrust.signaturecloud.de.controllers.signcontroller.post/)(ReceiptRequest request) in /src/src/fiskaltrust.SignatureCloud.DE/Controllers/SignController.cs:line 111
POSSystem (please complete the following information):
- b2460c21-f12b-4b2c-9bf4-a9487f8ede57 (Memento 8.225.0)
Cashbox Information (please complete the following information):
Whatever cashbox is currently processing a request.
Additional context
N/A