Failure to upload to storage service
mocsharp opened this issue · 1 comments
mocsharp commented
Description
The upload service occasionally fails to upload (C-STORE) received data.
<4> 10:35:18 Monai.Deploy.InformaticsGateway.Services.Storage.ObjectUploadService[4000] => File ID=1.2.826.0.1.3680043.2.1125.1.19616861412188316212577695277886020/1.2.826.0.1.3680043.2.1125.1.34918616334750294149839565085991567/1.2.826.0.1.3680043.2.1125.1.68845315069119806578612768053737857, Correlation ID=8c3a300c-aca0-45fd-9907-4b8f77f6f4c1 Failed to upload file 1.2.826.0.1.3680043.2.1125.1.19616861412188316212577695277886020/1.2.826.0.1.3680043.2.1125.1.34918616334750294149839565085991567/1.2.826.0.1.3680043.2.1125.1.68845315069119806578612768053737857; added back to queue for retry. System.NullReferenceException: Object reference not set to an instance of an object. at Minio.MinioClient.PutObjectAsync(PutObjectArgs args, CancellationToken cancellationToken) in /root/.q/sources/minio-dotnet/Minio/ApiEndpoints/ObjectOperations.cs:line 710 at Monai.Deploy.Storage.MinIO.MinIoStorageService.PutObjectUsingClient(MinioClient client, String bucketName, String objectName, Stream data, Int64 size, String contentType, Dictionary`2 metadata, CancellationToken cancellationToken) at Monai.Deploy.Storage.MinIO.MinIoStorageService.PutObjectAsync(String bucketName, String objectName, Stream data, Int64 size, String contentType, Dictionary`2 metadata, CancellationToken cancellationToken) at Monai.Deploy.InformaticsGateway.Services.Storage.ObjectUploadService.<>c__DisplayClass22_0.<<UploadData>b__1>d.MoveNext() in /app/src/InformaticsGateway/Services/Storage/ObjectUploadService.cs:line 226 --- End of stack trace from previous location --- at Polly.AsyncPolicy.<>c__DisplayClass40_0.<<ImplementationAsync>b__0>d.MoveNext() --- End of stack trace from previous location --- at Polly.Retry.AsyncRetryEngine.ImplementationAsync[TResult](Func`3 action, Context context, CancellationToken cancellationToken, ExceptionPredicates shouldRetryExceptionPredicates, ResultPredicates`1 shouldRetryResultPredicates, Func`5 onRetryAsync, Int32 permittedRetryCount, IEnumerable`1 sleepDurationsEnumerable, Func`4 sleepDurationProvider, Boolean continueOnCapturedContext) at Polly.AsyncPolicy.ExecuteAsync(Func`3 action, Context context, CancellationToken cancellationToken, Boolean continueOnCapturedContext) at Monai.Deploy.InformaticsGateway.Services.Storage.ObjectUploadService.UploadData(String identifier, StorageObjectMetadata storageObjectMetadata, String source, List`1 workflows, CancellationToken cancellationToken) in /app/src/InformaticsGateway/Services/Storage/ObjectUploadService.cs:line 228 at Monai.Deploy.InformaticsGateway.Services.Storage.ObjectUploadService.ProcessObject(FileStorageMetadata blob) in /app/src/InformaticsGateway/Services/Storage/ObjectUploadService.cs:line 179
Steps to reproduce
Please share a clear and concise description of the problem.
- C-STORE a DICOM dataset
- Review the logs
...
Expected behavior
All files are uploaded to storage service
Actual behavior
One or more files failed to upload
github-actions commented