minio/minio-dotnet

Failing to upload smaller streams having .WithObjectSize() as max possible stream size

w7rus opened this issue · 0 comments

I'm having an ASP.NET Controller incoming form data and request size is limited with following attributes:

//Assume Consts.AvatarMaxFileSize is 10MB

[RequestSizeLimit(Consts.AvatarMaxFileSize + 4096)]
[RequestFormLimits(MultipartBodyLengthLimit = Consts.AvatarMaxFileSize)]
...

Then the minioClient instance forces me to use .WithObjectSize(), assuming limits set on the controller, stream size would be less or equal to Consts.AvatarMaxFileSize, so:

...

//Upload file
var putObjectArgs = new PutObjectArgs()
    .WithBucket(bucketName)
    .WithStreamData(file)
    .WithObject(fileName)
    .WithContentType(fileContentType);
    .WithObjectSize(Consts.AvatarMaxFileSize);

await _minioClient.PutObjectAsync(putObjectArgs, cancellationToken).ConfigureAwait(false);

Which throws an exception when validating CompleteMultipartUploadArgs since ETags dictionary is null, because of following block of code at Minio.MinioClient.PutObjectPartAsync

// This shouldn't happen where stream size is known.
if (partCount != numPartsUploaded && args.ObjectSize != -1)
{
    var removeUploadArgs = new RemoveUploadArgs()
        .WithBucket(args.BucketName)
        .WithObject(args.ObjectName)
        .WithUploadId(args.UploadId);
    await RemoveUploadAsync(removeUploadArgs, cancellationToken).ConfigureAwait(false);
    return null;
}

//args.ObjectSize = 52428800 - stream max possible size
//partSize = 16777216 - stream actual size
//partCount = 4
//numPartsUploaded = 1

This happens when stream size is smaller than maximum possible, causing partCount != numPartsUploaded to return null ETags, removing an object

I can not determine actual file size since i read multipart request as stream, dropping it in memory/temp file does not sound like a good solution, rather than streaming directly

Can this check be removed since it is excessive?