Parallel upload in PutObject is only selected for os.File, not for other io.ReaderAt instances. Possibly bug?

[jcasc]

The parallel uploading functionality is not invoked for io.ReaderAt, only for os.Files, because the isReadAt function only returns true for os.Files, not for any other io.ReaderAt instance.

Expected Behavior

From the way the code and its comments look, it appears that the parallel upload implemented should work for any Reader which implements ReadAt.

// putObjectMultipartStream - upload a large object using
// multipart upload and streaming signature for signing payload.
// Comprehensive put object operation involving multipart uploads.
//
// Following code handles these types of readers.
//
//  - *minio.Object
//  - Any reader which has a method 'ReadAt()'
//
func (c Client) putObjectMultipartStream(ctx context.Context, bucketName, objectName string,
	reader io.Reader, size int64, opts PutObjectOptions) (n int64, err error) {
	
        if !isObject(reader) && isReadAt(reader) { // this last check returns false for generic ReaderAt instances
		// Verify if the reader implements ReadAt and it is not a *minio.Object then we will use parallel uploader.
		n, err = c.putObjectMultipartStreamFromReadAt(ctx, bucketName, objectName, reader.(io.ReaderAt), size, opts)
        ...

Current Behavior

Due to the implementation of the isReadAt function, this type of upload is only performed for os.File, not for other io.ReaderAt instances.

// Verify if reader is a generic ReaderAt
func isReadAt(reader io.Reader) (ok bool) {
	_, ok = reader.(io.ReaderAt)
	if ok {
		var v *os.File
		v, ok = reader.(*os.File) // this does not shadow the outer ok variable, instead it overwrites it!
		if ok {
			// Stdin, Stdout and Stderr all have *os.File type
			// which happen to also be io.ReaderAt compatible
			// we need to add special conditions for them to
			// be ignored by this function.
			for _, f := range []string{
				"/dev/stdin",
				"/dev/stdout",
				"/dev/stderr",
			} {
				if f == v.Name() {
					ok = false
					break
				}
			}
		}
	}
	return
}

This happens because the check for os.File overwrites the ok variable, which is the return variable.

Possible Solution

Modify these lines as such:

	v, ok_ := reader.(*os.File) // preserve value of return variable
	if ok_ {

This would still allow the function to exempt stdin etc. while accepting other non-file io.ReaderAt-implementing objects.

Steps to Reproduce (for bugs)

1. pass a bytes.Reader (which implementes ReaderAt) to PutObject
2. observe the code paths taken by debugging / print-debugging
3. observe that isReadAt returns FALSE

Context

In case that this is working as intended, the comments and/or documentation could potentially be improved by stating that this type of upload is only performed for os.Files, not for any ReaderAt.

Your Environment

go 1.22 on m1 mac
go.mod: github.com/minio/minio-go v6.0.14+incompatible

[harshavardhana]

Please send PR

[ferhatelmas]

It's already fixed in v6.0.18. I think resolution is to upgrade the client.

[jcasc]

Thanks, you're right. I did go get github.com/minio/minio-go without /v7 so i got v6.0.14. (not the newest v6). I'm on v7 now and the problem is gone. 👍

[harshavardhana]

You need to use v7 - go get github.com/minio/minio-go/v7 https://github.com/minio/minio-go?tab=readme-ov-file#download-from-github