omarryhan/aiogoogle

allow streaming of files / handling of download and uploads within python

Closed this issue · 6 comments

When uploading or downloading a file it is required to specify a path to the contents.
I would want to provide a file-like object instead to be able to stream the data.

So instead of:

async def download_file(file_id, path):
    async with Aiogoogle(user_creds=user_creds) as aiogoogle:
        drive_v3 = await aiogoogle.discover('drive', 'v3')
        await aiogoogle.as_user(
            drive_v3.files.get(fileId=file_id, download_file=path, alt='media'),
        )
asyncio.run(download_file('abc123', '/home/user/Desktop/my_file.zip'))

something like:

class MyFile:
    def write(self, data: bytes):
        print(data)
  

async def download_file(file_id, path):
    async with Aiogoogle(user_creds=user_creds) as aiogoogle:
        drive_v3 = await aiogoogle.discover('drive', 'v3')
        await aiogoogle.as_user(
            drive_v3.files.get(fileId=file_id, download_file=path, alt='media'),
        )
asyncio.run(download_file('abc123', MyFile()))

Hey @FlorianLudwig , I remember someone requesting this feature before 🤔. I recommend we use a different parameter than download_file for passing the writer object though, mainly because I want to maintain backward compatibility as much as possible.
May I know your use case? Do you want to use it as bytes? or just dump the data to a different destination?

@omarryhan my use-case is loading a big encrypted file and want to stream it into the decryption process. This saves disk space as well as time.

I think there might be more use-cases in regards to bigger files, where it is possible to start the processing of it without waiting for it to fully download.

I have no preference regarding the parameter, download_file could decide what to do depending on type or have a different name for it. So it would be backwards-compatible.

@FlorianLudwig yeah that makes complete sense! Just a bit uncertain about whether download_file is the best name for this? Maybe pipe_to or something similar?

Maybe pipe_to or something similar

Sounds good to me 👍

Great! I'll be happy to accept a PR @FlorianLudwig

Released in v3.2.1 thanks to @NabilMostafa