allow streaming of files / handling of download and uploads within python
Closed this issue · 6 comments
When uploading or downloading a file it is required to specify a path to the contents.
I would want to provide a file-like object instead to be able to stream the data.
So instead of:
async def download_file(file_id, path):
async with Aiogoogle(user_creds=user_creds) as aiogoogle:
drive_v3 = await aiogoogle.discover('drive', 'v3')
await aiogoogle.as_user(
drive_v3.files.get(fileId=file_id, download_file=path, alt='media'),
)
asyncio.run(download_file('abc123', '/home/user/Desktop/my_file.zip'))
something like:
class MyFile:
def write(self, data: bytes):
print(data)
async def download_file(file_id, path):
async with Aiogoogle(user_creds=user_creds) as aiogoogle:
drive_v3 = await aiogoogle.discover('drive', 'v3')
await aiogoogle.as_user(
drive_v3.files.get(fileId=file_id, download_file=path, alt='media'),
)
asyncio.run(download_file('abc123', MyFile()))
Hey @FlorianLudwig , I remember someone requesting this feature before 🤔. I recommend we use a different parameter than download_file
for passing the writer object though, mainly because I want to maintain backward compatibility as much as possible.
May I know your use case? Do you want to use it as bytes? or just dump the data to a different destination?
@omarryhan my use-case is loading a big encrypted file and want to stream it into the decryption process. This saves disk space as well as time.
I think there might be more use-cases in regards to bigger files, where it is possible to start the processing of it without waiting for it to fully download.
I have no preference regarding the parameter, download_file
could decide what to do depending on type or have a different name for it. So it would be backwards-compatible.
@FlorianLudwig yeah that makes complete sense! Just a bit uncertain about whether download_file is the best name for this? Maybe pipe_to
or something similar?
Maybe pipe_to or something similar
Sounds good to me 👍
Great! I'll be happy to accept a PR @FlorianLudwig
Released in v3.2.1 thanks to @NabilMostafa