Add download as stream option to get objects as readable streams

[its-eli-z]

Feature request

Is your feature request related to a problem? Please describe.

As a developer using the storage API, I want to have the ability to get the response body of the fetch request for an object as a stream instead of a final blob (current implementation).

Use cases:

• To have memory-efficient server-side code for returning large objects to client requests, the server should not read the blob into memory and only pipe the stream to the HTTP response body.
• When the server wants to return only parts of the object (according to some business rules). For example, there is an object which is a CSV file, and a server provides an export mechanism based on some filter coming as part of a client request. Ideally, the server would start to read rows from the large file, writing to the client response stream (filtered CSV) only the rows that matched the filter.

Describe the solution you'd like

Today's usage of the download function is like this:

const { data, error } = await supabase
  .storage
  .from(BUCKET)
  .download(PATH)

Data is the final blob, read into memory.

The download function could support an optional options object, initially with only one property, which defaults to false:

interface DownloadOptions {
  asStream?: boolean;
}

So now I can do (for example, stream directly to the client):

const { data, error } = await supabase
  .storage
  .from(BUCKET)
  .download(PATH, { asStream: true })

data.pipe(res)

Describe alternatives you've considered

Not using the supabase Storage API :(

A Suggestion for implementation

Per my understanding, it is easy to do with the standard Fetch API, which is supported in browsers, Node.JS (v17.5+), and deno.

The change should be pretty small, around here.

"Regular" downloads could return response.blob() from the fetch promise result (see docs)

Stream downloads calls could just return the response.body from the fetch promise result (see docs).

Another option is to have a separate downloadAsStream function, leaving the download stream always returning a blob.

I would be happy to help with that :)

[jorgeAgoiz]

hi fellas, on what state is this issue? It would be a very good feature

[Bernix01]

Hello, any update on this?

[amhci]

Would really love this. I'm developing an application that pulls ebooks files stored in a bucket in my server code and then sends that to the client. It would be great if I could stream the files instead of having to wait to download them fully on the server before sending to client!