frigus02/RESTer

Support for binary responses (e.g. PDFs)

jreed-cartago opened this issue · 8 comments

jreed-cartago commented

I have tested a rest endpoint that provides me with a response containing a raw PDF file. Since there is no way to save the response body to a file, I've attempted to copy and paste the value from the text field displaying the content. The entire content is present as I can see from the PDF, but only the first 18 lines (and the 18th line is not complete) of the content is copied out of the 422 lines of response body.

frigus02 commented

Hi there. That's a good point. There is no good support for binary responses.

For large chunks of text, Ctrl+A, Ctrl+C seems to work fine. I tried a PDF as well (https://www.w3.org/WAI/ER/tests/xhtml/testfiles/resources/pdf/dummy.pdf) and could reproduce the behavior you described.

image

What would be a helpful thing to do here? Is copying even useful? Or should RESTer offer that file as a download?

  • 👍1
jreed-cartago commented

Hi thanks for the fast response.

I'd love to see the ability to simply save the response body as a file. That would be the best option in my opinion. For what I do this is like the only thing that is lacking in this REST client for testing my endpoints.

  • 👍2
frigus02 commented

I had a quick stab at this. Sadly it's harder to implement than I hoped.

RESTer currently stores all responses in a local extension storage (chrome.storage.local). While Firefox seems to be able to store any objects in that, including ArrayBuffers, Chrome does not. I read that this storage is backed by IndexedDB in Firefox. Chrome documents the API like this:

Primitive values such as numbers will serialize as expected. Values with a typeof "object" and "function" will typically serialize to {}, with the exception of Array (serializes as expected), Date, and Regex (serialize using their String representation).

-- https://developer.chrome.com/docs/extensions/reference/storage/#type-StorageArea

I would actually like to switch RESTer to use IndexedDB instead of chrome.storage.local. But that is not something I'm going to have time for in the near future.

All in all this means, I don't have a good idea of how to support binary responses at the moment. Sorry. I'm happy to chat and review PRs, though.

zenglanmu commented

I would like to thank the author of this extension cause it really help me a lot in debugging backend rest apis.
However for this feature, from my personal experience that saving binary file response into history requests is something should not implicated. Because even if switch RESTer to use IndexedDB, the storage space might easily take out by very large file responses. Even display text content (retrieved from fetch response.text()) for binary file response is not very much needed, cause it is just nonsense utf8 string from binary.

I had a quick stab at this. Sadly it's harder to implement than I hoped.

RESTer currently stores all responses in a local extension storage (chrome.storage.local). While Firefox seems to be able to store any objects in that, including ArrayBuffers, Chrome does not. I read that this storage is backed by IndexedDB in Firefox. Chrome documents the API like this:

Primitive values such as numbers will serialize as expected. Values with a typeof "object" and "function" will typically serialize to {}, with the exception of Array (serializes as expected), Date, and Regex (serialize using their String representation).
-- https://developer.chrome.com/docs/extensions/reference/storage/#type-StorageArea

I would actually like to switch RESTer to use IndexedDB instead of chrome.storage.local. But that is not something I'm going to have time for in the near future.

All in all this means, I don't have a good idea of how to support binary responses at the moment. Sorry. I'm happy to chat and review PRs, though.

So for me a quick fix could be adding downloading file support when sending fresh requests. Like following code added into 'request.js' . It depends on the server side to set "Content-Disposition" header for checking if the response should download as file (also for retrieving filename), if so, opens the browser file save dialog for the user.

if (disposition && disposition.value.startsWith('attachment')) { const blob = await fetchResponse.blob(); const filename = getFilenameFromContentDispositionHeader( disposition.value ); downloadFile(blob, filename); } else { const fetchBody = await fetchResponse.text(); response.body = fetchBody; }

If the author feels this approach suitable, I can submit a full pr.

frigus02 commented

Hi @zenglanmu. That sounds like a nice idea. Some questions that popped into my head:

  • Are there requests that have a content-disposition:attachment header, where the user would not like to download the file? If so, how can we handle that? The "attachment" value sounds like a strong hint that this should be downloaded, though. So this might not be an issue.
  • How does a request look like in RESTer's history when then body has been downloaded as a file? The easiest option might be to set response.body to some text like "<body not available; it has been saved as a file because of the content-disposition header>"?

I'd be happy to review a PR for that.

zenglanmu commented

#717 pls check this PR.
if the user would not like to download the file, i think user just can choose to not save the file, like any other file save prompt dialog. A little annoying thorough.

zenglanmu commented

Maybe another way for saving is convert the utf8 encoded text response.body, which also saved in history response, back to blob binary. But this approach may face trucks of encode issues, at least I have not manage to get a correct binary version for text body

frigus02 commented

I just published version 4.11.0. RESTer now saves responses as files if the response has a Content-Disposition: attachment header. Thanks a lot for implementing this, @zenglanmu.