Support blob/large data
mlynch opened this issue · 40 comments
Investigate passing blobs from native to JS and vice versa.
There is this issue about improvements and one of the is about blobs. Should we close this one or better to have an issue per improvement?
#974
Great to see this on the roadmap. In the meantime, if working on a custom Capacitor plugin, are there any creative workarounds that might allow returning a blob/binary data without having to convert to JSON? Based on https://capacitor.ionicframework.com/docs/plugins/ios, it seems like the CAPPluginCall can only handle JSON right now, and that no amount of creative plugin writing could respond with a blob directly. Is that right?
The background here is that we're trying to let the user import and upload 20MB PDF files with our app. The current workaround is converting the binary data to base64 using FileSystem.readFile, and then converting it back to a blob in the app, and uploading to a remote server as XHR. Trying to avoid memory issues, and unable to do a direct Fetch on a non-HTTP URI (e.g. file://etc on iOS). Thanks
Fetch should work, but you have to use convertFileSrc on the file:// url
@jcesarmobile thank you! I was using convertFileSrc while running ng serve and getting a CORS error on the "capacitor://localhost" URL, but I see now that fetch is working with convertFileSrc on a production build. For ng serve, after doing convertFileSrc, replacing "capacitor://localhost" with window.location.origin gave me a URL that works just fine for fetch. Appreciate the quick help.
Sharing my workaround with you.
I'm using web-implementation of the Filesystem plugin.
Don't know if it's a good solution but it helps solving my problem.
import { FilesystemDirectory, FilesystemEncoding } from '@capacitor/core'
// const { Filesystem } = Plugins // => Doesn't work with blob in native
// Workaround: Use Web-Implementation of plugin
import { FilesystemPluginWeb } from '@capacitor/core/dist/esm/web/filesystem.js'
const Filesystem = new FilesystemPluginWeb()
I was able to create a service for writing and reading large blobs. (For web, blobs are already supported for indexeddb)
Reading is simply done with a fetch call.
Writing is a bit more tricky because we can only pass strings to that native layer.
Converting a large blob to a base64 string will crash the webview because the size is simply to big to handle. However we can split the blob into chunks and convert these smaller chunks to a base64 string. Then we can just use appendFile to stitch it back together.
Would it be possible to integrate this into the Filesystem plugin so it is supported directly? Would you consider a the PR if I make one? Or is there a better solution?
I just had an idea of how we could theoretically stream large amounts of binary data directly to the filesystem from the webview. We could avoid the strings-only bridge by making a POST request which is intercepted by Capacitor, and the body written to disk. This could either be implemented within Filesystem.writeFile and Filesystem.appendFile, and/or be exposed as an HTTP endpoint like so:
const res = await fetch('https://stuff.com/video.mp4')
const myVideoBlob = await res.body()
// write options like `recursive` could be passed in this query string?
const writeUrl = '/_capacitor_file_/data/user/0/com.example.app/files/videos/1.mp4?recursive=true'
await fetch(writeUrl, {
method: 'POST',
body: myVideoBlob,
})Thoughts?
I've implemented this functionality as a plugin, see https://github.com/diachedelic/capacitor-blob-writer
Blob writer plugin looks like an elegant solution, impressive benchmarks. Thanks @diachedelic for sharing!
I am upgrading to Capacitor 3 today and I was really hoping this made it in. Sad that it is not in there yet. Hope it makes it in there soon!
Some meta commentary: this is now the 6th (and related to the 4th) highest rated issue on Capacitor and blocking the oldest issue in Capacitor's plugin repo
Looking forward to this too. Meanwhile I am using:
- the cordova-plugin-x-socialsharing and file plugins for native apps
and
- the web share api on the web (when available and when file extensions are permitted)
Seems to me this is a gigantic hole in the capacitor ecosystem. I mean, binary data is a thing; and not everything is a "string"... The idea that capacitor has no real support for anything but strings seems pretty short-sighted. On our project we have a need to pass an ArrayBuffer from JavaScript to a capacitor plugin that we are writing. I thought for sure this would "obviously be possible", but alas here I am and I'm amazed that the only solution is kludgy workarounds. In my case I'll convert to base64 because the size of the binary data is always smaller than around 50kB. But come on man... how can this still not be fixed?
We have also managed to hit this problem. Filesystem should work with blobs - we cannot have files being saved and read as base64. Blobs are being used by the native code and then converted to base64 anyways.
@mlynch @jcesarmobile any chance this could be added to the 4.0 Milestone? 😍
Any update?
I'm in the middle of migrating my app from cordova to capacitor.
For offline usages I download a large file (100Mb) and then unzip it (in memory in JS code) and store its data in indexDB.
This unfortunately crashes when using the capacitor filesystem plugin.
I'm currently using two plugins which I wish I could remove:
- The ability to download a large file straight into the device filesystem (FileTransferPlugin as this fails in "classic" cordova)
- The ability to open a large file and use it as a blob (or arraybuffer)
Here's my current code and future code if anyone is interested to see how this is being used in my app:
Currently using cordova file plugin and file transfer plugin
https://github.com/IsraelHikingMap/Site/blob/e10acfbfcc15d7f4601bb923581254f3d1ee1f68/IsraelHiking.Web/src/application/services/file.service.ts#L346L354
https://github.com/IsraelHikingMap/Site/blob/7784673508d3ee8ce22e91e04b74af818295d7f5/IsraelHiking.Web/src/application/services/file.service.ts#L339L350
@HarelM I'm not sure about the unzipping part, but I was able to get large files (~100mb) to save to the file system by downloading the file in multiple chunks using the Range http header. My code saved the first chunk to the file system, then appending the following chunks to the end until the file was complete.
That's very nice! But I would love this to be a part of the file system plugin as I don't think every developer using capacitor should implement this.
If not implemented, I would suggest to add your example in the docs so that people can use it in these "edge" cases.
In my case, since I can't read large files and I need to download large files I use cordova-plugin-file and cordiva-plugin-file-transfer which work well.
@HarelM I've went back through an app I've made and grabbed some chunks that touch the capacitor fs plugin and removed anything that isn't capacitor related.
Some parts are a little messy.
Feel free to peruse it, hopefully it can help more people than the capacitor docs did.
https://gist.github.com/spartanatreyu/6ba9dd416b9a9a5b3ccd6026ecfd1de1
@spartanatreyu thanks for sharing!! What you wrote is amazing :-)
Having said that, I would prefer if some of it will happen in the native code in order to avoid blocking the UI thread in some cases and in general, while I can achieve using what you wrote there, I prefer to have it as part of the capacitor library.
The current solution of taking the file plugin and file transfer plugin is good enough so that I don't need to find alternatives.
This will not be the case if this was part of this plugin of course as I would prefer to use capacitor plugins and reduce the dependency on Cordova plugins (due to maintenance issues).
I have a similar abstraction layer here:
https://github.com/IsraelHikingMap/Site/blob/07cbd143b3839ff02b3545bbc768dd324a7b541a/IsraelHiking.Web/src/application/services/file.service.ts
Which in theory can be replaced by your code in some or most places, but again, I'm not sure there's a good benefit for replacing the code...
If the capacitor team would like to point me in the right direction in terms of where and maybe how to write this code I would be happy to donate my time as I think this is an important project and I will relay heavily on it once the migration is complete. but I'll need some guidance and help to achieve this... :-)
I encounter problems with UTF-8 encoding. Umlauts in German or currencies ($, €) are converted wrongly. I do not encounter this issue when using the Browser API to generate a download link.
if (content instanceof Blob) {
write_blob({
data: content,
path: filename,
directory: Directory.Cache
});
} else {
await Filesystem.writeFile({
path: filename,
data: content,
directory: Directory.Cache,
});
}
@mlynch @jcesarmobile any chance this could be added to the 5.0 Milestone? 😍
To address this issue, I created a plugin specifically designed to handle large files. It is based on the capacitor-blob-writer, but with added capabilities for reading and writing files in chunks, as well as encryption support for improved security. I encourage you to try out this plugin : https://www.npmjs.com/package/capacitor-file-chunk
Thanks @qrclip! Does this plugin supports downloading large files as well?
I'm looking into removing cordova file and cordova file transfer plugins.
I've opened the first issue in your repo and provided my feedback, thanks! :-)
qrclip/capacitor-file-chunk#1
Great, I will do my best ... after all, it's the first issue! 🥇
Stale, and this should be handled by plugins - re-file under @ionic-team/capacitor-plugins if it's still an issue and an issue isn't open there.
This is still very much an issue. I'm currently using the Cordova file plugin to overcome this issue. Is this planned as part of the file plugin of capacitor?
I would just like to remind everyone that this is a hard problem. A reliable and simple solution not currently possible, given iOS and Android's WebView APIs. (More info) I vaguely remember a (W3C?) proposal to support binary data transfer over the native bridge, but I can find no trace of it.
@diachedelic you sure? I know one method → WebWorker and FileSystemApi. Data can be send as blob to web worker which sends it to native platform file system. Idk if capacitor can hook into ww apis but its a way. Fallback would be the current method with base64.
Some reads:
That could work if native code is able to access the WebView's OPFS.
OPFS by itself does not seem any better than IndexedDB, because the data could be deleted at any time. From https://webkit.org/blog/12257/the-file-system-access-api-with-origin-private-file-system/:
storage lifetime is the same as other persistent storage types like IndexedDB and localStorage
Also from that page:
Based on implementation of different browsers, one entry in the origin private file system does not necessarily map to an entry in user’s local filesystem — it can be an object stored in some database. That means a file or directory created via the File System API may not be easily retrieved from outside of the browser.
So there is no guarantee that native code will have access to the contents of the OPFS.
But there must be a solution.
If I load an image / video via capacitor://localhost/_capacitor_file_/{path} into a img / video tag I can display local files in the WebView.
Why does that work?
And could there not be a similar solution for writing local files via local WebServer API?
Webworker also has a limit on the amount of data you can transfer. Even if you transfer(Transferable objects) 50 MB instead of copying, it will crash on many Android devices. If you use the QRClip plugin, you can read and write files of any size in chunks. You have two options: You can use the Capacitor Bridge, which is slower, especially on Android, or a web server (idea and base code come from the Capacitor Blob Writer). The web server has a big disadvantage if the application goes into the background, as in this case on iOS stops immediately. With the plugin you can encrypt and decrypt the data and thus avoid the localhost http security concerns. QRClip can easily handle large files with the plugin, but it only needs to save and read in chunks. When opening a video it would be more difficult as you need to be able to process chunks rather than store a large file in memory in its entirety.
In the case of a map application, it would probably be best if you could read the maps in chunks from the file system. This would reduce loading times etc. And the application would require very little memory.
@qrclip I think it is already possible to read a file in chunks, using HTTP Range requests.
I have noticed that the web server used by the Capacitor Blob Writer plugin is not fully reliable. Do you think it is only because the app goes to the background? How did you solve it?
@black-hawk85 Files requested from capacitor://localhost do not actually ever reach a web server. Rather, they are intercepted by Capacitor's native code configuring the WebView.
I had hoped that we could use the same trick for uploading files, but neither Android nor iOS's WebView APIs seem to support it. See the Android bug and the iOS bug. The iOS bug is marked as fixed, but I have not verified it.
@diachedelic , I needed a way to save and open large files that don't fit in memory. With Capacitor you can easily attach to a file, but there was no good option for reading, which I only needed when I receive shared files from other apps, because if they are selected within the app, I can easily read them in chunks with the HTML file. My first approach was the Capacitor Blog Writer approach. I took it as a basis and developed the plugin so that it can write and read in chunks via the web server. As far as I know, a 500 MB file is posted in full by Capacitor Blob Writer, my plugin does it in configurable chunks, and for me 10 MB is a good value. I also have the option of reading files in chunks without using a web server. This is simply using the Capacitor Bridge to read a chunk of a file, but with base64, which is not fast. At the moment this is the approach I am using. I plan to offer the other approach for Android in the future, but that has yet to be tested. On iOS, the base64 conversion speed is actually good, but on Android it's much slower. The problem is that every Android version works differently and it's a nightmare to release something that works on all situations(devices, android version, user options).
I haven't thought about using HTTP range requests yet. But the problem is that I'm dealing with some files outside the app range and don't know if it would work. I searched for alternatives back before I created the plugin, but this option didn't come up.
Thanks for the issue! This issue is being locked to prevent comments that are not relevant to the original issue. If this is still an issue with the latest version of Capacitor, please create a new issue and ensure the template is fully filled out.