emscripten-core/emscripten

Make file system abstraction pluggable

ryanackley opened this issue · 6 comments

ryanackley commented

I'm trying to use emscripten generated code in a browser extension that I eventually want to port to a full blown application that uses webkit.

It would be nice if there was an easy way to build my own file system abstraction that the emscripten generated code could use. My application will use the html5 filesystem or the local native file system.

Maybe there is a way to do this now but it's just not clear to me?

kripken commented

There is no way to do this currently. The emscripten filesystem is an "in-memory" filesystem, the goal is to make things look normal to a compiled C program, but really everything is in JS data structures.

It would be easy to export and import stuff to indexeddb to the html5 file api, if that is what you want. Or, if you want each read and write to go to those APIs, that would require abstracting things to make it truly pluggable. Not impossible but would take some refactoring.

dalbani commented

I am also in favor of a non "in-memory" filesystem. A good use case for that feature would be SQL.js.
It works great on small "memory" files but it doesn't scale to large files (which SQLite databases can be).
I plan to use SQL.js client-side, in the browser, so the HTML FileAPI could be leveraged.
In particular, the Blob interface and its slice() method could be much more efficient that having the whole database in memory.
What do you think?

ysangkok commented

@snotling Emscripten is not only used in the browser, and the Blob/ArrayBuffer is only for the browser. So some abstraction is needed.

dalbani commented

Indeed, I do agree.
In the meantime, is there an "easy" way I could hack a quick Blob-based replacement?
I couldn't figure out how it works, for instance in the context of SQL.js.

briandastous commented

This would be extremely useful for working with large files.

stale commented

This issue has been automatically marked as stale because there has been no activity in the past 2 years. It will be closed automatically if no further activity occurs in the next 7 days. Feel free to re-open at any time if this issue is still relevant.