mhx/dwarfs

Allow mounting of merged filesystems

Opened this issue · 5 comments

ruanformigoni commented

Greetings!

I would like to request a feature to mount merged filesystems, i.e.:

# Assume folder-0.dwarfs has 20000 bytes
mkdwarfs folder-0 folder-0.dwarfs
mkdwarfs folder-1 folder-1.dwarfs
cp folder-0.dwarfs folder.dwarfs
cat folder-1.dwarfs >> folder.dwarfs
dwarfs folder.dwarfs mount-0
dwarfs -o offset=20000 mount-1

Currently this is not possible, when trying to mount the folder.dwarfs file I get the error:

E 15:15:05.006702 error initializing file system: dwarfs::runtime_error: truncated section data [fs_section.cpp:56]

This is for both dwarfs commands, with and without the offset.

mhx commented

Something like this isn't going to work unless there's either a major change to the file system format, or you're willing to not only explicitly provide the offset, but also the length of the file system.

A DwarFS image is just a sequence of blocks; there are no explicit begin / end markers, although DwarFS assumes that the last block might be a section index. However, when simply concatenating multiple DwarFS images, there's no way to tell (other than by heuristics) where one image ends and another one starts. I'm not going to add any such heuristics, because they would very likely conflict with future extensions, such as being able to append/update DwarFS images.

So, the only option would be to explicitly provide offset/length of the image within the file.

ruanformigoni commented

To provide the length of the image would be no problem for my use case, is there an option for it?

mhx commented

No, there's no such option yet. It should be fairly simple to add, though.

  • 👍1
mhx commented

Please try the experimental/image-size branch. You should be able to do something like:

$ dwarfs merged.dwarfs /mnt/folder-1 -ooffset=123,imagesize=4321
$ dwarfs merged.dwarfs /mnt/folder-2 -ooffset=4711,imagesize=7999

Let me know if that works for you and I'll pull the change in for the next release.

(I've only written unit tests, but didn't actually test the FUSE driver (yet) — so definitely let me know if it's not working as expected.)

ruanformigoni commented

@mhx thanks for implementing this! I just finished compiling and got the novel option in the help menu -o imagesize=NUM filesystem image size in bytes. It worked on three concatenated filesystems perfectly.

  • 🚀3