Is Bincode The Best Serialization Format for Cooked Shader Packages?
zicklag opened this issue · 1 comments
Related to a comment ( bincode-org/bincode#221 (comment) ) in the bincode repository ( emphasis mine ):
I think you put it well in that this is mainly supposed to be used internally by Rust programs; I might go further and say that it should only be used by the same program because you must be sure that the serialization and deserialization must be implemented exactly the same.
I'm just wondering if, due to the instability of the format maybe it would be more suitable to use a format like CBOR that will be stable regardless of which version of bincode Rust, etc. that is used to cook the shaders.
It seems like it would be a really easy change to make.
Maybe we don't technically need the stability, but I'm immagining a situation where the baked shaders are commited to git, and then you upgrade your project without updating the shaders, and now the baked shaders fail to load. Even in that case, I suppose you would probably know right away that its because you need to re-compile your shaders.
The disadvantage of CBOR would be that it is less compact than bincode, which I just realized is also used for assets, where the compactness is probably a bigger deal.
On another note, there's also rkyv if we want a zero-copy format, but it doesn't natively handle different endianness.
Let me know if I'm just being annoying by bringing up ideas for changes when I've hardly used the library or gotten a solid understanding of what's best for the project yet. ;) It helps me identify project priorities and what the general design preferences are when I get into a new project.
The long term plan is that distill will read shader files directly, although it’s unclear how that would work with the rust code generation. This can not be done right now because a distill import currently cannot look at multiple files when importing an asset, and shaders can #include other shaders.
At some point I’d like to have a common wrapper around all binary data identifying a type/version of a format. I don’t think it’s worth changing the format to something else right now. But this topic might come up again when I start looking at exporting binary data from blender and importing it with distill.