corrade_add_resource slow compile
Opened this issue · 4 comments
I think clang and gcc struggle with the big strings in the generated cpp files? My Magnum projects with a few KB of shaders compiles in ~5sec but including a 25MB model and it takes 2.5min to build. Compile times could be kept low by embedding resources with incbin or ld or objcopy?
Hi! The tool has its limitations, I'm aware of that.
I was impatiently waiting for the C++ proposal for std::embed, but after seeing the latest non-progress on it I gave up all my hopes :(
Incbin, ld, objcopy are solutions, unfortunately each of them is tied to a particular subset of platforms, and I don't know for example what would work for webassembly / asm.js builds -- I don't think inline assembly would work for those. What could be doable is having the current way as a fallback, and opt-in for the other platform-specific ones where avavilable. But honestly it sounds like a maintenance nightmare 😅
Another idea I had in mind is figuring out what variant of specifying the data is the fastest to compile. Can't find a reference right now, but somewhere I saw that compilers parsed C string literals ("") faster than a list of byte values. I also see that the generated array has an implicit size, which might be causing the compiler to take more time than strictly necessary. Can't say when I'll be able to get to this, but I'll do some measurements and experiments to trim down the compile times a bit.
I like the design and workflow of the resource utility and can understand compile times being low priority vs its current cross platform, low maintenance implementation. In the mean time I cut and pasted something similar together that loads local resources based on last modified time to avoid recompiling while iterating on models or whatever.
avoid recompiling while iterating on models
I wanted to mention this but then forgot (sorry) -- for faster turnaround times, you can point the Resource to the original *.conf file, causing it to ignore the compiled-in data and fetching the files from the filesystem instead: Utility::Resource::overrideGroup(). This API is extremely buried and undiscoverable, I'll freshen up the docs a bit to make it easier to find.
By the way, regarding iteration times, have a look at Utility::Tweakable as well ;)
Utility::Resource::overrideGroup does what I need. Probably skimmed over its name and assumed it wasn't relevant to what I was doing. And Utility::Tweakable looks really interesting, I'll try it out. thanks!