Archives and custom patching
Closed this issue · 4 comments
Hi John,
Just curious with this new approach to packaging up dependencies, what would be the way to apply patches to archives, run a patch diff over the top via the build scripts after extracting? I've had to patch some cmake files (or bring in custom cmake build files) for some of these dependencies on the windows side, or symbol export some internals like with python. Maybe the linux/macOS side doesn't need many custom patches?
Cheers!
Hi Alex,
We're starting to use archives now because storing all the source files was making the repository really unwieldy. Another benefit I hadn't considered is that supporting different platforms should be easier with archives, because we can easily store different patches for each platform.
I think there's only one example of patching so far - for OSL on Mac. You should just be able to use diff to generate similar patch files for yourself...
Cheers...
John
Hey John,
I've been looking into this ExternalProject_Add feature of CMake right now and testing a few packages with it. It kind of looks like a nice way of abstracting the system's command line while also benefiting from some of CMake's features, and exporting out a FindXXX.cmake for another library to ingest. Downloading from a url or git cloning external libs is also an option with patching support. You could even clone this dependencies repo into Gaffer for instance and make it a part of Gaffer's (soon-to-be unofficial) CMake build scripts.
I managed to get a dedicated Linux machine again so I can prototype this on there first by porting the exact commands you issue in the shell scripts, I don't have macOS here to test but I'll make sure to add the custom flags for it as well in an OS check.
Cheers
Hey Alex,
That looks interesting - thanks for pointing it out. It seems like you still need to provide it with raw commands for the patch and build steps though - there's no magic, right? I'm still somewhat inclined to roll something similar ourselves so we can make it more data-driven and make just the minimal set of features we need, in a scripting language a bit more manageable than CMake. My plan is to try to make a simple prototype of that when I next have to add a new dependency...
Cheers...
John
Yeah agreed there are much better choices than CMake for syntax, it's the sheer widespread adoption of everyone using CMake now and easily integrating it into other things that I'd vouch for it still over something custom.
Conan.io was also my other option but it doesn't give a lot over this ExternalProject_Add other than it being written in Python and has a binary caching mechanism which you can deploy pre-built binaries to the cloud for any OS/compiler combo (it locally caches lib dependencies to a hidden .conan/ directory into a hashed directory with its include/lib directoreis exported). But the dependencies are all bundled here and you need them packaged up with Gaffer itself so that feature isn't really needed here.
I'm using this as a nice guide for making a gafferDependenecies CMake project, they make an ext_patch dependency on windows that downloads gnu patch on there, and I'll do the same for ext_zlib to dynamically download/compile but omit those on Linux/macOS:
https://github.com/KDE/krita/tree/master/3rdparty
Later I was thinking of making some rez config files with it for anyone who would prefer to use rez here also...