Build system doesn't seem to respect '-j' for parallel builds
VorpalBlade opened this issue · 5 comments
It seems that the build system mostly runs serially even though I'm passing -j30. It also seems to invoke pkg-config --cflags protobuf way too many times (again serially). It really only should need to invoke that once.
I would attempt to debug this, but since you use a home grown Makefile based solution (rather than autoconf, cmake, meson or something similar) I'd rather not.
I'm on commit ab9d6cf.
All I can say is that it works for me. I'm on a 24-core system and do parallel builds as a matter of course.
Regarding build systems... I'd love to use an established one, but every one I've established so far falls down in some way. autoconf/automake is almost impossible to make valid parallel makefiles with, meson isn't scriptable, bazel can't make mingw32 binaries (believe it or not, it's hard coded to use mingw64 exclusively) and in general is very hostile to open source builds, etc. cmake is a possibility but I'm having trouble getting past the awful scripting language; plus, it doesn't do combined release/debug builds, which I'd really like to do.
I might look into this (probably won't be for over a week at least). Do you have any documentation of how the build system is set up? I.e. what the overall design of it is.
Also, I think parallel builds used to work for me. It seems like a new thing that they broke. I use Arch Linux (and packaged fluxengine for AUR). It was when rebuilding fluxengine against newer system dependencies that I noticed the slow build and started investigating. Possibly due to being on a rolling release distro I noticed some issue sooner than you would have done.
It's a fairly simple non-recursive makefile using implicit patterns to actually build things based on variable names, automake-like. The only complex bit is the use of macros to add dependencies. src/build.mk shows how to build a binary, lib/build.mk shows a library. The main rules which actually do the work are in the top-level Makefile. It relies heavily on rule-specific variables.
There are bugs; sometimes it won't rebuild things when it should, and I need to sort those out. As I said, I'd love an alternative, but so far haven't seen anything. I wish the meson author would add proper scripting; it's so nearly exactly what I want...
I wish the meson author would add proper scripting; it's so nearly exactly what I want...
By proper scripting, do you mean functions?
Yes. Without them I can't generate abstract rules which can be used in multiple places in the build script. The language also has some other awkward gotchas, like a unified global scope meaning submeson files won't play nice with each other (or, at least, it did the last time I looked).
I do actually have a mostly-working Python ninja file generator I should look into polishing up, which ought to work here.