cmake global variables
Closed this issue · 39 comments
This line changes the output directory not only for portmidi but also for all the projects using cmake in the build
https://github.com/rbdannenberg/portmidi/blob/master/CMakeLists.txt#L14
I'll take it out. Should it be replaced with something else? What about the other lines after line 14? This came from another project and I thought it was harmless to support library installations.
I'm really not familiar with using cmake in a modular way like this: I would either just compile portmidi code into my application or build the library separately. Building a library AND making it a sub-project of another cmake project requires some conventions and is probably never going to work except where it has been debugged and tuned to a particular project. But with that caveat, I don't want to get in the way of using portmidi's CMakeLists.txt from another CMakeLists.txt file.
I'm also in the middle of changing the way virtual ports work -- I realized the design was not so good and creating virtual ports could/should be separate from opening them.
This is fixed by the CMake rewrite in https://github.com/mixxxdj/portmidi
But this rewrite dropped a lot of things. If you have some requirements or design docs, it would help to know what you were trying to achieve.
All it dropped were the bindings to other languages. I explained why this was done at mixxxdj/portmidi#8 (comment). The build system in this repository is a major pain to package. I could not get it to build in vcpkg on macOS or Linux and the old vcpkg port required a lot of hacks around the build system to make it work on Windows.
Listing every detail of what is wrong with the build system in this repository would take more effort than I already put into rewriting it from scratch.
OK, I'll see what I can do. I'm pretty sure I can't just drop in your cmake file because of file locations, and I'll have to guess what you are trying to do with it. I guess you are looking for vcpkg compatibility. I've never used it and don't know about its limitations.
Yes, you would need to move files to match https://github.com/mixxxdj/portmidi and copy the packaging directory for the CMakeLists.txt to work. I could make a pull request to do this.
vcpkg doesn't require anything special. A normal CMake build system that doesn't break standard CMake conventions is very easy to package in vcpkg or any other package manager.
OK. That's not much to go on, but obviously you know more about CMake than I do.
I could make a pull request for this repository reorganizing the files and using the new CMakeLists.txt. Would that be welcome?
Only if you support and test building the Java JNI stuff and test on all platforms. I don't want to throw that out without any feedback, as putting it back in would be more work.
I have no interest in supporting Java. Is anyone even using the Java bindings? I can't find any evidence of that. That is the biggest problem with the build system in this repository. Every packager has to patch that out of the build system.
Are you saying that supporting Java interfaces is incompatible with vcpkg or however you want to use CMake?
No, I am saying that requiring Java to build a C library is a major problem. Building the JNI bindings can be reimplemented if they are no longer a requirement to build the C library.
I didn't think Java was ever required, but it should be easy to change if that's the case -- maybe it was just a default.
For my needs only root CMakeList.txt has to be changed
Here's my current plan:
- rewrite virtual device code to separate creation function from open function (which will be done with existing Pm_OpenInput/Pm_OpenOutput function). ... currently debugging on Mac, haven't started Linux, nothing to do for Windows which does not support virtual device
- update repo/site to refer to status of other language bindings, e.g. ocaml
- bring Java JNI bindings/library up-to-date; we'll see if anyone is using this
- reorganize files a bit, move non-working bindings elsewhere
- update CMakeList.txt -- not clear what will happen hear: @sonoro1234 's CMakeList.txt is specific to a particular style of installation, so retaining current functionality and making it compatible with @sonoro1234 's requirements might be difficult. Features I'd like to retain: (1) building both static and dynamic libraries; without renaming static library to portmidi_s.a, CMake/XCode converts libportmidi.a to libportmidi to libportmidi.dylib, so attempts to link statically use dynamic library without warning (need to check if this terrible policy is still in place). (2) Java JNI library building. (3) Some sane approach to getting consistent MSVC runtime flags and avoiding warnings in the linker - MS supports at least 4 different runtime libraries. I think CMake now has MSVC_RUNTIME_LIBRARY property that may work, but not the library has to be specified differently for static and dynamic portmidi libraries. Not sure what to do here, but at least review the options.
- long term: unless there's any activity with Java, remove Java support and remove preferences support for Pm_GetDefaultInputDeviceID(void), replacing it with a policy to just return the first device found to avoid changing the API. Note that historically, Java support was introduced as a way to create a cross-platform GUI to set preferences. This is most useful to support CLI programs that use Pm_GetDefaultInputDeviceID(void) or perhaps allowing simple GUI programs to forgo implementing their own device preferences, put it really doesn't seem worth the trouble.
(1) building both static and dynamic libraries;
Please don't do this. This violates CMake conventions and makes the library hard to package and was another major reason why the build system had to be rewritten from scratch.
Note that historically, Java support was introduced as a way to create a cross-platform GUI
Java is pretty much dead for this purpose and I don't think this is a good reason to maintain Java bindings. But if you really want to maintain those despite nobody using them (as far as I can tell), the most straightforward way to do that and make this library not such a pain for packagers would be to reimplement support for building the Java bindings on my branch (#2). That should be optional and disabled by default.
If you want to make a cross platform GUI application for a C library then I suggest using Qt, not complicating the library's build system with Java bindings.
For the CMakeList.txt part, as the differences are smaller than @Be-ing's fork maybe consider pulling the patch from gentoo or maybe another distro as they likely have similar ones (repology.org is useful for this btw):
- Makes Java optional, as it should be so it's easier to build just the needed bits
- Makes the static version optional, not sure if the way it's done is the best
- Makes the test programs (
pm_test) optional - Apparently also fixes parallel builds with make
- Note that it still fails to compile with ninja, the current build recipe on gentoo forces it to make
EDIT: #2 has a lot of changes but from a quick review it looks like a better start
As for Java I would also lean towards removing it if you do not know if you have any downstreams actually using it as I would strongly doubt anyone would still be using it for GUIs (GUIs are probably one of the worst part of normal cross-platform Java).
Makes the static version optional, not sure if the way it's done is the best
That's not the best way. The build system should not hardcode static or dynamic linking without a technical reason. This should be left for the user/packager to set the conventional way with BUILD_SHARED_LIBS. Building both, even optionally, is weird and requires packagers to hack around that.
Note that it still fails to compile with ninja, the current build recipe on gentoo forces it to make
That's weird. My branch builds with Ninja fine.
Well yes but CMake rather annoyingly lacks the ability to have both, something which for a simple example, meson allows via defaulting to shared (default_library in https://mesonbuild.com/Builtin-options.html ) as having both static and shared can be rather unwieldy at times but is done often for the system packaging of distros as you get software preferring one or the other from time to time.
And building objects twice in the case of distros is rather wasteful but does happens, there is few examples like this in gentoo with different kinds of workarounds.
If both static and dynamic are needed for whatever reason, it's easy to do this with a conventional CMake build system by creating two different build directories, one with BUILD_SHARED_LIBS=OFF and the other with BUILD_SHARED_LIBS=ON.
Hi! I'm packaging portmidi for Arch Linux.
I would like to mention, that for all the years I have packaged portmidi there has been no other package requiring the java bindings (this might be different for other distributions) AFAIK and in fact I eventually dropped them.
I think that the effort to make them work using an improved cmake setup is probably not justified. In the case that someone still wants to use them it is still possible to add them back.
FWIW: I am happy about the improvements done by @Be-ing, as portmidi hadn't seen a release in several years and the svn/tarball based workflow was somewhat intransparent (no offense, it's an old project :)). This is one of the reasons why I switched to the mixxxdj/portmidi repo for Arch Linux and I much welcome the more transparent release workflow via tags. :)
In regards to static/shared libs: I'd follow whatever is the default for cmake.
Hi David,
Don't worry; as far as I know, PortMidi has never required Java bindings or Java and never will.
If you explain your workflow and what tags have to do with it, maybe I can support that too.
I think CMake has a slight bias towards static libraries (I certainly do) but in Linux, dynamic libraries seem to be the preferred way to offer libraries, so that should probably be the default but easily switchable to static.
Thanks for packaging portmidi for Arch Linux!
@Be-ing and @lanodan - thanks for comments. One of the technical issues with static vs shared libs is, at least in my experience, CMake gives them the same base name in XCode projects, and XCode will link to a dynamic library even when you build and specify a static one. There may be problems with Windows as well. Do you have better ways to address these problems? I gather it's important not to build both versions, which seems simple enough. I still have to look into the naming issue, but maybe you don't care if you don't plan to build or package static libraries anyway.
To clarify, are you talking about the XCode C/C++ LLVM toolchain, the XCode IDE, or the XCode CMake generator, or all of these?
XCode will link to a dynamic library even when you build and specify a static one
How do you specify this? By specifying BUILD_SHARED_LIBS=OFF to CMake?
I still have to look into the naming issue, but maybe you don't care if you don't plan to build or package static libraries anyway.
Using different names for static and dynamic libraries requires downstream applications' build systems to handle the additional complexity of searching for both library names. Downstream applications' build systems should just work regardless if the library was built statically or dynamically.
If you explain your workflow and what tags have to do with it, maybe I can support that too.
When working with git (in general, but as a downstream in particular) it is usually much easier to follow up on "which commit was used to create the tarball?", "which changes went into a given version?" and "what changes have been applied after that version?" due to a git tag (the version) pointing at a specific commit in the overall sequence of commits on the default branch of a project. Thus it also becomes much easier to e.g. identify any issues and to apply patches for hotfixes that are already added to the default branch or that are offered in open pull/merge requests (after the released version) using git cherry-pick or downloading that patch straight from github and applying it via patch.
All of this is... rather problematic when looking at svn and sourceforge in particular, as it does not offer the transparency of "which exact revision was the tarball created from?" (unless one versions by revision number, which is sort of a not so great compromise to make when there are more specific versioning schemes) and it makes applying specific patches on top of a given version rather complex as well (e.g. checkout entire source tree, create patch... but from which revision?). Maybe my svn skills just lack, but I always felt that the tooling was rather inferior to git in many regards (transparency being only one).
I'd be very happy if versioning could continue somewhere above the last versions (236) in https://github.com/mixxxdj/portmidi/releases once things have settled for you here (as I otherwise have to "downgrade" the package)! :)
Sorry for hijacking this issue with this rather off-topic information.
Good question. I'm talking about XCode IDE building a project created with XCode CMake generator:
To clarify, are you talking about the XCode C/C++ LLVM toolchain, the XCode IDE, or the XCode CMake generator, or all of these?
XCode will link to a dynamic library even when you build and specify a static one
"How do you specify this?" I really don't remember, and I'm not even sure XCode still behaves like this. It looks like I'll have to investigate.
Using different names for static and dynamic libraries requires downstream applications' build systems to handle the additional complexity of searching for both library names. Downstream applications' build systems should just work regardless if the library was built statically or dynamically.
This is true if you wish for downstream applications to be agnostic with respect to libraries being static or dynamic. But downstream applications often take great care to build one way or the other because it matters. To give a specific example, I built an application with -llibportmidi.a in XCode, copied the binary to another machine for a concert, and it failed because XCode found and linked to libportmidi.dylib. Yes, "Downstream applications' build systems should just work" but the result should not be surprising failure to run. Reading between the lines, I think you are trying to say that building static libraries with the same base names as dynamic libraries is preferable to eliminating the problems it creates. If XCode has not changed it's linking behavior, which would eliminate the need for renaming the static library to libportmidi_s.a, it should be pretty easy to make this an option for those like me who like to play it safe.
@dvzrv Very interesting. From the perspective of PortMidi, I think you would like to see stable releases designated as GitHub "releases" and given tags such as v236, v237, ...? I was planning to go to 2.1 when I get the virtual devices code working properly, but I can continue numbering from SVN numbers -- glad you raised the issue and let's figure it out before doing it.
What I don't understand is, if you want to link a static library, why was there a dynamic library on your system at all?
Because I developed and tested portmidi, which means I test a lot of things and create lots of versions.
I think you are trying to say that building static libraries with the same base names as dynamic libraries is preferable to eliminating the problems it creates.
Yes.
From the perspective of PortMidi, I think you would like to see stable releases designated as GitHub "releases" and given tags such as v236, v237, ...?
The github "releases" are absolutely no necessity (they just allow the developer to leave a note and add more additional artifacts based on a tagged version, which is not necessarily needed in the case of portmidi). Tagging a version will automatically create a tarball and that is usually enough :)
Somewhat related: If you want to go the extra mile and make a poor packager happy, sign your tags with a PGP public key that you establish as your default signing key for this project by making the public key available on keyservers, here on github and mentioning its fingerprint in e.g. the README.
I was planning to go to 2.1 when I get the virtual devices code working properly, but I can continue numbering from SVN numbers
That would be problematic in any way, as with portmidi we are already in the "above 200" version range: https://repology.org/project/portmidi/versions
You can subscribe to a repository and only subscribe for notifications of releases if the repo uses GitHub Releases. I have done this for all the dependencies of applications I maintain that are hosted on GitHub.
If I understand, @dvzrv interprets tags as a releases. That really limits the use of tags. I would make release tags with github's release mechanism in any case.
I think signing would need either a portmidi organization signature or the possibility of different signers or I remain the sole signer. I'm not sure how sharing signing privileges or even getting it to work is any more secure than github authentication. Github says "You can sign commits and tags locally, to give other people confidence about the origin of a change you have made." So is Github saying their storage and authentication is insecure, and they just display, say, "rbdannenberg" as a best guess? But, OK, I'll stop whining and at least try it.
You can subscribe to a repository and only subscribe for notifications of releases if the repo uses GitHub Releases. I have done this for all the dependencies of applications I maintain that are hosted on GitHub.
I use nvchecker for that. At a couple of hundred packages it would become slightly confusing with so many notifications I think 😆
But I do get why one would want to use "github releases" (from a project's perspective). They are usually just not that useful to me personally when packaging (depends on the project though).
If I understand, @dvzrv interprets tags as a releases.
If you mean "github releases" with that, then no. If you mean the general term "a release" with that, then yes :)
I think signing would need either a portmidi organization signature or the possibility of different signers or I remain the sole signer. I'm not sure how sharing signing privileges or even getting it to work is any more secure than github authentication.
It would really be down to just defining "this is the key used for signing releases, this is where you may retrieve it and this is how you can use it to verify a tag signed by it". Github's PGP signature is fairly useless as it does not guard against "other person also being a dev signs a tag with this through the web interface" and it is generally disregarded for packaging purposes.
I finally have virtual devices working the way they should and tested. The current code has been moved to github.com/PortMidi/portmidi. I'm going to close this repo later, but for now I'll change the README to point to the PortMidi project. If y'all want to beat me up about CMake, let's do it there instead of here (it has been significantly changed). Also @dvzrv, if you want to walk me through tags and signing, I'll try to adopt that, but I can't figure it out without specific instructions.
portmidi is now in the PortMidi project