Release 3.7.1 request
Closed this issue · 18 comments
Since last release, there were few bugs fixed and it's been a long time.
Can we have new release to Maven Central?
@pavlus I would like to. But the build infrastructure for releases is currently broken, and I do not have time to fix it. I did some investigation into it a few months ago, but didn't solve it in the time I had. It is on my radar to try again, but I cannot promise when I'll have time. In general, this project is seeking new maintainer(s); my group has mostly transitioned away from using NAR for our Java/native integration efforts.
@ctrueden is there any way I can help fix it?
Also, what are you using for Java/native integration now?
I took another crack at fixing it today. The deploy to OSS Sonatype had been failing with a 400 error, so I thought it might be the problem where the -sources JAR gets attached to the build twice and then deployed twice, with the second deploy failing due to duplicate version. So I made a commit to avoid that. Then I tried releasing 3.7.1.
The good news is: the Travis CI release build passed.
The bad news is: no artifact appeared on OSS Sonatype. Logging in to the web interface, it seems two staging repositories were created and artifacts were deployed (.jar and .pom to the first; -javadoc and -sources to the second), but the staging repositories were not subsequently closed. I tried closing them manually, and it didn't work; looking at the Activities, I see why: failed signature validation: "Missing Signature: '/com/github/maven-nar/nar-maven-plugin/3.7.1/nar-maven-plugin-3.7.1.pom.asc' does not exist for 'nar-maven-plugin-3.7.1.pom'." The .asc file is present in the second staging repository, whereas the .pom itself is in the first.
So, the fact that there are two staging repositories broke things. I wonder if there are two repositories because we use jar goal instead of jar-no-fork for the javadoc and sources artifacts. I will try switching it over to jar-no-fork now, to see if that makes a difference.
With 22e10a0, I made another attempt to keep the build lifecycle from forking. I am releasing 3.8.0 now; we'll see what ends up being staged where. With luck, it will all stage into a single repository, which will then be auto-closed and subsequently auto-released. 🤞
The release build failed again. I don't know why. Maybe we should rebuild the release configuration according to the latest guidelines from Sonatype. Or maybe we should give up on publishing to Maven Central in favor of jitpack. I don't have any more time to worry about it in the near future.
The next step is probably to file an issue on the OSS Sonatype JIRA asking for their help, since I do not know how to proceed in troubleshooting this. Maybe some web searches first, to see if we can find any hints without troubling a human being.
what are you using for Java/native integration now?
@pavlus For the ImageJ Launcher project, we are now using the antrun plugin to invoke CMake. That project builds a single executable with no dependencies apart from libjvm (or libjli), so NAR is of limited utility.
For SLIM Curve we now use exec-maven-plugin to invoke CMake, which uses SWIG to wrap native code for use from Java. We extract and load the native library from Java using native-lib-loader.
For another project (not yet open source, unfortunately, so no links), we are using CMake to build the native code and JNA to access it from Java, which is simple and convenient.
The spirvcrossj project also uses CMake for the native code and then Maven to package into JAR files with native classifiers, as well as for the Java code.
I don't love the complexity of any of those builds, but I also don't feel like NAR made our lives easier when we were using it instead. With the "CMake + Maven" approach, we can build the native code with a first-class native build tool, and the Java code with a first-class Java build+packaging tool, and it works.
@ctrueden I would like to help with this but it seems there isn't much I can do without access credentials to sonatype. Would this be something you would be willing to grant? I have never deployed to oss/sonatype before but do have some basic deploy experience with nexus at my company.
@bchiodo Thank you for the kind offer! I would be happy to configure the permissions you need.
However, what I'd like to do is create a dedicated maven-nar account on OSS Sonatype, which we could share instead with all admins. Right now, that shared account is scijava-ci, but it's also used when deploying artifacts from several other organizations primarily built around https://github.com/scijava. I'd prefer for NAR to have its own dedicated account.
So, I am in the process of creating that now, and will ping back again here when it's ready.
@bchiodo I created a new maven-nar Sonatype account, which we can use to do the deploys henceforth. The GitHub repo here has not yet been updated to use it, though. I sent you a Dropbox link via private mail with the credentials. Let me know if you need it shared some other way.
@ctrueden Thanks for getting me that info. Since this project uses the maven-release-plugin to prepare/do releases, it requires a releaser (me, potentially) to have push rights to this repo, which I do not have. How would you like me to handle this?
I can attempt to deploy locally by not doing a release first (change version manually to non-snapshot -> mvn clean deploy -Psonatype-oss-release) and this might be ok if the release fails but will be very awkward if it succeeds.
Otherwise I am not so sure how to debug this because I don't see any staging repo left on sonatype from the last failure.
@bchiodo I invited you to the org. Check your email, or visit here, to accept.
If you are going to make changes to the master branch, please work on eliminating the dependency on the travis-build.sh script of scijava-scripts. That branch assumes deployment takes place using the scijava-ci user, which is now no longer the case for us (attempting to deploy to OSS Sonatype as that user will now fail; we need to use maven-nar instead).
We likely also want to switch to deploying snapshots to OSS Sonatype instead of to the SciJava Maven repository. The NAR project used to deploy snapshots to OSS Sonatype, but it kept breaking, so to eliminate the headaches we switched over to maven.scijava.org.
If you cannot figure out how to deploy to Central with a reasonable amount of work/time, we could alternately switch to Jitpack.io. With Jitpack all our deploys would be automatic and on-demand. The downside is that people would need to add the jitpack repository to their <repositories> or settings.xml in order to use nar-maven-plugin, which would be unfortunate.
@bchiodo Thanks for digging into the CI situation. Keep us posted on what you find. BTW, I do not feel strongly about AppVeyor versus some other Windows-supporting CI platform. For example, I've seen projects increasingly switching to Azure pipelines lately, and would have no objection to that, if you think it would be better.
And just in case it's not clear already: the actual releases are currently driven by Travis CI on Linux, not Windows. So the broken releases and broken Windows builds are two separate issues.
@ctrueden So far I have no complaints about Appveyor -- it seems to work well and is fairly easy to use. I don't have access to any Windows systems for development so needed to use Appveyor itself to debug all the problems with Windows builds. I am not a Winows developer (haven't programmed for Windows since college) so it took a bit of poking around to figure out what was going on.
- The first problem was that the invoker plugin was using the system provided jna loader, instead of the project provided one. This caused all executions (tests) to fail. I defined the
<jna.nosys>true</jna.nosys>property to fix this. - After fixing this, some tests were still failing with "cannot open file" errors, usually during linking. This ended up being related to the max path length in Windows which is 260 (WTF, right?!). I modified Appveyor to set the LongPathsEnabled registry item, then reboot the VM before the build. This works some of the time. Just because the system supports longer paths, doesn't mean all system utilities do... so I ended up having to shorten test names.
- Several it-tests were either not using __declspec(dllexport) correctly, or at all, so the tests wouldn't link. Fixed all of these as well.
I will combining all of this into a single commit and merging into master. Glad to be able to help. Now that I have verified that Windows and Linux builds are succeeding, I will try to free up some time to tackle the sonatype problems.
@ctrueden So I took a stab at doing a release today... and have sort of mixed results. So I locally executed:
mvn release:clean release:prepare
mvn release:perform
And now we have version 3.9.1 on sonatype. I was kind of expecting that command to fail but strangely it did not. I could not figure out how to have Travis-CI do the release or why a local release succeeds and a Travis-CI one does not. Could you explain what steps you take to get Travis-CI to execute a release? I would like to get this automated to some degree and locally releasing may be a decent stop-gap measure but not a good long term solution.
Thanks.
@bchiodo Sorry for the delay in reply, and thank you for the progress you have made!
Right now, the NAR plugin borrows the infrastructure from the SciJava projects. The scheme is that the releaser runs release-version.sh locally, which does the mvn release:prepare step and pushes the resulting release.properties file to a new release tag. Then the Travis CI infrastructure picks up on that tag and completes the build using mvn release:perform via travis-build.sh. It works well for all the SciJava components, but fails for NAR, for reasons I don't understand.
If no one can figure it out, we could file an issue on the OSS Sonatype JIRA asking for help. But it would probably be for the best to stop leaning on the SciJava build infrastructure here anyway, since NAR is not directly part of SciJava. I don't have strong preferences at this point. Local releases are also fine, as long as there is someone active with credentials to do them.
Since version 3.9.1 is now available on Central, I am closing this issue.