gradlex-org/extra-java-module-info

Add option to automatically turn plain Jars into automatic modules based on Jar name

Closed this issue · 8 comments

mikewacker commented

Summary

Additional option:

extraJavaModuleInfo {
    deriveAutomaticModuleNamesFromFileNames = true
}
  • add the Automatic-Module-Name entry automatically to all Jars that are not Modules
  • based on the name of the Jar
  • fails, if Jar name is not a valid module name

tl;dr: I just want to add a few implied automatic modules to the module path without managing transitive dependencies.

To demonstrate, let me construct a very lightweight example module with three easily-met requirements:

  1. The module includes a module-info.java file.
  2. It uses Immutables to create @Value.Immutable interfaces (and generate their implementations).
  3. It needs Google Guava as a dependency.

To repro, we can create a minimalistic Java file (note that the generated code will differ based on whether Guava is present or not):

package org.example;

import org.immutables.value.Value;

@Value.Immutable
public interface Empty {}

Here's the corresponding module-info.java:

module org.example {
    requires static org.immutables.value;
    requires com.google.errorprone.annotations;
    requires jsr305;
}

(jsr305 is an implied automatic module generated by com.google.code.findbugs:jsr305.)

Let's try to get his code to build with Maven first, and then with Gradle.

Maven

I only have a passing knowledge of Maven, but it didn't take very long for me to generate a working pom.xml file:

<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0"
         xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
         xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
    <modelVersion>4.0.0</modelVersion>

    <groupId>org.example</groupId>
    <artifactId>maven-module</artifactId>
    <version>1.0-SNAPSHOT</version>

    <properties>
        <maven.compiler.source>17</maven.compiler.source>
        <maven.compiler.target>17</maven.compiler.target>
        <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
    </properties>

    <dependencies>
        <dependency>
            <groupId>org.immutables</groupId>
            <artifactId>value</artifactId>
            <version>2.10.0</version>
            <scope>provided</scope>
        </dependency>
        <dependency>
            <groupId>com.google.guava</groupId>
            <artifactId>guava</artifactId>
            <version>32.1.2-jre</version>
        </dependency>
    </dependencies>
</project>

Now granted, one could criticize Maven for including an implied automatic module on the module path, but this build logic just works.

Gradle

This won't build out-of-the-box; Gradle will complain that it can't find module jsr305. Now let's 1) temporarily get rid of the module-info.java file, and 2) just add the plugin to the build.gradle.kts:

plugins {
    `java-library`
    id("org.gradlex.extra-java-module-info") version "1.4.2"
}

repositories {
    mavenCentral()
}

dependencies {
    compileOnly("org.immutables:value-annotations:2.10.0")
    annotationProcessor("org.immutables:value:2.10.0")

    implementation("com.google.guava:guava:32.1.2-jre")
}

This actually will not build either:

Execution failed for task ':lib:compileJava'.
> Could not resolve all files for configuration ':lib:compileClasspath'.
   > Failed to transform failureaccess-1.0.1.jar (com.google.guava:failureaccess:1.0.1) to match attributes {artifactType=jar, javaModule=true, org.gradle.category=library, org.gradle.libraryelements=jar, org.gradle.status=release, org.gradle.usage=java-api}.
      > Execution failed for ExtraJavaModuleInfoTransform: /home/mike/.gradle/caches/modules-2/files-2.1/com.google.guava/failureaccess/1.0.1/1dcf1de382a0bf95a3d8b0849546c88bac1292c9/failureaccess-1.0.1.jar.
         > Not a module and no mapping defined: failureaccess-1.0.1.jar
   > Failed to transform jsr305-3.0.2.jar (com.google.code.findbugs:jsr305:3.0.2) to match attributes {artifactType=jar, javaModule=true, org.gradle.category=library, org.gradle.libraryelements=jar, org.gradle.status=release, org.gradle.usage=java-api}.
      > Execution failed for ExtraJavaModuleInfoTransform: /home/mike/.gradle/caches/modules-2/files-2.1/com.google.code.findbugs/jsr305/3.0.2/25ea2e8b0c338a877313bd4672d3fe056ea78f0d/jsr305-3.0.2.jar.
         > Not a module and no mapping defined: jsr305-3.0.2.jar
   > Failed to transform j2objc-annotations-2.8.jar (com.google.j2objc:j2objc-annotations:2.8) to match attributes {artifactType=jar, javaModule=true, org.gradle.category=library, org.gradle.libraryelements=jar, org.gradle.status=release, org.gradle.usage=java-api}.
      > Execution failed for ExtraJavaModuleInfoTransform: /home/mike/.gradle/caches/modules-2/files-2.1/com.google.j2objc/j2objc-annotations/2.8/c85270e307e7b822f1086b93689124b89768e273/j2objc-annotations-2.8.jar.
         > Not a module and no mapping defined: j2objc-annotations-2.8.jar

I was expecting a solution where I could add this plugin and this short snippet, and the build would just work like it did with Maven.

extraJavaModuleInfo {
    automaticModule("com.google.code.findbugs:jsr305", "jsr305")
}

What I actually got is a solution where I also have to do extra work to manage all my transitive dependencies—which will become a real pain for larger projects. Each dep added could potentially introduce new build errors.

Now perhaps there are some valid reasons for this design decision w.r.t. managing transitive dependencies. But if I have to do all this extra work just to get Java modules to work with Gradle, why not just use Maven—where things just work out-of-the-box.

For many real-world projects, we can realistically expect that you're going to have to take some dependencies that don't use named modules. (E.g., for Undertow and XNIO, you will have to consume an implied automatic module as well.)

jjohannes commented

This is unrelated to this plugin.

This plugin is about patching existing Jars to add a module-info.class so that all Jars are real modules and you can fully use the Java Module System as originally intended.

The behavior in Maven you describe is to put everything on the --module-path, no matter if the Jars are compatible or not. This may fail for certain setups (invalid names, split packages). In your example it works, but there are many constellations where it does not. And "it works" still means that you do not fully use the module system as there are Automatic Modules involved. The Automatic Modules mechanism in Java only exist as a kind of intermediate compatibility. A lot of the advantages of the Module System are lost once you involve automatic modules. So if you fully want to use the Module System, you should only use dependencies that are already real modules (Guava is not google/guava#2970).

But if you really want the behavior of Maven you describe in Gradle, this is a Gradle core issue on the topic: gradle/gradle#12630 (comment)

Feel free to share your argumentation there.

With current Gradle versions, you could re-configure tasks in your build to get the Maven behavior:

tasks.withType<JavaCompile>().configureEach {
    doFirst {
        options.compilerArgs.add("--module-path")
        options.compilerArgs.add(classpath.asPath)
        classpath = files()
    }
}

tasks.withType<JavaExec>().configureEach {
    doFirst {
        jvmArguments.add("--module-path")
        jvmArguments.add(classpath.asPath)
        classpath = files()
    }
}
  • 👍1
mikewacker commented

It's ultimately your call since this is your project, but I would ask that you reconsider.

I'd ask this: what is more likely to attract users to this plugin?

  1. One day, they woke up and had this burning desire to convert all JARs to real Java modules.
  2. They ran into a (well-)known issue where Gradle cannot not find a module when it's a filename-based automodule.

My money is on (2), especially when even javax.inject:javax.inject:1 is a filename-based automodule.

It's also worth noting that the benefits of Java modules are not all-or-nothing. E.g., you can still benefit from strong encapsulation for your module's code, even if some of your dependencies have filename-based automodules.

This could very well be a case where the perfect (real modules for all JARs) is the enemy of the good (strong encapsulation for your code), especially if people decide to either not use Java modules or switch to Maven. (In my case, I just switched to Maven.)

(Also, I had some issues getting your workaround to work.)

iherasymenko commented

@mikewacker, like @jjohannes mentioned, this is the way how Gradle works and treats non-modular JARs. This plugin in particular allows you to add the missing metadata either by defining the proper module-info.class or by adding the Automatic-Module-Name entry to the JAR Manifest. While the former is recommended to benefit from the module system the most, the latter will allow you to ("quickly") solve your problem and make Gradle place the JAR on the module path. The example is below.

   extraJavaModuleInfo {               
         automaticModule("javax.inject:javax.inject", "javax.inject")
   }
mikewacker commented

The main problem that I noted there is that you have to do this for all your transitive dependencies, not just the modules that have a requires in module-info.java. (See the error message from the original post.)

jjohannes commented

Maybe I misunderstood the request a litte bit. My point is that – from a technical perspective – this plugin cannot be changed to provide exactly the same behavior that Maven has to Gradle:

  • Gradle modifies JARs (when using this plugin)
  • Maven does not modify JARs

That being said, I could imagine that we add an option to add the Automatic-Module-Name entry automatically to all Jars that are not Modules. This option would use the name of the Jar file as Module Name (same as what Java does implicitly). It can fail at build time for invalid names and inform the user to explicitly define a name for the problematic Jar. I personally do not see much value in such an option. But maybe it is interesting as one step in migrating an existing project to Modules. Or just for experimentation.

I can imagine something like this:

extraJavaModuleInfo {
    autoCreateAutomaticModules = true
}

(Happy about bettter suggestions for how to call this option.)

If that is what you are looking for @mikewacker, I can reopen this issue (and adjust title and description accordingly).

mikewacker commented

It's moreso that things just work in Maven. From that perspective, it doesn't really matter if that's accomplished by adding plain JARs to the module path, or by converting all plain JARs to (explicit) automatic modules.

AFAIK, Maven includes plain JARs on the module path, but it generates a warning:

[WARNING] *********************************************************************************************************************************************************************************************************************
[WARNING] * Required filename-based automodules detected: [jsr305-3.0.2.jar, undertow-core-2.3.9.Final.jar, xnio-api-3.8.8.Final.jar, javax.inject-1.jar]. Please don't publish this project to a public artifact repository! *
[WARNING] *********************************************************************************************************************************************************************************************************************

(Note: This warning only lists plain JARs that the module directly requires in module-info.java. It doesn't list all the plain JARs that you transitively depend on.)

As a simple example, let's use one of the most common Java dependencies: Guava. Here is the error that you get if have a very simple build file with only the com.google.com.guava:guava:32.1.3-jre dep and an empty extraJavaModuleInfo{} section. (Let's assume that your module never requires a plain JAR in module-info.java.)

Execution failed for task ':lib:compileJava'.
> Could not resolve all files for configuration ':lib:compileClasspath'.
   > Failed to transform failureaccess-1.0.1.jar (com.google.guava:failureaccess:1.0.1) to match attributes {artifactType=jar, javaModule=true, org.gradle.category=library, org.gradle.libraryelements=jar, org.gradle.status=release, org.gradle.usage=java-api}.
      > Execution failed for ExtraJavaModuleInfoTransform: /home/mike/.gradle/caches/modules-2/files-2.1/com.google.guava/failureaccess/1.0.1/1dcf1de382a0bf95a3d8b0849546c88bac1292c9/failureaccess-1.0.1.jar.
         > Not a module and no mapping defined: failureaccess-1.0.1.jar
   > Failed to transform listenablefuture-9999.0-empty-to-avoid-conflict-with-guava.jar (com.google.guava:listenablefuture:9999.0-empty-to-avoid-conflict-with-guava) to match attributes {artifactType=jar, javaModule=true, org.gradle.category=library, org.gradle.libraryelements=jar, org.gradle.status=release, org.gradle.usage=java-api}.
      > Execution failed for ExtraJavaModuleInfoTransform: /home/mike/.gradle/caches/modules-2/files-2.1/com.google.guava/listenablefuture/9999.0-empty-to-avoid-conflict-with-guava/b421526c5f297295adef1c886e5246c39d4ac629/listenablefuture-9999.0-empty-to-avoid-conflict-with-guava.jar.
         > Not a module and no mapping defined: listenablefuture-9999.0-empty-to-avoid-conflict-with-guava.jar
   > Failed to transform jsr305-3.0.2.jar (com.google.code.findbugs:jsr305:3.0.2) to match attributes {artifactType=jar, javaModule=true, org.gradle.category=library, org.gradle.libraryelements=jar, org.gradle.status=release, org.gradle.usage=java-api}.
      > Execution failed for ExtraJavaModuleInfoTransform: /home/mike/.gradle/caches/modules-2/files-2.1/com.google.code.findbugs/jsr305/3.0.2/25ea2e8b0c338a877313bd4672d3fe056ea78f0d/jsr305-3.0.2.jar.
         > Not a module and no mapping defined: jsr305-3.0.2.jar
   > Failed to transform j2objc-annotations-2.8.jar (com.google.j2objc:j2objc-annotations:2.8) to match attributes {artifactType=jar, javaModule=true, org.gradle.category=library, org.gradle.libraryelements=jar, org.gradle.status=release, org.gradle.usage=java-api}.
      > Execution failed for ExtraJavaModuleInfoTransform: /home/mike/.gradle/caches/modules-2/files-2.1/com.google.j2objc/j2objc-annotations/2.8/c85270e307e7b822f1086b93689124b89768e273/j2objc-annotations-2.8.jar.
         > Not a module and no mapping defined: j2objc-annotations-2.8.jar

At this point, I'd just take the option to turn all plain JARs into automatic modules, rather than untangle this error.

Obviously, one of the big benefits of Java modules is strong encapsulation, i.e., limiting the packages that your module exports. If I want the benefits of strong encapsulation, but I also have an unavoidable dependency on a plain JAR (e.g., Undertow, javax.inject), this is where that feature becomes useful.

I just want to add a select few plain JARs to my module path (or turn them into automatic modules) so that the requires statement compiles; I don't want to manage all my transitive dependencies which are plain JARs.

jjohannes commented

This is in 1.6. Turn it on with this:

extraJavaModuleInfo {
    deriveAutomaticModuleNamesFromFileNames = true
}

@mikewacker let me know if you run into any issues using this.

  • 👍2
mikewacker commented

Thanks!

Seems to work with Guava and Immutables, but I ran into an issue with Dagger. I'll open a separate issue for that.