Support building code with external Java dependencies
mboes opened this issue · 12 comments
By default, all of Java's extensive standard library is in scope. So a fair number of code snippets can be written inline. But they're limited to the standard library.
Tools like Gradle and Maven, much like Cabal, bring other packages into scope. That way packages that uses classes defined in those packages can be built. Because inline-java calls javac directly, we need to replicate the classpath settings that those tools generate, so as to call javac the way they do.
There are two options here:
- detect the presence of
build.gradleor maven .pom files, and coax a classpath out of the respective build tools. - call those build tools in place of naked
javac, that way we don't fiddle with the classpath nor any other build setting. Problem is, we need to somehow convince those tools to build the out-of-band .java file that inline-java generates, despite not being part of the source sets a priori.
Does this mean that you can't use an external dependency such as the official Cassandra bindings with inline-java?
Does it mean that you can use an external dependency, you just need to ensure inline-java calls javac with a classpath that includes your external dependency? Any instructions for doing the latter if it is possible? I realize it could be some time before integration with Gradle/Maven to do it automatically.
Additionally understanding the manual process might allow others to try and add the functionality on their own as well.
Setting the environment variable CLASSPATH at build time allows to use any dependencies.
This issue is more about giving a solution to users so when they build packages that use inline-java, they need to do modest thought on how to have cabal or stack set the CLASSPATH appropriately.
detect the presence of build.gradle or maven .pom files, and coax a classpath out of the respective build tools.
Currently there is an implementation of this approach for build.gradle files. See https://github.com/tweag/inline-java/blob/master/src/Language/Java/Inline/Cabal.hs. This is used in https://github.com/tweag/inline-java/tree/master/examples/classpath
The user is still responsible for setting the classpath at runtime, though.
One approach to get the classpath at runtime is to implement a template haskell function:
-- | @$(classpath) :: String@ produces the classpath used at build time.
classpath :: Q ExpAnother option is to produce a text file during compilation that contains the classpath, but this looks less convenient.
We need to keep in mind, though, that the classpath used at build time might not be appropriate at runtime if the program is executed in a different machine or environment.
Yes, that should work.
After thinking it a bit more, I have just removed the classpath :: Q Exp though, but the classpath example still works the same (sets the build-time classpath).
The problem with classpath :: Q Exp is that it works well for examples but it is not a solution to defining packages. We usually don't wan't to hardcode the classpath in executables.
It would be more useful to provide a way for the user to know which java dependencies are necessary to run code for a given Haskell package. Any Haskell dependency might introduce java dependencies, so the mechanism needs to be smart enough to traverse the dependencies.
In addition, we don't want paths to jars in the build machine, that would only work on that particular machine. We either gather the jars in a folder and distribute that with the built code, or we provide a list of dependencies that can be put in a gradle file. Then the user can use the gradle file to install the java dependencies in any machine where he needs to run the built code.
Here is a scheme based on cabal hooks.
Each package requiring java dependencies provides a build.gradle file which lists the dependencies. This gradle file is listed in the cabal field data-files.
Each package which uses packages with java dependencies uses a custom cabal hook provided by jni or inline-java. The custom hook traverses the dependencies looking for the gradle files in the data dir. It has the gradle files parsed to produce a single gradle file with all the dependencies.
Are the data-files of dependencies queryable via the Cabal API?
There's InstalledPackageInfo.dataDir. In sparkle ghc-pkg describe says:
data-dir: /home/facundo/tweag/sparkle/.stack-work/install/x86_64-linux-nix/lts-10.0/8.2.2/share/x86_64-linux-ghc-8.2.2/sparkle-0.7.2.1
$ ls /home/facundo/tweag/sparkle/.stack-work/install/x86_64-linux-nix/lts-10.0/8.2.2/share/x86_64-linux-ghc-8.2.2/sparkle-0.7.2.1
sparkle.jar
After another couple of years working with inline-java, it is looking to me like it could work best if we didn't impose a particular build tool (gradle, sbt or maven) to manage java dependencies at the package level.
Instead, the owner of a project should be free to choose which tool to use. In return for this freedom, she would be responsible to provide a consistent set of java dependencies. This can be hard if there are a lot of Haskell packages depending on external Java libraries, but the situation is not much different with respect to dependencies on libraries written in C.
To address the initial motivation of this issue, I think we can document better how we specify dependencies with gradle for Haskell projects though, for those users willing to start with inline-java.
A plan follows. There are at least two cases in which a Java dependency might exist.
- A package uses an existing java library
- A package defines some auxiliary java library (jvm-batching, sparkle)
Every project requiring Java dependencies can use build tools for Java, and will have to build the Java dependencies prior to building the Haskell packages.
For the case of (2) we can either upload our packages to maven, or we can provide some example configuration to build the Java library from sources.
When the documentation and our builds are following the above style, we can drop the support to deal with gradle in custom cabal Setup scripts.
I disagree. Bazel is well-established enough, and build tools are finicky enough, with a user base for inline-java small enough, that spending any time maintaing multiple build systems over the long-term is not justified. Bazel is the only tool that was designed as a polyglot build system, and this is a polyglot project.
inline-java has dropped support for stack in #176. Java dependencies are managed with Bazel now.