Update rake-compiler for compiling JRuby exts

[headius]

JRuby extensions have not changed API in some time, but the mechanisms for building and loading them keep evolving. This issue will track getting rake-compiler to support JRuby extensions in more modern ways.

Currently (as I remember it), rake-compiler supports raw compilation of a JRuby ext by looking in ext for Java sources and running javac to compile them. The gem is expected to have a root class named the same as the extension, which JRuby then automatically picks up to load it.

This mechanism works for simple extensions, but falls short for a few reasons:

• More complex extensions will have dependencies on external libraries that may need to be fetched, added to classpath, vendored into the same jar, and so on.
• Most Java projects do not build with just javac, preferring a build tool like ant, maven, gradle, etc.
• The root-class mechanism for loading extensions has never worked particularly well; most newer extensions just instantiate the library using JRuby's Java integration. A new mechanism may come along for JRuby 9k, but the JI approach will always work.

I propose improving rake-compiler to build the extension based on expected Java build tool (using heuristics similar to Travis, looking for build.xml, pom.xml, etc) and aligning it with how recent extensions are built and loaded.

As an example, see https://github.com/headius/racc/tree/jruby_bits for an example of a JRuby ext built with maven and loaded via Java integration (in lib/racc/parser.rb).

Copying @mkristian, @enebo.

[mkristian]

I have quite something to say but need some more time to sort out my
thoughts ;)

On Tue, Jan 14, 2014 at 5:12 PM, Charles Oliver Nutter <
notifications@github.com> wrote:

JRuby extensions have not changed API in some time, but the mechanisms for
building and loading them keep evolving. This issue will track getting
rake-compiler to support JRuby extensions in more modern ways.

Currently (as I remember it), rake-compiler supports raw compilation of a
JRuby ext by looking in ext for Java sources and running javac to compile
them. The gem is expected to have a root class named the same as the
extension, which JRuby then automatically picks up to load it.

This mechanism works for simple extensions, but falls short for a few
reasons:

• More complex extensions will have dependencies on external libraries
  that may need to be fetched, added to classpath, vendored into the same
  jar, and so on.
• Most Java projects do not build with just javac, preferring a build
  tool like ant, maven, gradle, etc.
• The root-class mechanism for loading extensions has never worked
  particularly well; most newer extensions just instantiate the library using
  JRuby's Java integration. A new mechanism may come along for JRuby 9k, but
  the JI approach will always work.

I propose improving rake-compiler to build the extension based on expected
Java build tool (using heuristics similar to Travis, looking for build.xml,
pom.xml, etc) and aligning it with how recent extensions are built and
loaded.

As an example, see https://github.com/headius/racc/tree/jruby_bits for an
example of a JRuby ext built with maven and loaded via Java integration (in
lib/racc/parser.rb).

Copying @mkristian https://github.com/mkristian, @enebohttps://github.com/enebo
.

—
Reply to this email directly or view it on GitHubhttps://github.com//issues/87
.

[mkristian]

what I would like to see a way to declare jar dependencies in a formal way. what kind of tool is used to use those dependencies to compile a java extension for gem is less important.

so I wrote up a little blog post on what exactly I mean
http://blog.mkristian.tk/2014/01/rubygems-with-java-extensions.html

the formal way is to add those jar dependencies inside the gemspec as requirements. there is also an old wiki page about that https://github.com/mkristian/jbundler/wiki/Gem-with-declared-jars

what I did with these maven tasks for rake is basically make it easy for the gem authors to use those declared jar dependencies. of course the more tools use that kind of declaration the more likely it will be "the standard".

whether the tool include ant, gradle, buildr or whatever there is used to do it.

in the long term vendored jar will cause only problems in the future (IMO) and when I remember it right with the advent of bundler the vendored gems also disappeared. vendored gems and jars are still valid for deployment packages but not for regular library gems.

I am aware that my opinion is not in line with the above proposal of @headius but nonetheless I hope I am at least getting my point understood.

[luislavena]

@headius @mkristian please do not take my lack of response as lack of interest on this, I'm just a bit busy 😄

I've been thinking and doing some prototypes to change rake-compiler entirely and move into more Ruby-like objects instead of the rake-tasks spaghetti that is right now.

I would like to play a bit more on this and perhaps share some code in a few weeks (when some of the projects I'm working on move to live phase).

If you have more suggestions, please do not hesitate and share them here!

[mkristian]

so finally I got around and did my first gem with native-extension which basically installs the declared jar dependencies. see

https://github.com/mkristian/ruby-maven-demo/tree/no_vendored_jars

the simple gem has jar dependencies but does not vendor them. on install of the gem those jars will be installed in the local repository and then required by the jar-dependencies gem (which just keeps track of version conflicts)

the Rakefile setup task does that installing and it need ruby-maven which comes with jbundler already. ruby-maven is not a runtime dependency since is has nothing to do during runtime. it is only needed on install time ;)

the jar-dependencies is not on rubygems.org yet and I want to wait for some more feedbaclk on all this.