Xtext Standalone Maven Build

Introduction

This example Xtext project shows you how to create a standalone version of your Xtext DSL compiler, to be used without Eclipse (e.g. in continuous integration environments). The project remains compatible with Eclipse.

Since Xtext 2.9 or newer, it is possible to create a Maven-compatible project. However, it takes your project and uses Eclipse Tycho to manage dependencies using a manifest-first approach. The dependencies are not known to Maven, therefore plugins such as the Maven Dependency Plugin cannot assemble a runnable (fat) jar.

On the other hand, it is possible to use Eclipse and export the project as a runnable jar. This works, but requires Eclipse to be installed, and is not compatible with build automation tools.

Solution

In this repository, a solution is provided that comes close to what Eclipse produces when you export your project as a runnable jar. It works as follows:

It is based on a plain Xtext 2.12 project structure generated by Eclipse, with Maven as the preferred build system and default values for all other settings.
The main method is generated by setting generateJavaMain = true in org.xtext.example.mydsl/src/org/xtext/example/mydsl/GenerateMyDsl.mwe2. The actual main method is generated once you run that file as a MWE2 workflow.
The custom-made org.xtext.example.mydsl.standalone module is added. It includes a pom.xml that is similar to the repository facet, but it is configured to collect all dependencies into a single folder: org.xtext.example.mydsl.standalone/target/repository/plugins.

During the packaging phase of the Maven build system (when running mvn package):

All dependencies are copied to org.xtext.example.mydsl.standalone/target/classes/lib..
The main artifact (org.xtext.example.mydsl) is collected into org.xtext.example.mydsl.standalone/target/target/classes, together with the jar-in-jar loader (see below).
A Groovy script writes a manifest file, listing all the jars in org.xtext.example.mydsl.standalone/target/classes/lib.
The Maven AntRun Plugin is used to assemble a fat jar in org.xtext.example.mydsl.standalone/target.

While this solution is not perfect (see below), it produces a jar that is very similar to what Eclipse produces. You are not required to list all dependencies and all dependencies will be isolated (see below).

How to use

Clone this repository and run mvn clean package in the parent folder to build the fat jar. You can then run it using java ‐jar org.xtext.example.mydsl.standalone/target/org.xtext.example.mydsl.standalone‐1.0.0‐SNAPSHOT.jar.

Migrating an existing project

Unfortunately, there is no one-size-fits-all solution to migrate an existing project, because it depends on the original project nature. The easiest way is to start a new project (see above) and copy your existing source files to the new project. Alternatively, compare your project structure and files (pom.xml, .project and .settings/*) and ensure they match with this project.

Then, apply the following steps:

Integrate the standalone module by copying the org.xtext.example.mydsl.standalone folder to your project.
Change the package name org.xtext.example.mydsl to match your package name, including all occurrences in the source files (use find-replace all).
Modify the parent pom.xml file to include the standalone module (see the <modules /> section).
Ensure a main method is generated in your MWE2 workflow (see above).
Modify the assemble.groovy in the standalone folder. Ensure that line mentioning Rsrc-Main-Class points to the generated main class.

Limitations

The standalone build assumes you use a default Xtext 2.12 generated project structure. If your project depends on jar files that you include manually, ensure they reside in the lib/ folder (on the same level as your src/ folder, which would be org.xtext.example.mydsl/lib/ for this project), so they will be included when assembling the final jar using Maven.

If you have a different project structure, or if you depend on other resources, you can probably work around the above limitation using additional Maven tasks.

File encoding

By default, Eclipse generates project structures that match with your platform's file encoding. This may cause problems with your generators, because the Xtend language uses the « and » characters for formatting. Both characters have different byte representations, depending on whether you have created an Eclipse project in Windows (Windows-1252) or Linux (UTF-8).

This repository is configured (and assumes) UTF-8 encoded source files, in both Eclipse and Maven. If you mix up encodings, you may notice unformatted output produced by your generators (this does not generate a compile error or warning). Depending on your setup, you can use command line tools such as iconv to convert encodings (e.g. iconv -f [source encoding] -t UTF-8 [source file] > [destination file]).

Warnings and errors

The generated org.xtext.example.mydsl.generator.Main class will not produce sources when one or more validation warnings are present in your DSL. This behavior is different from running it from within Eclipse, where the sources will be produced if there are one or more validation warnings.

To change this behavior, it is necessary to modify org.xtext.example.mydsl/src/org/xtext/example/mydsl/generator/Main.xtend, which is generated after the first run. Change the section near line 44 to match below (but feel free to change it to your own needs):

    // Validate the resource
    val issues = validator.validate(resource, CheckMode.ALL, CancelIndicator.NullImpl)
    if (!issues.empty) {
        issues.forEach[System.err.println(it)]
    }

    // Exit on error
    val errors = issues.filter[it.severity == Severity.ERROR]
    if (!errors.empty) {
        return;
    }

Changes to this file will not be overwritten.

References

The design and implementation of the standalone module was inspired by two other projects.

The first one is ckulla/xtext-tycho-example. The fundamental structure of this approach has been based on this repository, but it unpacks all *.jar files in a single folder instead of bundling them. This has the disadvantage of overwriting files with the same name, such as the log4j.properties and plugin.properties, which will give errors when trying to run the standalone version.

The class loader to bootstrap the jar is raisercostin/eclipse-jarinjarloader. However, you still need to provide a MANIFEST.MF that will tell the class loader which jar files to load. This process is automated using the Groovy script during the packaging step.

microwan/xtext-standalone-maven-build