Migrate build system from Maven to Gradle

[ghost]

The official documentation suggests that migrating from Maven to Gradle can, in many cases, be accomplished using:

gradle init

Perhaps worth considering to get advantages of a simpler build syntax, a more powerful build engine, and a faster build system.

[carlosame]

This issue is related to #1: in order to reuse the old tests (and run them with a Security Manager), at least two Jar files have to be built.

One possibility is the Gradle Shadow Plugin, but its documentation is not very explicit about e.g. the handling of JPMS modular dependencies.

Another plugin: gradle-one-jar.

More information:

• https://mkyong.com/gradle/gradle-create-a-jar-file-with-dependencies/
• https://discuss.gradle.org/t/how-to-include-dependencies-in-jar/19571/4

The idea to move to Gradle looks good and I'll investigate it, not sure when though.

EDIT: the Gradle manual has a section about uber jars:

https://docs.gradle.org/current/userguide/userguide_single.html#sec:creating_uber_jar_example

[carlosame]

Studying the possible migration to Gradle, I found two issues:

1. It is a common Gradle practice to include a binary file among the source files of projects that are built with Gradle, gradle/wrapper/gradle-wrapper.jar, and I do not like that. My intention is to remove jar files from the repository, not adding new ones.
   See the following SO post for context (and workarounds): https://stackoverflow.com/questions/20348451/why-should-the-gradle-wrapper-be-committed-to-vcs

2. Gradle does not work with Java 16, even the (currently unreleased) 7.0 version is still having problems with it. Only a temporary issue I hope, but a stopper anyway.

I still see a potential value in moving to Gradle, but need to think more about it.

[ghost]

It is a common Gradle practice to include a binary file among the source files of projects that are built with Gradle, gradle/wrapper/gradle-wrapper.jar,

I don't use this approach with KeenWrite. Building requires calling "gradle clean build" as opposed to "./gradlew clean build". The difference is that the binary jar file is not required. (I don't like having it around either!) It also means gradle must be installed somewhere, similar to how ant or maven must be installed on the PATH.

Gradle does not work with Java 16,

Gradle 7 RC 2 works with Java 16.

[carlosame]

Gradle 7 RC 2 works with Java 16.

Perhaps there is something that I'm not doing correctly, but:

$ ./gradlew wrapper --gradle-version=7.0-rc-2

FAILURE: Build failed with an exception.

* Where:
Build file 'C:\***\project-snap\buildSrc\build.gradle'

* What went wrong:
Could not compile build file 'C:\***\project-snap\buildSrc\build.gradle'.
> startup failed:
  General error during semantic analysis: Unsupported class file major version 60

and if you look at the comment that I linked in my previous post, I'm not the only one having problems: gradle/gradle#13481 (comment)

[ghost]

Perhaps there is something that I'm not doing correctly, but:

Don't use the gradle wrapper? Here's my setup:

$ cd $HOME/archives
$ wget https://services.gradle.org/distributions/gradle-7.0-rc-2-bin.zip
$ sudo su -
$ cd /opt
$ unzip ~username/archives/gradle-7.0-rc-2-bin.zip
$ ln -s gradle-7.0-rc-2 gradle
$ exit
$ cat $HOME/.bashrc
export GRADLE_HOME=/opt/gradle
export PATH="$PATH:$GRADLE_HOME/bin"
$ source $HOME/.bashrc
$ gradle --version
Gradle 7.0-rc-2

This allows upgrades by changing a symbolic link to the new release directory: the environment variables need not change. I use the same trick for the JDK:

$ which java
/opt/jdk/bin/java
$ java --version
openjdk 16 2021-03-16

Then build (without using the gradlew script):

$ cd $HOME/dev/java/keenwrite
$ gradle clean build -x test
BUILD SUCCESSFUL in 20s

(The unit tests aren't working yet---something to do with an improper module config, nothing to do with Gradle 7---but the build succeeds nonetheless.)

You may need to purge the caches directory:

rm -rf $HOME/.gradle/caches

[carlosame]

Status update: I'm stuck with the following error:

echosvg (master)
$ gradle build

> Task :echosvg-script:compileJava FAILED
C:\****\echosvg\echosvg-script\src\main\java\module-info.java:25: error: module not found: jython
    requires jython;
             ^
1 error

FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':echosvg-script:compileJava'.
> Compilation failed; see the compiler error output for details.

* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.

* Get more help at https://help.gradle.org

BUILD FAILED in 4s
39 actionable tasks: 3 executed, 36 up-to-date

I successfully fixed quite a few of these already, and here are the relevant snippets of echosvg-script's build.gradle:

plugins {
     [...]
	id "de.jjohannes.extra-java-module-info" version "0.6"
}
[...]
extraJavaModuleInfo {
	failOnMissingModuleInfo.set(false)
	automaticModule('rhino-1.7.13.jar', 'rhino')
	automaticModule('jython-2.7.2.jar', 'jython')
	automaticModule('xml-apis-ext-1.3.04.jar', 'xml.apis.ext')
}

but somehow finds rhino and xml.apis.ext but not jython. I verified that the jar filename is correct, so that's not the issue. It is possibly a problem with a jython's transitive dependency which remains hidden.

In general, Gradle is a PITA for modular projects when some jar package does not declare a module name.

[ghost]

Gradle is a PITA for modular projects

I'd go even further: modular projects are a PITA.

jython

I didn't even realize the library had a Python interpreter dependency---seems like bloatware.

There's a Gradle plug-in that converts non-modular libraries into modular ones:

• https://github.com/moditect/moditect-gradle-plugin

There's also what looks to be a variation on automatic modules:

• https://youtu.be/FpC6zPmycgw?t=1710
• https://youtu.be/FpC6zPmycgw?t=1901

Another resource might be the dev mailing list. Or perhaps Sander Mak (sandermak) via email (@gmail.com), he's on GitHub.

[carlosame]

I'd go even further: modular projects are a PITA.

At least, Maven automatically derives filename-based module names, which IIRC is what the relevant JSR mandates. Gradle is not being compliant.

I didn't even realize the library had a Python interpreter dependency

It supports Javascript and Python 2.x scripts. It used to accept Tcl as well, but I do not recall seeing specific support when I looked at the code (perhaps it was removed).

The obsolete Python 2.x scripting seems an obvious target for removal: nobody is going to pay jython developers to implement Python 3.x (all the resources in that area are being dropped into GraalVM) and that 3.x upgrade is too big for the volunteer developers working in their spare time (see jython/jython/issues/24).

Unfortunately, there are bigger issues in the way to modularisation.

There's a Gradle plug-in that converts non-modular libraries into modular ones

The de.jjohannes.extra-java-module-info plugin that I'm using does just that, but the underlying problem is deeper than just lacking a module name. I forgot that I already hit the issue while I attempted to modularise with Maven, see jython/jython/issues/28 for details (you'll even see a comment that I posted a few months ago).

Another resource might be the dev mailing list. Or perhaps Sander Mak (sandermak) via email

Thanks for the pointer, but there is nothing that they can do. And the worst part is that jython is not the real issue here: even if I drop it, this project depends on Xalan both directly (echosvg-dom module) and via Apache FOP dependency (in echosvg-rasterizer module). And Xalan (a dead project) is unusable in JPMS modular projects.

Perhaps I should not attempt to build the module-info files (I'm currently filtering them out in the Maven build), and go the way of automatic module names (which I already wrote for Maven) but that's tedious and a waste of my previous effort. Too bad that gradle init only honored the dependencies (albeit assumed that they are all optional!) but not the rest of the POM files when creating the initial configuration files.

[carlosame]

After my recent removals of code (see especially #12) I managed to compile the project with Gradle (including the module-info.java files). Once everything is compiled, it fails the execution of the old integration tests but this was expected because they are broken (it seems that I'll have to remove most of those old tests in order to fix #10).

$ gradle clean build

> Task :echosvg-util:compileJava
Note: C:\***\echosvg\echosvg-util\src\main\java\io\sf\carte\echosvg\util\DoublyIndexedTable.java uses unchecked or unsafe operations.
Note: Recompile with -Xlint:unchecked for details.

> Task :echosvg-dom:compileJava
Note: C:\***\echosvg\echosvg-dom\src\main\java\io\sf\carte\echosvg\dom\util\SAXDocumentFactory.java uses or overrides a deprecated API.
Note: Recompile with -Xlint:deprecation for details.

> Task :echosvg-awt-util:compileJava
Note: C:\***\echosvg\echosvg-awt-util\src\main\java\io\sf\carte\echosvg\ext\awt\geom\RectListManager.java uses unchecked or unsafe operations.
Note: Recompile with -Xlint:unchecked for details.

> Task :echosvg-anim:compileJava
Note: C:\***\echosvg\echosvg-anim\src\main\java\io\sf\carte\echosvg\anim\dom\SVGOMElement.java uses unchecked or unsafe operations.
Note: Recompile with -Xlint:unchecked for details.

> Task :echosvg-bridge:compileJava
Note: C:\***\echosvg\echosvg-bridge\src\main\java\io\sf\carte\echosvg\bridge\BridgeContext.java uses or overrides a deprecated API.
Note: Recompile with -Xlint:deprecation for details.

> Task :echosvg-gui-util:compileJava
Note: C:\***\echosvg\echosvg-gui-util\src\main\java\io\sf\carte\echosvg\util\gui\xmleditor\XMLView.java uses or overrides a deprecated API.

> Task :echosvg-transcoder:compileJava
Note: C:\***\echosvg\echosvg-transcoder\src\main\java\io\sf\carte\echosvg\ext\awt\image\codec\util\SeekableStream.java uses or overrides a deprecated API.

> Task :echosvg-test-old:test FAILED

io.sf.carte.echosvg.test.RunTestSuite > run FAILED
    java.util.MissingResourceException at RunTestSuite.java:29

1 test completed, 1 failed

FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':echosvg-test-old:test'.
> There were failing tests. See the report at: file:///C:/***/echosvg/echosvg-test-old/build/reports/tests/test/index.html

* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.

* Get more help at https://help.gradle.org

BUILD FAILED in 49s
125 actionable tasks: 125 executed

The thing is not very clean yet (there are dependencies that should be made transitive, probably others are redundant because of coming transitive from another dependency, etc), but I've reached much far than I anticipated.

Next step is to fix tests, because I'm not betting on neither Maven nor Gradle without fixing #1 first :-)

[ghost]

If Apache FOP depends on Apache Batik, and Apache Batik uses FOP, doesn't that introduce a circular dependency?

From https://xmlgraphics.apache.org/batik/ :

Another possibility is to use Batik’s modules to convert SVG to various formats, such as raster images (JPEG, PNG or TIFF) or other vector formats (EPS or PDF, the latter two due to the transcoders provided by Apache FOP).

From https://xmlgraphics.apache.org/fop/0.95/graphics.html :

FOP uses Apache Batik for SVG support.

What would make for clear separation? Perhaps EchoSVG is responsible for:

• Reading and writing SVG documents
• Rasterizing SVG (JPEG, TIFF, GIF, PNG, MNG, java.awt.image.BufferedImage)
• Offering Java API hooks (reading DOM, painting Graphics2D)

Apache FOP is then all-things-PDF. This implies that FOP uses those Java API hooks to create EPS/PDF versions of SVG documents. If developers need to convert SVG to PDF, they'll have to include FOP---would that make EchoSVG easier to maintain, build, etc.?

[carlosame]

If Apache FOP depends on Apache Batik, and Apache Batik uses FOP, doesn't that introduce a circular dependency?

In #10 it is explained that Batik works around this with an old FOP binary (one that did not depend on Batik yet). BTW that old binary may have security vulnerabilities and other bugs.

would that make EchoSVG easier to maintain, build, etc.?

Another possibility would be to import the relevant FOP sources. They import the binaries, we import the sources :-)

(Anyway this discussion belongs to #10, not here)

[carlosame]

I committed [84c611d] which provides a functioning Gradle build, comments are welcome.

Requires Gradle 7, and it is no longer required to manually install any dependency.

To build, just run gradle build. And to produce a uberJar bundle containing all the packages with their dependencies, use gradle build uberjar.

The version is now 0.1-SNAPSHOT because no releases have been produced yet (and Gradle no longer visits the Maven Central Repository to attempt retrieving snapshots), but I wonder if having just 0.1 was more convenient to users. Note that this project lacks a Maven snapshot repository, so no SNAPSHOT releases are going to be formally produced.

Next steps are:

1. Remove the old pom.xml files to avoid confusing users (also, Eclipse Gradle projects get upset if there are POM files around).
2. Update the build instructions.
3. Follow with the effort to fix the testing infrastructure, but using Gradle instead of Maven.

[carlosame]

I plan to produce a 0.1 milestone release in the next few days (issue #14) and make it available through this project's Maven repository. If there is any additional feedback about the new Gradle build, please make sure that it is discussed before the new release is tagged (I'll close this issue before releasing).

[carlosame]

Milestone release 0.1 is now tagged and available from the Maven repository. I'll update the README about that point.