Migration to Bazel

@regisd is attempting to migrate the build of JFlex from Maven to Bazel.

Phases

1. Maven main build system.
   • Add Bazel BUILD on modules, one by one.
2. Bazel main build system.
   • Maven kept as a backup.
   • People who want to simply use jflex, can download the targz distribution. The targz doesn't contain java sources.
   • People who want to build clone the git repo and use bazel.
3. Bazel unique build system.
   • Remove obsolete code.
4. Optional: reorganise directory structure.

Tasks

Environnement

https://github.com/jflex-de/jflex/pull/395

jflex-maven-plugin

Bazel Rule instead

status: very easy, done PR #401

See bazel_rules

plugin itself

status: hard, not started

Ideally, we need to infer the deps to generate a pom.xml

Examples

status: Very easy, done on "simple". PR #401

CUP

cup Bazel rule

status: Easy, done. PR #442

cup-maven-plugin

status: obsolete, won't do

Jflex itself

status: easy (once cup rule and jflex rule are done), done PR #444

testsuite

jflex-testsuite-maven-plugin

status: obsolete, won't do

testsuite

status: hard, WIP

• custom test runner
• custom format to define test cases (.test)
• a lot of convention on the input name, the golden file name, etc.
• handling generation failure or compilation failures as expected

uberjar

status: needs investigation

Target directory structure

Source code, phases 2-3

Anyone who wants to build from source clones the git repo and uses Maven.

The repo looks like this:

├── bazel-*                           [REMOVED for targz]
├── cup-maven-plugin                  [OBSOLETE, removed]
├── bin                               [MOVED from jflex/bin]
├── docs                              [OK]
├── examples                          [MOVED from jflex/examples]
├── jflex                             [OK]
│   ├── common-testing                [REMOVED for targz]   
│   └── src                           [OK]
├── jflex-maven-plugin                [REMOVED for targz]
├── jflex-unicode-maven-plugin        [REMOVED for targz]
├── report-module                     [OBSOLETE, removed]
├── scripts                           [REMOVED for targz]
├── src                               [OBSOLETE, removed]
├── testsuite                         [REMOVED for targz]
│   ├── bzltestsuite                  [NEW, replaces jflex-testsuite-maven-plugin]
│   ├── jflex-testsuite-maven-plugin  [OBSOLETE, removed]
│   └── testcases
├── lib                               [UNCHANGED]
├── third_party                       [NEW]
└── tools                             [REMOVED for targz]

targz distribution

As a result, the targz has the following strucuture:

Distribution targz

├── bin
│   ├── jflex.bat
│   └── jflex.sh
├── docs                              [COPIED from bazel-bin]
│   ├── manual.html
│   └── manual.pdf
├── examples
├── jflex
│   └── src                           [OK]
├── lib                               [UNCHANGED; add jars copied from bazel-bin]
├── third_party 
│   ├── cup
│   ├── com
│   └── org                           [etc.]
└── README.md LICENSE etc.

Note: for the distribution to avoid the bootstrap question, we can also copy relevant files from bazel-genfiles:

bazel-genfiles
├── examples
│   └── simple
│       └── Yylex.java
└── jflex
    ├── LexParse.java
    ├── LexScan.java
    └── sym.java

Sources structure, phase 4 (optional)

This is the flattest structure that works well with Maven and makes releasing the targz easy.

├── bin
│   └── jflex.bat
├── docs
│   ├── grammar.md
│   ├── intro.md
│   └── template.tex
├── java
│   └── jflex
│       ├── LexGenerator.java
│       ├── gui
│       ├── lexparse.flex
│       ├── lexscan.cup
│       ├── maven
│       │   └── JflexMojo.java
│       ├── testing
│       │   └── testsuite
│       │       └── TestRunner.java
│       └── unicode
│           └── UnicodeProperies.java
├── javatests
│   └── jflex
│       └── testsuite
│           └── apiprivate
│               ├── ApiPrivateTest.java
│               └── private.lex
├── scripts
│   ├── release.sh
│   └── travis.sh
└── third_party
    ├── com
    │   └── google
    │       └── guava
    └── java
        └── java_cup
            ├── java_cup-11b.jar
            └── runtime
                ├── ComplexSymbolFactory.java
                └── lr_parser.java