Starting with ANTLR4

[gurucubano]

As we can now compile our ANTLR2 written code with Java 21, I'd like to do first steps with ANTLR4.

I've installed ANTLR4 in my MacOS as the tutorial https://tomassetti.me/antlr-mega-tutorial/#setup-antlr explains it, created an alias for my bash etc. and it works:

$ alias antlr4
alias antlr4='java -Xmx500M -cp "/usr/local/lib/antlr-4.9.3-complete.jar:$CLASSPATH" org.antlr.v4.Tool'
$ antlr4
ANTLR Parser Generator  Version 4.9.3
 -o ___              specify output directory where all output is generated
 -lib ___            specify location of grammars, tokens files
 -atn                generate rule augmented transition network diagrams
 -encoding ___       specify grammar file encoding; e.g., euc-jp
...

When I now pick-up the first example from doc/getting-started.md it simply failes:

$ cat InfoGuideQuery.g4

grammar InfoGuideQuery;
prog:	expr EOF ;
expr:	expr ('*'|'/') expr
    |	expr ('+'|'-') expr
    |	INT
    |	'(' expr ')'
    ;
NEWLINE : [\r\n]+ -> skip;
INT     : [0-9]+ ;

antlr4 InfoGuideQuery.g4 prog -tree
error(2):  unknown command-line option -tree
error(7):  cannot find or open file: prog

Perhaps I do something stupidly wrong. Please help me out. What is progexactly in this beginners example?

[kaby76]

You might want to skip the aliases and just type out the Java command completely to see what is going on. "TestRig" runs a compiled Java parser. For TestRig, you use the grammar name, not the grammar file name.

$ java -Xmx500M -cp 'c:/users/kenne/downloads/antlr-4.13.1-complete.jar;.' org.antlr.v4.gui.TestRig Fortran90 program ../../../examples/4105.f90 -tree
(program (executableProgram (programUnit (mainProgram (mainRange (body (bodyConstruct (specificationPartConstruct (declarationConstruct (typeDeclarationStmt (typeSpec integer (kindSelector ( (expression (level5Expr (equivOperand (orOperand (andOperand (level4Expr (level3Expr (level2Expr (addOperand (multOperand (level1Expr (primary (unsignedArithmeticConstant 4))))))))))))) ))) :: (entityDeclList (entityDecl (objectName (name status)))))))) (bodyConstruct (executableConstruct (actionStmt (callStmt call (subroutineNameUse some_routine) ( (subroutineArgList (subroutineArg (expression (level5Expr (equivOperand (orOperand (andOperand (level4Expr (level3Expr (level2Expr (addOperand (multOperand (level1Expr (primary (nameDataRef (name status)))))))))))))))) )))))) (endProgramStmt end))))) <EOF>)
09/30-06:51:11 ~/fortran/old/fortran90/Generated-Java-large
$ rm *.class
09/30-06:53:21 ~/fortran/old/fortran90/Generated-Java-large
$ ls
build.ps1  ErrorListener.java     Fortran90LexerBase.java  Fortran90ParserBaseListener.java  pom.xml    Test.java
build.sh   Fortran90Lexer.g4      Fortran90Parser.g4       Fortran90ParserListener.java      readme.md  test.ps1
clean.ps1  Fortran90Lexer.interp  Fortran90Parser.interp   makefile                          run.ps1    test.sh
clean.sh   Fortran90Lexer.java    Fortran90Parser.java     MyDiagnosticErrorListener.java    run.sh
desc.xml   Fortran90Lexer.tokens  Fortran90Parser.tokens   perf.sh                           small/
09/30-06:53:23 ~/fortran/old/fortran90/Generated-Java-large
$ java -Xmx500M -cp 'c:/users/kenne/downloads/antlr-4.13.1-complete.jar;.' org.antlr.v4.gui.TestRig Fortran90 program ../../../examples/4105.f90 -tree
Can't load Fortran90 as lexer or parser
09/30-06:53:29 ~/fortran/old/fortran90/Generated-Java-large

The alternative is to use the "easy" Python tool wrappers for Antlr4. In that case, after doing pip install antlr4-tools, run the parser interpreter thus: echo "10+20*30" | antlr4-parse -v 4.13.2 Expr.g4 prog -tree.

Note, the main issue with the Python wrapper tools for Antlr4 are timeouts. The tools try to download from "Maven Central", to get the latest version of Antlr, and then download that .jar file. But, it's been having problems lately. I just checked https://search.maven.org/solrsearch/select?q=a:antlr4-master+g:org.antlr where it goes to get the latest version of the .jar, but it times out. After it downloads the .jar and JRE, it places them in a cache directory ~/.m2. If I use the -v option, it works against the cache and avoids the download timeout issues.

[gurucubano]

I've downloaded

wget https://www.antlr.org/download/antlr-4.13.2-complete.jar

and can use it with Java as:

$ java -Xmx500M -cp "/usr/local/lib/antlr-4.13.2-complete.jar:$CLASSPATH" org.antlr.v4.Tool InfoGuideQuery.g4
OCLC-7YDFXP:antlr apitzm$ ls -ltr
total 112
-rw-r--r--  1 apitzm  32873736   177 30 Sep 12:27 InfoGuideQuery.g4
-rw-r--r--  1 apitzm  32873736  1501 30 Sep 13:42 InfoGuideQueryLexer.interp
-rw-r--r--  1 apitzm  32873736  5383 30 Sep 13:42 InfoGuideQueryLexer.java
-rw-r--r--  1 apitzm  32873736    94 30 Sep 13:42 InfoGuideQueryLexer.tokens
-rw-r--r--  1 apitzm  32873736   943 30 Sep 13:42 InfoGuideQuery.interp
-rw-r--r--  1 apitzm  32873736  9634 30 Sep 13:42 InfoGuideQueryParser.java
-rw-r--r--  1 apitzm  32873736   946 30 Sep 13:42 InfoGuideQueryListener.java
-rw-r--r--  1 apitzm  32873736  1726 30 Sep 13:42 InfoGuideQueryBaseListener.java
-rw-r--r--  1 apitzm  32873736    94 30 Sep 13:42 InfoGuideQuery.tokens

What I have to put below ~/.m2 to make the Python tool to avoid the download and the timeout?

[kaby76]

Actually, for the Python wrapper tools, just run the command antlr4-parse with -v 4.13.2 and it avoids the query to find the latest version of Antlr4. It'll still do a download, but there are two different addresses. One address is to get the latest version of Antlr4 at https://search.maven.org/solrsearch/select?q=a:antlr4-master+g:org.antlr (times out). The other address is https://repo1.maven.org/maven2/org/antlr/antlr4/, which is old html support for FTP, and that always works. Just run the command with a version option and all works! No need to copy anything.

[gurucubano]

$ export ANTLR4_TOOLS_ANTLR_VERSION="4.13.2"
$ /Users/apitzm/.local/bin/antlr4-parse
Downloading antlr4-4.13.2-complete.jar
java org.antlr.v4.gui.Intrepreter [X.g4|XParser.g4 XLexer.g4] startRuleName
  [-tokens] [-tree] [-gui] [-encoding encodingname]
  [-trace] [-profile filename.csv] [input-filename(s)]
Omitting input-filename makes rig read from stdin.

$ printf "10+20*30\n" | antlr4-parse InfoGuideQuery.g4 prog -tree
(prog:1 (expr:2 (expr:3 10) + (expr:1 (expr:3 20) * (expr:3 30))) <EOF>)

Success! Thanks.

[kaby76]

ANTLR4_TOOLS_ANTLR_VERSION

Glad all is working. You're right. There's a "magic" environment variable switch. https://github.com/antlr/antlr4-tools/blob/10a090aab6ec70217d1d8aa0c2f16c9f1e425861/antlr4_tool_runner.py#L128