Antlr Usage

[lmajano]

Just a quick question: Have you guys considered using ANTLR to build your AST instead of doing it manually?

[hectorlf]

Good question. Based on my puny tests (and my rusty knowledge on formal grammars), ANTLR grammars are not expressive enough to represent Pebble's template grammar. There's a piece of context in a template (namely: is this simple text or part of a directive) that presents a challenge when trying to model it.

Feel free to have a look and spot any stupid mistake you can find: https://github.com/hectorlf/shitty-template-engine

I'll close this for now. Reopen if you feel it can be done.

[DirkLachowski]

@hectorlf FYI: There's an antlr parser for liquid at https://github.com/bkiers/Liqp and liquid has a lot in common with pebble.

[hectorlf]

I see, guess I didn't dive deep enough into ANTLR and missed the "mode" thing. I'll reopen for discussion. Thanks!

[hectorlf]

My take on this. It would have been great to use ANTLR from the start, but I have my doubts now that the parsing code is done. Template compiling is just a small fraction of the execution time, changes to the template language should be scarce, and the effort to do the change has a very small ROI. If someone is bored and wants to tinker, I'd gladly have a look at the results.

[lmajano]

Hi @hectorlf. I just wanted to bring it up, just to see if there were steps to modernization of the parser. Maybe something to consider for the future. The ROI would be that more people could contribute to the parser and compiler if its based on standard libraries. Anyways, thanks for the input.

[hectorlf]

Been giving this a thought. If/when we get to make the engine reactive, that would discard the idea of using ANTLR. Correct me if I'm wrong.

[lmajano]

What do you mean by reactive Luis Majano CEO Ortus Solutions, Corp www.ortussolutions.com P/F: 1-888-557-8057

…

________________________________ From: Héctor López <[email protected]> Sent: Wednesday, April 25, 2018 7:46:43 PM To: PebbleTemplates/pebble Cc: Luis Majano; Author Subject: Re: [PebbleTemplates/pebble] Antlr Usage (#325) Been giving this a thought. If/when we get to make the engine reactive, that would discard the idea of using ANTLR. Correct me if I'm wrong. — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub<#325 (comment)>, or mute the thread<https://github.com/notifications/unsubscribe-auth/AAIXl-GpoQxzO6QOPV16jWv5R4-KpttVks5tsRjzgaJpZM4S3rIU>.

[hectorlf]

Reactive as in reactive programming. Specifically, reactive as in asynchronous non-blocking reactive Spring WebFlux. See #333

[cjbrooks12]

One other thing that might put another nail in the Antlr coffin: would extensions be able to extend the grammar without needing to mess with the .g4 file, and without parsing raw strings themselves?

The current parser allows the flexibility of plugins to define fully custom syntaxes for tags, that can be as simple or complicated as needed by giving the TokenParser direct access to consuming tokens one-at-a-time. I haven't worked much with Antlr, but I don't think it would be possible, or at least not very easy, which again go against the reason for wanting to reimplement the parser with Antlr.

My vote is to leave the parser as-is. It works well, mimics the parser used for PHP Twig, and is really easy to extend.

[bjansen]

I'm a bit late to the party, but the IntelliJ plugin I wrote is based on an ANTLR4 grammar.

I believe it can parse pretty much any "standard" Pebble template without errors, but it will likely fail to parse custom extensions that bring new operators etc. I'm no expert in grammar design, so the one I wrote is probably not state of the art, but I am pretty sure that it's not possible to add runtime language extensions to parsers generated by tools like ANTLR.

[thraidh]

I've done my fair share of language design and I second bjansen's statement. Dynamically adding new syntax or operators with configurable precedence does not work with ANTLR 3 or 4.

[bjansen]

I wonder how many people extend the core language to add their own operators...

[thraidh]

I do, so I assume there are others.