Skip to content

Commit

Permalink
v0.3 prerelease (#29)
Browse files Browse the repository at this point in the history
  • Loading branch information
rrevenantt authored Jul 22, 2022
1 parent 4eecc87 commit 4779134
Show file tree
Hide file tree
Showing 69 changed files with 4,320 additions and 809 deletions.
17 changes: 7 additions & 10 deletions .github/workflows/antlr.yml
Original file line number Diff line number Diff line change
Expand Up @@ -4,15 +4,10 @@ on:
push:
branches: [ master ]
pull_request:
branches:
- master
- v0.3

workflow_dispatch:
branches: [ master ]

env:
CARGO_TERM_COLOR: always
RUST_VERSION: nightly-2021-11-30

jobs:
tests-antlr:
Expand All @@ -35,7 +30,7 @@ jobs:
- name: Install Rust
uses: actions-rs/toolchain@v1
with:
toolchain: ${{ env.RUST_VERSION }}
toolchain: stable
default: true
- name: Cache local Maven repository
uses: actions/cache@v2
Expand Down Expand Up @@ -64,12 +59,15 @@ jobs:
runs-on: ubuntu-latest
outputs:
new_version: ${{ steps.check_ver.outputs.new_version }}
strategy:
matrix:
rust_version: [ 1.52 , stable]
steps:
- uses: actions/checkout@v2
- name: Install Rust
uses: actions-rs/toolchain@v1
with:
toolchain: ${{ env.RUST_VERSION }}
toolchain: ${{ matrix.rust_version }}
default: true
components: rustfmt
- name: Build
Expand All @@ -81,7 +79,6 @@ jobs:
- uses: actions/checkout@v2
if: ${{ github.event_name == 'push' }}
with:
# todo change to `fetch-depth: 2` and HEAD^
ref: ${{ github.event.before }}
path: base_branch
- name: check version bump
Expand Down Expand Up @@ -110,7 +107,7 @@ jobs:
- name: Install Rust
uses: actions-rs/toolchain@v1
with:
toolchain: ${{ env.RUST_VERSION }}
toolchain: stable
default: true
- name: Cargo publish check
run: |
Expand Down
5 changes: 3 additions & 2 deletions Cargo.toml
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
[package]
name = "antlr-rust"
version = "0.2.2"
version = "0.3.0-beta"
authors = ["Konstantin Anisimov <[email protected]>"]
homepage = "https://github.com/rrevenantt/antlr4rust"
repository = "https://github.com/rrevenantt/antlr4rust"
Expand All @@ -23,7 +23,8 @@ bit-set = "=0.5.*"
once_cell = "^1.2"
#backtrace = "=0.3"
typed-arena = "^2.0"
better_any = "=0.1"
better_any = "0.2.0-dev.1"
#better_any = "=0.1"
parking_lot = "0.11"
#qcell = { path="../qcell" }

Expand Down
37 changes: 10 additions & 27 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# antlr4rust
[![Crate](https://flat.badgen.net/crates/v/antlr-rust)](https://crates.io/crates/antlr_rust/0.2.0)
[![docs](https://flat.badgen.net/badge/docs.rs/v0.2.0)](https://docs.rs/antlr-rust/0.2.0)
[![Crate](https://flat.badgen.net/crates/v/antlr-rust)](https://crates.io/crates/antlr_rust/0.3.0-beta)
[![docs](https://flat.badgen.net/badge/docs.rs/v0.3.0-beta)](https://docs.rs/antlr-rust/0.3.0-beta)
![ANTLR4 testsuite](https://github.com/rrevenantt/antlr4rust/workflows/ANTLR4%20testsuite/badge.svg?event=push)
![cargo test](https://github.com/rrevenantt/antlr4rust/workflows/cargo%20test/badge.svg)
[![](https://tokei.rs/b1/github/rrevenantt/antlr4rust)](https://github.com/rrevenantt/antlr4rust)
Expand All @@ -27,29 +27,10 @@ But if you want to build or change generator yourself:
For now development is going on in this repository
but eventually it will be merged to main ANTLR4 repo

Currently, requires nightly version of rust.
This likely will be the case until `coerce_unsized` or some kind of coercion trait is stabilized.
There are other unstable features in use but only `CoerceUnsized` is essential.

Remaining things before merge:
- API stabilization
- [ ] Rust api guidelines compliance
- [ ] more tests for API because it is quite different from Java

Can be done after merge:
- Documentation
- [ ] Some things are already documented but still far from perfect, also more links needed.
- cfg to not build potentially unnecessary parts
(no Lexer if custom token stream, no ParserATNSimulator if LL(1) grammar)
- run rustfmt on generated parser
###### Long term improvements
- generate enum for labeled alternatives without redundant `Error` option
- option to generate fields instead of getters by default and make visiting based on fields
- make tree generic over pointer type and allow tree nodes to arena.
(requires GAT, otherwise it would be a problem for users that want ownership for parse tree)
- support stable rust
- support no_std(although alloc would still be required)

Since version `0.3` works on stable rust.
Previous versions are not maintained any more
so in case of nightly breakage you should migrate to the latest version.

### Usage

You should use the ANTLR4 "tool" to generate a parser, that will use the ANTLR
Expand All @@ -67,9 +48,8 @@ Then add following to `Cargo.toml` of the crate from which generated parser
is going to be used:
```toml
[dependencies]
antlr-rust = "=0.2"
antlr-rust = "0.3"
```
and `#![feature(try_blocks)]` in your project root module.

### Parse Tree structure

Expand Down Expand Up @@ -117,9 +97,11 @@ Also while structs used by generated lexer and parser were customized to track a
internals of the lexer cannot be customized enough yet and still track quite a lot of data that might not be used in particular case.
So there is still room for improvement.
```text
lexers:
large/large_xmlparser time: [1.8598 ms 1.8607 ms 1.8619 ms]
large/large_quick_xml time: [1.4623 ms 1.4645 ms 1.4675 ms]
large/large_antlr_xml_lexer time: [5.7866 ms 5.7877 ms 5.7891 ms]
parsers:
large/large_xmlrs time: [16.734 ms 16.748 ms 16.766 ms]
large/large_minidom time: [7.0639 ms 7.0792 ms 7.0975 ms]
large/large_roxmltree time: [4.9341 ms 4.9360 ms 4.9380 ms]
Expand All @@ -140,3 +122,4 @@ BSD 3-clause.
Unless you explicitly state otherwise,
any contribution intentionally submitted for inclusion in this project by you
shall be licensed as above, without any additional terms or conditions.

12 changes: 11 additions & 1 deletion build.rs
Original file line number Diff line number Diff line change
Expand Up @@ -9,14 +9,24 @@ use std::process::Command;

fn main() {
let grammars = vec![
"VisitorBasic",
"VisitorCalc",
"CSV",
"ReferenceToATN",
"XMLLexer",
"SimpleLR",
"Labels",
"FHIRPath",
];
let additional_args = vec![Some("-visitor"), None, None, None, None];
let additional_args = vec![
Some("-visitor"),
Some("-visitor"),
Some("-visitor"),
None,
None,
None,
None,
];
let antlr_path = "/home/rrevenantt/dev/antlr4/tool/target/antlr4-4.8-2-SNAPSHOT-complete.jar";

for (grammar, arg) in grammars.into_iter().zip(additional_args) {
Expand Down
7 changes: 7 additions & 0 deletions grammars/VisitorBasic.g4
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
grammar VisitorBasic;

s
: 'A' EOF
;

A : 'A';
18 changes: 18 additions & 0 deletions grammars/VisitorCalc.g4
Original file line number Diff line number Diff line change
@@ -0,0 +1,18 @@
grammar VisitorCalc;

s
: expr EOF
;

expr
: INT # number
| expr (MUL | DIV) expr # multiply
| expr (ADD | SUB) expr # add
;

INT : [0-9]+;
MUL : '*';
DIV : '/';
ADD : '+';
SUB : '-';
WS : [ \t]+ -> channel(HIDDEN);
4 changes: 2 additions & 2 deletions rustfmt.toml
Original file line number Diff line number Diff line change
@@ -1,3 +1,3 @@
edition = "2018"
fn_single_line = true
fn_params_layout = "Compressed"
#fn_single_line = true
#fn_args_layout = "Compressed"
4 changes: 2 additions & 2 deletions src/atn.rs
Original file line number Diff line number Diff line change
Expand Up @@ -55,7 +55,7 @@ impl Debug for ATN {
}

impl ATN {
crate fn new_atn(grammar_type: ATNType, max_token_type: isize) -> ATN {
pub(crate) fn new_atn(grammar_type: ATNType, max_token_type: isize) -> ATN {
ATN {
decision_to_state: Vec::new(),
grammar_type,
Expand Down Expand Up @@ -93,7 +93,7 @@ impl ATN {
analyzer.look::<Ctx>(s, None, _ctx)
}

crate fn add_state(&mut self, state: Box<dyn ATNState>) {
pub(crate) fn add_state(&mut self, state: Box<dyn ATNState>) {
debug_assert_eq!(state.get_state_number(), self.states.len());
self.states.push(state)
}
Expand Down
36 changes: 26 additions & 10 deletions src/atn_config.rs
Original file line number Diff line number Diff line change
Expand Up @@ -32,8 +32,8 @@ impl PartialEq for ATNConfig {
fn eq(&self, other: &Self) -> bool {
self.get_state() == other.get_state()
&& self.get_alt() == other.get_alt()
&& (Arc::ptr_eq(self.get_context().unwrap(), other.get_context().unwrap())
|| self.get_context() == other.get_context())
// Arc is optimized to not do a deep equalitiy if arc pointers are equal so that's enough
&& self.context == other.context
&& self.get_type() == other.get_type()
&& self.semantic_context == other.semantic_context
&& self.precedence_filter_suppressed == other.precedence_filter_suppressed
Expand Down Expand Up @@ -202,25 +202,41 @@ impl ATNConfig {
new
}

pub fn get_state(&self) -> ATNStateRef { self.state }
pub fn get_state(&self) -> ATNStateRef {
self.state
}

pub fn get_alt(&self) -> isize { self.alt }
pub fn get_alt(&self) -> isize {
self.alt
}

pub(crate) fn get_type(&self) -> &ATNConfigType { &self.config_type }
pub(crate) fn get_type(&self) -> &ATNConfigType {
&self.config_type
}

pub fn get_context(&self) -> Option<&Arc<PredictionContext>> { self.context.as_ref() }
pub fn get_context(&self) -> Option<&Arc<PredictionContext>> {
self.context.as_ref()
}

pub fn take_context(&mut self) -> Arc<PredictionContext> { self.context.take().unwrap() }
pub fn take_context(&mut self) -> Arc<PredictionContext> {
self.context.take().unwrap()
}

pub fn set_context(&mut self, _v: Arc<PredictionContext>) { self.context = Some(_v); }
pub fn set_context(&mut self, _v: Arc<PredictionContext>) {
self.context = Some(_v);
}

pub fn get_reaches_into_outer_context(&self) -> isize { self.reaches_into_outer_context }
pub fn get_reaches_into_outer_context(&self) -> isize {
self.reaches_into_outer_context
}

pub fn set_reaches_into_outer_context(&mut self, _v: isize) {
self.reaches_into_outer_context = _v
}

pub fn is_precedence_filter_suppressed(&self) -> bool { self.precedence_filter_suppressed }
pub fn is_precedence_filter_suppressed(&self) -> bool {
self.precedence_filter_suppressed
}

pub fn set_precedence_filter_suppressed(&mut self, _v: bool) {
self.precedence_filter_suppressed = _v;
Expand Down
56 changes: 42 additions & 14 deletions src/atn_config_set.rs
Original file line number Diff line number Diff line change
Expand Up @@ -86,7 +86,9 @@ impl PartialEq for ATNConfigSet {
impl Eq for ATNConfigSet {}

impl Hash for ATNConfigSet {
fn hash<H: Hasher>(&self, state: &mut H) { self.configs.hash(state) }
fn hash<H: Hasher>(&self, state: &mut H) {
self.configs.hash(state)
}
}

impl ATNConfigSet {
Expand All @@ -113,7 +115,9 @@ impl ATNConfigSet {
a
}

fn full_hash_key(config: &ATNConfig) -> Key { Key::Full(config.clone()) }
fn full_hash_key(config: &ATNConfig) -> Key {
Key::Full(config.clone())
}

fn local_hash_key(config: &ATNConfig) -> Key {
let mut hasher = MurmurHasher::default();
Expand Down Expand Up @@ -176,7 +180,9 @@ impl ATNConfigSet {
true
}

pub fn add(&mut self, config: Box<ATNConfig>) -> bool { self.add_cached(config, None) }
pub fn add(&mut self, config: Box<ATNConfig>) -> bool {
self.add_cached(config, None)
}

pub fn get_items(&self) -> impl Iterator<Item = &ATNConfig> {
self.configs.iter().map(|c| c.as_ref())
Expand All @@ -197,19 +203,33 @@ impl ATNConfigSet {
}
}

pub fn length(&self) -> usize { self.configs.len() }
pub fn length(&self) -> usize {
self.configs.len()
}

pub fn is_empty(&self) -> bool { self.configs.is_empty() }
pub fn is_empty(&self) -> bool {
self.configs.is_empty()
}

pub fn has_semantic_context(&self) -> bool { self.has_semantic_context }
pub fn has_semantic_context(&self) -> bool {
self.has_semantic_context
}

pub fn set_has_semantic_context(&mut self, _v: bool) { self.has_semantic_context = _v; }
pub fn set_has_semantic_context(&mut self, _v: bool) {
self.has_semantic_context = _v;
}

pub fn read_only(&self) -> bool { self.read_only }
pub fn read_only(&self) -> bool {
self.read_only
}

pub fn set_read_only(&mut self, _read_only: bool) { self.read_only = _read_only; }
pub fn set_read_only(&mut self, _read_only: bool) {
self.read_only = _read_only;
}

pub fn full_context(&self) -> bool { self.full_ctx }
pub fn full_context(&self) -> bool {
self.full_ctx
}

//duplicate of the self.conflicting_alts???
pub fn get_alts(&self) -> BitSet {
Expand All @@ -219,11 +239,19 @@ impl ATNConfigSet {
})
}

pub fn get_unique_alt(&self) -> isize { self.unique_alt }
pub fn get_unique_alt(&self) -> isize {
self.unique_alt
}

pub fn set_unique_alt(&mut self, _v: isize) { self.unique_alt = _v }
pub fn set_unique_alt(&mut self, _v: isize) {
self.unique_alt = _v
}

pub fn get_dips_into_outer_context(&self) -> bool { self.dips_into_outer_context }
pub fn get_dips_into_outer_context(&self) -> bool {
self.dips_into_outer_context
}

pub fn set_dips_into_outer_context(&mut self, _v: bool) { self.dips_into_outer_context = _v }
pub fn set_dips_into_outer_context(&mut self, _v: bool) {
self.dips_into_outer_context = _v
}
}
4 changes: 3 additions & 1 deletion src/atn_deserialization_options.rs
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,9 @@ pub struct ATNDeserializationOptions {
}

impl ATNDeserializationOptions {
pub fn is_verify(&self) -> bool { self.verify_atn }
pub fn is_verify(&self) -> bool {
self.verify_atn
}
}

impl Default for ATNDeserializationOptions {
Expand Down
Loading

0 comments on commit 4779134

Please sign in to comment.