Best way to halt rust peg on matching a rule?

[devdave]

I mostly have cPython's PEG grammar implemented but I am currently trying to implement their invalid_ rules here https://github.com/python/cpython/blob/main/Grammar/python.gram#L1077

I tried

rule invalid_import_from_targets()
            = import_from_as_names() comma() t:NEWLINE() {? Err("trailing comma not allowed without surrounding parentheses") }

And that seems pretty close as I get

Failed to parse: "test_fixtures/error_invalid_import_from_targets.py" - because ParseError { location: ParseLoc { start_pos: LineCol { line: 3, column: 1, offset: 2 }, end_pos: LineCol { line: 3, column: 2, offset: 2 } }, expected: ExpectedSet { expected: {"NameToken - Name rule", "trailing comma not allowed without surrounding parentheses"} } }

Tokens being parsed: 
Token: Token { type: Name, start: Position { col: 0, line: 1 }, end: Position { col: 4, line: 1 }, text: "from" }
Token: Token { type: Name, start: Position { col: 5, line: 1 }, end: Position { col: 8, line: 1 }, text: "foo" }
Token: Token { type: Name, start: Position { col: 9, line: 1 }, end: Position { col: 15, line: 1 }, text: "import" }
Token: Token { type: Name, start: Position { col: 16, line: 1 }, end: Position { col: 20, line: 1 }, text: "blah" }
Token: Token { type: Op, start: Position { col: 20, line: 1 }, end: Position { col: 21, line: 1 }, text: "," }
Token: Token { type: NL, start: Position { col: 21, line: 1 }, end: Position { col: 22, line: 1 }, text: "\n" }
Token: Token { type: EndMarker, start: Position { col: 1, line: 3 }, end: Position { col: 2, line: 3 }, text: "EOF" }

Note, I have a bug in my tokenizer currently where the line # is wrong for EndMarker but ideally I would like to halt the parser at the syntax error without using panic!() I saw the no_eof directive mentioned in various pull and issue discussions but I wasn't sure if that would work or how it would work in my situation.

My grammar file is here https://github.com/devdave/rython4/blob/master/src/parser/grammar.rs#L122 and it works well for syntactically correct code.

Any help/pointers is greatly appreciated!

[kevinmehall]

Currently the only ways to make a rust-peg parser return an error are the expected!("x") and {? Err("x") } constructs, which are equivalent. These add x to the set of expected tokens at the current parse location, and alternatives are tried. If overall parsing fails, the error message is reported with the set of tokens expected at the furthest parse location.

Some other PEG implementations have a construct that bails out of parsing entirely, without trying any alternatives. Is that what you're looking for here? This recently came up on #308.

It looks like the rule you have is doing some kind of error recovery. There's some discussion of that in #276. I think the best approach to an IDE/LSP ready parser may be to make the parser never fail, but include error nodes in the resulting output. I recently tried that and found that it worked, but it's tricky to avoid ambiguity between the normal rules and those added for error recovery.

[kevinmehall]

I saw the no_eof directive mentioned in various pull and issue discussions but I wasn't sure if that would work or how it would work in my situation.

I don't think that's applicable here. Normally if the parser succeeds without consuming all the input, it generates a failure expecting EOF at the final parse position. no_eof suppresses that, for instance if you want to parse the header of a file and don't care that the full file doesn't match.

[devdave]

Some other PEG implementations have a construct that bails out of parsing entirely, without trying any alternatives. Is that what you're looking for here? This recently came up on #308.

Indeed that's the direction I was heading by implementing cPython's invalid_* rules as they're the most common syntax errors and help keep the next stages in code compilation a little simpler. For my situation I think I am just going to prefix the {? Err("message") } to explain to user why parsing failed.

Unrelated I am at a pause point in my project and started profiling/optimizing what I've written so far. Rust-peg is remarkably fast and efficient.