Whitespace Question

[mcdenhoed]

Hello! A quick question about the whitespace argument in parsita:

Say I have the following code snippet:

class PrimitiveParsers(TextParsers, whitespace=None):
    int = reg(r'[-+]?[0-9]+') > int


class PairParsers(TextParsers, whitespace=r'[ ]*'):
    a = PrimitiveParsers.int
    b = PrimitiveParsers.int
    pair = a & b


if __name__ == "__main__":
    result = PairParsers.pair.parse('23 84')
    if isinstance(result, Success):
        print(result.value)
    elif isinstance(result, Failure):
        print(result.message)

If I run this, I get the following output:

Expected r'[-+]?[0-9]+' but found ' '
Line 1, character 3

23 84
  ^  

If I change PrimitiveParsers to not specify whitespace=None then this snippet correctly parses [23, 84]:

class PrimitiveParsers(TextParsers):
    int = reg(r'[-+]?[0-9]+') > int


class PairParsers(TextParsers, whitespace=r'[ ]*'):
    a = PrimitiveParsers.int
    b = PrimitiveParsers.int
    pair = a & b


if __name__ == "__main__":
    result = PairParsers.pair.parse('23 84')
    if isinstance(result, Success):
        print(result.value)
    elif isinstance(result, Failure):
        print(result.message)

[23, 84]

Is it expected that changing whitespace in a Parser effects the whitespace of "downstream" Parsers? Or am I just invoking this stuff incorrectly?

[drhagen]

The fundamental issue is that whitespace affects only lit and reg within its context. The whitespace regex is not invoked by other operators. I'd like to say this is for performance reasons, but truthfully, I had simply not considered this edge case where parsers from one context were used in another context with no delimiters. I'll have to think about this a bit longer before I decide if the current behavior is philosophically incorrect. But regardless, I agree that it is unexpected.

I would have to see more of your problem before deciding the best solution to your problem. (Thanks for the minimized problem.) Without seeing your real problem, I would suspect that the things in PrimitiveParsers should probably just go into PairParsers or have some whitespace enabled on them.

You probably already know that one workaround is to throw empty strings in. Depending on the number of locations where this is an issue in your parser, this may be satisfactory:

from parsita import *

class PrimitiveParsers(TextParsers, whitespace=None):
    int = reg(r'[-+]?[0-9]+') > int


class PairParsers(TextParsers, whitespace=r'[ ]*'):
    a = PrimitiveParsers.int
    b = PrimitiveParsers.int
    pair = a << "" & b


if __name__ == "__main__":
    result = PairParsers.pair.parse('23 84')
    if isinstance(result, Success):
        print(result.value)
    elif isinstance(result, Failure):
        print(result.message)