Generic parser over input

[alrz]

Right now, the Parse method only accepts an string. Accepting a generic ReadOnlySpan<T> would enable defining a parser over a non-string sequence (therefore current text based parsers would be defined on a ReadOnlySpan<char> input).

(note: accepting ReadOnlySpan<T> may be too big of a change because it's a ref struct, but I think any container type should work as long as it's a sequence, like an array)

[sebastienros]

Agreed.

There is a very old PR too that I kept for reference that use some streamable source too (don't recall if it was based on streams or pipes). This one was concerning me because it made the code more complex and slower, though it might be the best thing to do in the end whatsoever.

[alrz]

Support for an streamable source is most definitely useful but this issue is only about accepting a generic (buffered) sequence of T as the input instead of string. I don't think this support alone would be too distant from the existing implementation.

[alrz]

OK, I just experimented with this to see how much work it would be to do this, still not quite sure but I think it would make more sense to first start by only accepting ReadOnlySpan<char> to remove all the Subscring calls and then maybe go from there. This will make a few types a ref struct and may have greater impact further down the line. If that sounds good to you I may take a stab at it in the future if someone else doesn't get there first.

[sebastienros]

Also tried and the TextSpan class is the main issue. I first removed the Buffer to make it implicit but it's actually necessary to distinguish these coming from the input buffer or from a concrete string. Example is the Decode method that tries to evaluate char escapes. I think TextSpan could become some kind of abstraction, maybe even handling composites if necessary. Somehow like ReadOnlySpan but not ref struct.

[alrz]

Also tried and the TextSpan class is the main issue.

If that's because it's being used as a type arg, would it make sense to use ReadOnlyMemory<char> instead (either within TextSpan or even directly)?

[sebastienros]

ROM could be the solution. Could even be used directly since it might be what I have modeled without realizing.

If you are looking into it locally I made Scanner a ref struct and extracted it from ParseContext. Anything that tool a ParseContext would also take a Scanner.