Parse fails based on order

[jimmydevine]

I have a simple example which tries to parse a number, either an integer or float value. Based on the order, a float value may fail to parse based on the order they are presented in the sor<> statement and I'm not sure why as they both seem like valid grammars that should work with backtracking.

#include <tao/pegtl.hpp>
#include <tao/pegtl/contrib/parse_tree.hpp>
#include <iostream>
#include <string>


namespace pegtl = tao::pegtl;


struct GrammarInteger : pegtl::plus< pegtl::digit > { };
struct GrammarFloat   : pegtl::seq< GrammarInteger, pegtl::one< '.' >, GrammarInteger > { };
struct GrammarNumber  : pegtl::sor< GrammarInteger, GrammarFloat > { }; // Does not work
//struct GrammarNumber  : pegtl::sor< GrammarFloat, GrammarInteger > { }; // Does work
struct Grammar : pegtl::seq< GrammarNumber, pegtl::eof > { };


int main() {
	std::string str = "100.0";
	pegtl::memory_input in(str, "");
	std::unique_ptr<pegtl::parse_tree::node> root = pegtl::parse_tree::parse<Grammar>(in);
	if (!root)
	{
		std::cout << "Failed to build tree\n";
	}
	
	return 0;
}

[ColinH]

You need to take into account how backtracking works in PEGs, i.e. the only reason to choose a different (which will always be the next) branch of an sor is when one fails to match. PEGs will never go looking for a better (i.e. longer) match once they found one.

When you use the version of GrammarNumber that does not work it's because when you give it a something like 1.2 as input then the first branch of the sor with GrammarInteger will match and consume the 1 from the input, returning control to the sor rule signalling success. The sor will therefore not attempt to match the subsequent GrammarFloat.

That is why, with PEGs, you always need to take care that a rule B that can match a prefix of what another rule A can match comes after A in an sor.

[jimmydevine]

Fair enough. Does something like this make sense for pegtl::analyze to catch ?

[ColinH]

Make sense? Yes. Would be really nice to have. Possible in theory? No. Possible in practice? Perhaps something good enough to be useful could be implemented, but it's a different beast and would be a lot more complicated than the cycle analysis we have now.

[jimmydevine]

Understood, thanks