-
Notifications
You must be signed in to change notification settings - Fork 35
Getting started
This getting-started page will guide us through the implementation of a very simple parser. The parser will process simple text expressions adding several numbers. The working program returns the sum contained in the sample string.
For example
"1 + 2 + 3"
will be parsed to
6
Install from the NuGet gallery GUI or with the Package Manager Console using the following command:
Install-Package sly
or with dotnet core
dotnet add package sly
The first stage of a parser is the scanning step. From an input source the scanner extracts your language's tokens (words). In this example, our language will have a few simple words:
- numbers: we will stay with integers
- the "+" operator
We also need to add a skippable lexeme whitespace (WS) to makes the lexer ignore trivia. The whitespaces will not be sent to the parser to avoid cluttering the parser with whitespace terminals.
Csly encodes this token definitions as a mere C# enum
annotated with C# attributes. Each token is annotated with a regular expressions matching it. Here is the full lexer:
public enum ExpressionToken {
[Lexeme("[0-9]+")]
INT = 1,
[Lexeme("\\+")]
PLUS = 2,
[Lexeme("[ \\t]+",isSkippable:true)] // the lexeme is marked isSkippable : it will not be sent to the parser and simply discarded.
WS = 3
}
our grammar is quite easy (BNF notation) :
expression : INT
expression : term PLUS expression
term : INT
Csly uses BNF notation and attaches visitor methods to each rule. Visitor methods have the following properties:
- return type is the type of the parse result:
int
for our language, - parameters matches each clause of the right-hand side of a rule.
⚠️ read carefully the typing section to correctly write your visitor methods
here is the visitor for the first rule
[Production("expression: INT")]
public int Primary(Token<ExpressionToken> intToken)
{
return intToken.IntValue;
}
The whole parser is then
public class ExpressionParser
{
[Production("expression: INT")]
public int intExpr(Token<ExpressionToken> intToken)
{
return intToken.IntValue;
}
[Production("expression : term PLUS expression")]
public int Expression(int left, Token<ExpressionToken> operatorToken, int right) {
return left + right;
}
[Production("term : INT")]
public int Expression(Token<ExpressionToken> intToken) {
return intToken.IntValue;
}
}
using sly.parser;
using sly.parser.generator;
public class SomeClass {
public static Parser<ExpressionToken,int> GetParser() {
var parserInstance = new ExpressionParser();
var builder = new ParserBuilder<ExpressionToken, int>();
var Parser = builder.BuildParser(parserInstance, ParserType.LL_RECURSIVE_DESCENT, "expression").Result;
return Parser;
}
}
public class SomeTest {
public void TestCSLY() {
string expression = "42 + 42";
var Parser = SomeClass.GetParser();
var r = Parser.Parse(expression);
if (!r.IsError)
{
Console.WriteLine($"result of <{expression}> is {(int)r.Result}");
// outputs : result of <42 + 42> is 84"
}
else
{
if (r.Errors !=null && r.Errors.Any())
{
// display errors
r.Errors.ForEach(error => Console.WriteLine(error.ErrorMessage));
}
}
}
}
Next steps:
- lexers
- regex base lexer : slow but fully customizable
- generic lexer : fast and matches almost every lexing use case
- parsers: