Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

add parser notes #4064

Merged
merged 4 commits into from
Mar 25, 2022
Merged
Show file tree
Hide file tree
Changes from 3 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
15 changes: 15 additions & 0 deletions src/parser/parser.yy
Original file line number Diff line number Diff line change
@@ -1,9 +1,11 @@
// Bison options
%language "C++"
%skeleton "lalr1.cc"
%no-lines
%locations
%define api.namespace { nebula }
%define api.parser.class { GraphParser }
// Parameters of scanner and parser
%lex-param { nebula::GraphScanner& scanner }
%parse-param { nebula::GraphScanner& scanner }
%parse-param { std::string &errmsg }
Expand Down Expand Up @@ -54,6 +56,7 @@ static constexpr size_t kCommentLengthLimit = 256;
const nebula::GraphParser::location_type& loc);
}

// Define types of semantic values
%union {
bool boolval;
int64_t intval;
Expand Down Expand Up @@ -402,6 +405,18 @@ static constexpr size_t kCommentLengthLimit = 256;
%type <boolval> opt_with_properties
%type <boolval> opt_ignore_existed_index

// Define precedence and associativity of tokens.
// Associativity:
// The associativity of an operator op determines how repeated uses of the operator nest:
// whether ‘x op y op z’ is parsed by grouping x with y first or by grouping y with z first.
// %left specifies left-associativity (grouping x with y first) and %right specifies right-associativity (grouping y with z first).
// %nonassoc specifies no associativity, which means that ‘x op y op z’ is considered a syntax error.
//
// Precedence:
// The precedence of an operator determines how it nests with other operators.
// All the tokens declared in a single precedence declaration have equal precedence and nest together according to their associativity.
// When two tokens declared in different precedence declarations associate, the one declared later has the higher precedence and is grouped first.

%left QM COLON
%left KW_OR KW_XOR
%left KW_AND
Expand Down
21 changes: 21 additions & 0 deletions src/parser/scanner.lex
Original file line number Diff line number Diff line change
@@ -1,3 +1,5 @@
/* Flex definitions section */
jievince marked this conversation as resolved.
Show resolved Hide resolved
/* Flex options */
%option c++
%option yyclass="GraphScanner"
%option nodefault noyywrap
Expand All @@ -11,6 +13,7 @@
#include "GraphParser.hpp"
#include "graph/service/GraphFlags.h"

/* YY_USER_ACTION is called to advance location after each time a pattern is matched. */
#define YY_USER_ACTION \
yylloc->step(); \
yylloc->columns(yyleng);
Expand All @@ -19,6 +22,14 @@ static constexpr size_t MAX_STRING = 4096;

%}

/* Define some exclusive states.
jievince marked this conversation as resolved.
Show resolved Hide resolved
* Each state is referenced within a `<>` in the rules section
* <DQ_STR> double quoted string literal
* <SQ_STR> single quoted string literal
* <LB_STR> accent quoted label, eg. `v2`
* <COMMENT> comment
*/

%x DQ_STR
%x SQ_STR
%x LB_STR
Expand Down Expand Up @@ -60,6 +71,15 @@ LABEL_FULL_WIDTH {CN_EN_FULL_WIDTH}{CN_EN_NUM_FULL_WIDTH}*

%%

/* Flex rules section */
jievince marked this conversation as resolved.
Show resolved Hide resolved
/* How Does the input is matched?
* When the generated scanner is run, it analyzes its input looking for strings which match any of its patterns.
* 1. If it finds more than one match, it takes the one matching the most text.
* 2. If it finds two or more matches of the same length, the rule listed first in the flex input file is chosen.
* Once the match is determined, the text corresponding to the match is made available in the global character pointer yytext, and its length in the global integer yyleng.
* The action corresponding to the matched pattern is then executed, and then the remaining input is scanned for another match.
* The last rule `.` could match any single character except `\n`.
*/
/* Reserved keyword */
"GO" { return TokenType::KW_GO; }
"AS" { return TokenType::KW_AS; }
Expand Down Expand Up @@ -538,3 +558,4 @@ LABEL_FULL_WIDTH {CN_EN_FULL_WIDTH}{CN_EN_NUM_FULL_WIDTH}*
}

%%
/* Flex user code section */