Basic Syntax

proposals/p0162.md

@@ -0,0 +1,373 @@
+# Syntax of Basic Carbon Features
+
+<!--
+Part of the Carbon Language project, under the Apache License v2.0 with LLVM
+Exceptions. See /LICENSE for license information.
+SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
+-->
+
+[Pull request](https://github.com/carbon-language/carbon-lang/pull/0162)
+
+## Table of contents
+
+<!-- toc -->
+
+-   [Problem](#problem)
+-   [Background](#background)
+-   [Proposal](#proposal)
+
+<!-- tocstop -->
+
+## Problem
+
+The purpose of this proposal is to establish some basic syntactic
+elements of the Carbon language and make sure that the grammar is
+unambiguous and can be parsed by an LALR parser such as `yacc` or
+`bison`.  The grammar presented here has indeed been checked by
+`bison`. The language features in this basic grammar include control
+flow via `if` and `while`, functions, simple structures, choice, and
+pattern matching. The main syntactic categories are `declaration`,
+`statement`, and `expression`. Establishing these syntactic categories
+should help the other proposals choose syntax that is compatible with
+the rest of the language.
+
+## Background
+
+The grammar proposed here is based on the following proposals:
+
+* Carbon language overview []() 
+* proposals for pattern matching
+  [#87](https://github.com/carbon-language/carbon-lang/pull/87), 
+* structs [#98](https://github.com/carbon-language/carbon-lang/pull/98), 
+* tuples [#111](https://github.com/carbon-language/carbon-lang/pull/111), and
+* metaprogramming [#89](https://github.com/carbon-language/carbon-lang/pull/89).
+
+There may be places that this grammar does not accurately capture what
+was intended in those proposal, which should trigger some useful
+discussion and revisions.
+
+## Proposal
+
+We summarize the three main syntactic categories here and define the
+grammar in the next section.
+
+* `declaration` includes function, structure, and choice definitions.
+
+* `statement` includes local variable definitions, assignment, blocks, `if`,
+    `while`, `match`, `break`, `continue`, and `return`.
+
+* `expression` plays three roles. In an initial attempt these roles
+    were separate, with three different syntactic categories, but that
+    led to ambiguities in the grammar. Folding them into one category
+    resolved the ambiguities. The three roles will be teased apart in
+    the static and dynamic semantics.
+
+    1. The `expression` category plays the usual role of expressions
+       that produce a value, such as integer literals and arithmetic
+       expression.
+
+    2. To fascilitate metaprogramming and reflection, `expression`
+       also includes type expressions, such as function types
+       and variables that are aliases for types.
+
+    3. `expression` is also used for patterns, for example, in the
+        `case` of a `match`, in the parameters of a function, and on
+        the left-hand side of variable definitions.
+
+The proposal also specifies the abstract syntax.
+
+## Details
+
+### Expressions
+
+The following grammar defines the concrete syntax for
+expressions. Below we comment on a few unusual aspects of the grammar.
+
+    expression:
+      identifier
+    | expression '.' identifier
+    | expression '[' expression ']'
+    | expression ':' identifier
+    | integer_literal
+    | "true"
+    | "false"
+    | tuple
+    | expression "==" expression
+    | expression '+' expression
+    | expression '-' expression
+    | expression "&&" expression
+    | expression "||" expression
+    | '!' expression
+    | '-' expression
+    | expression tuple
+    | "auto"
+    | "fn" tuple "->" expression
+    ;
+    tuple:
+      '(' field_list ')'
+    ;
+    field_list:
+      /* empty */
+    | field
+    | field ',' field_list
+    ;
+    field:
+      expression
+    | identifier '=' expression
+    ;
+
+The grammar rule
+
+    expression:  expression ':' identifier
+
+is for pattern variables. For example, in a variable definition such as
+
+    var Int: x = 0;
+
+the `Int: x` is parsed with the grammar rule for pattern variables.
+In the above grammar rule, the `expression` to the left of the `:`
+must evaluate to a type at compile time.
+
+The grammar rule
+
+    tuple:  '(' field_list ')'
+
+is primarily for constructing a tuple, but it is also used for
+creating tuple types and tuple patterns, depending on the context in
+which the expression occurs.
+
+### Statements
+
+The following grammar defines the concrete syntax for statements.
+
+    statement:
+      "var" expression '=' expression ';'
+    | expression '=' expression ';'
+    | expression ';'
+    | "if" '(' expression ')' statement "else" statement
+    | "while" '(' expression ')' statement
+    | "break" ';'
+    | "continue" ';'
+    | "return" expression ';'
+    | '{' statement_list '}'
+    | "match" '(' expression ')' '{' clause_list '}'
+    ;
+    statement_list:
+      statement
+    | statement statement_list
+    ;
+    clause_list:
+      /* empty */
+    | clause clause_list
+    ;
+    clause:
+      "case" expression "=>" statement
+    | "default" "=>" statement 
+    ;
+
+In the grammar rule for the variable definition statement
+
+    statement:  "var" expression '=' expression ';'
+
+the left-hand-side `expression` is used as a pattern, so it would
+typically evaluate to a variable pattern or some other kind of value
+(such as a tuple) that contains variable patterns.
+
+Likewise, in the rule for `case`
+
+    clause:  "case" expression "=>" statement
+
+the `expression` is used as a pattern.
+
+
+### Declarations
+
+The following grammar defines the concrete syntax for declarations.
+
+    declaration:
+      "fn" identifier tuple return_type '{' statement_list '}'
+    | "fn" identifier tuple ARROW expression ';'
+    | "struct" identifier '{' member_list '}'
+    | "choice" identifier '{' alternative_list '}'
+    ;
+    return_type:
+      /* empty */
+    | ARROW expression
+    ;
+    member:
+      "var" expression ':' identifier ';'
+    | "method" identifier expression "->" expression '{' statement_list '}'
+    ;
+    member_list:
+      /* empty */
+    | member member_list
+    ;
+    alternative:
+      "alt" identifier expression ';'
+    ;
+    alternative_list:
+      /* empty */
+    | alternative alternative_list
+    ;
+    declaration_list:
+      /* empty */
+    | declaration declaration_list
+    ;
+
+In the grammar fule for function definitions
+
+    declaration:  "fn" identifier tuple return_type '{' statement_list '}'
+
+the `tuple` is used as a pattern to describe the parameters of the
+function whereas the `expression` in the `return_type` must evaluate
+to a type at compile-time. The grammar for function definitions does
+not currently include implicit parameters, but the intent is to add
+them to the grammar in the future.
+
+In the rule for field declarations
+
+    member:  "var" expression ':' identifier ';'
+
+the `expression` must evaluate to a type at compile time.
+The same is true for the `expression` in the grammar
+rule for an alternative:
+
+    alternative:  "alt" identifier expression ';'
+
+### Precedence and Associativity
+
+The following defines the precendence and associativity of symbols
+used in the grammar. The ordering is from lowest to highest
+precendence. The main goal of the choices here is to stay close to
+C++.
+
+    nonassoc '{' '}'
+    nonassoc ':' ','
+    left "||"
+    left "&&"
+    left "=="
+    left '+' '-'
+    right '!' '*' '&'
+    left '.' "->"
+    nonassoc '(' ')' '[' ']' 
+
+For more information on operators and precedence, see proposal
+[#168](https://github.com/carbon-language/carbon-lang/pull/168).
+
+### Abstract Syntax
+
+The output of parsing is an abstract syntax tree.  There are many ways
+to define abstract syntax. Here I'll simply use C-style `struct`
+definitions.
+
+
+#### Abstract Syntax for Expressions
+
+    enum ExpressionKind { Variable, PatternVariable, Dereference, Int, Bool, 
+                          PrimitiveOp, Call, Tuple, Index, GetField, 
+                          IntT, BoolT, TypeT, FunctionT, AutoT };
+    enum Operator { Neg, Add, Sub, Not, And, Or, Eq };
+
+    struct Expression {
+      ExpressionKind tag;
+      union {
+        struct { string* name; } variable;
+        struct { Expression* aggregate; string* field; } get_field;
+        struct { Expression* aggregate; Expression* offset; } index;
+        struct { string* name; Expression* type; } pattern_variable;
+        int integer;
+        bool boolean;
+        struct { vector<pair<string,Expression*> >* fields; } tuple;
+        struct { 
+          Operator operator_; 
+          vector<Expression*>* arguments; 
+        } primitive_op;
+        struct { Expression* function; Expression* argument; } call;
+        struct { Expression* parameter; Expression* return_type;} function_type;
+      } u;
+    };
+
+The correspondence between most of the grammar rules and the abstract
+syntax is straightforward. However, the parsing the `field_list`
+deserves some explanation.  The fields can be labeled with the grammar
+rule:
+
+    field:  identifier '=' expression
+
+or unlabeled, with the rule
+
+    field: expression
+
+The unlabeled fields are given numbers (represented as strings) for
+field labels, starting with 0 and going up from left to right.
+
+Regarding the rule for tuples:
+
+    tuple:  '(' field_list ')'
+
+if the field list only has a single unlabeled item, then the parse
+result for that item is returned. Otherwise a `tuple` AST node is
+created containing the parse results for the fields.
+
+
+#### Abstract Syntax for Statements
+
+    enum StatementKind { ExpressionStatement, Assign, VariableDefinition,
+                         Delete, If,  Return, Sequence, Block, While, Break, 
+                         Continue, Match };
+
+    struct Statement {
+      StatementKind tag;
+      union {
+        Expression* exp;
+        struct { Expression* lhs; Expression* rhs; } assign;
+        struct { Expression* pat; Expression* init; } variable_definition;
+        struct { Expression* cond; Statement* thn; Statement* els; } if_stmt;
+        Expression* return_stmt;
+        struct { Statement* stmt; Statement* next; } sequence;
+        struct { Statement* stmt; } block;
+        struct { Expression* cond; Statement* body; } while_stmt;
+        struct {
+          Expression* exp;
+          list< pair<Expression*,Statement*> >* clauses;
+        } match_stmt;
+      } u;
+    };
+
+#### Abstract Syntax for Declarations
+
+    struct FunctionDefinition {
+      string name;
+      Expression* param_pattern;
+      Expression* return_type;
+      Statement* body;
+    };
+
+    enum MemberKind { FieldMember };
+
+    struct Member {
+      MemberKind tag;
+      union {
+        struct { string* name; Expression* type; } field;
+      } u;
+    };
+
+    struct StructDefinition {
+      string* name;
+      list<Member*>* members; 
+    };
+
+    enum DeclarationKind { FunctionDeclaration, StructDeclaration, 
+                           ChoiceDeclaration };
+
+    struct Declaration {
+      DeclarationKind tag;
+      union {
+        struct FunctionDefinition* fun_def;
+        struct StructDefinition* struct_def;
+        struct { 
+          string* name; 
+          list<pair<string, Expression*> >* alts; 
+        } choice_def;
+      } u;
+    };