diff --git a/lang/spec.html b/lang/spec.html index 0e20d36f..ed1954bf 100644 --- a/lang/spec.html +++ b/lang/spec.html @@ -13,7 +13,7 @@ -

Ballerina Language Specification, 2022R4

+

Ballerina Language Specification, 2023R1 preview

Primary contributors:

@@ -27,7 +27,7 @@

Ballerina Language Specification, 2022R4

(Other contributors are listed in Appendix D.)

-Copyright © 2018-2022 WSO2 +Copyright © 2018-2023 WSO2

Licensed under the List constructor list-member-list := list-member (, list-member)* list-member := single-list-member | spread-list-member single-list-member := expression -spread-list-member := ... expression +spread-list-member := spread | aggregated-variable-reference +spread := ... expression

-A list-constructor-expr creates a new list value. The list is constructed by -processing each list-member in the order specified as follows. The expression in -the list-member is first evaluated resulting in a value v. If the -list-member is a single-list-member, then v is added to the list -being constructed. Otherwise v must be a list, and every member of -v is added in order to the list being constructed. It is a -compile-time error if the static type of the expression in a spread-list-member -is not a subtype of list. +A list-constructor-expr creates a new list value. It is a compile-time error if +the static type of the expression in a spread in a spread-list-member is not a +subtype of list. The list is constructed by processing each list-member in the +order specified as follows. If the list-member is a single-list-member, then the +expression in the list-member is first evaluated resulting in a value +v and v is added to the list being constructed. If the +list-member is a spread, then the expression in the spread is evaluated and each +member of the resulting list is added in order to the list being constructed. An +aggregated-variable-reference is handled like the equivalent spread.

If there is a contextually expected type, then the inherent type of the newly @@ -4830,7 +4833,7 @@

List constructor

member of the union that is a list type descriptor and that allows list shapes of a length that the static types of the expressions in the list-member-list make it possible for the constructed list to have; this list type descriptor is -used as the inherent type. If there is a spread-list-member, and the type of its +used as the inherent type. If there is a spread, and the type of its expression is not fixed length, then the minimum of the total number of members that may be added to the list being constructed by this spread-list-member and any previous list-member must be greater than or equal to the minimum length @@ -4885,7 +4888,7 @@

Mapping constructor

value-expr := expression computed-name-field := [ field-name-expr ] : value-expr field-name-expr := expression -spread-field := ... expression +spread-field := spread

A mapping-constructor-expr creates a new mapping value. @@ -5400,6 +5403,11 @@

Variable reference expression

module-var-decl that includes isolated is only allowed within a lock-stmt.

+

+A variable-reference-expr cannot refer to an aggregated variable; an aggregated variable can +be referenced only by an aggregated-variable-reference. +

Field access expression

@@ -5732,6 +5740,29 @@

Function call expression

contextually expected type for the expression comes from the static type required for the expression as specified below.

+

+If +

+ +

+then the function-reference is treated as a reference to that langlib function. +In this case, if the langlib function requires at least one argument, but the +type of the aggregated-variable-reference allows an empty list, then the +evaluation of the function-call-expr will first check whether the +aggregated-variable-reference refers to an empty list, and, if so, the result of +the function-call-expr will be nil; the return type of the langlib function will +accordingly be unioned with nil. +

positional-args := positional-arg (, positional-arg)*
 positional-arg := expression
@@ -5782,17 +5813,18 @@ 
Function call expression

 field-descriptor.
 

 
rest-arg := ... expression
+class="grammar">rest-arg := spread | aggregated-variable-reference
 

 

-The static type of the expression in a rest-arg must be either a list type or a
-mapping type. If it is a list type, then the rest-arg is equivalent to
-specifying each member of the list as a positional-arg. If it is a mapping type,
-then the rest-arg is equivalent to specifying each field of the mapping as a
-named-arg, with the name and value of the named-arg coming from the name and
-value of the field. In either case, the static type of the expression must be
-such as to ensure that the equivalent named-args or positional-args would be
-valid.
+The static type of the expression in a spread in a rest-arg must be either a
+list type or a mapping type. If it is a list type, then the rest-arg is
+equivalent to specifying each member of the list as a positional-arg. If it is a
+mapping type, then the rest-arg is equivalent to specifying each field of the
+mapping as a named-arg, with the name and value of the named-arg coming from the
+name and value of the field. In either case, the static type of the expression
+must be such as to ensure that the equivalent named-args or positional-args
+would be valid. An aggregated-variable-reference is handled like the equivalent spread.
 

 

 If there is neither a named-arg nor a positional-arg for a parameter that is not
@@ -6749,112 +6781,54 @@ 
Trap expression

 

 
Query expression

 

-A query expression provides a language-integrated query feature using SQL-like
-syntax. In this version of Ballerina, the functionality is similar to a list
-comprehensions; future versions will provided richer functionality.
+Query expressions provide a language-integrated query feature using SQL-like
+syntax.
 

-
-
query-expr := [query-construct-type] query-pipeline select-clause [on-conflict-clause]
-query-construct-type :=
-  map
-  | table key-specifier
-  | stream
+
query-expr := query-select-expr | query-collect-expr
+query-select-expr := [query-construct-type] query-pipeline select-clause [on-conflict-clause]
+query-collect-expr := query-pipeline collect-clause
 query-pipeline := from-clause intermediate-clause*
 intermediate-clause :=
    from-clause
    | where-clause
    | let-clause
    | join-clause
-   | order-by-clause
    | limit-clause
+   | order-by-clause
+   | group-by-clause
 

 

 A query expression consists of a sequence of clauses. The semantics of clauses
 is specified in terms of transforming a sequence of frames, where a frame is a
 binding of variables to values. The input to each clause is a sequence of
-frames. As each clause is executed, it iterates over its input frames and emits
-output: the final clause, which is a select clause, emits output
-values; the other clauses emit output frames. When a
-query-expr is evaluated, its clauses are executed in a pipeline by making the
-sequence of frames emitted by one clause be the input to the next clause. Each
-clause in the pipeline is executed lazily, pulling input from its preceding
-clause. The input to the first clause is a single empty frame.
+frames.
 

 

-When a from or join clause is executed, it makes calls
-to the next method of an Iterator object. If any of these calls
-return an error value e, then we say that the clause completes
-early with error value e.
+The initial part of a query expression is a pipeline, where each clause outputs
+a sequence of frames; the sequence of frames output by one clause become the
+input to the next clause. As each clause in the pipeline is executed, it
+iterates over its input frames and emits output frames. The pipeline is executed
+lazily, with each clause pulling input from its preceding clause. The input to
+the first clause is a single empty frame. The output of the pipeline is the
+output of its last clause.
 

 

-A query-expr that does not construct a stream is evaluated as described in this
-paragraph. If a clause completes early with error value e, then the
-result of evaluating the query-expr is e. The normal rules apply if
-the execution of any clause results in the evaluation of an expression
-completing abruptly. Otherwise, the result of evaluating a query expression is a
-single value, which is constructed from the sequence of values emitted by the
-last clause. The result must be one of the following basic types:
-

-
    
-
  • list - the constructed list has a member for each emitted value; every
-emitted value must belong to type T, where T[] is the type of the constructed
-value
    • 
-
  • mapping - the constructed mapping has a member for each emitted value; every
-emitted value must belong to type [string, T], where map<T> is the type
-of the constructed value 
    • 
-
  • table - the constructed table has a member for each emitted value; every
-emitted value must belong to type T, where table<T> is the type of the
-constructed value
    • 
-
  • string - the constructed string is the concatenation of the emitted values;
-every emitted value must be of type string
    • 
-
  • xml - the constructed xml value is the concatenation of the emitted values;
-every emitted value must be of type xml
    • 
-
  • stream - the stream generates the emitted values
    • 
-

-

-A query-expr that constructs a stream must start with the stream
-keyword and is evaluated differently from a query-expr that constructs a value
-of other basic types. The clauses in the query-expr are executed lazily: clauses
-in the query-expr becomes closures, which are called as a result of next
-operations being performed on the stream. If a next operation causes the
-execution of a clause that completes early with an error value, then the error
-value is returned by the next operation; the evaluation of the query-expr will
-already have resulted in a stream value. If the next operation results in the
-evaluation of an expression within the query-expr completing abruptly with a
-check-fail, the associated error value will be returned as the result of the
-next operation. If the next operation results in the evaluation of an expression
-within the query-expr completely abruptly with a panic, then the next operation
-will complete abruptly with a panic.
-

-

-A query-expr that constructs a mapping must start with the map keyword.
-An emitted value [k, v] results in adding a
-field with key k and value v.
+The remainder of the query expression says how to compute the result of the
+query expression using the sequence of frames output by the query pipeline.
+There are two kinds of query expression: query-select expressions and
+query-collect expressions. In a query-select expression the pipeline is followed
+by a select clause; in a query-collect expression, the pipeline is
+followed by a collect clause.
 

 

-If the query-expr starts with table, then the query-expr will
-construct a table; the key-specifier specifies the key sequence of the
-constructed table in the same way as with a table-constructor-expr. Otherwise,
-the applicable contextually expected type determines the basic type of the value
-constructed. If there is no contextually expected type, then the basic type of
-the value constructed is the basic type of expression following in;
-it is a compile-time error if the static type of this expression is not a
-subtype of one of the basic types that a query expression can construct.
-

-

-When the query-expr is constructing a value of basic type list, mapping, table
-or xml, and the applicable contextually expected type is a subtype of readonly,
-then the query-expr will construct a value with its read-only bit set and the
-static type of the query-expr will be a subtype of readonly. Furthermore, the
-contextually expected type for the expression in the select-clause will also be
-a subtype of readonly.
-

-

-When during the construction of a map or table, an emitted value is added as a new
-member, it replaces any existing member with the same key value; when a new
-member replaces an existing member, it will have the same position in the order
-of members as the existing member. This behavior may be controlled by an
-on conflict clause.
+The execution of certain intermediate clauses can result in the pipeline being
+terminated prematurely. Specifically, when a from or
+join clause is executed, it makes calls to the next
+method of an Iterator object; if any of these calls return an error value
+e, then the clause is said to complete early with error
+value e. A query-select or query-collect expression will handle this
+early completion by terminating the execution of the pipeline, as described
+below.
 

 

 Variables bound by the clauses of a query-pipeline are implicitly final, and
@@ -7010,6 +6984,35 @@ 
Join clause

 
 

 
+

+
Limit clause

+
+limit-clause := limit expression
+

+

+A limit clause limits the number of frames emitted by a query pipeline.
+

+

+A limit clause is executed as follows:
+

+
    
+
  • evaluate expression resulting in a value n; variable bindings
+from input frames are not in scope for this evaluation;
    • 
+
  • if n is less than zero, then panic;
    • 
+
  • for each input frame f, while n is greater than 0
+
      
+
    • decrement n;
      • 
+
    • emit f.
      •  
+
    
+
    •  
+

+
+

+The static type of expression must be a subtype of int.
+

+
+

+
 

 
Order by clause

 
@@ -7074,51 +7077,232 @@ 
Order by clause

 
 
 

+
 

-
Limit clause

-
-limit-clause := limit expression
+
Aggregated variable bindings

+
+

+Frames constructed by collect and group-by clauses can contain
+aggregated variable bindings. The type of an aggregated variable
+binding is always a list type; as usual, the
+value bound to the variable belongs to this type. An aggregrated variable is
+special in that it is only allowed to be referenced by an
+aggregated-variable-reference; the semantics of such a reference are different
+from the semantics of a variable-reference-expr.
+

+

+An aggregated variable binding for a variable x is aggregated
+from a sequence of frames S, by binding x to a list
+with the same length as S, where the i-th member of the
+list is the value bound to x in the i-th frame in
+S. The type of the aggregated variable binding is derived from the
+type T of x in S and the kind of clause
+C that is constructing the binding. When C is a collect
+clause, S can be empty and the type is T[].
+When C is a group-by clause, S cannot be empty and so the
+type is [T,T...].
+

+
+
aggregated-variable-reference := identifier
 

 

-A limit clause limits the number of frames emitted by a query pipeline.
+A reference to an aggregated variable binding is implicitly spread: an
+identifier x that refers to an aggregated variable
+binding is treated like a spread ...y, where the
+variable y has the same type and value as
+x but is not aggregated.
 

 

-A limit clause is executed as follows:
+The grammar shows this by using the aggregated-variable-reference production. A
+reference to an aggregated variable is allowed only in a syntactic context where
+the grammar allows an aggregated-variable-reference. In each such syntactic
+context, the grammar also allows both a variable-reference-expr and a spread.
+When a reference to an aggregated variable occurs in an allowed context, it
+should be parsed as a aggregated-variable-reference rather than as a
+variable-reference-expr, and should be transformed into the equivalent spread.
+Any other reference to an aggregated variable should be parsed as a
+variable-reference-expr, and then rejected as semantically incorrect.
+

+

+
+

+
Group by clause

+
group-by-clause := group by grouping-key (, grouping-key)*
+grouping-key :=
+   variable-name
+   | inferable-type-descriptor variable-name = expression
+

+

+A group-by-clause that uses an inferable-type-descriptor is transformed into a a
+let-clause followed by a group-by-clause where every grouping-key is a
+variable-name. For example:
+

+
group by var x1 = E1, var x2 = E2
+

+

+is transformed to
+

+
let x1 = E1, x2 = E2 group by x1, x2
+

+

+A group-by clause where every grouping-key is a variable-name is executed by
+first partitioning the sequence of input frames S into a set
+P of non-empty sequences of frames, such that:
 

 
    
-
  • evaluate expression resulting in a value n; variable bindings
-from input frames are not in scope for this evaluation;
    • 
-
  • if n is less than zero, then panic;
    • 
-
  • for each input frame f, while n is greater than 0
-
      
-
    • decrement n;
      • 
-
    • emit f.
      •  
-
    
-
    •  
+
  • every member P is a subsequence (not necessarily consecutive) of S;
    • 
+
  • every constituent of S is a constituent of exactly one member of P;
    • 
+
  • two constituents f1 and f2 of
+S are constituents of the same member of P if and only if
+for every variable x that is a grouping-key,
+DeepEquals(x1,x2) is true, where
+x1 is the value bound to x in
+f1, and x2 is the value bound to
+x in f2.
    • 
 

+

+Next, for each sequence G in P, ordered by the order in
+which the first constituent of G occurs in S, an output
+frame is constructed from G and emitted. An output frame g
+is constructed from a sequence of input frames G as follows. The same
+variables are bound by g as are bound by G. For each
+variable that is a grouping-key, the value and type in g will be the
+same as the value and type in the first constituent of G. The binding
+for every other variable is aggregated from
+G.
+

+
+

+
+

+
Collect clause

+
collect-clause := collect expression
+

 
 

-The static type of expression must be a subtype of int.
+A query-collect expression is used to perform aggregation. It is evaluated as
+follows.
 

 
+
    
+
  1. The query-pipeline is executed to produce a sequence of frames S.
+If during this any clause completes early with error value e, then
+the result of evaluating the query-collect-expr is e. In addition,
+the normal rules apply if the execution of any clause results in the evaluation
+of an expression completing abruptly.
    2. 
+
  2. A frame f is constructed. For each variable x that is
+bound by S, f has a variable binding for x that
+is aggregated from S.
    3. 
+
  3. The expression in the collect-clause is evaluated with f in scope
+resulting in a value v.
    4. 
+
  4. The result of the query-collect expression is v.
    5. 
+

+
+

+The contextually expected type for the expression in the collect-clause is the
+contextually expected type for the query-collect-expr.
+

 

 
 

 
Select clause

+
 
 select-clause := select expression
 

 

-A select clause is executed as follows:
+When a query-select expression is evaluated, the select clause is
+executed as follows, so as to emit a sequence of values:
 

 
    
-
  • for each input frame f 
+
  • for each frame f output by the query pipeline
 
      
-
    • evaluate the expression with f in scope resulting in value v
      • 
+
    • evaluate the expression that follows select with f
+in scope resulting in value v
      • 
 
    • emit v
      •  
 
    
 
    •  
 

+

+The sequence of values emitted by the execution of the select
+clause is used to construct a value, which becomes the result of the
+query-select-expr. The constructed result value must be one of the following
+basic types:
+

+
    
+
  • list - the constructed list has a member for each emitted value; every
+emitted value must belong to type T, where T[] is the type of the constructed
+value;
    • 
+
  • mapping - the constructed mapping has a member for each emitted value; every
+emitted value must belong to type [string, T], where map<T> is the type
+of the constructed value; an emitted value [k,
+v] results in adding a field with key
+k and value v;
    • 
+
  • table - the constructed table has a member for each emitted value; every
+emitted value must belong to type T, where table<T> is the type of the
+constructed value;
    • 
+
  • string - the constructed string is the concatenation of the emitted values;
+every emitted value must be of type string;
    • 
+
  • xml - the constructed xml value is the concatenation of the emitted values;
+every emitted value must be of type xml;
    • 
+
  • stream - the stream generates the emitted values.
    • 
+

+

+The clauses in a query-select-expr are executed either eagerly or lazily. If a
+query-select-expr is constructing a stream, they are executed lazily; otherwise,
+they are executed eagerly. When the clauses in a query-select-expr are being
+executed eagerly, the execution happens during the evaluation of the
+query-select-expr. When the clauses are being executed lazily, the evaluation of
+the query-select-expr constructs a stream object without executing any of the
+clauses; the clauses becomes closures, which are called later as a result of
+next operations being performed on the stream.
+

+

+With eager execution, if any clause completes early with error value
+e, then the result of evaluating the query-select-expr is
+e. In addition, the normal rules apply if the execution of any clause
+results in the evaluation of an expression completing abruptly.
+

+

+With lazy execution, when a next operation on the stream results in the
+execution of a clause that completes early with error value e, then
+that next operation returns e. Similarly, if the next operation
+results in the evaluation of an expression within the query-select-expr
+completing abruptly with a check-fail, the associated error value will be
+returned as the result of the next operation; if the next operation results in
+the evaluation of an expression within the query-select-expr completely abruptly
+with a panic, then the next operation will complete abruptly with a panic.
+

+
+
query-construct-type :=
+  map
+  | table key-specifier
+  | stream
+

+

+If a query-construct-type is specified, then it determines the basic type of the
+result. A query-select-expr that constructs a mapping or stream must specify a
+query-construct-type of map or stream respectively.
+When constructing a table, the key-specifier specifies the key
+sequence of the constructed table in the same way as with a
+table-constructor-expr.
+

+

+If no query-construct-type is specified, the applicable contextually expected
+type determines the basic type of the value constructed. If there is no
+contextually expected type, then the basic type of the value constructed is the
+basic type of expression following in; it is a compile-time error
+if the static type of this expression is not a subtype of one of the basic types
+list, table, string or xml.
+

+

+When the query-select-expr is constructing a value of basic type list, mapping, table
+or xml, and the applicable contextually expected type is a subtype of readonly,
+then the query-select-expr will construct a value with its read-only bit set and the
+static type of the query-select-expr will be a subtype of readonly. Furthermore, the
+contextually expected type for the expression in the select-clause will also be
+a subtype of readonly.
+

 

 
 

@@ -7127,21 +7311,29 @@ 
On conflict clause

 on-conflict-clause := on conflict expression

-An on conflict clause is allowed only for a query expression that -constructs a map or table with a key sequence. The expression is evaluated when the -select clause emits a value that conflicts with a previous value, -in the sense that both values have the same key value in the map or table. The -expression is evaluated with the same frame in scope as the select clause that -emitted the value that conflicts with the previous value. The static type of the -expression must be a subtype of error?. If the result of evaluating -the expression is an error e, then the result of evaluating the -query-expr is e. Otherwise, the result must be nil, and the earlier -new value replaces the earlier conflicting value, in the same way as if there -not been an on conflict clause. +During the execution of the select clause of query-select-expr that is +constructing a map or table, when value emitted by the select clause is added as +a new member of the map or table being constructed, it replaces any existing +member with the same key value; when a new member replaces an existing member, +it will have the same position in the order of members as the existing member. +This behavior may be controlled by an on conflict clause.

-Note that the expression may have side-effects; for example, it may call a -function that logs an error. +An on conflict clause is allowed only for a query-select-expr that +constructs a table with a key sequence or a map. The expression is evaluated +when the select clause emits a value that conflicts with a previous +value, in the sense that both values have the same key value in the map or +table. The expression is evaluated with the same frame in scope as the select +clause that emitted the value that conflicts with the previous value. The static +type of the expression must be a subtype of error?. If the result +of evaluating the expression is an error e, then the result of +evaluating the query-select-expr is e. Otherwise, the result must be +nil, and the earlier new value replaces the earlier conflicting value, in the +same way as if there not been an on conflict clause. +

+

+Note that the expression in an on-conflict-clause may have side-effects; for +example, it may call a function that logs an error.

@@ -7340,21 +7532,24 @@

Actions

query-action-or-expr :=
-   [query-construct-type] query-action-pipeline select-action-clause [on-conflict-clause]
+class="grammar">query-action-or-expr := query-select-action-or-expr | query-collect-action-or-expr
+query-select-action-or-expr := [query-construct-type] query-action-pipeline select-action-clause [on-conflict-clause]
+query-collect-action-or-expr := query-action-pipeline collect-action-clause
 query-action-pipeline := from-action-clause intermediate-action-clause*
 intermediate-action-clause :=
    from-action-clause
    | let-action-clause
    | where-clause
+   | limit-clause
    | join-clause
    | order-by-clause
-   | limit-clause
+   | group-by-clause
 
 from-action-clause := from typed-binding-pattern in action-or-expr
 let-action-clause := let let-action-var-decl [, let-action-var-decl]*
 let-action-var-decl := [annots] typed-binding-pattern = action-or-expr
 select-action-clause := select action-or-expr
+collect-action-clause := collect action-or-expr

A query-action-or-expr is a query-expr in which @@ -8278,7 +8473,7 @@

Resources

computed-resource-access-path-segment := [ expression ] -resource-access-rest-segment := [ ... expression ] +resource-access-rest-segment := [ spread ]

A client-resource-access-action performs a A. References

B. Changes since previous releases

+
+

Summary of changes from 2022R4 to 2023R1

+
    +
  1. Query expressions now support grouping and aggregation, using new +group by and collect clauses.
    2. +
+
+

Summary of changes from 2022R3 to 2022R4