Isolated module level variables and objects

[jclark]

This builds on #145 and adds

• isolated objects
• isolated module level variables

as described in the proposal.

[jclark]

This is required to support service concurrency #116

[jclark]

When we do this, remove "concurrency safety" from future functionality appendix.

[MaryamZi]

Regarding copying out values, the proposal says

Values are transferred out by writing to a variable defined outside the lock statement or by a return statement. A value can only be transferred in or out only if it is readonly, is the result of a calling the clone method or is an isolated object. In addition, within the lock statement only isolated functions can be called.

Consider the following example:

isolated class Foo {
    private map<int> m = {};

    isolated function baz() {
        map<int> x = {};
        
        map<int>[] y = [x];
        Bar bar = new;

        lock {
            x = self.m; // Case I - error, OK

            // Also errors?
            y[1] = self.m; // Case II
            y.push(self.m); // Case III
            bar.setMap(self.m); // Case IV
        }
    }
}

class Bar {
    int i = 1;
    private map<int> m = {};

    isolated function setMap(map<int> m) {
        self.m = m;
    }
}

While Case I is an error (since it is "writing to a variable defined outside the lock statement"), I assume the rest should also result in errors?

Would appreciate clarification regarding the condition that would make them errors?

• can case II and case III be considered as "writing to a variable defined outside the lock statement"?
• even though case III and IV call isolated methods, they are still transferring mutable values outside the lock statement?

[jclark]

Case I is an error because you are writing to variable x and the type of x is not readonly/cloned/isolated.

In case II and III you are reading the value of variable y and so transferring it in. This is an error because it is not readonly/cloned/isolated. Similarly case IV is an error because you are reading variable bar.

[MaryamZi]

Yes, missed that. Thank you.

[MaryamZi]

Follow up question regarding using isolated with an object-type-descriptor (#145 (comment)).

Since object-type-descriptors don't allow private fields or final fields, we can't have a private mutable field or an immutable field, right? Does this effectively mean we can only use isolated with an object-type-descriptor that has no fields?

[jclark]

This would be OK:

type IO isolated object {
   int x;
};
isolated class IC {
   *IO;
   final int x = 10;
}

So the answer to your last question is no.

[MaryamZi]

Noted, will allow this.

[jclark]

@MaryamZi I think I gave the wrong answer to you in #145 (comment) as regards isolated on an object-constructor-expr. I think we should allow it there, since it is drastically affects how you write the methods of the object. In general, a listener is not going to require an isolated service type, so I won't work well for the service declaration to infer isolated from that. Instead, the service declaration should be allowed to be declared to be isolated (which would desugar into isolated on the object constructor). Then we can support inference of isolated for service declarations just as we already do inference of isolated for module level functions.

[jclark]

Still need to figure out when a query-expr or object-constructor-expr should be treated as isolated; can probably push till later. I haven't yet made variables that are only referenced once be isolated; it's better just to rely on let expressions (where we can allow multiple references).

[jclark]

Things still to do:

• module level variable with initializer can be declared as isolated
  • can only be referred to be within a lock statement
  • isolated functions can refer to it
  • initializer must be isolated expression
  • syntactic ambiguity between isolated belonging to variable and to type
  • extend inference to handle this
• isolated objects
  • object type can be isolated
  • isolated functions can refer to final variable whose value is isolated object
  • expression of type isolated object {} is isolated
  • class can be isolated
  • object constructor can be isolated
  • fields of an isolated object that are not final or not readonly|isolated object{} must be private
  • all methods can refer to self only within a lock statement; but self.x where x is final and readonly or isolated is OK
  • every field must be initialized with isolated expression
  • service declarations can be isolated
  • extend inference to cover this
• if a lock statement refers to a protected variable (i.e. the self of an isolated method or an isolated module level variable)
  • the lock statement cannot refer to another protected variable,
  • define allowed transfers in/out of lock statement in terms of isolated expressions
  • only isolated functions can be called

[jclark]

I wonder if clone and Cloneable should treat isolated objects like immutable values. So inter-worker message send could send an isolated object between workers.

[MaryamZi]

@MaryamZi I think I gave the wrong answer to you in #145 (comment) as regards isolated on an object-constructor-expr. I think we should allow it there, since it is drastically affects how you write the methods of the object. In general, a listener is not going to require an isolated service type, so I won't work well for the service declaration to infer isolated from that. Instead, the service declaration should be allowed to be declared to be isolated (which would desugar into isolated on the object constructor). Then we can support inference of isolated for service declarations just as we already do inference of isolated for module level functions.

Noted, we will allow these.

[MaryamZi]

Since the module variable declaration section says "The keyword isolated occurring immediately before the typed-binding-pattern is parsed as part of isolated-final-quals rather than as part of typed-binding-pattern.", is it correct for the following to result in an error, since it would mean duplicate isolated qualifiers in isolated-final-quals?

e.g.,

isolated isolated object {} x = new Foo();

isolated class Foo {
}

[jclark]

That sentence is only to address the ambiguity in

isolated object {} x = new Foo();

The grammar would allow isolated in the above to be parsed two ways. There is no ambiguity in:

isolated isolated object {} x = new Foo();

[MaryamZi]

Noted, thank you.

Would also appreciate clarification regarding the following, for isolated expressions.

The spec says

An expression is always isolated if its static type is immutable, i.e. a subtype of readonly, or an isolated object, i.e. a subtype of isolated object {}.

In addition, the following expressions are also isolated if all their subexpressions are isolated:

• list-constructor-expr
• table-constructor-expr
• mapping-constructor-expr
• xml-template-expr
• raw-template-expr
• type-cast-expr
• checking-expr
• trap-expr
• conditional-expr

The section for each kind of expression will describe any additional conditions under which that kind of expression is isolated.

Does this mean that for one of these explicitly mentioned expressions, or for one of the expression kinds where there are additional rules, we do not have to check the sub-expressions if the static type is a subtype of readonly/isolated object {}?

For example, in the following scenarios the static type of the expression is a subtype of readonly, but if we analyze the sub-expressions, they do not meet the criteria for isolated expressions.

int i = <int> nonIsolatedVarRef;

or

string s = nonIsolatedFunction(nonIsolatedExpr);

[jclark]

The point of an isolated expression is to ensure that the result of the expression is a guaranteed to be an isolated root and unaliased. For a result that is a subtype of readonly | isolated object {} the former condition is automatically satisfied and the latter is irrelevant (it's only freely reachable mutable state that musn't be aliased).

So if an expression satisfies the condition in the first para (its static type is immutable, i.e. a subtype of readonly, or an isolated object, i.e. a subtype of isolated object {}), no further checking is needed: it's guaranteed to be an isolated expression.

The next paragraph is giving another way in which particular expressions can be isolated. For example:

[1, 2, 3]

is isolated, because all the sub-expressions are (trivially) isolated.

Both these examples are fine:

int i = <int> nonIsolatedVarRef;
string s = nonIsolatedFunction(nonIsolatedExpr);

because an int or a string is immutable and so is automatically an isolated root.

Please have a careful look at the definition of isolated root here: https://ballerina.io/spec/lang/master/#section_5.1.3 . It's the key to the whole design.

[MaryamZi]

Noted, this is clear now.

[jclark]

I have pushed two possible compatible improvements into separate issues: #629, #630.