proposal: Go2: check and catch as built-in macros

[ohir]

The check and the catch revisited as a macro

The implicit version of the check already compiles at playground.
This proposal is not a direct counter-proposal to #32437. It stands by itself.
The check macro is not a one-liner: it helps the most where many repetitive
checks using the same expression should be performed in close proximity.
Recent use of the catch word was spotted in: #32813 and #32611.

Design constraints (met)

It is a built-in, it does not nest in a single line, it allows for way more flows than try and has no expectations about the shape of a code within. It does not encourage naked returns.

usage example

// built-in macro signature: 
func check(Condition bool) {}

check(err != nil) // explicit catch: label.
{
    ucred, err := getUserCredentials(user)
    remote, err := connectToApi(remoteUri)
    err, session, usertoken := remote.Auth(user, ucred)
    udata, err := session.getCalendar(usertoken)

  catch:               // sad path
    ucred.Clear()      // cleanup passwords
    remote.Close()     // do not leak sockets
    return nil, 0,     // dress before leaving
      fmt.Errorf("Can not get user's Calendar because of: %v", err)
}
// happy path

// implicit catch: label is above last statement
check(x < 4) 
  {
    x, y = transformA(x, z)
    y, z = transformB(x, y)
    x, y = transformC(y, z)
    break // if x was < 4 after any of above
  }

• The func check(Condition bool) {} built-in macro inserts a check for the Condition after any matching statement in the anonymous { block } that immediately follows the macro invocation.
• "Matching" are statements where a variable identifier used in the "Condition" expression of the check appears on the left to the "=" or ":=" assignment operator within the block.
• If the Condition at any inserted check evaluates to true, statements under the "catch:" label inside the block are executed immediately.
• If the "catch:" is reached naturally, all statements under it to the end of the block are skipped.
• If there is none "catch:" label given within the block, it is assumed to be present right before the last statement of the block.

Implementation uses none magic:

1. The LN "label mutator" string is made of the line number where check macro was invoked. Then catchˁLN and okˁLN labels are used.
2. After any matching statement, the ; if Condition { goto catchˁLN }; is put in the place of the single 'invisible semicolon'.
3. When the catch: label is spotted (parsed), it gets replaced with a goto okˁLN; statement followed by a catchˁLN: label.
4. If parser got to the last statement of the block and there was no catch: label yet, this statement gets prepended with the goto okˁLN; catchˁLN:.
5. The okˁLN: label is put right before the end of the block.
6. The block node, and all autoinserted nodes are simply flagged with check context annotation for AST tools (gofmt, gopls and others).

Limitations

Variables local to the block must be declared with var at top of the block.

This is a direct consequence of keeping to 'no-magic' principle.
Go's sane goto usage safeguards should not be circumvented.

Thats all spec.

Macro expansion explained:

check(err != nil)     // line 123 → catchˁ123, okˁ123 labels.
  {
    x, y, err := fa() // '\n' → ";" → "; if err != nil; { goto catchˁ123 };"
    y += 2            // '\n' → ";"       //  no err at lhs, no check
    x = fb(y)         // '\n' → ";"       //  no err at lhs, no check
    err = fc(x, y)    // '\n' → ";" → "; if err != nil; { goto catchˁ123 };"

    goto okˁ123       // happy path, added by the macro
  catchˁ123:          // "catch:" → "catchˁ123" // subst by the macro
    cleanup()         //
    return 0, 0, err  // dressed returns are better than naked
  okˁ123:             // happy exit point, added by the macro
  }

check(x < 4)                // line 234, catchˁ234, okˁ234 labels.
  {
    x, y = transformA(x, z) // check: if x < 4 { goto catchˁ234 }
    y, z = transformB(x, y) // no x, no check
    x, y = transformC(y, z) // check: if x < 4 { goto catchˁ234 }
                            // goto okˁ234 (implicit)
                            // catchˁ234:  (implicit)
    break                   // last statement
                            // okˁ234:     (implicit)
  }

---

changes:

-"Matching" are statements that assign a value to at least one of variables that are checked in the "Condition" expression
+"Matching" are statements where a variable identifier used in the "Condition" expression of the check appears on the left to the "=" or ":=" assignment operator within the block.

+"Limitations" section.

[beoran]

Instead of built in macros, I think we need hygienic macros in go, as described in #32620, so go developers can experiment with macros themselves, and use the ones they like themselves in their own projects.

[ohir]

Instead of built in macros, I think we need hygienic macros in go, as described in #32620, so go developers can experiment with macros themselves, and use the ones they like themselves in their own projects.

Ah, wouldn't it be a wonderful change?! Giving to every person ability to make its own flavor of Go? It would also enable readers to feel again this peculiar joy of jumping from file to file, to wonder the beauty of the macros and to adore the genius of their creators... https://docs.racket-lang.org/reference/

[natefinch]

This has the same problems as try/catch in other languages, namely that you wrap a whole bunch of lines in a handler and then the handler code can't know what part failed. It discourages people from handling every error individually. And just like exception in other languages, (which is actually worse than the try proposal) - there's zero indication on each line that you might suddenly exit if that line fails.

[bcmills]

CC @griesemer

[griesemer]

@ohir Built-ins in Go have a clearly specified syntax: Syntactically, they look like function calls with expressions as arguments. There may be additional rules for the arguments, but they remain expressions. There may be special behavior to the built-ins, but syntactically they remain function calls.

What you are proposing does not look like a function call, nor an expression: there's a {} block following the check condition. This cannot fit into the existing syntax of Go and certainly cannot be considered a built-in - macro or not.

In order to make something like check work inside Go you'd have to extend the statement syntax. That is a significant, non-backward compatible language change. Claiming that something is a "macro" does not make it work automatically. Claiming that something it a "built-in" does not make it one.

What you are proposing is a new form of statement for Go; a check statement. There are a lot of open questions in this proposal, starting with the exact syntax specification, to the detailed semantics of the new statement.

For instance, you say that check automatically inserts a check for all matching statements inside the block, where a matching statement is an assignment that at least assigns to one variable checked in the check. I don't know how to make this work: What happens if one assigns to a pointer variable that happens to be an alias to the variable checked? (This cannot be easily detected in general). What kind of expressions are permitted in the check? Can it be function calls? What if those function calls access the variables which are being assigned to in the check block? Etc.

In short, I don't believe this proposal can be implemented in any reasonable way. It is also not very Go like as it introduces invisible code to be executed upon assignments.

More importantly, you are proposing a fairly general new mechanism with all kinds of ramifications without an argument as to why this is needed. Certainly, error handling can be done with less effort.

I am against this proposal.

[ohir]

@natefinch Thank you. It is a valid concern about possible abuse of this construct. Though I do (I might) know exactly which what part failed from the err (or other variable being checked) if I am really interested in it.

[ohir]

@griesemer

For instance, you say that check automatically inserts a check for all matching statements inside the block, where a matching statement is an assignment that at least assigns to one variable checked in the check.

What happens if one assigns to a pointer variable that happens to be an alias to the variable checked? (This cannot be easily detected in general).

It needs not to be detected. The check is envisioned as a real macro, expanded at first parse to the marked nodes[1] and with none magic underhood. check will work exactly as explained in the 'Implementation' section, and one can reason about it using real labels and gotos.

Though this exposed the vagueness of the "matching" definition I gave. After correction it should say:

• "Matching" are statements where a variable identifier used in the
  "Condition" expression of the check appears on the left to the "="
  or ":=" assignment operator within the block.

What kind of expressions are permitted in the check?

As it is a macro, it is expanded, it allow any and all expressions that have a bool result. Ie. any expression that can be parsed in the expanded if.

The "Implementation" section of the proposal is straight. Where it says "insert ; if Condition { goto catchˁLN };" it will make it to the AST using the check's Condition source string (what in turn will allow Condition to operate on identifiers local to the block).

Can it be function calls? What if those function calls access the variables which are being assigned to in the check block? Etc.

There is no magic in check. A source of:

check(err != nil || rv == nil)
{
    rv, err := fn()
    continue retry
}

will result in same AST as

{
    rv, err := fn()
    if err != nil || rv == nil {
        goto catchˁ1
    }
    goto okˁ1
  catchˁ1:
    continue retry
  okˁ1:
}

([1] same AST spare flag annotations on nodes check..., if... and down)

In order to make something like check work inside Go you'd have to extend the statement syntax.

I am pleased to inform you that check already compiles on current playground. At least in its short (no label) form.

I don't believe this proposal can be implemented in any reasonable way. I don't know how to make this work:

I do. The 'Implementation' part is the implementation sketch. The check expansion belongs mostly to cmd/compile/internal/syntax/parser.go

More work will be with the fmt printer. All other go tools will stay untouched, including debuggers, where the real identifiers like goto okˁ234 should be given in plain. One of the goals of check construct is to allow mere mortals to understand fully what is under the check's hood.

It is also not very Go like as it introduces invisible code to be executed upon assignments.

With all due respect, the check invisible code is no less visible than try's return. If users can be expected to learn and understand

It returns the first n arguments (if any) if the (final) error argument is nil, otherwise it returns from the enclosing function with that error.

they should also be trusted to learn and understand

"the if Condition goto catch is added after any line where you assigned to a variable mentioned in the Condition".

you are proposing a fairly general new mechanism

Agreed, its new, but it reads Go: check(Condition) { block }

with all kinds of ramifications without an argument as to why this is needed.

I will not beat around the bush: the real impulse to spending a week on thinking about simplest solution to 'if err != nil' whinning, and one that will benefit the most of users was my deep concern about try's ability to nest. Try is being pushed and its implementation is a WiP. And - as try can be abused, it will be abused for sure. Without an ample alternative to nested try (to its abuse), it will sooner than later pollute the OSS codebase. IMO.

The check construct in my opinion can be an easier and more Gooish alternative to nested trys, while leaving it (try) for places where it shines: ie. single line checks for err then return err.

Certainly, error handling can be done with less effort.

Yes, indeed. Though check is not only about error handling, there are other (business logic, math) places where it might be of use.

[griesemer]

Thanks @ohir for your explanations.

So what happens if the condition in check refers to variables that have not been defined yet in the block? Say, the condition refers to variables a and b and a and b are not introduced in assignments at the same time? For instance:

check(a == b) {
   a := f1()
   b := f2()
}

when is the check a == b executed? Presumably only after the second assignment?

As you say, any and all expressions can be used with check - it's just a macro. In

check(f()) {
   ...
}

where f() returns a boolean, and I never refer to f anywhere in the check block - presumably f() get's never called? What if one assigns to f?

check(f()) {
   f := g()
}

What if f is not a function? What it f() doesn't return a boolean value? Etc. etc.

Also, this construct would be the first construct where we can refer to an identifier, f in this example, before it's been declared. In Go, all identifiers inside a function must be declared and in scope before they can be used.

This proposal seems to be open many more questions and problems than it seeks to solve.

[ohir]

@griesemer

So what happens if the condition in check refers to variables that have not been defined
yet in the block? Say, the condition refers to variables a and b and a and b are
not introduced in assignments at the same time?

check(a == b) {
   a := f1()
   b := f2()
}

For instance: when is the check a == b executed? Presumably only after the second assignment?

With this example? Never. It is an invalid code and effects in a compile error:

• ./prog.go:2:55: goto okˁ1 jumps over declaration of b at ./prog.go:3:11

check hides nothing

as I said. Neither from the tools nor from the user. It gets expanded before
the syntax checking phase and stays in its expanded form till gofmt printer will
collapse it again. The most frequent place for user to see her check expanded will
be in the -cover output.

So lets expand it now to the form used in the syntax checking phase:

check(a == b) // marked node, it gets thru (as a node string for the Condition) 
{
        a := f1(); if a == b { goto catchˁ1; }; goto okˁ1; catchˁ1:
        b := f2(); okˁ1:
}

1. Playground error /aͺ, bͺ local declaration/
2. Playground valid /no locals/

Lets correct above snippet
(the b := f2() as a catch: statement was an error likely)

check(a == b)
{  // this fmt rule stays, to highlight we're using a macro
   a := f1()
   b := f2()
   break     // Condition == true statement (implicit catch:)
}
./prog.go:11:43: goto catchˁ1 jumps over declaration of bͺ at ./prog.go:12:17
./prog.go:11:61: goto okˁ1 jumps over declaration of bͺ at ./prog.go:12:17

See at playground

As you say, any and all expressions can be used with check - it's just a macro. In

check(f()) {
...
}

where f() returns a boolean, and I never refer to f anywhere in the check block - presumably f() get's never called?

The spec says now identifier of a variable. So bare (f()) will effect in compiler (or vet) error (tbd):

• ./prog.go:1:1: "check(expression)" needs at least one variable to check for.

What if one assigns to f? What if f is not a function? What if f() doesn't return a boolean value? Etc. etc.

Irrelevant. Check considers only varible assignments. I amended the spec already (thanks for pointing ambiguity in previous "matching" definition).

Also, this construct would be the first construct where we can refer to an identifier, [...] before it's been declared.

Yes, the check would be a first built-in fullblood macro and first construct of its kind. Just like either the make or append (cant recall which one) was the first built-in function once upon a time.

In Go, all identifiers inside a function must be declared and in scope before they can be used.

The check Condition expression makes a promise that any that is not already in scope will be
declared. If they won't, there will⁽ˣ⁾ be a compiler error about undeclared variable used in an if expression. (⁽ˣ⁾Now it is a goto over error, but this gonna change in 1.14 afaik).

This proposal seems to be open many more questions

I welcome tough questions and discussion. There can be flaws lurking I could not seen
and I would like this proposal to either fall or pass on merit and on merit only.

and problems than it seeks to solve.

It eg. shortens the (much frequent in production) code where one must acquire a set of remote services before doing a business on all of them:

  for tries > 0 {
    check(err != nil)
    {
      s1, err = getService1(uriAuth)
      s2, err = getService2(uriLog)
      s3, err = getService3(uriBasket)
      s4, err = getService4(uriCheckout)
    catch:
      metricsTimeWait(tries)
      tries--
      continue
    }
  }
  if err != nil { metricsAcquireFailed(err) // user can't shop }
  // use services

Without check above snippet usually takes a 5L+ for each getService to acquire.

[griesemer]

Thanks for the explanations.

Contrary to what you say, check clearly hides jumps and labels. That by itself may be ok (consider it an "abstraction"), but it's clearly not ok that the compiler can produce an error message such as goto okˁ1 jumps over declaration of b when there is no goto to be seen in the code (i.e., the abstraction is leaky).

As I said before, this proposal seems to open more questions and problems than it tries to solve, with little motivation as to why one would want such a mechanism.

[ianlancetaylor]

@ohir You seem to be saying that := may not be used inside a check block, because it will lead to a "jumps over declaration" error. That seems like a serious drawback.

[ohir]

@ianlancetaylor
Yes. Local scope variables can be used but until #26058 will resolve, variables local to the block need to be var declared before the first check. Even after #26058 there might be cases where such an explicit declaration should stay. Ie. if the catch part would do something with a local variable.

This is a real flaw that should either be addressed or excluded. Thank you.
Likely it means that, against my assumptions, check will be in need of a helping hand from the syntax checker.

Edit: added Limitations section to the spec.

saying that := may not be used inside a check block, because it will lead to a "jumps over
declaration" error. That seems like a serious drawback.

@griesemer

it's clearly not ok that the compiler can produce an error message such as goto okˁ1 jumps over declaration of b when there is no goto to be seen in the code
This is the current state and current compiler. As nodes are marked as being expanded, syntax checking MAY produce an error message pertaining to the macro invocation. Eg.

• ./prog.go:1:1: "Variable "x" in check(expression) is neither in scope nor declared in the 'check' block.

But - again - I personally would like users of a macro to know they are using a macro and how does this macro expands.

[ohir]

Update. Added Limitations section.

Rationale: A macro may not change language specs. Even if #26058 would loosen goto safeguards, some code that is not "checked" within the block might trigger spurious error message too. What means that issue pointed to by @griesemer and @ianlancetaylor is valid. It may not be addressed in other than by specifying its limitations.

[ohir]

@natefinch

wrap a whole bunch of lines in a handler and then the handler code can't know what part failed. It discourages people from handling every error individually.

With just added (natural) limitation the whole bunch of lines should be fairly limited. The potential to gross abuse you pointed at, as in enclosing in check huge blocks of code, diminished. IMO.

@griesemer

with little motivation as to why one would want such a mechanism.

The "must acquire with retries" pattern I have shown is the bread and butter of microservice architectures (and many "bigger-services" too). As it happens "cloud" backend is the domain where Go has most applications.

[ianlancetaylor]

This proposal as it stands relies on simple macro expansion. That means that we can't use := in the check block. It makes it hard to understand what happens when check blocks are nested. It makes it hard to understand what happens if a variable declared in the check expression is shadowed by a different variable with the same name inside the check block. Error messages from the compiler expose the macro expansion in ways that make them very hard to understand.

All of these issues could perhaps be addressed. But we would still have a Go statement that defines a condition that is then checked dynamically during execution of the block. There is nothing else in Go that acts that way. We would have to clearly define exactly when the checks occur. We also have to specify what kinds of check expressions are permitted.

There does not seem to be much enthusiasm for this proposal.

-- writing for @golang/proposal-review

[ohir]

@ianlancetaylor
Fair assessment.
I can add that goto seems to scary way too many people.