Experimental annotation behaviour redesign

[prolativ]

We need to rethink how @experimental should work to:

• fix the problems with bootstrapping the compiler (until then 3.0.0 has to be used as the reference compiler)
• eliminate strange behaviours and unexpected limitations (e.g. Experimental trait cannot be instantiated anonymously #13091)
• take into account the expectations of the community regarding this feature

Related discussion: https://contributors.scala-lang.org/t/experimental-annotation-behaviour-redesign/5187

[odersky]

I think @experimental should behave like erased. To refer to an identifier that is annotated @experimental you need to be either in a definition marked @experimental or you need to run a snapshot compiler. That's it. I don't see the need to check more than that.

That would solve the bootstrap problem since tests of experimental features can be marked experimental themselves.

[prolativ]

As mentioned here https://contributors.scala-lang.org/t/experimental-annotation-behaviour-redesign/5187/7?u=prolativ
it seems to me that we should distinguish here between experimental features of the compiler and experimental parts of APIs of libraries, as in the latter case the requirement of using a snapshot compiler doesn't seem to make sense. Unless @experimental was not designed with this use case in mind at all and it's always supposed to mean "directly or transitively using experimental compiler features".

[prolativ]

@odersky I believe the bootstrapping problem would be solved in this case only if we allowed compiling experimental definitions with a non-snapshot compiler but not using experimental features.

[smarter]

in the latter case the requirement of using a snapshot compiler doesn't seem to make sense

We require a snapshot compiler to use experimental APIs in the standard library because we want to be able to change or remove these APIs at anytime without breaking binary and tasty compatibility for non-experimental code. Forcing the use of a snapshot compiler for experimental APIs ensure that even if someone publishes a library that uses these APIs, a project that uses a release compiler cannot accidentally depend on them, because the tasty version will be different (we always have a non-zero ExperimentalVersion for nightly builds).

[prolativ]

What I meant is whether when someone designs the API of their library and they introduce e.g. some method whose signature is quite likely to change in the future, is it OK to annotate such method with @experimental even if it doesn't use any experimental features of the compiler? Then @experimental be something kind of dual to @deprecated.
Or is @experimental ONLY for using/declaring experimental features of the compiler?

[smarter]

So far we've only been considering @experimental for compiler usecases, it could also be useful for external libraries but that seemed like something that should be discussed with library authors and iterated on first before becoming official which is why using @experimental itself was made experimental.

[dwijnand]

Just realised: even if we did rework the checks so they only checked usages of experimental APIs, rather than the definition of experimental APIs, the fact that @experimental is experimental means those definitions (e.g. in scala3-library) need a snapshot compiler because they're using @experimental.

Perhaps we really should do like Rust and give the compiler a backdoor so a non-snapshot compiler can bootstrap...

[prolativ]

So maybe we could allow using experimental features inside definitions which are experimental themselves no matter what compiler we use but usage of experimental features outside of experimental context would require using a snapshot compiler?

[odersky]

@dwijnand Maybe @experimental should not be experimental?

@prolativ

So maybe we could allow using experimental features inside definitions which are experimental themselves no matter what compiler we use but usage of experimental features outside of experimental context would require using a snapshot compiler?

That's exactly what I proposed.

[odersky]

For language imports: You can treat language.experimental as @experimental. That is, you can import experimental features with a snapshot compiler or inside an @experimental definition. Maybe you want to customize the error message if the condition is not satisfied.

[prolativ]

@odersky

That's exactly what I proposed.
Oh, you're right. For some reason I didn't get the emphasis on the fact that inside an experimental block you wouldn't need to be using a snapshot compiler.

But even if we mark tests of experimental features as @experimental could that potentially cause some problems for the test framework to run them?

Another question is whether we would like to have some more fine-grained control over which experimental features are used in a particular scope. Then the @experimental annotation would probably take some parameters indicating the used experimental features and for the imports one would also need to mention them explicitly like import language.experimental.{foo, bar} or use a wildcard import import language.experimental.*

[smarter]

could that potentially cause some problems for the test framework to run them?

there's an escape-hatch for the tests: https://github.com/lampepfl/dotty/blob/3dd91d1b7810ce1dd6a453314f52976e2e545a51/compiler/src/dotty/tools/dotc/config/Feature.scala#L100

the @experimental annotation would probably take some parameters indicating the used experimental features

but you can have experimental APIs which are independent of any particular feature, the annotation is used whenever an API isn't stable yet.

[prolativ]

What about end users who would like to write experimental definitions and still be able to test them when compiling with a non-snapshot compiler? Then each test framework would need its own escape hatch hardcoded in the compiler?

[smarter]

That's one of the reason we kept @experimental for compiler-only usages so far :)

[prolativ]

So that would mean that there should be no definitions marked as @experimental outside of the compiler codebase? From your comments above I understood that it would be OK to mark e.g. a method defined in an external library's API as experimental to indicate that it uses experimental features of the compiler (rather than to indicate its uncertain status in the API). But that seems not to be the case, from what you're saying now, unless I missed something

[odersky]

That's one of the reason we kept @experimental for compiler-only usages so far :)

I think @prolativ has a point. Anything that needs an escape hatch for compiler use is a cop out. But with the proposed rules you can mark a test as @experimental and that makes it compile with a stable compiler. I don't see how it would cause problems for test runners.

[dwijnand]

As a hypothesis: what's stopping me and my mates creating a little ecosystem of libraries, all using experimental features and APIs, all compiled with the stable compiler, published to Maven Central, and running in production, because we all added @experimental on our library packages (assuming that works - on all our classes, if not)?

[dwijnand]

I guess that's no worst than adding a RUSTC_BOOSTRAP=1 backdoor to the compiler to have the same effect.

[prolativ]

Even for the compiler itself we don't run all the tests with Vulpix, there are also some unit tests using junit, right? Actually I've never dived into how it works under the hood but I would guess it uses reflection to find methods annotated with @Test and then calls them, also via reflection, so it would somehow bypass the restriction on calling an experimental method from nonexperimental code when using a non-snapshot compiler anyway. But can we assume all test frameworks are reflection based?

[odersky]

As a hypothesis: what's stopping me and my mates creating a little ecosystem of libraries, all using experimental features and APIs, all compiled with the stable compiler, published to Maven Central, and running in production, because we all added @experimental on our library packages (assuming that works - on all our classes, if not)?

(I think you'd need to add it to classes and objects, not packages)

Nothing would stop you. But would that be bad?

Experimental additions are not evil, they are the (as yet unstable) future of Scala. As long as people are aware what they are it's fine. And having to mark all your classes and objects as @experimental should be blindingly obvious.

[nicolasstucki]

Note that there were 2 reasons why we made @experimantal experimental

• To not break compatibility with 3.0.0-RC1
• To ensure that anyone using @experimental is using it from an unstable version. We assume that a project compiled with a stable versions should not be able to depend on projects compiled with unstable versions.

[nicolasstucki]

The current issue with the compiler bootstrapping is that our assumeExperimentalIn escape hatch is not good enough (#13112 (comment)). We created it this way to only allow the use of experimental in our code base, but it only covers tests and not compiling the compiler itself.

[nicolasstucki]

What we need is to be able to use experimental features in RC versions. This way we can use the latest RC (i.e. the released version) as reference compiler. Or we need to have a way to enable experimental on that version of the compiler.

[nicolasstucki]

Here is the informal spec of @experimenal proposed in #13112 (comment) and some examples. @odersky did I miss something?

Experimental definitions

The @experimental annotation allows the definition of an API that is not guaranteed backward binary or source compatibility.
This annotation can be placed on term or type definitions.

References to experimental definitions

Experimental definitions can only be referenced in an experimental scope. Experimental scopes are defined as follows:

1. The RHS of an experimental def, val, var, given or type is an experimental scope.
2. The signatures of an experimental def, val, var, given and type, or constructors of class and trait are experimental scopes.
3. The extension clause of an experimental class, trait, object are experimental scopes.
4. Members of an experimental class, trait or object are in experimental scopes.
5. Annotations of an experimental definition are in experimental scopes.
6. Any code compiled using a Nightly or Snapshot version of the compiler is considered to be in an experimental scope.

In any other situation, a reference to an experimental definition will cause a compilation error.

Question: Should we disallow @experimental def main(...)?

Experimental inheritance

All subclasses of an experimental class or trait must be marked as @experimental even if they are in an experimental scope.
Anonymous classes and SAMs of experimental classes are considered experimental.

We require explicit annotations to make sure we do not have completion or cycles issues with nested classes. This restriction could be relaxed in the future.

Experimental overriding

For an overriding member M and overridden member O, if O is non-experimental then M must be non-experimental.

This makes sure that we cannot have accidental binary incompatibilities such as the following change.

class A:
  def f: Any = 1
class B extends A:
-  @experimental def f: Int = 2

Test frameworks

Tests can be defined as experimental. Tests frameworks can execute tests using reflection even if they are in an experimental class, object or method.

Test that touch experimental APIs can be written as follows

import scala.annotation.experimental

@experimental def x = 2

class MyTests {
  /*@Test*/ def test1 = x // error
  @experimental /*@Test*/ def test2 = x
}

@experimental
class MyExperimentalTests {
  /*@Test*/ def test1 = x
  /*@Test*/ def test2 = x
}

Examples / Tests cases

For rule (1.)

The RHS of an experimental def, val, var, given or type are experimental scopes.

import scala.annotation.experimental

@experimental
def x = ()

def d1 = x // error: value x is marked @experimental and therefore ...
@experimental def d2 = x

val v1 = x // error: value x is marked @experimental and therefore ...
@experimental val v2 = x

var vr1 = x // error: value x is marked @experimental and therefore ...
@experimental var vr2 = x

lazy val lv1 = x // error: value x is marked @experimental and therefore ...
@experimental lazy val lv2 = x

import scala.annotation.experimental

@experimental
val x = ()

@experimental
def f() = ()

@experimental
object X:
  def fx() = 1

def test1: Unit =
  f() // error: def f is marked @experimental and therefore ...
  x // error: value x is marked @experimental and therefore ...
  X.fx() // error: object X is marked @experimental and therefore ...
  import X.fx
  fx() // error: object X is marked @experimental and therefore ...

@experimental
def test2: Unit =
  // references to f, x and X are ok because `test2` is experimental
  f()
  x
  X.fx()
  import X.fx
  fx()

import scala.annotation.experimental

@experimental type E

type A = E // error
@experimental type B = E

import scala.annotation.experimental

@experimental class A
@experimental type X
@experimental type Y = Int
@experimental opaque type Z = Int

def test: Unit =
  new A // error: class A is marked @experimental and therefore ...
  val i0: A = ??? // error: class A is marked @experimental and therefore ...
  val i1: X = ??? // error: type X is marked @experimental and therefore ...
  val i2: Y = ??? // error: type Y is marked @experimental and therefore ...
  val i2: Z = ??? // error: type Y is marked @experimental and therefore ...
  ()

@experimental
trait ExpSAM {
  def foo(x: Int): Int
}
def bar(f: ExpSAM): Unit = {} // error: error form rule 2

def test: Unit =
  bar(x => x) // error: reference to experimental SAM
  ()

For rule (2.)

The signatures of an experimental def, val, var, given and type, or constructors of class and trait are experimental scopes.

import scala.annotation.experimental

@experimental def x = 2
@experimental class A
@experimental type X
@experimental type Y = Int
@experimental opaque type Z = Int

def test1(
  p1: A, // error: class A is marked @experimental and therefore ...
  p2: List[A], // error: class A is marked @experimental and therefore ...
  p3: X, // error: type X is marked @experimental and therefore ...
  p4: Y, // error: type Y is marked @experimental and therefore ...
  p5: Z, // error: type Z is marked @experimental and therefore ...
  p6: Any = x // error: def x is marked @experimental and therefore ...
): A = ??? // error: class A is marked @experimental and therefore ...

@experimental def test2(
  p1: A,
  p2: List[A],
  p3: X,
  p4: Y,
  p5: Z,
  p6: Any = x
): A = ???

class Test1(
  p1: A, // error
  p2: List[A], // error
  p3: X, // error
  p4: Y, // error
  p5: Z, // error
  p6: Any = x // error
) {}

@experimental class Test2(
  p1: A,
  p2: List[A],
  p3: X,
  p4: Y,
  p5: Z,
  p6: Any = x
) {}

trait Test1(
  p1: A, // error
  p2: List[A], // error
  p3: X, // error
  p4: Y, // error
  p5: Z, // error
  p6: Any = x // error
) {}

@experimental trait Test2(
  p1: A,
  p2: List[A],
  p3: X,
  p4: Y,
  p5: Z,
  p6: Any = x
) {}

For rule (3.)

The extension clause of an experimental class, trait, object are experimental scopes.

import scala.annotation.experimental

@experimental def x = 2

@experimental class A1(x: Any)
class A2(x: Any)


@experimental class B1 extends A1(1)
class B2 extends A1(1) // error: class A1 is marked @experimental and therefore marked @experimental and therefore ...

@experimental class C1 extends A2(x)
class C2 extends A2(x) // error def x is marked @experimental and therefore

For rule (4.)

Members of an experimental class, trait or object are in experimental scopes.

import scala.annotation.experimental

@experimental def x = 2

@experimental class A {
  def f = x // ok because A is experimental
}

@experimental class B {
  def f = x // ok because A is experimental
}

@experimental object C {
  def f = x // ok because A is experimental
}

@experimental class D {
  def f = {
    object B {
      x // ok because A is experimental
    }
  }
}

For rule (5.)

Annotations of an experimental definition are in experimental scopes.

import scala.annotation.experimental

@experimental class myExperimentalAnnot extends scala.annotation.Annotation

@myExperimentalAnnot // error
def test: Unit = ()

@experimental
@myExperimentalAnnot
def test: Unit = ()

For rule (6.)

Any code compiled using a Nightly or Snapshot version of the compiler is considered to be in an experimental scope.

All tests are executed in snapshot mode. We can use the -Yno-experimental to disable it and run as a proper release.

[prolativ]

@nicolasstucki should point 2 also cover signatures of types? E.g.

import scala.annotation.experimental
@experimental trait Foo
type Bar[F <: Foo] = Option[F] // error
@experimental type Baz[F <: Foo] = Option[F] // ok

[dwijnand]

• Test frameworks can execute experimental tests

Does this mean tests don't need to be marked experimental to reference experimental APIs? How is that achieved?

• Any code compiled using a Nightly or Snapshot version of the compiler is considered experimental.

Seems unnecessary. Is it necessary? All snapshots have experimental tasty versions, so there's already some guarding. What does this point do and/or look to solve?

The rest looks on point to me. 👏🏼

[prolativ]

The examples for point 6 seem to show that you would need to to explicitly mark a test case or a test suite experimental to test experimental features. But I'm still not sure how this could be done in a generic way that would work for different test frameworks.
And are we dropping the idea of making all RCs experimental as well?

[nicolasstucki]

Test frameworks can execute experimental tests

Does this mean tests don't need to be marked experimental to reference experimental APIs? How is that achieved?

Tests that reference experimental definitions need to be marked as experimental. They could be marked on the def or class level.

The test framework will call these test definitions reflectively and hence will do not need to reference the experimental definitions explicitly.

Any code compiled using a Nightly or Snapshot version of the compiler is considered experimental.

Seems unnecessary. Is it necessary? All snapshots have experimental tasty versions, so there's already some guarding. What does this point do and/or look to solve?

This rule is intended for users that do not necessarily know about the tasty internals. I would say that the tasty version is part of the implementation of the rule.

[nicolasstucki]

The examples for point 6 seem to show that you would need to to explicitly mark a test case or a test suite experimental to test experimental features. But I'm still not sure how this could be done in a generic way that would work for different test frameworks.

The test framework will call these test definitions reflectively and hence will do not need to reference the experimental definitions explicitly. As long as the experimental code is contained in an experimental definition, the tests framework can see that definition and call it reflectively.

[prolativ]

I think rule 1 should also mention given

[nicolasstucki]

I think rule 1 should also mention given

Updated and added other missing definitions

[nicolasstucki]

I updated #13112 (comment). @odersky and @smarter could you have a quick look to see if all seems in order. I will move the contents of that comment into .md document when I open the PR.

[smarter]

LGTM (I made a few small edits)

The RHS of an experimental def, val, var, given or type are experimental scopes.

This should be "is an experimental scope" or the beginning of the sentence should be changed (same in a few other rules)

Question: Should we disallow @experimental def main(...)?

I think it's not too big a deal to leave that escape hatch in since at the point where you're adding@experimental everywhere in your program it should be obvious you're doing something weird :)

[nicolasstucki]

This was closed by #13305