Mixin fields with trait setters shouldn't be JVM final #7028

retronym · 2018-08-09T01:37:55Z

Following on from #6425, also a release fence to the constructor of classes that mixin traits with val fields, and changed the backing JVM field to be non-final. This lets us comply with the JVM spec ("assignment to a final field must lexically scoped in the class constructor"), which is enforced under Classfile format 53

Fixes scala/scala-dev#408

Before:

defined class C

scala> :javap -private -c C Compiled from "<console>" public class
$line3.$read$$iw$$iw$C implements $line3.$read$$iw$$iw$T {
 private final int foo;

  public int foo();
   Code:
      0: aload_0
      1: getfield      #21                 // Field foo:I
      4: ireturn

  public void $line3$$read$$iw$$iw$T$_setter_$foo_$eq(int);
   Code:
      0: aload_0
      1: iload_1
      2: putfield      #21                 // Field foo:I
      5: return

  public $line3.$read$$iw$$iw$C();
   Code:
      0: aload_0
      1: invokespecial #30                 // Method
java/lang/Object."<init>":()V
      4: aload_0
      5: invokestatic  #34                 // InterfaceMethod
$line3/$read$$iw$$iw$T.$init$:(L$line3/$read$$iw$$iw$T;)V
      8: return
}

The assignment to the final field foo has always contravened the JVM
spec, and this rule is enforced for any classfiles of format 53 and higher.

After this patch:

defined class C

scala> :javap -private -c C Compiled from "<console>" public class
$line3.$read$$iw$$iw$C implements $line3.$read$$iw$$iw$T {
 private int foo;

  public int foo();
   Code:
      0: aload_0
      1: getfield      #21                 // Field foo:I
      4: ireturn

  public void $line3$$read$$iw$$iw$T$_setter_$foo_$eq(int);
   Code:
      0: aload_0
      1: iload_1
      2: putfield      #21                 // Field foo:I
      5: return

  public $line3.$read$$iw$$iw$C();
   Code:
      0: aload_0
      1: invokespecial #30                 // Method
java/lang/Object."<init>":()V
      4: aload_0
      5: invokestatic  #34                 // InterfaceMethod
$line3/$read$$iw$$iw$T.$init$:(L$line3/$read$$iw$$iw$T;)V
      8: getstatic     #40                 // Field
scala/runtime/ScalaRunTime$.MODULE$:Lscala/runtime/ScalaRunTime$;
     11: invokevirtual #43                 // Method
scala/runtime/ScalaRunTime$.releaseFence:()V
     14: return
}

adriaanm

A few questions, mostly because I still haven't fully internalized the memory model :-)

adriaanm · 2018-08-09T07:57:10Z

src/compiler/scala/tools/nsc/transform/Fields.scala

-            mkAccessor(setter)(Assign(Select(This(clazz), fieldSel), castHack(Ident(setter.firstParam), fieldSel.info)))
-          }
+          case fieldSel =>
+            if (!fieldSel.hasFlag(MUTABLE)) {


This could use a comment: If the field is mutable, it won't be final, so we can write to it in a setter. If it's not, we still need to initialize it, and make sure it's safely published. Since initialization is performed (lexically) outside of the constructor (in the trait setter), we have to make the field mutable starting with classfile format 53 (it was never allowed, but the verifier enforces this now).

updated with this comment

adriaanm · 2018-08-09T08:01:33Z

src/library/scala/collection/immutable/List.scala

@@ -560,6 +567,7 @@ sealed abstract class List[+A]

 final case class :: [+A](override val head: A, private[scala] var next: List[A @uncheckedVariance]) // sound because `next` is used only locally
  extends List[A] {
+  ScalaRunTime.releaseFence()


Let's add a comment to the class that every time next is mutated and the resulting :: instance is published, you should first call releaseFence. Maybe, maybe, we can actually add a method publishNext or something to make this more clear.

adriaanm · 2018-08-09T08:02:50Z

src/library/scala/collection/immutable/Vector.scala

@@ -215,6 +216,7 @@ final class Vector[+A] private[immutable] (private[collection] val startIndex: I
    s.dirty = dirty
    s.gotoPosWritable(focus, idx, focus ^ idx)  // if dirty commit changes; go to new pos and prepare for writing
    s.display0(idx & 31) = elem.asInstanceOf[AnyRef]
+    ScalaRunTime.releaseFence()


is this to publish the mutation of dirty? Should we also do this in the constructor?

adriaanm · 2018-08-09T08:04:40Z

src/library/scala/collection/mutable/ListBuffer.scala

@@ -61,6 +62,7 @@ class ListBuffer[A]
  // Avoids copying where possible.
  override def toList: List[A] = {
    aliased = nonEmpty
+    ScalaRunTime.releaseFence()


Why no fence for the other methods that mutate? I see they're private, but they are called from public methods, I assume?

SethTisue · 2018-08-15T16:03:47Z

@retronym should we merge this for M5, or hold it for RC1?

…hable

Before: ``` scala> trait T { val foo = 24 }; class C extends T defined trait T defined class C scala> :javap -private -c C Compiled from "<console>" public class $line3.$read$$iw$$iw$C implements $line3.$read$$iw$$iw$T { private final int foo; public int foo(); Code: 0: aload_0 1: getfield #21 // Field foo:I 4: ireturn public void $line3$$read$$iw$$iw$T$_setter_$foo_$eq(int); Code: 0: aload_0 1: iload_1 2: putfield #21 // Field foo:I 5: return public $line3.$read$$iw$$iw$C(); Code: 0: aload_0 1: invokespecial #30 // Method java/lang/Object."<init>":()V 4: aload_0 5: invokestatic #34 // InterfaceMethod $line3/$read$$iw$$iw$T.$init$:(L$line3/$read$$iw$$iw$T;)V 8: return } ``` The assignment to the final field `foo` has always contravened the JVM spec, and this rule is enforced for any classfiles of format 53 and higher. After this patch: ``` scala> trait T { val foo = 24 }; class C extends T defined trait T defined class C scala> :javap -private -c C Compiled from "<console>" public class $line3.$read$$iw$$iw$C implements $line3.$read$$iw$$iw$T { private int foo; public int foo(); Code: 0: aload_0 1: getfield #21 // Field foo:I 4: ireturn public void $line3$$read$$iw$$iw$T$_setter_$foo_$eq(int); Code: 0: aload_0 1: iload_1 2: putfield #21 // Field foo:I 5: return public $line3.$read$$iw$$iw$C(); Code: 0: aload_0 1: invokespecial #30 // Method java/lang/Object."<init>":()V 4: aload_0 5: invokestatic #34 // InterfaceMethod $line3/$read$$iw$$iw$T.$init$:(L$line3/$read$$iw$$iw$T;)V 8: getstatic #40 // Field scala/runtime/ScalaRunTime$.MODULE$:Lscala/runtime/ScalaRunTime$; 11: invokevirtual #43 // Method scala/runtime/ScalaRunTime$.releaseFence:()V 14: return } ```

retronym · 2018-08-16T04:39:56Z

Rebased, this still shares the preliminary commits with #6425.

I think we should get this into M5.

This is a forward port of relevant parts of the upstream PR scala/scala#7028

scala-jenkins added this to the 2.13.0-RC1 milestone Aug 9, 2018

retronym force-pushed the topic/trait-setter-fence branch from 74122e1 to d8c0dad Compare August 9, 2018 01:48

adriaanm reviewed Aug 9, 2018

View reviewed changes

adriaanm modified the milestones: 2.13.0-RC1, 2.13.0-M5 Aug 9, 2018

SethTisue assigned retronym Aug 10, 2018

retronym mentioned this pull request Aug 14, 2018

Make MODULE$ Final Again? scala/scala-dev#537

Closed

SethTisue added the prio:blocker release blocker (used only by core team, only near release time) label Aug 15, 2018

retronym added 2 commits August 15, 2018 11:30

First attempt to write a jcstress test show List is not safely publis…

3783fe8

…hable

Expose VarHandle.releaseFence (or the JDK8 equivalent) in ScalaRuntime

00d9e14

retronym force-pushed the topic/trait-setter-fence branch from d8c0dad to cebfe1a Compare August 16, 2018 04:36

retronym force-pushed the topic/trait-setter-fence branch from cebfe1a to f9ecece Compare August 16, 2018 04:38

retronym mentioned this pull request Aug 16, 2018

Offer stronger guarantees under the Java Memory Model for List & Vector (using releaseFence) #6425

Merged

lrytz approved these changes Aug 16, 2018

View reviewed changes

SethTisue added the release-notes worth highlighting in next release notes label Aug 16, 2018

SethTisue merged commit 69b307a into scala:2.13.x Aug 16, 2018

SethTisue added release-notes worth highlighting in next release notes and removed release-notes worth highlighting in next release notes labels Aug 22, 2018

smarter mentioned this pull request Jul 23, 2020

releaseFence needs to be added at the end of constructors of classes that contain mixin fields scala/scala3#9424

Closed

sjrd added a commit to dotty-staging/dotty that referenced this pull request Nov 25, 2020

Fix scala#9424: Add releaseFence() call when mixing in a trait val.

ef6aa52

This is a forward port of relevant parts of the upstream PR scala/scala#7028

sjrd added a commit to dotty-staging/dotty that referenced this pull request Nov 25, 2020

Fix scala#9424: Add releaseFence() call when mixing in a trait val.

a335cd8

This is a forward port of relevant parts of the upstream PR scala/scala#7028

sjrd mentioned this pull request Nov 25, 2020

Fix #9424: Add releaseFence() call when mixing in a trait val. scala/scala3#10493

Merged

sjrd added a commit to dotty-staging/dotty that referenced this pull request Nov 26, 2020

Fix scala#9424: Add releaseFence() call when mixing in a trait val.

f256aa8

This is a forward port of relevant parts of the upstream PR scala/scala#7028

This was referenced Jan 31, 2022

(2.13.4 -target:11) java.lang.IllegalAccessError: Update to non-static final field ... attempted from a different method than the initializer method <init> scala/bug#12340

Closed

Make fields for locals captured in traits non-final #9915

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Mixin fields with trait setters shouldn't be JVM final #7028

Mixin fields with trait setters shouldn't be JVM final #7028

retronym commented Aug 9, 2018 •

edited

Loading

adriaanm left a comment

adriaanm Aug 9, 2018

retronym Aug 16, 2018

adriaanm Aug 9, 2018

adriaanm Aug 9, 2018

adriaanm Aug 9, 2018

SethTisue commented Aug 15, 2018

retronym commented Aug 16, 2018 •

edited

Loading

Mixin fields with trait setters shouldn't be JVM final #7028

Mixin fields with trait setters shouldn't be JVM final #7028

Conversation

retronym commented Aug 9, 2018 • edited Loading

adriaanm left a comment

Choose a reason for hiding this comment

adriaanm Aug 9, 2018

Choose a reason for hiding this comment

retronym Aug 16, 2018

Choose a reason for hiding this comment

adriaanm Aug 9, 2018

Choose a reason for hiding this comment

adriaanm Aug 9, 2018

Choose a reason for hiding this comment

adriaanm Aug 9, 2018

Choose a reason for hiding this comment

SethTisue commented Aug 15, 2018

retronym commented Aug 16, 2018 • edited Loading

retronym commented Aug 9, 2018 •

edited

Loading

retronym commented Aug 16, 2018 •

edited

Loading