correct outer check for match on final inner class

[scabug]

=== What steps will reproduce the problem (please be specific and use wikiformatting)? ===

The following code crashes with a NoSuchMethodError:

final class Outer {

  sealed trait Inner

  final case class Inner1(foo: Int) extends Inner

  val inner: Outer#Inner = Inner1(0)

  def bar = inner match {
    case Inner1(i) => i
  }
}

object Crash {
  def main(args: Array[String]) {
    println((new Outer).bar)
  }
}

The crash does not happen when one of these changes is applied:

• Inner is a sealed abstract class instead of a sealed trait
• the type of val inner is not a type projection but just Inner
• the class Inner1 is not marked final

The NoSuchMethodError at runtime:

java.lang.NoSuchMethodError: Outer$$Inner1.Outer$$Inner1$$$$$$outer()LOuter;
	at Outer.bar(finalcase.scala:9)
	at Crash$$.main(finalcase.scala:16)
	at Crash.main(finalcase.scala)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
	at java.lang.reflect.Method.invoke(Method.java:597)
	at scala.tools.nsc.util.ScalaClassLoader$$$$anonfun$$run$$1.apply(ScalaClassLoader.scala:81)
	at scala.tools.nsc.util.ScalaClassLoader$$class.asContext(ScalaClassLoader.scala:24)
	at scala.tools.nsc.util.ScalaClassLoader$$URLClassLoader.asContext(ScalaClassLoader.scala:91)
	at scala.tools.nsc.util.ScalaClassLoader$$class.run(ScalaClassLoader.scala:81)
	at scala.tools.nsc.util.ScalaClassLoader$$URLClassLoader.run(ScalaClassLoader.scala:104)
	at scala.tools.nsc.ObjectRunner$$.run(ObjectRunner.scala:33)
	at scala.tools.nsc.ObjectRunner$$.runAndCatch(ObjectRunner.scala:40)
	at scala.tools.nsc.MainGenericRunner$$.runTarget$$1(MainGenericRunner.scala:61)
	at scala.tools.nsc.MainGenericRunner$$.process(MainGenericRunner.scala:85)
	at scala.tools.nsc.MainGenericRunner$$.main(MainGenericRunner.scala:33)
	at scala.tools.nsc.MainGenericRunner.main(MainGenericRunner.scala)

=== What versions of the following are you using? ===

• Scala: 2.9.0.RC1
• Java: 1.6.0_24
• Operating system: OS X 10.6.7

[scabug]

Imported From: https://issues.scala-lang.org/browse/SI-4440?orig=1
Reporter: Moritz Uhlig (muhlig)

[scabug]

@magarciaEPFL said:
[[syntax trees at end of cleanup]]// Scala source: bt4.scala

package <empty> {
  final class Outer extends java.lang.Object with ScalaObject {
    @volatile protected var bitmap$$0: Int = 0;
    <synthetic> lazy private[this] var Inner1$$module: object Outer$$Inner1 = _;
    <synthetic> <stable> <accessor> lazy def Inner1(): object Outer$$Inner1 = {
      if (Outer.this.bitmap$$0.&(1).==(0))
        {
          Outer.this.synchronized({
            if (Outer.this.bitmap$$0.&(1).==(0))
              {
                Outer.this.Inner1$$module = new object Outer$$Inner1(Outer.this);
                Outer.this.bitmap$$0 = Outer.this.bitmap$$0.|(1);
                ()
              };
            scala.runtime.BoxedUnit.UNIT
          });
          ()
        };
      Outer.this.Inner1$$module
    };
    private[this] val inner: Outer$$Inner = _;
    <stable> <accessor> def inner(): Outer$$Inner = Outer.this.inner;
    def bar(): Int = {
      <synthetic> val temp5: Outer$$Inner = Outer.this.inner();
      if (temp5.$$isInstanceOf[Outer$$Inner1]().&&(temp5.$$asInstanceOf[Outer$$Inner1]().Outer$$Inner1$$$$$$outer().eq(Outer.this)))
        {
          temp5.$$asInstanceOf[Outer$$Inner1]().foo()
        }
      else
        throw new MatchError(temp5)
    };
    def this(): Outer = {
      Outer.super.this();
      Outer.this.inner = new Outer$$Inner1(Outer.this, 0);
      ()
    }
  };
  final object Crash extends java.lang.Object with ScalaObject {
    def main(args: Array[java.lang.String]): Unit = scala.this.Predef.println(scala.Int.box(new Outer().bar()));
    def this(): object Crash = {
      Crash.super.this();
      ()
    }
  };
  sealed abstract trait Outer$$Inner extends java.lang.Object;
  final case class Outer$$Inner1 extends java.lang.Object with Outer$$Inner with ScalaObject with Product with Serializable {
    def productIterator(): Iterator = scala.Product$$class.productIterator(Outer$$Inner1.this);
    @deprecated("use productIterator instead") def productElements(): Iterator = scala.Product$$class.productElements(Outer$$Inner1.this);
    <caseaccessor> <paramaccessor> private[this] val foo: Int = _;
    <stable> <caseaccessor> <accessor> <paramaccessor> def foo(): Int = Outer$$Inner1.this.foo;
    <synthetic> def copy(foo: Int = foo): Outer$$Inner1 = new Outer$$Inner1(Outer$$Inner1.this.$$outer, foo);
    <synthetic> def copy$$default$$1(): Int = Outer$$Inner1.this.foo();
    override def hashCode(): Int = ScalaRunTime.this._hashCode(Outer$$Inner1.this);
    override def toString(): java.lang.String = ScalaRunTime.this._toString(Outer$$Inner1.this);
    override def equals(x$$1: java.lang.Object): Boolean = Outer$$Inner1.this.eq(x$$1).||({
      {
        <synthetic> val temp1: java.lang.Object = x$$1;
        if (temp1.$$isInstanceOf[Outer$$Inner1]())
          {
            <synthetic> val temp2: Outer$$Inner1 = temp1.$$asInstanceOf[Outer$$Inner1]();
            <synthetic> val temp3: Int = temp2.foo();
            val foo$$1: Int = temp3;
            if (Outer$$Inner1.this.gd1$$1(foo$$1))
              {
                x$$1.$$asInstanceOf[Outer$$Inner1]().canEqual(Outer$$Inner1.this)
              }
            else
              {
                false
              }
          }
        else
          {
            false
          }
      }
    });
    override def productPrefix(): java.lang.String = "Inner1";
    override def productArity(): Int = 1;
    override def productElement(x$$1: Int): java.lang.Object = {
      <synthetic> val temp4: Int = x$$1;
      if (temp4.==(0))
        {
          scala.Int.box(Outer$$Inner1.this.foo())
        }
      else
        {
          throw new java.lang.IndexOutOfBoundsException(scala.Int.box(x$$1).toString())
        }
    };
    override def canEqual(x$$1: java.lang.Object): Boolean = x$$1.$$isInstanceOf[Outer$$Inner1]();
    <synthetic> <paramaccessor> private[this] val $$outer: Outer = _;
    final <synthetic> private[this] def gd1$$1(x$$1: Int): Boolean = x$$1.==(Outer$$Inner1.this.foo());
    def this($$outer: Outer, foo: Int): Outer$$Inner1 = {
      Outer$$Inner1.this.foo = foo;
      if ($$outer.eq(null))
        throw new java.lang.NullPointerException()
      else
        Outer$$Inner1.this.$$outer = $$outer;
      Outer$$Inner1.super.this();
      scala.Product$$class./*Product$$class*/$$init$$(Outer$$Inner1.this);
      ()
    }
  };
  final <synthetic> object Outer$$Inner1 extends scala.runtime.AbstractFunction1 with ScalaObject with Serializable {
    final override def toString(): java.lang.String = "Inner1";
    case <synthetic> def unapply(x$$0: Outer$$Inner1): Option = if (x$$0.==(null))
      scala.this.None
    else
      new Some(scala.Int.box(x$$0.foo()));
    case <synthetic> def apply(foo: Int): Outer$$Inner1 = new Outer$$Inner1(Outer$$Inner1.this.$$outer, foo);
    <synthetic> <paramaccessor> private[this] val $$outer: Outer = _;
    case <synthetic> <bridge> def apply(v1: java.lang.Object): java.lang.Object = Outer$$Inner1.this.apply(scala.Int.unbox(v1));
    def this($$outer: Outer): object Outer$$Inner1 = {
      if ($$outer.eq(null))
        throw new java.lang.NullPointerException()
      else
        Outer$$Inner1.this.$$outer = $$outer;
      Outer$$Inner1.super.this();
      ()
    }
  }
}

[scabug]

@lrytz said:
paul, we believe this is a pattern matcher bug

[scabug]

@lexn82 said:
I believe this is related, but does not require a sealed trait or a case class:

object Test {
  def main(args: Array[String]) {
    (new Clazz).matcher()
  }
}

final class Clazz {
  private final class SubClazz {
    def foo() = 1
  }
  
  private val array = new Array[AnyRef](2)
  (0 until 2).foreach(i => array(i) = new SubClazz) 
  
  def matcher() {
    var i = 0; while (i < array.size) { val c = array(i)
      c match {
        case s: SubClazz => println(s)
        case _ => println("no match")
      }
    }
  }
}

[scabug]

@soc said (edited on Apr 20, 2012 5:55:53 PM UTC):
Another snippet with probably the same problem, although tha stacktrace seems to be a bit different:

trait ContextContainer {
  def Contexts: Contexts

  abstract class Contexts extends swing.Publisher {
    trait Context extends swing.Publisher {
      Contexts.this.listenTo(this)
      def fireChanged() {publish(ContextChanged(this))}
    }
    reactions += {case c @ ContextChanged(_) => publish(c)}
  }
  final case class ContextChanged(c: Contexts#Context) extends swing.event.Event
}

object ContextsTest extends App with ContextContainer {
  object Contexts extends Contexts {
    case class NamedContext(name: String) extends Context
    val C1 = NamedContext("C1")
  }
  
  Contexts.C1.fireChanged() //java.lang.NoSuchMethodError: ContextContainer$ContextChanged.ContextContainer$ContextChanged$$$outer()LContextContainer;
}

java.lang.NoSuchMethodError: ContextContainer$ContextChanged.$line37$$read$ContextContainer$ContextChanged$$$outer()LContextContainer;
	at ContextContainer$Contexts$$anonfun$1.isDefinedAt(<console>:18)
	at ContextContainer$Contexts$$anonfun$1.isDefinedAt(<console>:18)
	at scala.swing.Reactions$Impl$$anonfun$apply$1.apply(Reactions.scala:25)
	at scala.swing.Reactions$Impl$$anonfun$apply$1.apply(Reactions.scala:25)
	at scala.collection.LinearSeqOptimized$class.foreach(LinearSeqOptimized.scala:59)
	at scala.collection.immutable.List.foreach(List.scala:77)
	at scala.collection.generic.TraversableForwarder$class.foreach(TraversableForwarder.scala:31)
	at scala.collection.mutable.ListBuffer.foreach(ListBuffer.scala:45)
	at scala.swing.Reactions$Impl.apply(Reactions.scala:25)
	at scala.swing.Reactions$Impl.apply(Reactions.scala:19)
	at scala.Function1$class.apply$mcVL$sp(Function1.scala:39)
	at scala.swing.Reactions.apply$mcVL$sp(Reactions.scala:46)
	at scala.swing.Publisher$$anonfun$publish$1.apply(Publisher.scala:47)
	at scala.swing.Publisher$$anonfun$publish$1.apply(Publisher.scala:47)
	at scala.collection.Iterator$class.foreach(Iterator.scala:697)
	at scala.swing.SingleRefCollection$$anon$4.foreach(Publisher.scala:108)
	at scala.collection.IterableLike$class.foreach(IterableLike.scala:73)
	at scala.swing.RefSet.foreach(Publisher.scala:165)
	at scala.swing.Publisher$class.publish(Publisher.scala:47)
	at ContextsTest$Contexts$NamedContext.publish(<console>:26)
	at ContextContainer$Contexts$Context$class.fireChanged(<console>:16)
	at ContextsTest$Contexts$NamedContext.fireChanged(<console>:26)
	at ContextsTest$delayedInit$body.apply(<console>:30)
	at scala.Function0$class.apply$mcV$sp(Function0.scala:40)
	at scala.runtime.AbstractFunction0.apply$mcV$sp(AbstractFunction0.scala:12)
	at scala.App$$anonfun$main$1.apply(App.scala:61)
	at scala.App$$anonfun$main$1.apply(App.scala:61)
	at scala.collection.LinearSeqOptimized$class.foreach(LinearSeqOptimized.scala:59)
	at scala.collection.immutable.List.foreach(List.scala:77)
	at scala.collection.generic.TraversableForwarder$class.foreach(TraversableForwarder.scala:31)
	at scala.App$class.main(App.scala:61)
	at ContextsTest$.main(<console>:24)
	at .<init>(<console>:35)
	at .<clinit>(<console>)
	at .<init>(<console>:7)
	at .<clinit>(<console>)
	at $print(<console>)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:616)
	at scala.tools.nsc.interpreter.IMain$ReadEvalPrint.call(IMain.scala:775)
	at scala.tools.nsc.interpreter.IMain$Request$$anonfun$16.apply(IMain.scala:1039)
	at scala.tools.nsc.interpreter.Line.scala$tools$nsc$interpreter$Line$$runAndSetState(Line.scala:41)
	at scala.tools.nsc.interpreter.Line$$anonfun$2.apply$mcV$sp(Line.scala:47)
	at scala.tools.nsc.io.package$$anon$2.run(package.scala:22)
	at java.lang.Thread.run(Thread.java:679)

[scabug]

@hubertp said:
Still crashes with the new virt pattern matcher.

[scabug]

@retronym said (edited on Jun 22, 2012 6:35:35 AM UTC):
A pinch of minimization and a sprinkle of analysis:

class Outer {
  final class Inner {
    // Main$$anon$Outer$Inner1$$$outer removed in the constructors phase.
    // because Inner1 is effectively final. (Remove `final` and things work.)
    // See:
    //   $outer method elimination: https://github.com/scala/scala/blob/3631f1d/src/compiler/scala/tools/nsc/transform/Constructors.scala#L226
    //   outer test in pattern matcher: https://github.com/scala/scala/blob/8284486/src/compiler/scala/tools/nsc/typechecker/PatternMatching.scala#L937
    //   Supposed elimintion of the outer test: https://github.com/scala/scala/blob/8284486/src/compiler/scala/tools/nsc/transform/ExplicitOuter.scala#L525
  }

  def bar2 = (new Inner: Any) match {
    case i: Inner =>
  }

}

println((new Outer).bar2)
// java.lang.NoSuchMethodError: Main$$anon$1$Outer$Inner1.Main$$anon$Outer$Inner1$$$outer()LMain$$anon$1$Outer;
//               <synthetic> <stable> def Main$$anon$Outer$Inner1$$$outer(): Outer = Inner1.this.$outer

[scabug]

@adriaanm said (edited on Jul 19, 2012 10:17:39 AM UTC):
we used to drop outer accessors from nested final classes if the outer-accessor wasn't accessed anywhere
however, for the sake of separate compilation and the outer checks inserted by the pattern matcher,
we must assume they might be accessed anywhere

Why single out final classes in the first place?

[scabug]

@adriaanm said:
scala/scala#951

[scabug]

@adriaanm said:
workaround in scala/scala#976

(#951 was rejected because a proper fix would increase memory footprint)

[scabug]

@adriaanm said:
In 2.11 we'll do the right thing: the final modifier is not a sufficient condition for omitting the outer pointer. If we can come up with some kind of optimization (hard to do if you can't assume a closed world), we'll optimize memory usage there.

In the mean time, if you don't want an outer pointer, don't nest your class.

[scabug]

@retronym said (edited on Mar 2, 2013 8:11:25 AM UTC):
Perhaps a class annotation {{@noouter}}?

[scabug]

@Blaisorblade said:
I and Miguel Garcia discussed this bug again on a pull request. Since the current situation was not obvious, here's a link to the post where I explain the current problem:

scala/scala#2634 (comment)
Code and results are at https://gist.github.com/Blaisorblade/5745026.

I might post here an updated summary in my copious free time, but until then, I hope that link helps.

[scabug]

@Blaisorblade said:
The optimization should still be valid when any of the following conditions holds:

• the Inner class is only visible within the compilation unit, say because Inner (or Outer) is private (but not private[x]), and the compilation unit doesn't use @outer
• the inner class is anonymous (so it can't be used in a pattern match - right?)
• the inner class is annotated with @noouter (Jason Zaugg's proposal).

Here's also a link to the email conversation, for completeness (not sure it contains anything more):
https://groups.google.com/d/msg/scala-internals/jQiRINTvhZQ/GrxAJjccdmUJ

[scabug]

@dcsobral said:
So, 2.11 is out and the compiler isn't doing "the right thing", so the bug persists. I assume no annotations came forth either?

[scabug]

Malcolm Greaves (malcolmgreaves) said (edited on Nov 24, 2015 12:13:45 AM UTC):
Any update on this bug? I just ran into a problem that's related to this one (I think). In my code, if I do the following, I do not get any errors:

trait BinaryTree {
  trait Payload  
  sealed trait Node
  case class Parent(left: Option[Node], right: Option[Node]) extends Node
  case class Leaf(data: Payload) extends Node
}

However, if I add final to the case classes, then I get the following compiler error:

The outer reference in this type test cannot be checked at run time.
final case class Leaf(d: Payload) extends Node

(Note: I get the same error message for both Leaf and Parent.)

My current solution is to use sealed instead of final. Unlike final, using sealed actually works! Specifically, the following compiles:

trait BinaryTree {
  trait Payload  
  sealed trait Node
  sealed case class Parent(left: Option[Node], right: Option[Node]) extends Node
  sealed case class Leaf(data: Payload) extends Node
}

Any thoughts on this? I'd like to express the fact that this ADT definition cannot be changed. Including the definitions for Leaf and Parent. Since I'm controlling the definition of the ADT here, I think that using sealed suffices. I'd welcome any other ideas though!

Also, I hope that this can jump-start this stale ticket.

[scabug]

@DarkDimius said:
This bug is non-existent in Dotty.
See also discussion in scala/scala3#2156 for discusion of outer-checking and pattern matching

[xeno-by]

Here's another instance of this bug along with another workaround: scalameta/scalameta#900 (comment).

[johnynek]

It seems something like this bug can also be hit when you have aliases then try to pattern match on the val alias to the type:

zoolander-util/com/stripe/zoolander/util/scalding/Chunky.scala:33: warning: The outer reference in this type test cannot be checked at run time.
          case (TypedPipe.Fork(t), rec) => rec(t).flatMap(_.forceToDiskExecution)
                              ^
zoolander-util/com/stripe/zoolander/util/scalding/Chunky.scala:34: warning: The outer reference in this type test cannot be checked at run time.
          case (TypedPipe.ForceToDisk(t), rec) => rec(t).flatMap(_.forceToDiskExecution)

For example:

https://github.com/twitter/scalding/blob/181dc646f7f6ad1ed9d61b012b3bbbdaa2dbaf5f/scalding-core/src/main/scala/com/twitter/scalding/typed/TypedPipe.scala

in TypedPipe above we have a number of final case classes inside the object. We reexport com.twitter.scalding.typed.TypedPipe as com.twitter.scalding.TypedPipe with the following in the package object:

https://github.com/twitter/scalding/blob/181dc646f7f6ad1ed9d61b012b3bbbdaa2dbaf5f/scalding-core/src/main/scala/com/twitter/package.scala#L26

val TypedPipe = com.twitter.scalding.typed.TypedPipe
type TypedPipe[+T] = com.twitter.scalding.typed.TypedPipe[T]

using the un-aliased types seems to side-step the the issue.

[xeno-by]

@johnynek This is how we work around the problem in Scalameta. Annoying but effective: https://github.com/scalameta/scalameta/blob/v3.2.0/langmeta/langmeta/shared/src/main/scala/org/langmeta/inputs/Api.scala.

[johnynek]

spores use macros (which I thought are kind of on the chopping block), and have not been published for 2.12: https://search.maven.org/#search%7Cga%7C1%7Cch.epfl.scala.spores

Is there any chance to get a supported version of spores into scala? It would be great for GC sensitive apps (controlling what you close over) as well as serialization cases (spark, scalding, etc...)

[Blaisorblade]

@johnynek FWIW for spores 2.12, see scalacenter/spores#21 (that's currently the master repo), but I'll leave your other questions to others.

[SethTisue]

@dwijnand did this progress in #12398 ?

[dwijnand]

No. The original issue was a duplicate of this, but I repurposed it for the specific case where the outer check isn't required (with Singleton):

@dwijnand changed the title Type prefix outer reference is not checked in type tests Path-dependent type on singleton value emits spurious outer test warnings 4 days ago

This case is unfixable in 2.13, because we can't change the binary format of nested final inner classes mid-2.13.

[SethTisue]

meanwhile over in Scala 3, scala/scala3#13096

[He-Pin]

[som-snytt]

Since scala/scala#951 is now in the public domain, we can merge it under -Xsource:3-X for dotty compatibility.