SI-3452 Correct Java generic signatures for mixins, static forwarders

[Parts of this patch and some of the commentary are from @paulp]

This took me so long to figure out I can't even tell you. Partly because
there were two different bugs, one which only arose for trait forwarders
and one for mirror class forwarders, and every time I'd make one set
of tests work another set would start failing. The runtime failures
associated with these bugs were fairly well hidden because you usually
have to go through java to encounter them: scala doesn't pay that much
attention to generic signatures, so they can be wrong and scala might still
generate correct code. But java is not so lucky.

Bug #1)

During mixin composition, classes which extend traits receive forwarders
to the implementations. An attempt was made to give these the correct
info (in method "cloneBeforeErasure") but it was prone to giving
the wrong answer, because: the key attribute which the forwarder
must capture is what the underlying method will erase to *where the
implementation is*, not how it appears to the class which contains it.
That means the signature of the forwarder must be no more precise than
the signature of the inherited implementation unless additional measures
will be taken.

This subtle difference will put on an unsubtle show for you in test
run/t3452.scala.

    trait C[T]
    trait Search[M] { def search(input: M): C[Int] = null }
    object StringSearch extends Search[String] { }
    StringSearch.search("test");  // java
    // java.lang.NoSuchMethodError: StringSearch.search(Ljava/lang/String;)LC;

The principled thing to do here would be to create a pair of
methods in the host class: a mixin forwarder with the erased
signature `(String)C[Int]`, and a bridge method with the same
erased signature as the trait interface facet.

But, this turns out to be pretty hard to retrofit onto the
current setup of Mixin and Erasure, mostly due to the fact
that mixin happens after erasure which has already taken
care of bridging.

For a future, release, we should try to move all bridging
after mixin, and pursue this approach. But for now, what can
we do about `LinkageError`s for Java clients?

This commit simply checks if the pre-erasure method signature
that we generate for the trait forward erases identically to
that of the interface method. If so, we can be precise. If not,
we emit the erased signature as the generic signature.

Bug #2) The same principle is at work, at a different location.
During genjvm, objects without declared companion classes
are given static forwarders in the corresponding class, e.g.

    object Foo { def bar = 5 }

which creates these classes (taking minor liberties):

    class Foo$ { static val MODULE$ = new Foo$ ; def bar = 5 }
    class Foo  { static def bar = Foo$.MODULE$.bar }

In generating these, genjvm circumvented the usual process whereby one
creates a symbol and gives it an info, preferring to target the bytecode
directly. However generic signatures are calculated from symbol info
(in this case reusing the info from the module class.) Lacking even the
attempt which was being made in mixin to "clone before erasure", we
would have runtime failures of this kind:

    abstract class Foo {
      type T
      def f(x: T): List[T] = List()
    }
    object Bar extends Foo { type T = String }
    Bar.f("");    // java
    // java.lang.NoSuchMethodError: Bar.f(Ljava/lang/String;)Lscala/collection/immutable/List;

Before/after this commit:

    <   signature                                     f  (Ljava/lang/String;)Lscala/collection/immutable/List<Ljava/lang/String;>;
    ---
    >   signature                                     f  (Ljava/lang/Object;)Lscala/collection/immutable/List<Ljava/lang/Object;>;

This takes the warning count for compiling collections under
`-Ycheck:jvm` from 1521 to 26.

src/compiler/scala/tools/nsc/backend/jvm/GenASM.scala

@@ -1064,7 +1064,18 @@ abstract class GenASM extends SubComponent with BytecodeWriters with GenJVMASM {
       )
 
       // TODO needed? for(ann <- m.annotations) { ann.symbol.initialize }
-      val jgensig = if (m.isDeferred) null else getGenericSignature(m, module); // only add generic signature if method concrete; bug #1745
+      val jgensig = (
+        // only add generic signature if method concrete; bug #1745
+        if (m.isDeferred) null else {
+          val clazz = module.linkedClassOfClass
+          val m1 = (
+            if ((clazz.info member m.name) eq NoSymbol)
+              enteringErasure(m.cloneSymbol(clazz, Flags.METHOD | Flags.STATIC))
+            else m
+          )
+          getGenericSignature(m1, clazz)
+        }
+      )
       addRemoteExceptionAnnot(isRemoteClass, hasPublicBitSet(flags), m)
       val (throws, others) = m.annotations partition (_.symbol == ThrowsClass)
       val thrownExceptions: List[String] = getExceptions(throws)

src/compiler/scala/tools/nsc/transform/Mixin.scala

@@ -172,18 +172,23 @@ abstract class Mixin extends InfoTransform with ast.TreeDSL {
       // info) as they are seen from the class.  We can't use the member that we get from the
       // implementation class, as it's a clone that was made after erasure, and thus it does not
       // know its info at the beginning of erasure anymore.
-      //   Optimize: no need if mixinClass has no typeparams.
-      mixinMember cloneSymbol clazz modifyInfo (info =>
-        if (mixinClass.typeParams.isEmpty) info
-        else (clazz.thisType baseType mixinClass) memberInfo mixinMember
-      )
+      val sym = mixinMember cloneSymbol clazz
+
+      val erasureMap = erasure.erasure(mixinMember)
+      val erasedInterfaceInfo: Type = erasureMap(mixinMember.info)
+      val specificForwardInfo       = (clazz.thisType baseType mixinClass) memberInfo mixinMember
+      val forwarderInfo =
+        if (erasureMap(specificForwardInfo) =:= erasedInterfaceInfo)
+          specificForwardInfo
+        else {
+          erasedInterfaceInfo
+        }
+      // Optimize: no need if mixinClass has no typeparams.
+      // !!! JZ Really? What about the effect of abstract types, prefix?
+      if (mixinClass.typeParams.isEmpty) sym
+      else sym modifyInfo (_ => forwarderInfo)
     }
-    // clone before erasure got rid of type info we'll need to generate a javaSig
-    // now we'll have the type info at (the beginning of) erasure in our history,
-    // and now newSym has the info that's been transformed to fit this period
-    // (no need for asSeenFrom as phase.erasedTypes)
-    // TODO: verify we need the updateInfo and document why
-    newSym updateInfo (mixinMember.info cloneInfo newSym)
+    newSym
   }
 
   /** Add getters and setters for all non-module fields of an implementation

test/files/neg/t4749.check

@@ -25,6 +25,10 @@ t4749.scala:26: warning: Fail6 has a main method with parameter type Array[Strin
 
   object Fail6 {
          ^
+t4749.scala:42: warning: Win3 has a main method with parameter type Array[String], but bippy.Win3 will not be a runnable program.
+  Reason: main method must have exact signature (Array[String])Unit
+  object Win3 extends WinBippy[Unit] { }
+         ^
 error: No warnings can be incurred under -Xfatal-warnings.
-6 warnings found
+7 warnings found
 one error found

test/files/pos/t3452f.scala

@@ -0,0 +1,10 @@
+class Base[Coll] {
+  trait Transformed[S] {
+    lazy val underlying: Coll = ???
+  }
+}
+
+class Derived extends Base[String] {
+  class C extends Transformed[Any]
+}
+

test/files/run/mixin-signatures.check

@@ -0,0 +1,59 @@
+class Test$bar1$ {
+  public java.lang.String Test$bar1$.f(java.lang.Object)
+  public java.lang.Object Test$bar1$.f(java.lang.Object) <bridge> <synthetic>
+  public java.lang.String Test$bar1$.g(java.lang.String)
+  public java.lang.Object Test$bar1$.g(java.lang.Object) <bridge> <synthetic>
+  public java.lang.String Test$bar1$.g(java.lang.Object) <bridge> <synthetic>
+  public java.lang.Object Test$bar1$.h(java.lang.Object)
+}
+
+class Test$bar2$ {
+  public java.lang.Object Test$bar2$.f(java.lang.String)
+  public java.lang.Object Test$bar2$.f(java.lang.Object) <bridge> <synthetic>
+  public java.lang.String Test$bar2$.g(java.lang.String)
+  public java.lang.Object Test$bar2$.g(java.lang.Object) <bridge> <synthetic>
+  public java.lang.Object Test$bar2$.g(java.lang.String) <bridge> <synthetic>
+  public java.lang.Object Test$bar2$.h(java.lang.Object)
+}
+
+class Test$bar3$ {
+  public java.lang.String Foo3.f(java.lang.Object)
+    generic: public java.lang.String Foo3.f(T)
+  public java.lang.Object Foo3.f(java.lang.Object) <bridge> <synthetic>
+  public java.lang.String Test$bar3$.g(java.lang.String)
+  public java.lang.Object Test$bar3$.g(java.lang.Object) <bridge> <synthetic>
+  public java.lang.String Test$bar3$.g(java.lang.Object) <bridge> <synthetic>
+  public java.lang.Object Foo3.h(java.lang.Object)
+}
+
+class Test$bar4$ {
+  public java.lang.Object Foo4.f(java.lang.String)
+    generic: public R Foo4.f(java.lang.String)
+  public java.lang.Object Foo4.f(java.lang.Object) <bridge> <synthetic>
+  public java.lang.String Test$bar4$.g(java.lang.String)
+  public java.lang.Object Test$bar4$.g(java.lang.Object) <bridge> <synthetic>
+  public java.lang.Object Test$bar4$.g(java.lang.String) <bridge> <synthetic>
+  public java.lang.Object Foo4.h(java.lang.Object)
+}
+
+class Test$bar5$ {
+  public java.lang.String Test$bar5$.f(java.lang.String)
+  public java.lang.Object Test$bar5$.f(java.lang.Object) <bridge> <synthetic>
+  public java.lang.Object Test$bar5$.f(java.lang.String) <bridge> <synthetic>
+  public java.lang.String Test$bar5$.f(java.lang.Object) <bridge> <synthetic>
+  public java.lang.String Test$bar5$.g(java.lang.String)
+  public java.lang.Object Test$bar5$.g(java.lang.Object) <bridge> <synthetic>
+  public java.lang.Object Test$bar5$.g(java.lang.String) <bridge> <synthetic>
+  public java.lang.String Test$bar5$.g(java.lang.Object) <bridge> <synthetic>
+  public java.lang.Object Test$bar5$.h(java.lang.Object)
+}
+
+class Foo1$class {
+  public static java.lang.String Foo1$class.f(Foo1,java.lang.Object)
+}
+
+class Foo2$class {
+  public static java.lang.Object Foo2$class.f(Foo2,java.lang.String)
+}
+
+000000000000000000000000000000000000

test/files/run/mixin-signatures.scala

@@ -0,0 +1,105 @@
+trait Base[T, R] {
+  def f(x: T): R
+  def g(x: T): R
+  def h(x: T): R = null.asInstanceOf[R]
+}
+
+trait Foo1[T] extends Base[T, String] {
+  def f(x: T): String = null
+  def g(x: T): String
+}
+trait Foo2[R] extends Base[String, R] {
+  def f(x: String): R = { print(x.length) ; null.asInstanceOf[R] }
+  def g(x: String): R
+}
+abstract class Foo3[T] extends Base[T, String] {
+  def f(x: T): String = ""
+  def g(x: T): String
+}
+abstract class Foo4[R] extends Base[String, R] {
+  def f(x: String): R = { print(x.length) ; null.asInstanceOf[R] }
+  def g(x: String): R
+}
+
+object Test {
+  object bar1 extends Foo1[String] { def g(x: String): String = { print(x.length) ; "" } }
+  object bar2 extends Foo2[String] { def g(x: String): String = { print(x.length) ; "" } }
+  object bar3 extends Foo3[String] { def g(x: String): String = { print(x.length) ; "" } }
+  object bar4 extends Foo4[String] { def g(x: String): String = { print(x.length) ; "" } }
+
+  // Notice that in bar5, f and g require THREE bridges, because the final
+  // implementation is (String)String, but:
+  //
+  //   inherited abstract signatures: T(R), (T)String, and (String)R
+  //   which erase to: (Object)Object, (Object)String, and (String)Object
+  //
+  // each of which must be bridged to the actual (String)String implementation.
+  //
+  // public java.lang.String Test$bar5$.g(java.lang.String)
+  // public java.lang.Object Test$bar5$.g(java.lang.String) <bridge> <synthetic>
+  // public java.lang.Object Test$bar5$.g(java.lang.Object) <bridge> <synthetic>
+  // public java.lang.String Test$bar5$.g(java.lang.Object) <bridge> <synthetic>
+  object bar5 extends Foo1[String] with Foo2[String] {
+    override def f(x: String): String = { print(x.length) ; x }
+    def g(x: String): String = { print(x.length) ; x }
+  }
+
+  final def m1[T, R](x: Base[T, R], y: T)   = { x.f(y) ; x.g(y) ; x.h(y) }
+  final def m2[T](x: Base[T, String], y: T) = { x.f(y) ; x.g(y) ; x.h(y) }
+  final def m3[R](x: Base[String, R])       = { x.f("") ; x.g("") ; x.h("") }
+  final def m4(x: Base[String, String])     = { x.f("") ; x.g("") ; x.h("") }
+
+  final def m11[T](x: Foo1[T], y: T) = { x.f(y) ; x.g(y) ; x.h(y) }
+  final def m12(x: Foo1[String])     = { x.f("") ; x.g("") ; x.h("") }
+  final def m21[T](x: Foo2[T], y: T) = { x.f("") ; x.g("") ; x.h("") }
+  final def m22(x: Foo2[String])     = { x.f("") ; x.g("") ; x.h("") }
+  final def m31[T](x: Foo3[T], y: T) = { x.f(y) ; x.g(y) ; x.h(y) }
+  final def m32(x: Foo3[String])     = { x.f("") ; x.g("") ; x.h("") }
+  final def m41[T](x: Foo4[T], y: T) = { x.f("") ; x.g("") ; x.h("") }
+  final def m42(x: Foo4[String])     = { x.f("") ; x.g("") ; x.h("") }
+
+  def go = {
+    m1(bar1, "") ; m2(bar1, "") ; m3(bar1) ; m4(bar1)
+    m1(bar2, "") ; m2(bar2, "") ; m3(bar2) ; m4(bar2)
+    m1(bar3, "") ; m2(bar3, "") ; m3(bar3) ; m4(bar3)
+    m1(bar4, "") ; m2(bar4, "") ; m3(bar4) ; m4(bar4)
+
+    m11(bar1, "") ; m12(bar1)
+    m21(bar2, "") ; m22(bar2)
+    m31(bar3, "") ; m32(bar3)
+    m41(bar4, "") ; m42(bar4)
+    ""
+  }
+
+  def flagsString(m: java.lang.reflect.Method) = {
+    val str = List(
+      if (m.isBridge) "<bridge>" else "",
+      if (m.isSynthetic) "<synthetic>" else ""
+    ) filterNot (_ == "") mkString " "
+
+    if (str == "") "" else " " + str
+    //
+    // val flags = scala.reflect.internal.ClassfileConstants.toScalaMethodFlags(m.getModifiers())
+    // scala.tools.nsc.symtab.Flags.flagsToString(flags)
+  }
+
+  def show(clazz: Class[_]) {
+    print(clazz + " {")
+    clazz.getMethods.sortBy(x => (x.getName, x.isBridge, x.toString)) filter (_.getName.length == 1) foreach { m =>
+      print("\n  " + m + flagsString(m))
+      if ("" + m != "" + m.toGenericString) {
+        print("\n    generic: " + m.toGenericString)
+      }
+    }
+    println("\n}")
+    println("")
+  }
+  def show(x: AnyRef) { show(x.getClass) }
+  def show(x: String) { show(Class.forName(x)) }
+
+  def main(args: Array[String]): Unit = {
+    List(bar1, bar2, bar3, bar4, bar5) foreach show
+    List("Foo1$class", "Foo2$class") foreach show
+    println(go)
+  }
+}

test/files/run/t3452.check

@@ -0,0 +1 @@
+4

test/files/run/t3452.scala

@@ -0,0 +1,21 @@
+trait IStringPair[T] {
+  def a : String
+  def b : String
+  def build(a : String, b : String) : T
+  def cat(that : IStringPair[T]) = build(this.a + that.a, this.b + that.b)
+  override def toString = a + b
+}
+
+class StringPair(val a : String, val b : String) extends IStringPair[StringPair] {
+  def build(a : String, b : String) = new StringPair(a, b)
+  def len = a.length + b.length
+}
+
+object Test {
+  def main(args: Array[String]): Unit = {
+    val a = new StringPair("A", "B")
+    val b = new StringPair("1", "2")
+    val c = a cat b
+    println(c.len)
+  }
+}

test/files/run/t3452a.check

@@ -0,0 +1 @@
+BulkSearch.searchFor called.

test/files/run/t3452a/J_2.java

@@ -0,0 +1,5 @@
+public class J_2 {
+       public static void main(String[] args) {
+               BulkSearchInstance.searchFor(new UpRelation());
+       }
+}

test/files/run/t3452a/S_1.scala

@@ -0,0 +1,24 @@
+abstract class BulkSearch {
+       type R   <: Row
+       type Rel <: Relation [R]
+       type Corr <: Correspondence[R]
+
+       def searchFor(input: Rel): Mapping[Corr] = { println("BulkSearch.searchFor called.") ; null }
+}
+
+object BulkSearchInstance extends BulkSearch {
+       type R   = UpRow
+       type Rel = UpRelation
+       type Corr = UpCorrespondence
+}
+
+class Row
+class UpRow extends Row
+
+class Relation [R <: Row]
+class UpRelation extends Relation [UpRow]
+
+class Correspondence [R <: Row]
+class UpCorrespondence extends Correspondence [UpRow]
+
+class Mapping[MC <: Correspondence[_]]

test/files/run/t3452a/S_3.scala

@@ -0,0 +1,5 @@
+object Test {
+  def main(args: Array[String]): Unit = {
+    J_2.main(args)
+  }
+}

test/files/run/t3452b.check

@@ -0,0 +1 @@
+Search received: test

test/files/run/t3452b/J_2.java

@@ -0,0 +1,5 @@
+public class J_2 {
+  public static void j() {
+    StringSearch.search("test");
+  }
+}

test/files/run/t3452b/S_1.scala

@@ -0,0 +1,10 @@
+trait Search[M] {
+  def search(input: M): C[Int] = {
+    println("Search received: " + input)
+    null
+  }
+}
+
+object StringSearch extends Search[String]
+
+trait C[T]

test/files/run/t3452b/S_3.scala

@@ -0,0 +1,5 @@
+object Test {
+  def main(args: Array[String]): Unit = {
+    J_2.j()
+  }
+}

test/files/run/t3452c.check

@@ -0,0 +1,8 @@
+3
+3
+3
+3
+3
+3
+3
+3