proposal: Go 2: default methods

[neild]

(I don't currently believe the following proposal has a strong case for adoption; I'm filing it because various people have convinced me that it's worth writing down if only to have a place to attach arguments against it. And perhaps there's a better case for it than I'm currently seeing.)

Proposal

Permit declaring methods with a receiver of interface type. These methods become "default methods" of the interface type. Method with a default are not required for interface satisfaction, with the default being used when a type does not implement a method.

Methods with a receiver of type *T where T is an interface type would be disallowed.

Example:

type Interface interface {
  Add(x int)
  Inc()
}

func (i Interface) Inc() {
  i.Add(1)
}

type Num int
func (n *Num) Add(n int) {  *n += x }

// Num satisfies Interface, even though it does not have an Inc method.
// If Num did have an Inc method, it would take precedence over the default one defined on the interface.
var _ Interface = (*Num)(nil)

Motivation: Interface extension

A well-known problem in Go API design is that adding a method to an interface requires updating all implementations of that interface. This can make API changes prohibitively difficult. As a concrete example, we have a database package which defines a Table interface with a broad set of methods:

type Table interface{
  Apply(ctx context.Context, row string, m *Mutation) error
  Read(ctx context.Context, ranges []ReadRange, f (ReadItem) bool) error
  // ...and 12 more methods.
}

Adding a new method to this interface is exceedingly difficult, because there are many implementations of it used in tests.

Default methods would make this much simpler: New interface methods would be added with a corresponding default method returning an error.

The Go community has, however, developed a number of best practices around API design to minimize or avoid this problem: Interfaces should be small and precisely defined, APIs should return concrete types rather than interfaces, and interfaces which cannot obey these guidelines should include an unexported method to hinder uncontrolled reimplementation of the interface. The Table interface is in violation of all these guidelines. Therefore, it stands as both an argument for and against this feature--default methods would clearly help get us out of the corner we've painted ourselves into here, but we shouldn't have painted ourselves into the corner in the first place.

Motivation: Optional extension

A common pattern in languages with method inheritance is to define a base class with a default set of behaviors and override a subset of those behaviors in subclasses.

For example, in the example above we have a type with an Inc method which may be implemented in terms of an Add method. An implementation of this type might choose to either accept the default implementation or to include a more efficient specialized one.

We can, however, accomplish the same results in Go as it stands today with a standalone Inc function and a type assertion:

type Interface interface {
  Add(x int)
}

func Inc(i Interface) {
  if x, ok := i.(interface{Inc()}); ok {
    // Use an Inc method if one is present.
    x.Inc()
    return
  }
  // Default implementation.
  i.Add(1)
}

While this is arguably a bit more clumsy than the approach with default methods, it works well enough that this does not strike me as a strong motivating case for adding default methods to the language.

Conclusion

I am unconvinced that default methods would be a good addition to Go. However, perhaps others will see a stronger case for them than I do. I'm filing this as a proposal so we have a place to collect arguments for or against the feature.

[jimmyfrasche]

Another variant (albeit one that would make code generation harder) would be to have methods defined on interfaces only available when they are nil, making the zero values of interfaces more useful. For example a nil fmt.Stringer could return "" or a nil sort.Interface's Len method could return 0.

I don't think either variant would be useful enough to compensate for the introduced complexity.

Either would be at odds with #8082 which seems like it would produce the most benefit

[neild]

Defining methods only for nil interface values doesn't seem particularly useful, since you can get exactly the same behavior with an explicit nil check.

s := ""
if stringer != nil {
  s = stringer.String()
}

The most compelling argument I can think of is to permit extension of interfaces without updating existing implementations, which is something which genuinely cannot be done today.

[jba]

Embedding is the accepted way to future-proof your test implementations. Since nearly all default methods will have to return an error (there's not much else they can do, unless they're simply convenience methods), and embedding already results in a runtime error, default methods don't add anything we don't already have.

I think instead we should work harder at educating the community about embedding.

About those convenience methods: that's one way that default methods will be harmful. People will write grotesquely non-minimal interfaces like those of the Java collection classes, with all but a couple having default implementations in terms of the others.

[neild]

A concrete use case which occurred to me this morning: It would be nice to have a way to plumb a per-operation cancellation signal into io.Reader et al, probably via a Context parameter (#20280). Default methods would permit doing this in a backwards compatible fashion:

type Reader interface {
  Read(p []byte) (n int, err error)
  CRead(ctx context.Context, p []byte) (n int, err error)
}

func (r Reader) CRead(ctx context.Context, p []byte) (n int, err error) {
  return r.Read(p)
}

[davecheney]

This example doesn’t sound very compelling. The default method lets you bolt on a method to every io reader out there—100% of the population at the time of release—which ignores the context parameter it was given. Debugging failures from this would almost guarantee finding a horses head in your bed from angry developers.

[bcmills]

we shouldn't have painted ourselves into the corner in the first place.

Other similarly-painted corners include many of the magic-method interfaces in the Go standard library: http.CloseNotifier, http.Hijacker and http.Flusher (which should arguably be methods on http.ResponseWriter), flag.Getter and flag.boolFlag (should be methods on flag.Value), flate.Reader, and perhaps driver.NamedValueChecker

[bcmills]

For comparison, a similar feature was added to Java 8 to address an analogous problem there.

(But the problem in Java may have been even more acute, because the workaround — applying an instanceof test and type-cast — required the implementing class to explicitly satisfy the interface.)

[urandom]

@davecheney
Since io.Reader is defined in the stdlib, I would imagine that only suitable methods would be added.
Useful default methods for that specific interface would be the ReadByte/ReadBytes/ReadRune/ReadString/ReadSlice/WriteTo methods. They can be implemented for any io.Reader, and the user will get the benefit of using them (thus having greater freedom of using io.Reader), while benefiting from any implementation optimizations that might be present, depending on what the underlying object is (e.g. bytes.Buffer or bufio.Reader). Similarly, io.Writer can also have the corresponding Write* default methods, as well as ReadFrom

[bcmills]

@urandom That brings up an interesting question, actually. At the moment, programs can use type-assertions to interfaces like io.ReaderFrom to detect whether an optimized implementation exists and fall back to an entirely different algorithm otherwise (see also #16474).

The interaction between default methods and type assertions might be fairly subtle: on the one hand, one might expect to be able to type-assert a non-nil instance of an interface to another interface with a strict subset of its methods, but on the other hand it would be odd for passing a value as an interface with a default method to permanently attach that method to the value.

[urandom]

@bcmills I've always assumed that when you do type assertion of an interface, its underlying value is actually checked. If that's the case, an interface having default methods would be irrelevant to the check, in that io.ReaderFrom would be implemented for a given io.Reader only if the underlying type actually implemented the methods of the former interface.

[neild]

It might be odd, but I think "permanently attach" would be the only reasonable approach.

type Fooer interface { Foo() }

type DFooer interface { Foo() }
func (DFooer) Foo() { /* default implementation */ }

var x DFooer = struct{}{}
x.Foo() // DFooer.Foo(x)

var y Fooer = x // Surely this is fine; x has a Foo method.
y.Foo()

// y is a Fooer and you can call y.Foo, so for this type assertion to fail would be confusing indeed.
_, ok := y.(Fooer)

Perhaps this is an argument against default methods. It's certainly subtle.

[davecheney]

That’s a brilliant observation. Default methods would make interface assertions operate bizarrely.

…

On 10 Jan 2018, at 09:50, Viktor Kojouharov ***@***.***> wrote: @bcmills I've always assumed that when you do type assertion of an interface, its underlying value is actually checked. If that's the case, an interface having default methods would be irrelevant to the check, in that io.ReaderFrom would be implemented for a given io.Reader only if the underlying type actually implemented the methods of the former interface. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or mute the thread.

[urandom]

@neild
That might've been just a type in your original proposal, but I assumed that the interface's default methods were not part of the interface "definition" itself. That is to say:

type Fooer interface {
    Foo()
}

type FooAter interface {
    FooAt()
}

type F int
type FA int

func (f Fooer) FooAt() {
    ...
}

func (f F) Foo() {
}

func (f FA) Foo() {
}

func (f FA) FooAt() {
}

Up to here, it makes it a bit clearer what a type has to implement in order to satisfy each interface method.

Now, if the type assertions are done on the underlying type itself, nothing would change in the language for the following cases:

var x Fooer = F{}
x.Foo() // Foo from F
x.FooAt() // FooAt from Fooer

var y FooAter = x // Compile time error, FooAter doesn't have FooAt as part of its definition

_, ok := x.(FooAter) // ok = false since the underlying type does not define FooAt

var z Fooer = FA{}
z.Foo() // Foo from FA
z.FooAt() // FooAt from FA

var u FooAter = z //Not a problem
_, ok := z.(FooAt) // ok = true since the underlying type defines a FooAt method

In fact, the spec seems to suggest exactly that:

x.(T) asserts that x is not nil and that the value stored in x is of type T

The following example illustrates that: https://play.golang.org/p/P5LM23tlzGm

I can't really say whether in general, this scenario would cause confusion. I personally find it straightforward. The default methods only come into play during the method dispatch. Everything else stays unchanged. Thus, it shouldn't be confusing, since that's how the language works right now.

[neild]

@urandom There was indeed a typo in my original proposal; I've corrected it. (s/Increment/Int/)

It seems quite confusing to me for var _ string = x.String() to be valid while var _ fmt.Stringer = x is invalid for the same x.

[urandom]

@neild
I personally like your approach (var _ fmt.Stringer = x) better, since it is more versatile and makes the interface behave like any other type, with its methods playing a role in type assertion. Though I would still prefer if the default methods are not part of the interface definition, since then they are actually part of the interface type, and they are not needed to be present in a type when trying to implement an interface.

As for detecting whether the underlying value actually implements the default methods, what about allowing the TypeSwitchGuard to be used outside of a switch for that purpose? Going from the previous example:

var x Fooer = F{}
v := x.(type)

_, ok := x.(FooAter) // ok = true, since FooAt implements FooAter by way of its default method
_, ok := v.(FooAter) // ok = false, since v (the underlying value) doesn't implement FooAter
// Shorthand:
_, ok := x.(type).(FooAter) // ok = false

[ianlancetaylor]

This idea fundamentally changes the meaning of a method set. Today, every non-interface type has a (possibly empty) method set. Interface types do not have method sets; they instead express a kind of structural typing as a way to work with all types that share certain characteristics (namely, all types that have a method set that is a (possibly improper) superset of the interface type's methods).

Defining a method on an interface type changes this. If a "default method" somehow adheres to the value of a variable of interface type, that would give us a bizarre value whose method set would change depending on the set of conversions to interface type that were applied. It would mean that the very notion of an interface type would change. It would no longer be possible to pass an interface value to a function that expects an interface{} parameter by simply passing the value pointer and the type. We would have to construct a value with a dynamically created type with a dynamically created method set. Otherwise reflect.TypeOf would not work correctly, and neither would fmt.Print.

But if the "default method" does not adhere to the type, then adding a default String method would not work if the value were passed to fmt.Print.

So I don't really see how this proposal is feasible.

[neild]

@iant Interface types do have a method set: "The method set of an interface type is its interface."

I agree with your point about the need to dynamically construct a type, which is a compelling argument (to me, at least) against this proposal, at least in that form.

[bcmills]

Yeah, I think it wouldn't be sensible to attach the method to the value.

And if we don't attach the method to the value, then default methods can't be used for assignability, and provide no advantage over a package-level function that does a type assertion.

[neild]

You wouldn't actually need to dynamically construct a type, now that I think about it--you just need one type per interface type with default methods, which can be constructed at compile time.

(It's still subtle and complicated, however.)

[ianlancetaylor]

I think in the general case you need to dynamically construct a type. Consider assigning a value to an interface type with a default method, then assigning to an empty interface type, then calling a function in a different package that does a type assertion to a third interface type. The last type assertion would need to see the default method of the first interface type plus any arbitrary method of the original type.

[ianlancetaylor]

The additional functionality of this proposal is real but limited, since approximately the same functionality is available by using a function or by embedding the type in a struct with its own methods. The complexity seems significant, per the discussion about dynamically constructing a type.