-
Notifications
You must be signed in to change notification settings - Fork 17.7k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
proposal: spec: interface literals #25860
Comments
How would this work in terms of reflection? What would be the result of calling |
I assume you mean to ask that if You are right in that there isn't an underlying value being wrapped. That being said, since type switches can already switch on interfaces, doing so on an interface literal would simply not satisfy cases that check for concrete types.
As to the representation of a literal's reflected value, if the same reflect package is used for Go 2, the underlying value can be a MethodSet. This does not have to correspond to its runtime representation, but this is a simple abstraction for the reflect package. A MethodSet is just an interface that references all methods in the underlying value. Operations on a MethodSet are nearly identical to operations on an Interface. From the above example, if uv.Kind() is a MethodSet, then uv.Interface() is no longer a valid operation. ut := uv.Type() will return a type with all of the underlying methods. Similar to an interface type, ut.Method and ut.MethodByName will return Methods whose signatures do not have a receiver and whose Func fields are nil. |
I think the primary problem with this proposal is that it allows nil methods, which panic when invoked.
On the other hand, nil interfaces are extremely common, even though one could argue that they are also a violation of the contract described above, since users expect to be able to invoke its methods. Additionally, I think allowing nil methods to prevent a downcasted interface from type-asserting into the original interface sounds nice since it allows the promotion of known methods. However, this behavior only exists because nil methods are allowed, and allows the runtime to convert an "unsafe" (non-invocable) interface to a "safe" (invocable) interface. This behavior implies that non-invocable interfaces shouldn't exist in the first place, and is too subtle and surprising. The only alternative I can think of is to make a nil method provide some default no-op implementation of a method. Although this is safer than the previously mentioned, it seems just as subtle and surprising, but less powerful. Ultimately, it appears impossible to require that no field in an interface literal is nil, since a value that is assigned to it could be nil at run-time. The only way to guarantee no field is nil would be to restrict each field to be a function literal or top-level function. However, this pretty much loses all of the power of interface literals, since it is only a marginal improvement (LoC-wise) over a type which is defined to implement an interface. |
Can this be added to #33892 for review? |
/cc @golang/proposal-review |
This was reviewed and marked as NeedsInvestigation. At some point we'll re-review these issues. |
I came here from #47487 which might be seen as competing with this issue. Seeing that the interface literals proposed here have a serious problem with nil safety, as @smasher164 admits here: #25860 (comment) and considering that nil problems are a significant problem when developing software (look up "Null References: The Billion Dollar Mistake"), I strongly believe that any changes to the Go language should make it more nil safe and not less so. Therefore I respectfully oppose this proposal, unless a way can be found to avoid the nil problem. #47487 on the other hand does not seem to have this nil problem, so I support that as an alternative. |
This is not necessarily true. For example: package main
type T struct{}
func (*T) Foo() {}
func (T) Bar() {}
type FooBarer interface {
Foo()
Bar()
}
func main() {
var x FooBarer = (*T)(nil)
// Does not panic
x.Foo()
// Panics - a nil *T is dereferenced when calling a method with value receiver.
x.Bar()
} I included two methods here, to demonstrate that this isn't simply "calling a method on a
This is also not a worry. The runtime does not do
I would argue it's similar, if not the same, as a pointer being stored in an interface, where all methods panic when called with a |
@Merovius Good point about the dereferencing when the method has a value receiver. |
Under this proposal, given an interface literal with a nil method, is the result of the method expression selecting that method nil or non-nil? E.g., is the output of the following program package main
func main() {
println(interface{ M() }{}.M == nil)
} If the output of this program is |
If nil methods are allowed, the output of the above program would be |
#47487 can actually be used to replicate the functionality of this proposal in some circumstances, though it's a lot clunkier: v := []int{3, 2, 5}
type Lesser interface{ Less(i1, i2 int) bool }
type Swapper interface{ Swap(i1, i2 int) }
type Lenner interface{ Len() int }
sort.Sort(struct {
Lesser
Swapper
Lenner
}{
Lesser: Lesser(func(i1, i2 int) bool { return v[i1] < v[i2] }),
Swapper: Swapper(func(i1, i2 int) { v[i1], v[i2] = v[i2], v[i1] }),
Lenner: Lenner(func() int { return len(v) }),
}) |
Is there any particular reason that this proposal can't be modified to require exhaustive implementation? I see a lot of arguments for and against it being accepted with What's the benefit of allowing methods to be |
Even if we wanted to disallow var readerFunc func([]byte) (int, error)
if runtimeCondition {
readerFunc = func(p []byte) (int, error) { /* implementation */ }
}
reader := io.Reader{
Read: readerFunc,
} We could conservatively restrict methods in interface literals to either be top-level functions or function literals, but these rules are arbitrary and hard to teach. Additionally, a lot of the power of interface literals comes from the ability to change the implementation of some method by changing the variable binding for some closure. If you're going to define all of the functions up-front anyways, why not just make them methods on some basic type definition?
The reason is to allow the promotion of known methods. Consider the example that @neild provides here: #21670 (comment). Basically, you know that some library wants to type-assert on your value for specific functionality (like Normally, you would have to type-assert for all combinations of methods you care about, and return a concrete type that implements that combination of methods. So if you wanted to wrap an The question is if this is even a problem worth solving for interface literals. However, even if it wasn't, |
|
Ah, duh. I was mostly confused by the defaulting of unspecified methods to sort.Sort(sort.Interface{
Less(i1, i2 int) bool { ... }
Swap(i1, i2 int) { ... }
Len() int { ... }
}) Maybe inverting the problem and making it anonymous type literals instead of interface literals would make more sense. I remember the Edit: To clarify, I'm saying ditch the interface name completely and just define an inline method set, then let the type checker do the work: // sort.Sort() already defines its argument as being sort.Interface,
// so the regular type checker can easily sort this out.
sort.Sort(type {
Less(i1, i2 int) bool { ... }
Swap(i1, i2 int) { ... }
Len() int { ... }
}) |
@adonovan honestly, I don't very get your last reply. Maybe I over-worried about it. Is it possible that the implementation struct type is a distinct zero-size struct (with an incomparable zero-size field) and the functions list is maintained outside of the struct values? |
@griesemer I like your construction, but why it is important for "each interface literal has its own user-inaccessible type"? What does that add over having each interface type used in a literal have its own type? I ask because when hand writing mocks/fakes that use this pattern today I usually only make one type per interface I am mocking/faking. Obviously that saves me effort that the compiler doesn't care about as much, but does making each literal generate a new type have a specific advantage? |
@griesemer I would weakly argue in favor of requiring every method to be provided - if nothing else, then on the grounds that it is the more conservative choice and we can always relax it later, if we want to, but not the other way around. Also, personally I would be in favor to have interface-literals just always be non-comparable. |
@go101 There is no need to have func-valued fields. func F() (*string, error) {
x := new(string)
return x, error{func() string { return *x }}
} (for example) could be de-sugared into type _tmp struct {
p *string
}
func (v *_tmp) Error() string {
return *v.p
}
func F() (*string, error) {
x := new(string)
return x, &_tmp{x}
} But also, I don't think this is a super important argument. If it can be implemented efficiently, great. If it can't be implemented efficiently, then there isn't a way for the user to do so either (it's not like the user can write code the compiler couldn't generate), so that wouldn't be counted against this proposal. It's a red herring. |
@ChrisHines This may not be a sufficiently convincing answer, but in the model translation, if there's exactly one struct type per interface type, comparison of interface literal values behaves differently. For instance, given an interface
Yet if It seems a bit odd that comparing interface literals is ok if they are different interface types and the result is always false, but panics if they are the same interface type, unless the type is If each interface literal has its own dynamic type, comparing interface literals is always ok and the result is always false. This seems easier to understand. It may also be simpler to implement. |
Is it a good idea to make |
The comparison only panics if both have the same type. It is allowed to compare uncomparable values of different types. Requiring them to always panic comes with one theoretical complexity, which is that two interface literals from different packages would have to have the same type. I kind of get why that might be harder. |
@Merovius I just note that with other literals we are not required to provide all elements. So the proposal matches existing "behavior". I don't have a stronger argument than that, and this may not be strong enough. And yes, we wouldn't be able to tighten the rules after the fact, while with your suggestion we would. With respect to comparisons, if we panic when comparing interface literals, it seems that this leads to odd behavior in some cases. For instance, It seems simpler to always have a new inaccessible dynamic type for each interface literal. Then they can always be compared and the result is always false. |
Without the different type per literal rule, there is also a question of identity: is I think a similar question applies given |
@zephyrtronium Indeed. Consider multiple interface literals created in a loop (playground). So even with different types per literal, this probably has to be lexical. Which leaves some odd behavior after all, either way. |
Yes, @adonovan provided a machinery: #25860 (comment)
But what the problem is it? It looks normal to me. BTW, I have no strong opinions on the implementation details. I mean, any design is okay.
I prefer
In my opinion, all of the dummy struct types should be declared in the universal space. |
I'm also a bit unclear on what "with a type name D that is inaccessible to the user" means. I assume "inaccessible" here means "unable to be referred to," along the lines of an unexported name in an imported package. But what is the result of |
Also, does the "different type per interface-literal" imply that every interface-literal has to allocate a new |
I fairly strongly think interface literals should be required to provide implementations of all methods. It is true that with other literals we are not required to provide all elements. However, we do require implementations of an interface to implement all methods of the interface. We can argue consistency from either direction here. Leaving consistency aside, a compile time error explaining that a method isn't implemented is substantially safer and more comprehensible than a run-time panic. A panic can occur at a point far distant from where the interface value was created. A compile error will immediately point at the source of the problem. I also don't see much practical value to permitting implicit partial implementation of an interface. There may be some cases in tests where partial implementation is useful, but this doesn't seem common enough or useful enough to justify the reduction in comprehensibility and safety. If you need to partially implement an interface, you can do so explicitly: |
@neild Partial implementations in tests is very common in my experience and literals would be especially helpful in that case where the ratio of types/method to implement is very close to 1. if it's always 1 yes we can use the #47487 but very often in tests I need to implement 2/3 methods of some extensive interfaces. |
We should not require that a given interface type has a single (unnameable) struct type for its closure, as this precludes optimizations that eliminate the double indirections both in control (method calls) and data (accesses to free variables). I like @griesemer's desugaring as an example of a legal (if suboptimial) implementation. In other words, the identity of |
@Merovius By "different type per interface-literal" I meant a different type per lexical interface literal - this is bounded by the size of the source and statically know. |
@griesemer I think I'm confused, then by
ISTM that if we are talking "one type per lexical interface literal", with the model translation, this code would panic: type J interface{ M() }
func F() J {
return J{func() {}}
}
func main() {
x, y := F(), F()
x == y // panics: interface value have same, non-comparable type
} |
I think I'd be most inclined to support just specifying "interface literals are always comparable and compare as |
The source-to-source transformation I proposed earlier can actually be adjusted easily to address some of the shortcomings that have been found so far: instead of using a dummy struct, we use a pointer to a dummy struct (playground). Using a pointer in the interface variable (which an implementation would do anyway via boxing) addresses the comparison problems: two interface literals are equal only if they are in fact the same literal. Comparing an interface literal with any other interface will return false. This leaves still open the following questions:
For 1) we just have to make a decision. The conservative approach is to require that all methods are present. This would be different from precedent for other composite literals. For 2) I think the type name should be some artificial (compiler-created) name that makes it clear that these are special types. It may include the address of the actual type. The actual type may be marked such that it cannot be used via reflection (maybe?). |
@Merovius I think my previous comment addresses most of your concerns. |
https://go.dev/cl/573795 defines an analyzer to find types that are candidates for interface literals. Here are 25 random results from the module mirror corpus. https://go-mod-viewer.appspot.com/github.com/dop251/[email protected]/func_test.go#L214: goja.testAsyncContextTracker is a one-off goja.AsyncContextTracker
Update: the job identified a whopping 100,277 one-off types in 6456 modules (from a total corpus of around 20K modules) That's an average of 16 in those modules, which seems almost implausibly high, though the ones I looked at by hand seemed plausible. I was tempted to adjust the analyzer to reject candidates which have more methods than the interface type (such as a one-off io.Reader that also has a String method), but that would falsely reject one-off types with helper methods that might be more neatly expressed as local closures; also, one could easily locally define a broader interface such as ReaderStringer and use a literal of that type. Perhaps the analyzer should put a bound on the total number of lines of the one-off type's methods? A file containing the first 10,000 is attached. This file https://go-mod-viewer.appspot.com/github.com/greenpau/[email protected]/pkg/acl/rule.go is the largest (apparently) non-generated example, with over 480 one-off types. Interface literals would save about 7000 lines of code in that file alone. There are dozens of files with over with 100 one-offs. |
Change https://go.dev/cl/573795 mentions this issue: |
One possibility to permit partial implementations without compromising on the "fail open" principle would be to use an ellipsis to affirm that the interface demands more methods than are provided. A call to any of them would panic. rw = io.ReadWriter{
Read: readFromStdin,
...
} Of course, it's a change to the syntax of composite literals, which would make everything much more costly. |
I'm just not seeing compelling use cases here yet. I clicked on a handful at random and all of them look better to me with an explicit type. You need very trivial functions for this to be a win. After that I think the benefit of having a real type name and therefore a real method name in debuggers and debug prints becomes far more important than saving a few lines. https://go-mod-viewer.appspot.com/github.com/aacfactory/[email protected]/barriers/barrier.go#L42 The only defensible ones are the calls to sort.Sort but even there having to write three methods gets to the point where it's probably not a win to inline it. I've always found it much clearer to move that code to a separate place in the file and have the call site just say
That's less to skim at a glance when reading it than
Writing the literal seems to optimize for writing the code and not for reading it, and that's the wrong optimization. And as I noted these should use slices.SortStableFunc or sort.Slice anyway, and then there's no interface at all anymore. Any use of sort.Sort should be dropped from the analysis, since we already have a better API for sorting. (Making a language change to improve the use of an API we've already added a better replacement for isn't worth it.) |
Filing this for completeness sake, since it was mentioned in #21670 that this proposal had been discussed privately prior to Go 1. Just like literals allow the construction of slice, struct, and map values, I propose "interface literals" which specifically construct values that satisfy an interface. The syntax would mirror that of struct literals, where field names would correspond to method names. The original idea is proposed by @Sajmani in #21670 (comment).
Conceptually, this proposal would transform the following initializer
to the following type declaration and struct initializer
As an extension to what’s mentioned in #21670, I propose that fields be addressable by both method names and field names, like in this example:
The default value for a method is
nil
. Calling a nil method will cause a panic. As a corollary, the interface can be made smaller to be any subset of the original declaration. The value can no longer be converted back to satisfy the original interface. See the following modified example (from @neild in #21670 (comment)):The nil values for
ReadAt
andWriteTo
make it so the “downcasted”io.Reader
can no longer be recast to anio
. This provides a clean way to promote known methods, with the side effect that the struct transformation described above won't be a valid implementation of this proposal, since casting does not work this way when calling a nil function pointer through a struct.Although this proposal brings parity between struct and interface initialization and provides easy promotion of known methods, I don’t think this feature would dramatically improve the way Go programs are written.
We may see more usage of closures like in this sorting example (now obviated because of sort.Slice):
Promotion of known methods also avoids a lot of boilerplate, although I’m not sure that it is a common enough use case to warrant a language feature.
For instance, if I wanted to wrap an
io.Reader
, but also let through implementations ofio.ReaderAt
,io.WriterTo
, andio.Seeker
, I would need seven different wrapper types, each of which embeds these types:Here is the relevant change to the grammar (under the composite literal section of the spec):
The text was updated successfully, but these errors were encountered: