reflect: MakeInterface #4146
Comments
While we're listing use cases, I'd also be able to take an arbitrary http.ResponseWriter
value from a client (which may implement optional http.Hijacker and/or io.WriterTo
and/or http.Flusher) and change a method or two on it, while still forwarding all other
methods on it.
Currently whenever I want to implement an http.ResponseWriter wrapper (which I continue
to do regularly), I have to defensively implement all 3 optional interfaces, with the
implementation bodies checking at runtime whether the wrapper ResponseWrapper implements
those, otherwise I break people's flushing or sendfile or hijacking, when perhaps all I
want to do is record Writes that go by, or change the underlying transport, or watch for
errors, etc.
Doing something like:
type myWrapper struct {
io.Writer
http.ResponseWriter
}
... is an error, due to Write ambiguity, and something like:
type myWrapper struct {
http.ResponseWriter
}
func (myWrapper) Write([]byte) (int, error) { ... }
... hides Flush, WriteTo, Hijack, etc.
I could see where this road leads to abuse and performance problems, though.
But it's painful either way.
|
Eliminating half the generation for gomock may be compelling for you as the author but it doesn't change at all the way people interact with gomock. It's still a separate preprocessor. I'd be much more excited if we could get rid of all the Go file generation entirely. That's a qualitative difference. However, even with runtime type construction I don't believe you can eliminate the other half. You need something at compile time for the mock user to import. Russ |
You don't. The existing API for GoMock generated code only exports the mock implementation of the interface; the recorder is returned by an EXPECT method on that. I don't see any reason why something needs to be imported. Instead of foo := mock_foo.NewMockFoo(ctrl) you could write foo := gomock.NewMock(ctrl, (*Foo)(nil)) // passing a pointer to the interface and get back a dynamically created mock. |
Yes but keep going. What does the code using foo look like? Suppose it
says foo.EXPECT(). The compiler must see a definition of a struct or
interface with an EXPECT method at compile time in order to build
that. Where did that come from? Here's a line from
sample/user_test.go:
mockIndex.EXPECT().Put("b", gomock.Eq(2)) // matchers work too
I don't see how the compiler can compile that line without a
gomock-generated import defining something with a Put method that will
accept gomock.Eq(2) as an argument.
|
That 'qualitative difference' can also be achieved by other means, such as build tool support for custom pre-compilation phases, possibly specified by build directives (e.g. a // +prebuild comment in GoMock could specify that the GoMock preprocessor be run on anything that imports the mocking package). |
|
Comment 16 by [email protected]: Wouldn't it be possible to write something like Mockito's API:
ctrl := ...
defer ctrl.Finish()
mockFoo := gomock.NewMock(ctrl, (*Foo)(nil))
gomock.When(mockFoo.Put("b", gomock.Eq(2))).
ThenReturn(PutResponse{}, nil)
myTestCode()
I.e. NewMock creates a mock that returns zero values for all calls. When() returns an
Expectation interface that allows users to set expectations, e.g. using functions like
"ThenReturn(retVals ...interface{})". NewMock internally tracks the last call to any
method, calls on the returned Expectation interface set up the return values for any
subsequent calls.
You lose type safety on the values, i.e. ThenReturn is essentially untyped, but that
might be acceptable in return for no code generation. In Java-land, that is solved using
generics - When() could be parameterized as "func template <T> When(argument
<T>) Expectation<T>" (pseudo syntax), so that "ThenReturn()" calls could
require the correct types.
|
|
@bradfitz You mention having to implement wrappers for optional interfaces of http.ResponseWriter, are there any examples in http standard library? I've come across this problem as you've described it: https://groups.google.com/d/topic/golang-nuts/HKY3mI7Q2jY/discussion |
|
@crawshaw, any interest? :) |
|
@bradfitz I don't have time for this in the 1.7 window. |
|
Based on the idea that you should be able to cache pairs of types, I tried writing out a more complete implementation using I wanted to use reflect to do the equivalent of: That way I could cache The code was mostly straightforward. Create a struct with two fields Collect the relevant methods and create proxies that call them on I ran into a problem though. In order to create the proxies I needed the result of Also, I'm not sure how unexported methods would work with this. If I have a "Wrap" func defined in package A and in package B I create a value from a type defined in package B and one in C, then can I assert for interfaces containing unexported methods in package B? What if I pass the value to package C? |
Right, we'd need an explicit carve-out to allow the receivers of those methods to be the underlying type rather than the actual receiver type. (In my comment, I described that as 'the "underlying" type (or a pointer to that type) must be assignable to the function's first argument'.)
I think you'd have to use embedding, which implies that |
|
I think we need
That let's us create a value with an arbitrary method set and type from regular functions. Let's also say there was a way to create a method value† from a given Then given two arbitrary
† I don't see this used much in the wild so if anyone reading is unfamiliar, given a type |
|
@bcmills yes, sorry—missed the point about the underlying type to break the loop. That does have the disadvantage that you couldn't create a method which invokes another method using reflection as the receiver doesn't have the method set yet. Nothing to do with the use-case under discussion, but a limitation nonetheless. Maybe it needs to be a builder pattern where Back to the use-case, as far as embedding, since we're, by definition, trying to combine two types with intersecting method sets we could only embed one of the two types so you'd have the same problems just with one less type to worry about except you'd also need to worry about a method colliding the embedded types name. But that made me realize that in order for any of this to work we'd potentially need to create the equivalent of since the types we're combining come in interfaces so the underlying type could very well be unexported and is likely from another package. For that to work the constructed type would have to operate by different rules than ordinary Go types, which I'm sure would cause all kinds of subtle problems. The |
|
Field names conflicting with methods is potentially unfortunate. However, the solution already exists, since As for what the receiver type would need to be for As for unexported methods, the behaviour of promotion in the language is that unexported methods are promoted, so you should be able to assert to an interface containing those methods (provided the method name in the interface and the method name on the type originate in the same package).
val1 := reflect.ValueOf(Wrap{ inner: err })
val2 := val1.Field(0).Elem() |
|
@stevenblenkinsop good point about The issue with the receiver type is that instead of you have to create the methods like but that should actually be fine if The interesting thing about this wrt to unexported methods is that if you create a type from two types defined in separate packages, you could have two unexported methods on the created type with the same Name but different PkgPaths so it could, say, assert to Having thought about this more I think @bcmills solution could work. I'll write up a version using it and my |
|
Still playing with this, but I think All of the other alternatives (including first class support in the language) become O(n) when composed. If you had a function for combining two values in some way, say However, with |
|
I've created a gist of a rough implementation. Warning: a little over 300 LOC. While it's an optimizing implementation, the implementation is far from optimized. I pessimized it fairly extensively to make it easier to understand what it's doing since I can't actually run it to verify. It should be relatively straightforward to reduce allocations and the number of passes over the data, though. To avoid creating a tree of combined values that has to proxy and re-proxy method invocations up to the depth of the tree, it adds a marker field as the 0th field of the struct it creates. When it's called on a value whose type has that marker field, like A simpler, non-optimizing implementation would work, too, but it would have unfortunate performance implications in all but the simplest of cases making it less likely to be used. |
|
If something like that was added to the reflect package itself, then fmt.Errorf could be rewritten to something like: That's not a light change to consider, it could have some weird consequences, but it would mean that a lot of code that hides optional methods on errors would stop doing so with no modification. |
|
Another issue: (should have tested this earlier) |
|
Let's say for the moment that reflect can create new types with methods, something equivalent to what I wrote above is added to the stdlib, and that it becomes the blessed way, by convention, of composing errors. This approach works well for errors with additional methods but not so well for errors with additional fields or sentinel errors, like Errors with additional fields can be made to work with this approach by adding an unexported method and an exported helper. For example, let's examine and the OS package adds a func like then an error, under any number of combinations (with non-PathErrors), is still fully inspectable. Helpers like There's a small increase in API and implementation but everything works and is Go1 compatible. Sentinel errors are a bit more annoying. They could follow the same basic approach, but each one would need its own type, like I don't think that's strictly Go1 compatible, but the only code I can think of that that would break is something using reflect. Perhaps it would be permissible as part of the transition to Go2? |
|
It looks like the specific approach taken in the gist I posted relied on a bug ( #22073 ). If it wants to expose unexported methods, it would need to use embedding to handle the method promotion (once that is in place) and only create proxy methods as tie-breakers for exported methods. However that has several unhappy implications:
1 is unfortunate but non-essential. However, 2 means it just cannot handle unexported methods correctly. It needs to stick to only exported methods and always hide all unexported methods. So the approach in the gist is correct and it can discard values that don't add to the method set, but it's less useful than I'd hoped because it can't handle unexported methods. In order to handle unexported methods in the same manner as exported methods, it would need to either be a builtin or a function defined in the reflect package that circumvents the usual protections. |
|
I hit a use case for this while working on google/go-cmp#88. The gist of the problem is that b := new(bytes.Reader)
reflect.TypeOf(b) // *bytes.Reader
reflect.TypeOf(io.Reader(b)) // io.Reader expected, but got *bytes.ReaderThis causes an issue when doing something like: equateNotExist := cmp.Comparer(func(x, y error) bool { ... })
var e1, e2 error = syscall.ENOENT, os.ErrNotExist
cmp.Equal(e1, e2, equateNotExist) // falseThe user expected this to be true since there is a custom comparer that knows how to compare If you so much as pass even a pointer to the interface, then it works as expected: However, doing so is not intuitive at all. In an effort to make this more user-friendly, my approach was to try to promote the initial top-level value into an interface with the common set of methods to both values: // Determine the set of common methods.
type method struct {
Name string
Type reflect.Type
}
mm := make(map[method]int)
for i := 0; i < vx.NumMethod(); i++ {
mm[method{vx.Type().Method(i).Name, vx.Method(i).Type()}]++
}
for i := 0; i < vy.NumMethod(); i++ {
mm[method{vy.Type().Method(i).Name, vy.Method(i).Type()}]++
}
var ms []reflect.Method
for m, n := range mm {
if n == 2 {
ms = append(ms, reflect.Method{Name: m.Name, Type: m.Type})
}
}
// Promote to interface type with common method set.
iface := reflect.InterfaceOf(ms)
return vx.Convert(iface), vy.Convert(iface)but this does not work since there is no |
|
@dsnet I might misread this, but the documentation says that
That would seem to cover the case you mention? In particular this works. |
|
@Merovius, so as to keep this thread about |
|
I don't want to be a party pooper, but adding a feature like I skimmed over the issue's feedback and wondered: is there value in having a discussion 1. whether we should have this feature at all and 2. whether the benefits disproportionately outweigh the likely costs. |
|
@matttproud I assume you are talking about the case of adding methods to a type created by using the reflect package. E.g., adding methods to a type created via Personally I'm also not too worried about being able to add methods to a type created by reflect. That is nothing like Java's cglib. In Go, the code for the actual methods will be, essentially, a function literal. Adding methods to a type will amount to a different way of calling a function literal. Yes, the stack trace will be complex, but this is still much less powerful and much less complex than cglib. |
This comment was marked as off-topic.
This comment was marked as off-topic.
This comment was marked as off-topic.
This comment was marked as off-topic.
This comment was marked as off-topic.
This comment was marked as off-topic.
seankhliao
added
the
NeedsInvestigation
The text was updated successfully, but these errors were encountered: