reflect: StructOf doesn't generate wrapper methods for embedded fields

[adonovan]

% go version
go version devel +bbd1dcd Wed Jun 1 21:32:46 2016 +0000 linux/amd64
% cat ~/a.go
package main

import (
        "bytes"
        "io"
        "reflect"
        "sync"
)

func main() {
        t := reflect.StructOf([]reflect.StructField{
                {Type: reflect.TypeOf(sync.Mutex{}), Anonymous: true},
                {Type: reflect.TypeOf(bytes.Buffer{}), Anonymous: true},
        })
        x := reflect.New(t).Interface()
        _ = x.(io.Writer)
}

% go run ~/a.go
panic: interface conversion: *truct { sync.Mutex; bytes.Buffer } is not io.Writer: missing method Write

I expected that x would have generated wrapper methods for Mutex.Lock, etc.

Interestingly, adding this declaration inside func main (not at package level):

var _ interface{} = new(struct {
      sync.Mutex
      bytes.Buffer
})

causes the program to succeed. Presumably it causes the compiler to generate the correct type information, which reflect.StructOf then finds.

[ianlancetaylor]

This is a known deficiency, but it should be documented (or we should unexport StructOf until it is fixed).

CC @sbinet

[sbinet]

let me send a CL that documents this temporary (ie: go-1.7) limitation.

[gopherbot]

CL https://golang.org/cl/23681 mentions this issue.

[crawshaw]

cc me. I had a prototype of this that I didn't send out in the 1.7 cycle, hopefully I can get to it in 1.8.

[rsc]

Too late.

[crawshaw]

Requires creating methods at run time, which is discussed in #16522.

[bradfitz]

Dup with more repros in #20824

[gopherbot]

CL https://golang.org/cl/47035 mentions this issue.

[mdempsky]

FYI, as of CL 100846, methods are now sorted with exported names before non-exported names, and there's an extra field to track the number of exported methods.

The current code works because it only allows exported methods from one type, so they're already in the same sort order, and we don't need to worry about counting how many exported methods there are. If either of those constraints change, we'll need to change the code (e.g., to sort methods and/or count the exported methods). I've left TODOs to mark where I expect this needs to be done.

Also, one other comment: currently, StructOf only allows promoting methods from the first field, but it could be generalized slightly to allowing promoting methods of any field with offset==0. This would allow methods from multiple zero-width types to be promoted.

[gopherbot]

Change https://golang.org/cl/121316 mentions this issue: reflect: prevent additional StructOf embedded method cases

[cosmos72]

while implementing reflect.NamedOf, I noticed that reflect.StructOf creates wrapper methods for embedded interfaces, and I was surprised that declaring methods at runtime would simply require calling reflect.MakeFunc (see https://github.com/golang/go/blob/master/src/reflect/type.go#L2386).

Alas, it seems not.

The following example crashes with SIGSEGV on both my local Go 1.11.4 installation and on playground https://play.golang.org/p/jny1YKZsHlp

package main

import (
	"fmt"
	"reflect"
)

type MyInt int

func (n MyInt) String() string {
	return fmt.Sprint("MyInt(value = ", int(n), ")")
}

func main() {
	embed()
	embed_reflect()
}

func embed() {
	type S = struct {
		fmt.Stringer
	}
	s := S{
		MyInt(3),
	}
	fmt.Println(s.Stringer.String()) // works, prints 3

	var i fmt.Stringer = s
	fmt.Println(i.String()) // works, prints 3
}

func embed_reflect() {
	tstringer := reflect.TypeOf((*fmt.Stringer)(nil)).Elem()

	t := reflect.StructOf(
		[]reflect.StructField{
			reflect.StructField{
				Name:      tstringer.Name(),
				Type:      tstringer,
				Anonymous: true,
			},
		},
	)
	v := reflect.New(t).Elem()
	v.Field(0).Set(reflect.ValueOf(MyInt(4)))

        i := v.Field(0).Interface().(fmt.Stringer)
	fmt.Println(i.String()) // works, prints 4

	i = v.Interface().(fmt.Stringer)
	fmt.Println(i.String()) // crashes
}

on my local Go 1.11.4, the output is:

MyInt(value = 3)
MyInt(value = 3)
MyInt(value = 4)
unexpected fault address 0xc00000a080
fatal error: fault
[signal SIGSEGV: segmentation violation code=0x2 addr=0xc00000a080 pc=0xc00000a080]

goroutine 1 [running]:
runtime.throw(0x4eebf7, 0x5)
	/usr/local/go/src/runtime/panic.go:608 +0x72 fp=0xc000055e00 sp=0xc000055dd0 pc=0x4288a2
runtime.sigpanic()
	/usr/local/go/src/runtime/signal_unix.go:397 +0x275 fp=0xc000055e50 sp=0xc000055e00 pc=0x43b7f5
main.embed_reflect()
	/home/max/go/src/github.com/cosmos72/gomacro/_example/namedof.go:51 +0x386 fp=0xc000055f88 sp=0xc000055e50 pc=0x4b5486
main.main()
	/home/max/go/src/github.com/cosmos72/gomacro/_example/namedof.go:16 +0x25 fp=0xc000055f98 sp=0xc000055f88 pc=0x4b4f65
runtime.main()
	/usr/local/go/src/runtime/proc.go:201 +0x207 fp=0xc000055fe0 sp=0xc000055f98 pc=0x42a217
runtime.goexit()
	/usr/local/go/src/runtime/asm_amd64.s:1333 +0x1 fp=0xc000055fe8 sp=0xc000055fe0 pc=0x452a31

on playground https://play.golang.org/p/jny1YKZsHlp the output is instead:

MyInt(value = 3)
MyInt(value = 3)
MyInt(value = 4)
panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0xffffffff addr=0x0 pc=0x40c0e0]

goroutine 1 [running]:
main.embed_reflect()
	/tmp/sandbox101416977/main.go:51 +0x400
main.main()
	/tmp/sandbox101416977/main.go:16 +0x40

I would expect that either reflect.StructOf fails (due to unsupported interface embedding) or that its returned type does not implement fmt.Stringer - I was surely not expecting that the method call crashes at runtime.

[mihaiav]

83092a4#diff-cdbb65f700222dd514ff85e972c88be2R2470 broke my program after I've updated the Go version.

I didn't use any of the the struct methods. The struct was built only to create dynamic data structures matching a specific format when encoded with various encoders such encoding/json.

[ianlancetaylor]

@mihaiav I'm sorry that happened but given the choice between "incorrectly claim we did what the program requested" and "give an error" I think it's better to give an error.

You should be able to fix your program by making a regular field rather than an embedded field.

[lestrrat]

I was reading https://groups.google.com/forum/#!topic/golang-nuts/e9ij7B_c5DE and thought this interesting, so I wrote some test cases: https://play.golang.org/p/Z93JIKEpkgv https://play.golang.org/p/C8nTXcQ7UUN

Seems like if
(1) The wrapper type is given as value
(2) The embedded type is a value
(3) But the method receiver (in the embedded type) is declared as a pointer

We cannot call the method from the embedded type.

[dxq174510447]

while implementing reflect.NamedOf, I noticed that reflect.StructOf creates wrapper methods for embedded interfaces, and I was surprised that declaring methods at runtime would simply require calling reflect.MakeFunc (see https://github.com/golang/go/blob/master/src/reflect/type.go#L2386).

Alas, it seems not.

The following example crashes with SIGSEGV on both my local Go 1.11.4 installation and on playground https://play.golang.org/p/jny1YKZsHlp

package main

import (
	"fmt"
	"reflect"
)

type MyInt int

func (n MyInt) String() string {
	return fmt.Sprint("MyInt(value = ", int(n), ")")
}

func main() {
	embed()
	embed_reflect()
}

func embed() {
	type S = struct {
		fmt.Stringer
	}
	s := S{
		MyInt(3),
	}
	fmt.Println(s.Stringer.String()) // works, prints 3

	var i fmt.Stringer = s
	fmt.Println(i.String()) // works, prints 3
}

func embed_reflect() {
	tstringer := reflect.TypeOf((*fmt.Stringer)(nil)).Elem()

	t := reflect.StructOf(
		[]reflect.StructField{
			reflect.StructField{
				Name:      tstringer.Name(),
				Type:      tstringer,
				Anonymous: true,
			},
		},
	)
	v := reflect.New(t).Elem()
	v.Field(0).Set(reflect.ValueOf(MyInt(4)))

        i := v.Field(0).Interface().(fmt.Stringer)
	fmt.Println(i.String()) // works, prints 4

	i = v.Interface().(fmt.Stringer)
	fmt.Println(i.String()) // crashes
}

on my local Go 1.11.4, the output is:

MyInt(value = 3)
MyInt(value = 3)
MyInt(value = 4)
unexpected fault address 0xc00000a080
fatal error: fault
[signal SIGSEGV: segmentation violation code=0x2 addr=0xc00000a080 pc=0xc00000a080]

goroutine 1 [running]:
runtime.throw(0x4eebf7, 0x5)
	/usr/local/go/src/runtime/panic.go:608 +0x72 fp=0xc000055e00 sp=0xc000055dd0 pc=0x4288a2
runtime.sigpanic()
	/usr/local/go/src/runtime/signal_unix.go:397 +0x275 fp=0xc000055e50 sp=0xc000055e00 pc=0x43b7f5
main.embed_reflect()
	/home/max/go/src/github.com/cosmos72/gomacro/_example/namedof.go:51 +0x386 fp=0xc000055f88 sp=0xc000055e50 pc=0x4b5486
main.main()
	/home/max/go/src/github.com/cosmos72/gomacro/_example/namedof.go:16 +0x25 fp=0xc000055f98 sp=0xc000055f88 pc=0x4b4f65
runtime.main()
	/usr/local/go/src/runtime/proc.go:201 +0x207 fp=0xc000055fe0 sp=0xc000055f98 pc=0x42a217
runtime.goexit()
	/usr/local/go/src/runtime/asm_amd64.s:1333 +0x1 fp=0xc000055fe8 sp=0xc000055fe0 pc=0x452a31

on playground https://play.golang.org/p/jny1YKZsHlp the output is instead:

MyInt(value = 3)
MyInt(value = 3)
MyInt(value = 4)
panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0xffffffff addr=0x0 pc=0x40c0e0]

goroutine 1 [running]:
main.embed_reflect()
	/tmp/sandbox101416977/main.go:51 +0x400
main.main()
	/tmp/sandbox101416977/main.go:16 +0x40

I would expect that either reflect.StructOf fails (due to unsupported interface embedding) or that its returned type does not implement fmt.Stringer - I was surely not expecting that the method call crashes at runtime.

i meet the same question, how to solve it at last ?

[cosmos72]

I am not aware of any general mechanism to create methods at runtime - if I remember correctly, it would require code generation.

The function reflect.StructOf can "create" - actually "pretend to create" - only wrapper methods for some special cases that allow reusing the existing methods of embedded fields.

And that's not the only difficulty that reflect.NamedOf is expected to solve: there's also the issue of recursive types - see issue #39717

[infogulch]

A wrapper struct with one embedded member that has methods behaves differently if defined at compile time vs constructed at runtime. In particular, methods defined on the inner struct sometimes cannot be found with .MethodByName on a wrapper struct created by reflect.StructOf, whereas they can be found with .MethodByName on a wrapper struct defined at compile time.

This playground demonstrates: https://go.dev/play/p/lDE49fZuk5N

Lookup methods on wrapper struct created at runtime with reflect: (struct { S main.S }) -- (Some methods are missing)
Lookup ValRcv() with value receiver  : 0x47d100 (found, OK)
Lookup ValRcv() with pointer receiver: <invalid reflect.Value> (missing, unexpected!)
Lookup PtrRcv() with value receiver  : <invalid reflect.Value> (missing, OK)
Lookup PtrRcv() with pointer receiver: <invalid reflect.Value> (missing, unexpected!)

Lookup methods on Wrapper struct defined at compile time: (main.Wrapper) -- (All OK)
Lookup ValRcv() with value receiver  : 0x47d100 (found, OK)
Lookup ValRcv() with pointer receiver: 0x47d100 (found, OK)
Lookup PtrRcv() with value receiver  : <invalid reflect.Value> (missing, OK)
Lookup PtrRcv() with pointer receiver: 0x47d100 (found, OK)

In a debug session I can see that ft.Uncommon().Methods() in the relevant case branch only finds the value receiver method, not the pointer receiver method. I'm not familiar with Go's reflect and abi types; maybe the value and pointer receiver methods need to be queried and appended separately?