cgo: SetUnhandledExceptionFilter is broken when using CGO on Windows x64

[arg0d]

What version of Go are you using (go version)?

$ go version
go version go1.17.6 windows/amd64

Does this issue reproduce with the latest release?

Yes

What operating system and processor architecture are you using (go env)?

go env Output

$ go env
set GO111MODULE=
set GOARCH=amd64
set GOBIN=
set GOCACHE=C:\Users\T1347\AppData\Local\go-build
set GOENV=C:\Users\T1347\AppData\Roaming\go\env
set GOEXE=.exe
set GOEXPERIMENT=
set GOFLAGS=
set GOHOSTARCH=amd64
set GOHOSTOS=windows
set GOINSECURE=
set GOMODCACHE=C:\Users\T1347\go\pkg\mod
set GONOPROXY=
set GONOSUMDB=
set GOOS=windows
set GOPATH=C:\Users\T1347\go
set GOPRIVATE=
set GOPROXY=https://proxy.golang.org,direct
set GOROOT=C:\workspace\sentryBugReport\go
set GOSUMDB=sum.golang.org
set GOTMPDIR=
set GOTOOLDIR=C:\workspace\sentryBugReport\go\pkg\tool\windows_amd64
set GOVCS=
set GOVERSION=go1.17.6
set GCCGO=gccgo
set AR=ar
set CC=gcc
set CXX=g++
set CGO_ENABLED=1
set GOMOD=NUL
set CGO_CFLAGS=-g -O2
set CGO_CPPFLAGS=
set CGO_CXXFLAGS=-g -O2
set CGO_FFLAGS=-g -O2
set CGO_LDFLAGS=-g -O2
set PKG_CONFIG=pkg-config
set GOGCCFLAGS=-m64 -mthreads -fmessage-length=0 -fdebug-prefix-map=C:\Users\T1347\AppData\Local\Temp\go-build2855518025=/tmp/go-build -gno-record-gcc-switches

What did you do?

The issue seems to be present on x64, but not on x86. This could be related to the differences between how x86 and x64 handles exceptions: https://sourceforge.net/p/mingw-w64/support-requests/29/

I have made a sample project to make it easier to test this issue: https://github.com/arg0d/sentryProblem

The gist of it is:

• Set exception handler with SetUnhandledExceptionFilter
• Use CGO to call another library
• The other library raises an exception

What did you expect to see?

Exception filter set by SetUnhandledExceptionFilter is invoked after the other library has raised an exception.

What did you see instead?

Exception filter set by SetUnhandledExceptionFilter did not get invoked when the other library raised an exception.

[bcmills]

CC @alexbrainman @hanchaoqun

[arg0d]

I have made an implementation that uses static linkage instead of dynamic. The branch is named static. The behavior is identical to dynamic linkage. Exception filter set with SetUnhandledExceptionFilter is not invoked.

[gopherbot]

Change https://go.dev/cl/525475 mentions this issue: runtime: support SetUnhandledExceptionFilter on Windows

[qmuntal]

It looks like unhandled exception can't flow to the callback defined by SetUnhandledExceptionFilter if the stack can't be unwind when the exception occurs. CGO unwinding is tricky, and the Go runtimes missed some frame pointer chaining that I've fixed in CL 525475.

The fix doesn't apply to windows/arm64, as it still doesn't support SEH unwinding, which is required to unwind the cgo stack (see #57302)

[gopherbot]

Change https://go.dev/cl/548319 mentions this issue: doc: document SetUnhandledExceptionFilter interop