runtime: fixup darwin/arm64 framepointer assembly

[eliasnaur]

Quoting @cherrymui from CL 220588:

For example, in runtime/sys_darwin_arm64.s, function open_trampoline, the frame pointer is not saved. One possibility is to change the frame size (the $0 on the TEXT line) to 16, so the assembler will automatically allocate a frame and save the frame pointer, and remove the SUB and ADD to RSP. Do the same to other functions as well.

Another example is runtime/preempt_arm64.s, currently the frame pointer is only saved #ifdef GOOS_linux. We may want to do this for #ifdef GOOS_darwin as well.

[steeve]

That may explain why we saw weird unwinding on iOS.

[cherrymui]

I'll try to do this for Go 1.16. Thanks.

[steeve]

Hey @cherrymui, I just tried to do what you said, and ended up with a crash, using Go 1.13.8.

Here is the diff, just in case I'm missing something from your directives:

diff --git a/src/runtime/sys_darwin_arm64.s b/src/runtime/sys_darwin_arm64.s
index ac3ca74f63..63becfb630 100644
--- a/src/runtime/sys_darwin_arm64.s
+++ b/src/runtime/sys_darwin_arm64.s
@@ -15,14 +15,12 @@ TEXT notok<>(SB),NOSPLIT,$0
 	MOVD	R8, (R8)
 	B	0(PC)
 
-TEXT runtime·open_trampoline(SB),NOSPLIT,$0
-	SUB	$16, RSP
+TEXT runtime·open_trampoline(SB),NOSPLIT,$16
 	MOVW	8(R0), R1	// arg 2 flags
 	MOVW	12(R0), R2	// arg 3 mode
 	MOVW	R2, (RSP)	// arg 3 is variadic, pass on stack
 	MOVD	0(R0), R0	// arg 1 pathname
 	BL	libc_open(SB)
-	ADD	$16, RSP
 	RET
 
 TEXT runtime·close_trampoline(SB),NOSPLIT,$0

Also, what was the reason for the frame to be manually allocated, rather than via declaration?

[steeve]

Also, do you think asmcgocall should have it? We saw unwinders unable to go past it (main thread starting at this function).

[gopherbot]

Change https://golang.org/cl/241158 mentions this issue: runtime: adjust frame pointer on stack copy on ARM64

[aclements]

@cherrymui , you have some experimental CLs out for this. Should we still try to fix this in 1.16?

[cherrymui]

I could give it a try. More work needs to be done (besides the CL above), e.g. saving frame pointers in syscall wrappers.

Question: how can I test it? how do I know if I got everything?

There is also an open question about what to do at stack switch (e.g. systemstack, asmcgocall, etc.).

[randall77]

Not really a test per se, but there's now a vet check for this for amd64, and a TODO in that check for arm64.

https://github.com/golang/go/blob/master/src/cmd/vendor/golang.org/x/tools/go/analysis/passes/framepointer/framepointer.go#L34

Not an exhaustive check to be sure, but using it with go vet std cmd found some cases that needed to be fixed.

[randall77]

(And I have a really invasive test-only CL that rebases all the stack frame references to use FP instead of SP. That finds problems with the FP mighty quickly. Let me know if you want me to drag that CL back to life.)

[cherrymui]

Thanks, I could give that a try. There are probably not many cases that clobbers the frame pointer register on ARM64, because ARM64 has a lot of registers and frame pointer is R29.

Other case may be: manually allocate a frame without saving the frame pointer, or smash the frame pointer slot on stack.

[randall77]

smash the frame pointer slot on stack

There was one of these on amd64: https://go-review.googlesource.com/c/go/+/248262

[steeve]

@cherrymui I have a repro on amd64 at #40044 if that's any help, as the issue seems caused when the stack is swapped in asmcgocall and not forwarding the frame pointer list for some reason

not that I'm using libunwind but backtrace() does expose the issue also

[cherrymui]

when the stack is swapped in asmcgocall and not forwarding the frame pointer list for some reason

That is the "open question" I mentioned earlier. I don't know what is the correct answer here.

[aclements]

There's still a bunch of work to do here and we're late in the freeze, so don't want to introduce risk. Pushing to 1.17.

[ianlancetaylor]

Are we going to do anything on this issue for 1.17? Should we move this to Backlog? Thanks.

[cherrymui]

No, nothing is done for 1.17. Thanks.

[gopherbot]

Change https://go.dev/cl/481635 mentions this issue: runtime: save frame pointer to the stack in signal handlers for arm64

[gopherbot]

Change https://go.dev/cl/481636 mentions this issue: runtime: zero saved frame pointer when reusing goroutine stack on ARM