runtime: SIGILL: illegal instruction when running any Go apps (notably GIT) on High Sierra

[NerdyDeedsLLC]

What version of Go are you using (go version)?

I'm NOT. At least, not intentionally. I have reinistalled go some 10 times now, though, in an effort to figure out why, after a random Homebrew update I found myself unable to hit any of my repos. At present, I'm running (pinned, per 37478's instructions, though that was after undertaking a half dozen versions of 1.4)

go version go1.13.15 darwin/amd64

Does this issue reproduce with the latest release?

No clue. I'm not able to get the latest release.

What operating system and processor architecture are you using (go env)?

go env Output

GO111MODULE=""
GOARCH="amd64"
GOBIN=""
GOCACHE="/Users/z/Library/Caches/go-build"
GOENV="/Users/z/Library/Application Support/go/env"
GOEXE=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="darwin"
GONOPROXY=""
GONOSUMDB=""
GOOS="darwin"
GOPATH="/Users/z/go"
GOPRIVATE=""
GOPROXY="https://proxy.golang.org,direct"
GOROOT="/usr/local/Cellar/[email protected]/1.13.15/libexec"
GOSUMDB="sum.golang.org"
GOTMPDIR=""
GOTOOLDIR="/usr/local/Cellar/[email protected]/1.13.15/libexec/pkg/tool/darwin_amd64"
GCCGO="gccgo"
AR="ar"
CC="clang"
CXX="clang++"
CGO_ENABLED="1"
GOMOD=""
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fno-caret-diagnostics -Qunused-arguments -fmessage-length=0 -fdebug-prefix-map=/var/folders/hq/6j7zt7pj4dd4h94mjrwx750m0000gp/T/go-build519708015=/tmp/go-build -gno-record-gcc-switches -fno-common"

What did you do?

I brew installed mackup, which, as per typical behavior, brew updated. This in turn has resulted in the following error when I run any git-related commands:

What did you expect to see?

My git status

What did you see instead?

$git status

SIGILL: illegal instruction
PC=0x1065350 m=0 sigcode=1

goroutine 1 [running, locked to thread]:
runtime.asyncPreempt()
	/usr/local/Cellar/go/1.14/libexec/src/runtime/preempt_amd64.s:8 fp=0xc0000cdc30 sp=0xc0000cdc28 pc=0x1065350
regexp/syntax.Parse(0x13eec0f, 0x32, 0x2000d4, 0xc0000cddf8, 0xc0000cde00, 0x100d856)
	/usr/local/Cellar/go/1.14/libexec/src/regexp/syntax/parse.go:788 +0xeb fp=0xc0000cdd40 sp=0xc0000cdc30 pc=0x10e571b
regexp.compile(0x13eec0f, 0x32, 0x10000d4, 0x179f7d0, 0x0, 0x18)
	/usr/local/Cellar/go/1.14/libexec/src/regexp/regexp.go:170 +0x5a fp=0xc0000cddd8 sp=0xc0000cdd40 pc=0x10f86aa
regexp.Compile(...)
	/usr/local/Cellar/go/1.14/libexec/src/regexp/regexp.go:133
regexp.MustCompile(0x13eec0f, 0x32, 0x147b5c0)
	/usr/local/Cellar/go/1.14/libexec/src/regexp/regexp.go:309 +0x4b fp=0xc0000cde60 sp=0xc0000cddd8 pc=0x10f945b
gopkg.in/yaml%2ev2.init()
	vendor/gopkg.in/yaml.v2/encode.go:298 +0x1fe fp=0xc0000cdec8 sp=0xc0000cde60 pc=0x12d194e
runtime.doInit(0x1688d80)
	/usr/local/Cellar/go/1.14/libexec/src/runtime/proc.go:5414 +0x8a fp=0xc0000cdef8 sp=0xc0000cdec8 pc=0x1042a2a
runtime.doInit(0x168b280)
	/usr/local/Cellar/go/1.14/libexec/src/runtime/proc.go:5409 +0x57 fp=0xc0000cdf28 sp=0xc0000cdef8 pc=0x10429f7
runtime.doInit(0x168c1c0)
	/usr/local/Cellar/go/1.14/libexec/src/runtime/proc.go:5409 +0x57 fp=0xc0000cdf58 sp=0xc0000cdf28 pc=0x10429f7
runtime.doInit(0x16866a0)
	/usr/local/Cellar/go/1.14/libexec/src/runtime/proc.go:5409 +0x57 fp=0xc0000cdf88 sp=0xc0000cdf58 pc=0x10429f7
runtime.main()
	/usr/local/Cellar/go/1.14/libexec/src/runtime/proc.go:190 +0x1ce fp=0xc0000cdfe0 sp=0xc0000cdf88 pc=0x1035f0e
runtime.goexit()
	/usr/local/Cellar/go/1.14/libexec/src/runtime/asm_amd64.s:1373 +0x1 fp=0xc0000cdfe8 sp=0xc0000cdfe0 pc=0x1063af1

rax    0xc0000a6300
rbx    0x32
rcx    0x32
rdx    0x13eec0f
rdi    0x0
rsi    0x13eec0f
rbp    0xc0000cdd30
rsp    0xc0000cdc28
r8     0x0
r9     0x0
r10    0x2
r11    0x11
r12    0xf1
r13    0x0
r14    0x1463254
r15    0x0
rip    0x1065350
rflags 0x10202
cs     0x2b
fs     0x0
gs     0x0

I find this particularly remarkable because there IS NO /usr/local/Cellar/go/1.14/ DIRECTORY on the box (though I don't doubt at one time there likely WAS before I uninstalled go in the [email protected] fix attempt). Further:

echo "$PATH"
/usr/local/opt/[email protected]/bin:/usr/local/sbin:/usr/local/opt/ruby/bin:/Users/z/.nvm/versions/node/v12.8.1/bin:/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin

I tried a go clean -cache as well.

Finally, if I brew uninstall git hub then brew install git hub it WILL restore git's functionality... for about 15 minutes or so.

Final noteworthy piece here: I'm not a Go developer. I've NEVER used Go. I AM a professional software engineer, and I entirely grasp that other applications that I'm utilizing have been built IN Go, but I myself have done ZERO in the way of tinkering environmentally, nor do I have any background or experience in the language. I cite this only to point out that people completely outside the community are being impacted in new and exciting ways.

I found and read 37459 independently, and I've tried to effect any of the suggestions I've found from both it and the other issues that relate to it. I'm fairly confident this remains the same Darwin/amd64 on pre-AVX chipset error, but nothing else I've heard of seems to be seeing their git (or, I suspect more likely, their homebrew) impacted.

[randall77]

#34759 was fixed in 1.14.1.
Are you running on an old mac that's pre-AVX?

You'll need to get any of the software that you use that's built with go 1.14 to instead build with at least go1.14.1. I'm not sure how you would go about that.

[NerdyDeedsLLC]

1. This box IS in fact a pre-AVX box, yes.
2. I'm aware that it was declared fixed in 1.14.1.
3. The issue here is, as a consequence of a normal usage of Homebrew - entirely unrelated to Go - this issue has manifested itself.

I haven't the foggiest idea what software I'm using that was built with 1.14. Nor do I have even the vaguest inkling as to how to ascertain same. When I attempt to run anything within the git program, I get the above SIGILL: illegal instruction, which I suppose serves as little more than a symptom of the overarching issue. Although, again: reinstalling git will have it functioning again... for 10-15 minutes, until this goroutine 1 reinitializes again.

What I'm most baffled by is the fact that whatever's throwing the error doesn't appear to exist:

I mean, sure, I can get a missing file throwing an error, but rarely does it come back with a specific LINE NUMBER of said absentee file.

[randall77]

That code path is presumably on the machine that built the binary in question, not the machine it is running on.
Although maybe I don't understand homebrew enough. Does it build binaries, or just download them? If it builds binaries, does it leave tools (like /usr/local/Cellar/go/1.14) around, or does it clean them up after it is done building?

You're right, there's no obvious indication of what binary this is. It's some binary that imports gopkg.in/yaml%2ev2. (Normally you would see a main in the traceback with a package path that indicates the binary, but this code is crashing during package initialization which is before main is called.)

The only thing I can think of is to run your git command with strace, and look for what files it opens. One of those files is the bad binary in question.

[randall77]

Note that this is really a general homebrew problem - if some dependency of something you installed is bad, how do you tell homebrew that and ask it to rebuild with a different version of said dependency?

[NerdyDeedsLLC]

Copy that. I'll see what I can produce and report back. Please understand: this is not the hill I'm gonna die on here; this came to my attention on an older iMac I honestly just use as a MONITOR 99% of the time. I'm reporting this here because in my attempt to research the matter, I ran into the 1.14 AVX issue. Since I'd taken no action that was related to Go, but Go was being flagged as the problem, I thought you guys might be interested, too.

[NerdyDeedsLLC]

Alas, homebrew is not known for its "version selection" capabilities (nothing like node, as a for-instance). Unless the publisher intentionally creates a different distro of the application - as you guys did with [email protected] - what you have is what you get. There's ways to rebuild a package off an older version, but only if one has the cached version locally, or one knows where to find the older source code.

Mainly, I'm assuming I'm not going to be the only one impacted by this, and most of the discussions surrounding the resolutions presuppose a great deal more knowledge surrounding matter than most - or indeed, I myself - possess.

[randall77]

Sorry about all this.
This issue is definitely good documentation to have if other people run into a similar problem.

I'm going to close, as I don't think there's anything the Go project can do here.

[NerdyDeedsLLC]

Fair enough. For those other people running into a similar problem: it appears the offending application in question is Hub, GitHub's (shockingly-obscure) own command-line extension to Git (which is, perhaps unsurprisingly under the circumstances, built in Go). Part of Hub's installation overrides the default git behavior, preprocessing any commands sent to git to see if they're hub-specific prior to releasing them to the vanilla git cli. It then aliases the hub behaviors OVER the git command, thereby necessitating it to process the offending go commands prior to ever getting to the git portions.

I'm going to go open an issue with the guys at hub now I've worked out what I can actually report to anyone and to whom, but for anyone stuck similarly to the way I've been the last couple days, unalias git appears to have solved the problem on this box (as would brew uninstall hub, I imagine).

@randall77 - Thank you, Keith. I've been beating me head against a wall for hours on this one. You pointed me right towards where I needed to be looking, intentionally or not. :) Thanks for taking the time! <3