all: figure out OpenBSD support story

[mdempsky]

Unlike other Go target OSes, OpenBSD doesn't define a stable long-term ABI that Go can utilize. E.g., Linux provides a stable kernel ABI, and Solaris and Windows provides stable userland ABIs, but OpenBSD only maintains userland C API compatibility. Each release has its own one-off kernel and userland ABI.

That said, OpenBSD does (best effort) try to maintain backwards compatibility between consecutive releases, and it doesn't gratuitously break compatibility either. So in combination with the Go runtime's relatively modest OS requirements and OpenBSD's non-first-class port status, treating "OpenBSD" as a single target OS has generally worked out okay so far.

The main hiccup is package syscall. Since this package exposes kernel ABI structs like syscall.IfData, this struct definition can't be correct for all OpenBSD releases.

Another example is how Go implements TLS for cgo support. Because OpenBSD doesn't (yet) support ELF's PT_TLS segment type, cmd/link knows to not emit PT_TLS on OpenBSD and runtime/cgo instead provides a pthread_create wrapper that adds enough TLS support for the Go runtime to work correctly. But once OpenBSD does implement PT_TLS, we'll need to change both cmd/link and runtime/cgo, and there will be no way to produce a single Go executable that runs on both pre- and post-PT_TLS OpenBSD releases.

On the upside, OpenBSD upstream only maintains three releases: the current development version (referred to as "-current"), and the last two versioned releases. Upstream provides no support for versioned releases older than a year, so Go shouldn't feel obligated to either.

This issue is about deciding a way to handle this. Brainstormed solutions:

1. Treat OpenBSD releases as separate OS targets. E.g., we could have GOOS=openbsd58 and GOOS=openbsd59 to target OpenBSD 5.8 and 5.9, respectively, and GOOS=openbsd could just default to the most recent supported version. Alternatively, we could just use GOOS=openbsd, but add a GOOPENBSD version variable like GOARM.
2. Only explicitly support one OpenBSD release. If compiled binaries happen to work on other releases too, that's just a bonus.

[binarycrusader]

While Solaris generally guarantees binary compatibility, it doesn't guarantee source compatibility (obviously, that's not generally true for POSIX interfaces).

This means that header file definitions can change incompatibly between versions, which is going to start creating problems in the future as Solaris starts to change significantly, since the generated definitions that Go has checked into the gate don't actually match what version of Solaris that Go might be built on.

I've largely been ignoring this problem for the moment since it only tends to affect very specific areas of the system, but a better answer is needed. This seems like it might fit to a certain degree with what is mentioned here.

To add to this, historical OpenSolaris-based derivatives (e.g. SmartOS) are about five years behind Oracle Solaris in terms of ABI, so the ability to expose those will be necessary even if they are not supported by older derivatives. Supporting only the most current production release of Solaris might be one choice, but the actual support lifetimes for a given Solaris release are measured in decades, so that might reduce Go's usefulness significantly on those platforms.

[binarycrusader]

/cc @4ad

[4ad]

Regarding Solaris, we should use ELF symbol versioning, targetting
the oldest supported release (currently some old OpenSolaris build).

Obviously when we regenerate the Go definitions we should do it
on that particular old release.

That way, what we do is no more different that compiling a C program
on an old release and running it on the new release. Because Solaris
has binary compatibility for old binaries, this should just work.

This is very easy to implement. Just add symbol versions to
cgo_import_dynamic lines.

[4a6f656c]

I think the OpenBSD support story is already reasonably well defined - as you note, OpenBSD only officially supports two releases (the current release and the past release - for now, 5.9 and 5.8). Since the Go openbsd port has existed, we've always targeted and attempted to support the same releases. Many of the Go impacting changes made in OpenBSD have also been done in such a way that this has been possible. As far as I recall, there has only been one case in the past 5 years where this has not been possible (the 64-bit time_t conversion), at which point Go has only supported the most current release.

At this stage I see no reason to diverge from this approach. The syscall package is the only area that I don't have a good answer for - we're largely stuck with that due to the Go API promise.

But once OpenBSD does implement PT_TLS, we'll need to change both cmd/link and runtime/cgo, and there will be no way to produce a single Go executable that runs on both pre- and post-PT_TLS OpenBSD releases.

This is somewhat inaccurate - I have a single Go runtime that will work both pre-PT_TLS and post-PT_TLS, however the linked binary is dependent on a version of libpthread that is specific to a particular release, so it will fail to load libpthread if compiled on 5.9 and then run on 5.8 (same goes if you compile on 5.8 and run on pristine 5.9). If you upgraded from 5.8 to 5.9 and have the libraries from 5.8 available, it will would likely be possible to run a Go binary compiled on 5.8. That said, binary compatibility is not what OpenBSD targets and is not really what Go should be aiming for here. Having a Go release (e.g. 1.7) that works on both 5.9 (pre-PT_TLS) and 6.0 (post-PT_TLS) is what we want to achieve (and that's possible to do).

[mikioh]

kernel ABI structs like syscall.IfData,

Yup, ABI changes between major releases on BSD variants are pretty common and both syscall and x/sys/unix cannot follow the change completely. I think we should replace what the syscall package provides with new (external and more feature specific) packages in another way. For example, for IP routing stuff including if_data, rt_stats and rt_metrics: https://go-review.googlesource.com/22446/.

[4a6f656c]

@bradfitz what are the next steps here?

[4a6f656c]

@bradfitz ping?

[bradfitz]

@4a6f656c, sounds like the policy is that each released version of Go supports the past two OpenBSD releases.

Just document that on https://github.com/golang/go/wiki/OpenBSD and https://github.com/golang/go/wiki/MinimumRequirements#openbsd and then you can close this issue.

Each Go release will document which versions of OpenBSD we currently support, and let's keep the table at https://github.com/golang/go/wiki/OpenBSD up to date.

[kevinburke]

(People reading this issue may be interested in this commit, which adds PT_TLS support for Go1.9 9417c02)

[ertw]

I added longterm support information to https://github.com/golang/go/wiki/OpenBSD, and https://github.com/golang/go/wiki/MinimumRequirements#openbsd already had the correct information. I think this is okay to close now.