android: different ClientHello between debug and release mode

[bassosimone]

Now that I understand the problem, it's actually quite simple. (It's always like that.)

1. The TLS codebase chooses whether to put AES at top or not depending on hardware flags.

2. Hardware flags are initialized from the system auxv.

3. Reading the system auxv on Android depends on /proc/self/auxv being readable.

4. If /proc/self/auxv is not readable, the code will not initialize the hardware capabilities and it will just attempt to determine the page size.

5. It's documented that Go will not be able to open /proc/self/auxv in release mode (see x/mobile/app: cannot open /proc/self/auxv: Permission denied golang/go#9229) in many (all?) cases.

6. It's documented that in some networks not having the AES cipher at top causes a TLS protocol mismatch that prevents any TLS communication (see Deep packet inspection to classify V2Ray traffic in Dec, 2020 v2fly/v2ray-core#557).

The way in which v2ray developers fixed the issue seem to be using uTLS. We are already investigating this possibility. We are also considering modifying Go and using the modified Go for now, while we continue working on uTLS.

[bassosimone]

One option that I am considering is to patch the src/runtime of the Go used to build for Android as follows:

diff --git a/src/runtime/os_linux.go b/src/runtime/os_linux.go
index 058c7daf9c..9afc6ca2a3 100644
--- a/src/runtime/os_linux.go
+++ b/src/runtime/os_linux.go
@@ -234,6 +234,7 @@ func sysargs(argc int32, argv **byte) {
 			physPageSize = size
 		}
 		munmap(p, size)
+		sethwcap()
 		return
 	}
 	var buf [128]uintptr
diff --git a/src/runtime/sethwcap_android_arm64.go b/src/runtime/sethwcap_android_arm64.go
new file mode 100644
index 0000000000..ab15bcd429
--- /dev/null
+++ b/src/runtime/sethwcap_android_arm64.go
@@ -0,0 +1,38 @@
+// +build android
+// +build arm64
+
+// Copyright 2019 The Go Authors. All rights reserved.
+// Use of this source code is governed by a BSD-style
+// license that can be found in the LICENSE file.
+//
+// See https://go-review.googlesource.com/c/sys/+/197540/
+
+package runtime
+
+import "internal/cpu"
+
+/*
+#include <sys/auxv.h>
+
+// getauxval is not available on Android until API level 20. Link it as a weak
+// symbol and use other methods as fallback.
+unsigned long getauxval(unsigned long type) __attribute__((weak));
+*/
+import "C"
+
+// getauxval returns the value of the getauxval auxiliary vector getter, or
+// zero where the functionality is unavailable.
+func getauxval(t uint) uint {
+	if C.getauxval == C.NULL {
+		return 0
+	}
+	return uint(C.getauxval(C.ulong(t)))
+}
+
+// sethwcap uses getauxval to set cpu.HWCap on
+// linux_arm64 and does nothing otherwise.
+func sethwcap() {
+	// getauxval returns zero on failure and, if we end up here, we
+	// already have a zero cpu.HWCap anyway.
+	cpu.HWCap = getauxval(_AT_HWCAP)
+}
diff --git a/src/runtime/sethwcap_otherwise.go b/src/runtime/sethwcap_otherwise.go
new file mode 100644
index 0000000000..a6bffbef97
--- /dev/null
+++ b/src/runtime/sethwcap_otherwise.go
@@ -0,0 +1,7 @@
+// +build !android !arm64
+
+package runtime
+
+// sethwcap uses getauxval to set cpu.HWCap on
+// linux_arm64 and does nothing otherwise.
+func sethwcap() {}

This diff is annoying because it makes runtime depend on C. But, if it's working as intended, we have now a clean way to correctly initialize the hardware capabilities of the processor on Android arm64 devices. Tomorrow I'll test.

(I also explored forcing AES as the default in every case, but it makes me nervous knowing that the non-ASM implementation of AES in Golang is not constant time. I do not fully understand the implications but it does not seem right to unconditionally promote AES to be on top when there's no hardware AES support.)

[bassosimone]

It may be that the right fix for this issue is to patch golang/mobile rather than golang/go. The current solution is patching golang/go and it seems the effort to research how to patch golang/mobile would further postpone shipping a fix. We want to engineer the next release using the current patch for golang.go (implemented in ooni/probe-cli#311) and to continue investigating whether there are simpler fixes to this issue after that. We should also consider whether there is a way to upstream the fix.

[bassosimone]

The PR at ooni/probe-cli#315 introduces a new build script that takes care of using our Go fork when we are compiling for Android. It might be that there are better ways to do that, perhaps patching golang/mobile instead. For now, it's fine to use this solution, which is based on a variation of the above patch (see github.com/ooni/go for more info). We are going to close this issue soon.