crypto/x509: not finding CA from SystemCertPool

[wzzhu]

cat main.go

package main
  
import (
        "log"
        "net/http"
)

func main() {
        if _, err := http.Get("https://www.1823.gov.hk/common/ical/sc.json"); err != nil {
                log.Fatalf("Error downloading HK calendar: %v", err)
        }
}

go run main.go

Error downloading HK calendar: Get "https://www.1823.gov.hk/common/ical/sc.json": x509: certificate signed by unknown authority

The simple http get to download a JSON through https protocol failed.
If tracing it down into the x509 package, will find that it does not find the right CA in SystemCertPool. But the CA is valid in Safari and it is valid in MacOS KeyChain's SystemRoots (in attached screen shot)

What version of Go are you using (go version)?

$ go version
go version go1.16.3 darwin/amd64

Does this issue reproduce with the latest release?

Yes

What operating system and processor architecture are you using (go env)?

go env Output

$ go env
GO111MODULE=""

GOARCH="amd64"

GOBIN=""

GOCACHE="/Users/wzzhu/Library/Caches/go-build"

GOENV="/Users/wzzhu/Library/Application Support/go/env"

GOEXE=""

GOFLAGS=""

GOHOSTARCH="amd64"

GOHOSTOS="darwin"

GOINSECURE=""

GOMODCACHE="/Users/wzzhu/go/pkg/mod"

GONOPROXY=""

GONOSUMDB=""

GOOS="darwin"

GOPATH="/Users/wzzhu/go"

GOPRIVATE=""

GOPROXY="https://proxy.golang.org,direct"

GOROOT="/usr/local/Cellar/go/1.16.3/libexec"

GOSUMDB="sum.golang.org"

GOTMPDIR=""

GOTOOLDIR="/usr/local/Cellar/go/1.16.3/libexec/pkg/tool/darwin_amd64"

GOVCS=""

GOVERSION="go1.16.3"

GCCGO="gccgo"

AR="ar"

CC="clang"

CXX="clang++"

CGO_ENABLED="1"

GOMOD="/dev/null"

CGO_CFLAGS="-g -O2"

CGO_CPPFLAGS=""

CGO_CXXFLAGS="-g -O2"

CGO_FFLAGS="-g -O2"

CGO_LDFLAGS="-g -O2"

PKG_CONFIG="pkg-config"

GOGCCFLAGS="-fPIC -arch x86_64 -m64 -pthread -fno-caret-diagnostics -Qunused-arguments -fmessage-length=0 -fdebug-prefix-map=/var/folders/57/ntq8cyn57qq_2b532lr9cl_40000gn/T/go-build931250835=/tmp/go-build -gno-record-gcc-switches -fno-common"

What did you do?

run the main.go above

What did you expect to see?

It should download the json file correctly.

What did you see instead?

It failed to download due to error in verifying the SSL certificate.

[ZekeLu]

It seems that the server is misconfigured and serves the wrong intermediate certs.

$ openssl s_client -connect www.1823.gov.hk:443
CONNECTED(00000003)
depth=0 C = HK, ST = Hong Kong, L = Hong Kong, O = Hong Kong SAR Government, OU = 0002175468, OU = 000000000000000000000000EFFO, OU = Hongkong Post e-Cert (Server), OU = Efficiency Office, CN = www.1823.gov.hk
verify error:num=20:unable to get local issuer certificate
verify return:1
depth=0 C = HK, ST = Hong Kong, L = Hong Kong, O = Hong Kong SAR Government, OU = 0002175468, OU = 000000000000000000000000EFFO, OU = Hongkong Post e-Cert (Server), OU = Efficiency Office, CN = www.1823.gov.hk
verify error:num=21:unable to verify the first certificate
verify return:1
---
Certificate chain
 0 s:C = HK, ST = Hong Kong, L = Hong Kong, O = Hong Kong SAR Government, OU = 0002175468, OU = 000000000000000000000000EFFO, OU = Hongkong Post e-Cert (Server), OU = Efficiency Office, CN = www.1823.gov.hk
   i:C = HK, ST = Hong Kong, L = Hong Kong, O = Hongkong Post, CN = Hongkong Post e-Cert SSL CA 3 - 17
 1 s:C = HK, ST = Hong Kong, L = Hong Kong, O = Hongkong Post, CN = Hongkong Post e-Cert CA 1 - 15
   i:C = HK, O = Hongkong Post, CN = Hongkong Post Root CA 1
 2 s:C = HK, O = Hongkong Post, CN = Hongkong Post Root CA 1
   i:C = HK, O = Hongkong Post, CN = Hongkong Post Root CA 1

This issue is probably a duplicate of #31791.

[cagedmantis]

/cc @FiloSottile

[FiloSottile]

Duplicate of #35631.