net/http: Client.Do() panics when URL includes HTTP basic auth

[bradleyjkemp]

What version of Go are you using (go version)?

$ go version
go version go1.13.1 linux/amd64

Does this issue reproduce with the latest release?

Yes (and on play.golang.org)

What operating system and processor architecture are you using (go env)?

go env Output

$ go env
GO111MODULE=""
GOARCH="amd64"
GOBIN=""
GOCACHE="/home/bradley/.cache/go-build"
GOENV="/home/bradley/.config/go/env"
GOEXE=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="linux"
GONOPROXY=""
GONOSUMDB=""
GOOS="linux"
GOPATH="/home/bradley/go"
GOPRIVATE=""
GOPROXY="https://proxy.golang.org,direct"
GOROOT="/home/linuxbrew/.linuxbrew/Cellar/go/1.13.1/libexec"
GOSUMDB="sum.golang.org"
GOTMPDIR=""
GOTOOLDIR="/home/linuxbrew/.linuxbrew/Cellar/go/1.13.1/libexec/pkg/tool/linux_amd64"
GCCGO="gccgo"
AR="ar"
CC="gcc-5"
CXX="g++-5"
CGO_ENABLED="1"
GOMOD=""
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build417703076=/tmp/go-build -gno-record-gcc-switches"

What did you do?

Try to make a HTTP request using a URL including authentication/user info:

package main

import (
	"net/http"
	"net/url"
)

func main() {
	u, err := url.Parse("http://bradleyjkemp:[email protected]")
	if err != nil {
		panic(err)
	}
	req := &http.Request{
		Method: http.MethodGet,
		URL:    u,
	}
	http.DefaultClient.Do(req)
}

Runnable link: https://play.golang.org/p/ODXO4wBONJi

What did you expect to see?

Request is made successfully.

What did you see instead?

panic: assignment to entry in nil map

goroutine 1 [running]:
net/textproto.MIMEHeader.Set(...)
	/usr/local/go/src/net/textproto/header.go:22
net/http.Header.Set(...)
	/usr/local/go/src/net/http/header.go:37
net/http.send(0x85a510, 0x3aa6d0, 0x526ee0, 0x5bfa, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, ...)
	/usr/local/go/src/net/http/client.go:242 +0x3e0
net/http.(*Client).send(0x52d3a0, 0x85a510, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x2, 0x5bfa, ...)
	/usr/local/go/src/net/http/client.go:174 +0x120
net/http.(*Client).do(0x52d3a0, 0x85a510, 0x0, 0x0, 0x0, 0x5bfa)
	/usr/local/go/src/net/http/client.go:641 +0x480
net/http.(*Client).Do(...)
	/usr/local/go/src/net/http/client.go:509
main.main()
	/tmp/sandbox697076589/prog.go:17 +0xe0

[mvdan]

Why are you not using https://golang.org/pkg/net/http/#NewRequest? That sets up many fields, so I'm assuming that you're forgetting to initialise some of them properly.

[odeke-em]

Thank you for the report @bradleyjkemp!

This a new panic in Go1.13 that stems from the fact that invoking Header.Clone() when Header is nil returns a nil map as we implemented in CL https://go-review.googlesource.com/c/go/+/188022 to solve the inconsistency behavior of Clone as per #33141, but we didn't go through all the call sites to check that the map was nil before assignment.

The code posted up used to work in Go1.12 and before, now fails in Go1.13.
/cc @andybons @bradfitz @FiloSottile for awareness

@gopherbot please backport this issue to Go1.13.

[gopherbot]

Backport issue(s) opened: #34881 (for 1.12), #34882 (for 1.13).

Remember to create the cherry-pick CL(s) as soon as the patch is submitted to master, according to https://golang.org/wiki/MinorReleases.

[bradleyjkemp]

@mvdan you're right NewRequest() or even Get() are better to use for this use case. I actually found this panic while testing out a new fuzzing utility I'm working on (that lets you fuzz with structs rather than just []byte) hence the non-idiomatic repro case.

[gopherbot]

Change https://golang.org/cl/200977 mentions this issue: net/http: fix panic from nil Header.Clone