net/http: Request to certain server hangs, works fine with curl #21301

wjdp · 2017-08-03T21:40:42Z

Please answer these questions before submitting your issue. Thanks!

What version of Go are you using (`go version`)?

go version go1.8.3 linux/amd64

What operating system and processor architecture are you using (`go env`)?

GOARCH="amd64"
GOBIN=""
GOEXE=""
GOHOSTARCH="amd64"
GOHOSTOS="linux"
GOOS="linux"
GOPATH="/home/will/go"
GORACE=""
GOROOT="/usr/local/go"
GOTOOLDIR="/usr/local/go/pkg/tool/linux_amd64"
GCCGO="gccgo"
CC="gcc"
GOGCCFLAGS="-fPIC -m64 -pthread -fmessage-length=0 -fdebug-prefix-map=/tmp/will/go-build795220856=/tmp/go-build -gno-record-gcc-switches"
CXX="g++"
CGO_ENABLED="1"
PKG_CONFIG="pkg-config"
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"

What did you do?

If possible, provide a recipe for reproducing the error.
A complete runnable program is good.
A link on play.golang.org is best.

package main

import (
	"fmt"
	"net/http"
	"time"
)

func main() {
	client := &http.Client{
		Timeout: 30 * time.Second,
	}
	resp, err := client.Get("https://www.southampton.ac.uk/")
	if err != nil {
		panic(err)
	}
	fmt.Println(resp)
}

What did you expect to see?

A response within a second or two

What did you see instead?

Timeout after 30 seconds, will happily hang for longer if timeout extended.

The following is the output of curl on the same machine / network. This timeout issue with this server has also been observed on a Travis test runner.

~ ❯ curl -v https://www.southampton.ac.uk/
*   Trying 152.78.118.52...
* Connected to www.southampton.ac.uk (152.78.118.52) port 443 (#0)
* found 173 certificates in /etc/ssl/certs/ca-certificates.crt
* found 697 certificates in /etc/ssl/certs
* ALPN, offering http/1.1
* SSL connection using TLS1.2 / ECDHE_RSA_AES_256_GCM_SHA384
* 	 server certificate verification OK
* 	 server certificate status verification SKIPPED
* 	 common name: www.southampton.ac.uk (matched)
* 	 server certificate expiration date OK
* 	 server certificate activation date OK
* 	 certificate public key: RSA
* 	 certificate version: #3
* 	 subject: OU=Domain Control Validated,CN=www.southampton.ac.uk
* 	 start date: Tue, 13 Jan 2015 00:00:00 GMT
* 	 expire date: Fri, 12 Jan 2018 23:59:59 GMT
* 	 issuer: C=NL,ST=Noord-Holland,L=Amsterdam,O=TERENA,CN=TERENA SSL CA 2
* 	 compression: NULL
* ALPN, server did not agree to a protocol
> GET / HTTP/1.1
> Host: www.southampton.ac.uk
> User-Agent: curl/7.47.0
> Accept: */*
> 
< HTTP/1.1 200 OK
< Server: nginx
< Date: Thu, 03 Aug 2017 21:38:44 GMT
< Content-Type: text/html;charset=UTF-8
< Transfer-Encoding: chunked
< Connection: keep-alive
< Set-Cookie: JSESSIONID=9E437837C159A9276F8DA64CFF182DE7; Path=/; HttpOnly
< X-XSS-Protection: 1; mode=block
< X-Frame-Options: SAMEORIGIN
< X-Content-Type-Options: nosniff
< Referrer-Policy: unsafe-url
< Content-Security-Policy-Report-Only: default-src 'self' 'unsafe-inline' *.atbar.org *.issuu.com ddfbm2kt5ml6l.cloudfront.net cdn.soton.ac.uk cdn.southampton.ac.uk corporate.wordpress.soton.ac.uk blog.soton.ac.uk; script-src 'self' 'unsafe-inline' 'unsafe-eval' code.jquery.com www.googleadservices.com www.googletagmanager.com *.google-analytics.com connect.facebook.net static.ads-twitter.com j.flxpxl.com hj.flxpxl.com d2hlpp31teaww3.cloudfront.net d1d7fjtb6d4i2m.cloudfront.net analytics.twitter.com ib.adnxs.com www.intelliworkschat.com access.ecs.soton.ac.uk *.atbar.org www.youtube.com s.ytimg.com *.issuu.com ddfbm2kt5ml6l.cloudfront.net cdn.soton.ac.uk cdn.southampton.ac.uk stats.wp.com soton.disqus.com stats.wp.com corporate.wordpress.soton.ac.uk blog.soton.ac.uk s0.wp.com use.typekit.net connect.facebook.com platform.twitter.com; font-src 'self' data: ddfbm2kt5ml6l.cloudfront.net cdn.soton.ac.uk cdn.southampton.ac.uk use.typekit.net; img-src data: 'self' www.southampton.ac.uk cdn.soton.ac.uk cdn.southampton.ac.uk t.co mpp2.vindicosuite.com *.google-analytics.com *.facebook.com pixel.adsafeprotected.com *.doubleclick.net www.google.com www.google.co.uk *.atbar.org *.issuu.com image.isu.pub www.intelliworkschat.com pixel.wp.com blog.soton.ac.uk platform.twitter.com pbs.twimg.com syndication.twitter.com p.typekit.net d1d7fjtb6d4i2m.cloudfront.net; frame-src 'self' www.google.com www.google.co.uk www.youtube.com *.issuu.com ddfbm2kt5ml6l.cloudfront.net cdn.soton.ac.uk cdn.southampton.ac.uk; child-src 'self' www.google.com www.google.co.uk www.youtube.com *.issuu.com ddfbm2kt5ml6l.cloudfront.net cdn.soton.ac.uk cdn.southampton.ac.uk;
< Set-Cookie: BIGipServer~CorpWeb~pool-corpweb-prod-80=2678630592.20480.0000; path=/; Httponly; Secure
< 
<!DOCTYPE html>

The text was updated successfully, but these errors were encountered:

bradfitz · 2017-08-03T21:48:51Z

This is a dup of #20979

The server negotiates "h2" and has "BIGipServer" in its response headers.

I can verify with GODEBUG=http2debug that it does the immediate GOAWAY thing. Closing this bug and moving to #20979

wjdp · 2017-08-03T21:52:13Z

Apologies for the duplicate and thanks for the quick response 😄

odeke-em · 2017-08-03T21:54:47Z

Oops, nvm @bradfitz beat me to it. @wjdp if you want to use http1, please use

package main

import (
	"crypto/tls"
	"flag"
	"fmt"
	"net/http"
	"time"
)

func main() {
	var http1 bool
	flag.BoolVar(&http1, "http1", false, "use http1 to invoke the server")
	flag.Parse()

	tr := &http.Transport{}
	if http1 {
		tr.TLSClientConfig = &tls.Config{
			NextProtos: []string{"h1"},
		}
	}

	client := &http.Client{
		Timeout:   30 * time.Second,
		Transport: tr,
	}
	resp, err := client.Get("https://www.southampton.ac.uk/")
	if err != nil {
		panic(err)
	}
	fmt.Println(resp)
}

and invoke it with flag http1 ie go run main.go --http1 and that should return your result ASAP.

wjdp · 2017-08-03T22:00:11Z

Thanks @odeke-em, provided snippet does the trick!

HaraldNordgren · 2018-07-26T19:06:03Z

I had a problem with a hanging host as well. On the surface very similar to this case although setting the protocol version did not work.

For me it was necessary in the TLS config to use

	DynamicRecordSizingDisabled: true

I posted a more fleshed out comment about it in: #20979 (comment) for debugging similar issues.

bradfitz · 2018-07-26T22:45:33Z

@HaraldNordgren, please file a separate crypto/tls bug for your case. It sounds like a crypto/tls issue instead of a net/http issue.

…imeout. 1. Adcording to golang/go#21301, we use http1 in transport layer to prevent timeout. 2. Set meta parser timeout to 5 second, this prevent API blocking.

bradfitz closed this as completed Aug 3, 2017

bradfitz mentioned this issue Aug 3, 2017

x/net/http2: connecting to non-compliant HTTP2 server returns Client.Timeout exceeded; fallback to HTTP/1? #20979

Open

mikioh changed the title ~~net/http Request to certain server hangs, works fine with curl~~ net/http: Request to certain server hangs, works fine with curl Aug 3, 2017

golang locked and limited conversation to collaborators Sep 18, 2019

gopherbot added the FrozenDueToAge label Sep 18, 2019

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

net/http: Request to certain server hangs, works fine with curl #21301

net/http: Request to certain server hangs, works fine with curl #21301

wjdp commented Aug 3, 2017

bradfitz commented Aug 3, 2017

wjdp commented Aug 3, 2017

odeke-em commented Aug 3, 2017

wjdp commented Aug 3, 2017

HaraldNordgren commented Jul 26, 2018 •

edited

Loading

bradfitz commented Jul 26, 2018

net/http: Request to certain server hangs, works fine with curl #21301

net/http: Request to certain server hangs, works fine with curl #21301

Comments

wjdp commented Aug 3, 2017

What version of Go are you using (go version)?

What operating system and processor architecture are you using (go env)?

What did you do?

What did you expect to see?

What did you see instead?

bradfitz commented Aug 3, 2017

wjdp commented Aug 3, 2017

odeke-em commented Aug 3, 2017

wjdp commented Aug 3, 2017

HaraldNordgren commented Jul 26, 2018 • edited Loading

bradfitz commented Jul 26, 2018

What version of Go are you using (`go version`)?

What operating system and processor architecture are you using (`go env`)?

HaraldNordgren commented Jul 26, 2018 •

edited

Loading