Extreme slowdowns (x20) in web spec tests with warp==3.3.4

[hasufell]

Bug Report

We are running some webspecs with a scotty/warp/wreq/http2 stack.

The freeze file can be seen at: https://gist.github.com/hasufell/0d709bbbdc06af9a86809d64be69dd6c#file-cabal-freeze-broken

Our entire test suite used to take 8 minutes, with the bump to warp==3.3.4 it takes about 3 hours. There is no other bump that can cause it, all other dependencies are exactly the same.

• OS name and version: ubuntu bionic docker containers (reproducible outside of docker as well)
• Tools: ghc-8.6.5, cabal-3

I wasn't able to create a minimal test case yet, please bear with me.

[hasufell]

Through bisecting I found the commit: aee7377
related issue: #673

[kazu-yamamoto]

@hasufell Thank you for reporting.

gracefulClose adds extra 5 seconds to close a connection in worst case. Does webspec include about 2,000 test cases?

[hasufell]

Does webspec include about 2,000 test cases?

I think about ~1k, but some of them have multiple requests.

Shouldn't gracefulClose be specific to HTTP/2?

[kazu-yamamoto]

Yes, gracefulClose helps HTTP/2. But it should not harm HTTP/1.1.
Could you look into packet dumps to see if clients send TCP FINs?
It FINs are coming, this is a bug of gracefulClose. If FINs are not coming, this is a bug of the client.

[hasufell]

I don't see any FINs from the client https://gist.github.com/hasufell/d87db04ada989c34bba37beebfadde01

The client is wreq ==0.5.3.2

But it should not harm HTTP/1.1.

IMO, it's a huge assumption. I think such a big change should be as narrow in scope as possible.

[kazu-yamamoto]

OK. I will think workaround for HTTP/1.1.

[hasufell]

@kazu-yamamoto afais wreq uses http-client to do the calls and this doesn't seem to send FIN either. @snoyberg?

[snoyberg]

I don't know offhand, but the keepalive/connection manager code in http-client would probably prevent that.