net: support DNS-over-HTTPS

[jmhodges]

DNS-over-HTTPS is an evolution of DNS that allows us to secure the stream of DNS requests from a system or user in the same way we secure HTTP traffic with TLS. As of Sept 2018, it's currently deployed in two of the major browsers (Mozilla[1], Chrome) and 2 major service owners (Cloudflare[2], Google[3]) and has the backing of lot of folks in the field as what we need to fix DNS.

There are a few Go implementations[4] already on GitHub, but those require buy-in and awareness to be used. Having it baked into the Go standard library would be an incredible boon in securing our systems and users.

[1] https://blog.nightly.mozilla.org/2018/06/01/improving-dns-privacy-in-firefox/
[2] https://developers.cloudflare.com/1.1.1.1/dns-over-https/
[3] https://developers.google.com/speed/public-dns/docs/dns-over-https
[4] e.g. coredns/coredns#1619

[mvdan]

I'd imagine that it would be best to first implement this outside the standard library, much like how HTTP2 was first implemented in x/net. So perhaps #16218 should be done first.

I also wonder if #12503 would be a requirement for this, to be able to swap between different resolvers.

I am also personally a bit confused by DNS over TLS versus DNS over HTTPS. Do we want to support both? If not, which one should we prefer and why?

[jmhodges]

I think much of #16218 is in https://godoc.org/golang.org/x/net/dns/dnsmessage

DNS over TLS is a different protocol. The trick with DNS over TLS is that it requires a new port to be opened up in people's firewalls. Chrome experimented with deploying HTTP/2 and SPDY on ports other than HTTPS's 443 and found out that for a large percentage of their user base, they couldn't make connections out over those other ports. For the same reason, tunneling over 443 and reaping the benefits of solid HTTP (especially HTTP/2) clients with DNS over HTTPS has been taking the mindshare.

[blinsay]

I'd imagine that it would be best to first implement this outside the standard library, much like how HTTP2 was first implemented in x/net. So perhaps #16218 should be done first.

I've been playing with a toy cli-client that uses net/http and x/net/dns/dnsmessage. it was quite pleasant.

[artyom]

I am also personally a bit confused by DNS over TLS versus DNS over HTTPS. Do we want to support both? If not, which one should we prefer and why?

@mvdan DNS over TLS is already supported with trivial amount of code: https://github.com/artyom/dot/blob/master/dot.go

[shuLhan]

As side note, another DoH client and server implementation in Go is also available at https://godoc.org/github.com/shuLhan/share/lib/dns.

[ncruces]

These are a hack based on ideas expressed in #24796, but I'll share them anyway:

net.Resolver instances with caching, opportunistic encryption, DNS-over-TLS and/or DNS-over-HTTPS: github.com/ncruces/go-dns

The DoH resolver takes advantage of Go's HTTP machinery to resolve issues like #23866.
Both DoT/DoH should be unaffected by things like #23873, since the 512 byte limit does not apply.

OTOH (and IMO) any serious DoH/DoT implementation needs to consider caching. Neither will benefit from any local resolver cache. Also needing consideration is how the resolver is configured. DoH requires specifying a URI template, DoT a server name for certificate verification.

Go's net.Resolver API isn't the best fit to implementing these, IMO.