Hostnames for localhost HTTP gateway

[lidel]

Motivation

• At some point in the future we may have native protocol handler (libdweb+[PoC] libdweb experiment: protocol handler API ipfs-companion#533 or similar), but for now the URL in address bar remains fugly less than pretty
• The latest Companion extension supports DNSLink-based redirects from URLs like http://docs.ipfs.io to a local go-ipfs gateway at http://127.0.0.1:8080/ipns/docs.ipfs.io/
• Our public gateways start to support cidv1b32 in Subdomains (CID as a Subdomain #89), which restores Origin-based isolation between websites. This also means we don't want to redirect such URLs to local gateway as it would remove this isolation.

Requirements

I wonder if there is a way of generating vanity hostnames pointing at localhost that:

• does not require lookups to an external DNS server
• keeps HTML content in secure context
• works out-of-the box on windows, mac and linux
  • alternatively we can add heuristics to pick method based on user's platform
• lets us introduce subdomain-driven Origin isolation for locally running gateway
• (nice to have, but we can work around this for embedds) does not trigger mixed-content warnings when IPFS resource from localhost is embedded on HTTPS website
  • it is what happens when /ipfs/{cid} is redirected from public gateway running on https to a custom one running on http
  • this may be the blocker: AFAIK only 127.0.0.1 and ::1 are whitelisted by browser vendors (eg. firefox #903966) but localhost is not (that is why Companion uses raw IP)
  • as of 2019-10 localhost is a secure context in both Firefox and Chrome. Firefox does not support subdomains (foo.bar.localhost is not a secure context)

If anyone has unconventional ideas, feel free to comment.

Candidates

*.localhost

Current status

🔒 – window.isSecureContext === true
💢 – window.isSecureContext === false

Status	Firefox	Chrome
127.0.0.1	🔒	🔒
localhost	🔒	🔒
foo.bar.localhost	💢	🔒

References

• https://w3c.github.io/webappsec-secure-contexts/#localhost:

  Section 6.3 of [RFC6761] lays out the resolution of localhost. and names falling within .localhost. as special, and suggests that local resolvers SHOULD/MAY treat them specially. For better or worse, resolvers often ignore these suggestions, and will send localhost to the network for resolution in a number of circumstances.

• Let 'localhost' be localhost
  • [DNSOP] DNSOP Call for Adoption - draft-west-let-localhost-be-localhost
• bugzilla:
  • Stop treating 'localhost' as securely delivered for the purposes of Secure Contexts
  • Consider hardcoding localhost names to the loopback address
  • Stop treating "http://localhost/" (by name) as mixed content
• chromium
  • Intent to Implement and Ship: Treat http://localhost as a secure context
  • Issue 589141: *.localhost should be considered secure
• go-ipfs
  • localhost subdomains do not work on Firefox and Safari

[lidel]

Candidate: http://{fqdn}.ipns.localhost
Test URL: http://docs.ipfs.io.ipns.localhost:8080/ipns/docs.ipfs.io
Result:

• Linux 👎 (Works on a box with unbound, does not resolve on stock Debian)
• Windows: ?
• Mac: ?

[olizilla]

There are some occult tricks for browsers on mac and windows; you can do some automagic with this https://en.wikipedia.org/wiki/Proxy_auto-config

I found it via https://github.com/KishanBagaria/frwrd

[lidel]

[..] you can do some automagic with this https://en.wikipedia.org/wiki/Proxy_auto-config

Changing proxy setting globally is not a good idea, however Firefox provides browser.proxy API with a very interesting property:

use the proxy.onRequest event listener to intercept web requests, and return an object that describes whether and how to proxy them.

This would enable us to use PAC-like processing only for IPFS requests.
(Right now we just redirect them)

[lockedshadow]

Linux -1 (Works on a box with unbound, does not resolve on stock Debian)

Seems like that it also depends on myhostname plugin for /etc/nsswtich.conf, which is not installed by default in Debian. With this plugin installed and activated, the test URL actually redirects to http://{local_gateway}/ipns/docs.ipfs.io/ when IPFS integrations and DNSLink support is enabled in IPFS Companion, and just open as is when IPFS integrations disabled in add-on.

From man nss-myhostname:

The hostnames "localhost" and "localhost.localdomain" (as well as any hostname ending in ".localhost" or ".localhost.localdomain") are resolved to the IP addresses 127.0.0.1 and ::1.

[lidel]

I think switching from 127.0.0.1 to ::1 where possible is the best we can do right now, given cross-platform constraints.

Having local gateway at ::1 looks less intimidating:

For now, this can be enabled manually via:

$ ipfs config --json Addresses.Gateway '["/ip4/127.0.0.1/tcp/8080","/ip6/::1/tcp/8080"]'

A proposal to extend default Addresses.Gateway with /ip6/::1/tcp/8080 is at ipfs/kubo#5905

[lidel]

Update: there may be a way to support http://<cidv1b32>.ipfs.localhost on every platform supporting HTTP Proxies – see proposal in ipfs/kubo#5982

[lidel]

Notes on Secure Context and .ipfs.localhost – ipfs/ipfs-companion#667 (comment)

[lidel]

The ".dweb" Special-Use Domain Name: arewedistributedyet/arewedistributedyet#34

[lidel]

Interesting development:
As of 2019-10 localhost (hostname) is a secure context in both Firefox and Chrome.

Caveat: Firefox does not include subdomains (foo.bar.localhost is not a secure context), but I posted our use case in related bug to match behavior from Chrome:
https://bugzilla.mozilla.org/show_bug.cgi?id=1433933#c6

There is also a discussion to skip DNS lookup for *.localhost (https://bugzilla.mozilla.org/show_bug.cgi?id=1220810) and hardcode it to loopback, which would make it work on all platforms, regardless of local DNS setup.

This makes *.localhost nearly ready for use (waiting for Firefox to match Chrome).
I updated the first post in this issue to reflect current status.

Demo

Loaded http://en.wikipedia-on-ipfs.org.ipfs.localhost:8080/ipns/en.wikipedia-on-ipfs.org/wiki/ today and Chrome marks it as secure context:

Firefox 69	Chrome 78
isSecureContext: false	isSecureContext: true

[lidel]

Quick update (in the context of https://github.com/ipfs/go-ipfs/issues/7527):

Since go-ipfs 0.5, when user runs local IPFS node, they can load websites and content via subdomains like this:

http://bafybeiemxf5abjwjbikoz4mc3a3dla6ual3jsgpdr4cjr3oz3evfyavhwq.ipfs.localhost:8080/wiki/
http://bafybeid6grcjmnvsdaeklremt2lcqk7jevbjtwrerzgq46nzxfgn5isrne.ipfs.localhost:8080/images/ipfs-logo.svg

There are two known problems with *.localhost at the moment:

1. If browser vendor does not hardcode localhost to point at loopback, then it is resolved by OS DNS resolver, and on some platforms that can produce error. Fix here is to implement section 6.3.3 from https://tools.ietf.org/html/rfc6761#section-6.3 (Chromium-based browsers already do this)

2. Content at *.localhost should be marked as Secure Context. Right now some vendors such as Firefox handle localhost and *.localhost differently, partially due to legacy code, partially due to my previous point.

Implementing "let localhost be localhost" (6.3.3 from rfc6761) would ensure localhost can resolve on every platform and can be marked as secure context – very likely solving both issues.

Vendors are mostly ok with this change:

• Firefox/Gecko already is tracking this as part of https://bugzilla.mozilla.org/show_bug.cgi?id=1220810.
• Safari/Webkit:
  • https://bugs.webkit.org/show_bug.cgi?id=171934
  • https://bugs.webkit.org/show_bug.cgi?id=160504

Unfortunately the movement on Safari and Firefox fronts to resolve it turned out to be slower than we expected. Looking into ways we can accelerate that.

[lidel]

Good news: Chromium officially locked *.localhost to loopback as per https://tools.ietf.org/html/draft-ietf-dnsop-let-localhost-be-localhost-02 🥳

Details from https://bugs.chromium.org/p/chromium/issues/detail?id=589141#c15:

We addressed this in https://bugs.chromium.org/p/chromium/issues/detail?id=691930 but didn't close this bug. There, we locked *.localhost to loopback as per https://tools.ietf.org/html/draft-ietf-dnsop-let-localhost-be-localhost-02 (see https://source.chromium.org/chromium/chromium/src/+/master:net/dns/host_resolver_manager.cc;drc=25292af6cee762330490f47e79e09a071dc4b5a9;l=3091). This means that we also consider *.localhost secure (see https://source.chromium.org/chromium/chromium/src/+/master:services/network/public/cpp/is_potentially_trustworthy.cc;drc=c70af83bc44f6829277cdc3621e7015d6e0d7530;l=228).

This increases likelihood of other vendors to follow.

[ianopolous]

https://bugzilla.mozilla.org/show_bug.cgi?id=1220810#c89

[autonome]

merges to mozilla-central - can't trust them until 24hrs old ;)

[lidel]

Some updates and reading for anyone interested in following *.localhost space :-)

The best support can be found in Chromium (no need for HTTP proxy, local DNS resolver is ignored)
Firefox will soon catch up thanks to our partnership with Igalia, we are also reaching out to Webkit to follow.

• Chromium
  • Intent to Implement and Ship: Treat http://localhost as a secure context.
• Firefox
  • Intent to prototype & ship: Treat localhost addresses as "Potentially Trustworthy"
  • "Hardcode localhost to loopback" patch landed last week, sounds like it is here to stay 🥳
    https://bugzilla.mozilla.org/show_bug.cgi?id=1220810 → https://phabricator.services.mozilla.com/D92716
    • some remaining fixes need to land before its 100% hardcoded. Work tracked in:
      https://bugzilla.mozilla.org/show_bug.cgi?id=1673364 → https://phabricator.services.mozilla.com/D94726
  • Tests to ensure localhost and *.localhost can actually not be overridden by non-local IP:
    https://bugzilla.mozilla.org/show_bug.cgi?id=1673315 → https://phabricator.services.mozilla.com/D94712
  • Create a WPT test to verify handling of locahost and *.locahost
    https://bugzilla.mozilla.org/show_bug.cgi?id=1672323
• Webkit (Apple)
  • Localhost subdomains don't work
    https://bugs.webkit.org/show_bug.cgi?id=160504
  • Content from loopback addresses (e.g. 127.0.0.1) should not be considered mixed content
    https://bugs.webkit.org/show_bug.cgi?id=171934

[montanaflynn]

I get this error: I can access it from chrome but not my Golang program:

"http://zzz.ipfs.localhost:8080/1": dial tcp: lookup zzz.ipfs.localhost: no such host

[lidel]

This was resolved in ipfs/kubo#6096

We went with *.ipfs.localhost and *.ipns.localhost, as it works nearly everywhere: thanks to collaboration with Igalia we now have secure context on subdomains (window.isSecureContext) n both Chromium-based browsers (Edge, Chrome, Brave, Opera), and Firefox.

• Webkit/Safari issues are actually a problem with macOS DNS resolver – tracked in localhost subdomains do not work on Safari (macOS) #206
• Some linux distributions may require additional configuration – tracked in gateway subdomains: *.localhost not found kubo#7290