/etc/mime.types impacts Content-Type returned by Gateway

[RubenKelevra]

Describe the bug
I tried to open one of the most recent blogposts on the ipfs blog which was redirected to my local gateway by my ipfs browser plugin. My browser then started to download the HTML file, instead of rendering it.

To Reproduce
Steps to reproduce the behavior:

1. Open this URL

Expected behavior
Rendering of webpage

Screenshot

Desktop (please complete the following information):

• OS: ArchLinux
• Browser Brave
• Browser version: 1.8.96 Chromium: 81.0.4044.138 (Official Build) unknown (64-bit)
• Plugin version: 2.11.0

[sinkuu]

Opening the url http://blog.ipfs.io.ipns.localhost:8080/2020-05-20-gossipsub-v1.1 with Firefox(78.0a1) or Chromium (83.0.4103.61) without IPFS Companion installed downloads the page too, so that could be unrelated to redirection.

My local IPFS daemon (0.6.0-dev/53739faee) seems to be responding with Content-Type: application/x-troff-man, while the official gateway (https://ipfs.io/ipns/blog.ipfs.io/2020-05-20-gossipsub-v1.1) responds with Content-Type: text/html.

[lidel]

I suspected this is unexpected side-effect of recent change in go-ipfs:
ipfs/kubo#7262 but I was unable to reproduce this with latest master of go-ipfs:

$ docker run --rm -it --net=host ipfs/go-ipfs:master-2020-05-25-707b550
...
Daemon is ready

local gateway returns text/html

$ curl -sD - http://blog.ipfs.io.ipns.localhost:8080/2020-05-20-gossipsub-v1.1 | grep -i content-type
Content-Type: text/html

@RubenKelevra @sinkuu I assume you both run bleeding edge – are you able to update to the latest master of go-ipfs and check if the issue is still there?

[sinkuu]

Upgraded to ipfs/kubo@707b550, still the same. This behavior may be system dependent (I'm using ArchLinux), because mime.TypeByExtension function utilizes local /etc/mime.types entries.

On ArchLinux (https://www.archlinux.org/packages/extra/any/mailcap/):

application/x-troff-man                            man 1 2 3 4 5 6 7 8

On Ubuntu (https://packages.ubuntu.com/focal/mime-support):

application/x-troff-man				man

[lidel]

Thank you @sinkuu, I was able to reproduce by updating /etc/mime.types on my Linux box restarting ipfs daemon after and setting:

application/x-troff-man                            man 1 2 3 4 5 6 7 8

This confirms its go-ipfs issue, unrelated to ipfs-companion.

---

@Stebalien @hsanjuan @gowthamgts I am unsure if this is a bug or a feature introduced in ipfs/kubo#7262 / ipfs/kubo#7252, but this type of OS-dependent / nondeterministic behavior feels like a bug to me.

Perhaps we should disable use of OS-level /etc/mime.types and instead ship implicit list with go-ipfs itself? In needed, we could support overrides via $IPFS_PATH/mime.types, but the default content-types should be deterministic, and not depend on OS-level configs.

[Stebalien]

I agree that disabling OS level mime type checking is a good solution.

Note: Ultimately, the problem was that we weren't generating posts as /foo.1/index.html. Instead, we were outputting files like foo.1. See https://github.com/ipfs/blog/issues/423.

[Stebalien]

I've filed ipfs/kubo#7418 to summarize this issue.

[lidel]

Thank you!

This issue was originally filled about inability to open http://blog.ipfs.io.ipns.localhost:8080/2020-05-20-gossipsub-v1.1 and that bug was fixed by https://github.com/ipfs/blog/issues/423 (I am no longer able to replicate), so I am closing this as fixed.

Impact of /etc/mime.types on Content-Type returned by go-ipfs continues in ipfs/kubo#7418