HTTP/3 does not work properly. Browsers mostly do not choose h3 over h2. #4372

Forza-tng · 2021-10-06T10:43:48Z

Hi. I am experiencing some odd behaviour with regard to http/3.

Caddy version: v2.4.5 h1:P1mRs6V2cMcagSPn+NWpD+OEYUYLIf6ecOa48cFGeUg=

Caddyfile:

{
    debug
    #   http_port 60001
    auto_https off
    log {
        output file /var/log/caddy/main/caddy_main.log {
            roll_size 100MiB
            roll_keep_for 100d
        }
        format json
    }
    servers :443 {
        protocol {
            experimental_http3
        }
    }
    servers :80 {
        protocol {
            allow_h2c
        }
    }
}

## Hosts section

import vhosts/*.caddy

Vhost Caddyfile:

wiki.tnonline.net:443 {
    tls /etc/letsencrypt/live/wiki.tnonline.net/fullchain.pem /etc/letsencrypt/live/wiki.tnonline.net/privkey.pem {
    }
    log {
        output file /var/log/caddy/wiki.tnonline.net_443.log {
            roll_size 100MiB
            roll_keep_for 100d
        }
        format json
    }

    encode zstd gzip
    @title {
        not file {
            try_files {path} {path}/
            split_path .php
        }
        path_regexp title ^/(.*)$
    }
    rewrite @title /mediawiki/index.php?title={re.title.1}&{query}
    redir / /w/Main_Page

    root * /var/www/domains/wiki.tnonline.net/htdocs
    php_fastcgi unix//var/run/php-fpm/fpm-wiki.socket
    file_server
}

wiki.tnonline.net:80 {
    log {
        output file /var/log/caddy/wiki.tnonline.net_80.log {
            roll_size 100MiB
            roll_keep_for 100d
        }
        format json
    }
    root * /var/www/domains/wiki.tnonline.net/htdocs
    file_server
    @https not path /.well-known/*
    redir @https https://wiki.tnonline.net/ permanent
}

When I use cURL I can force http3 connection:

# curl -I --http3 https://wiki.tnonline.net/w/Main_Page
HTTP/3 200
alt-svc: h3=":443"; ma=2592000,h3-29=":443"; ma=2592000
expires: Thu, 01 Jan 1970 00:00:00 GMT
cache-control: no-cache, no-store, max-age=0, must-revalidate
pragma: no-cache
x-request-id: 96c8318b1c515c2243c6cf95
x-content-type-options: nosniff
content-language: en-GB
server: Caddy
vary: Accept-Encoding, Cookie
content-type: text/html; charset=UTF-8

However browsers such as FireFox, Chromium and Opera do not always choose h3 connection. I could not understand why, so I tried checking with the online service https://www.http3check.net/ which shows some weird result. It allowed me to export the qlog file (attached).

https://www.http3check.net/?host=wikidev.tnonline.net

While https://www.http3check.net/?host=http3.is gives the expected results

I've added a QLog output.
wikidev.tnonline.net.qlog-json.txt

Caddy Logfile:

{"level":"debug","ts":1633518503.0010457,"logger":"tls.handshake","msg":"choosing certificate","identifier":"wikidev.tnonline.net","num_choices":1}
{"level":"debug","ts":1633518503.0011086,"logger":"tls.handshake","msg":"custom certificate selection results","identifier":"wikidev.tnonline.net","subjects":["wikidev.tnonline.net"],"managed":false,"issuer_key":"","hash":"01809400870f643468157753337bae8360f7e532f0ada502e82077fd0b0b0938"}
{"level":"debug","ts":1633518503.0011237,"logger":"tls.handshake","msg":"matched certificate in cache","subjects":["wikidev.tnonline.net"],"managed":false,"expiration":1640897208,"hash":"01809400870f643468157753337bae8360f7e532f0ada502e82077fd0b0b0938"}
{"level":"debug","ts":1633518503.3504133,"logger":"tls.handshake","msg":"choosing certificate","identifier":"wikidev.tnonline.net","num_choices":1}
{"level":"debug","ts":1633518503.350483,"logger":"tls.handshake","msg":"custom certificate selection results","identifier":"wikidev.tnonline.net","subjects":["wikidev.tnonline.net"],"managed":false,"issuer_key":"","hash":"01809400870f643468157753337bae8360f7e532f0ada502e82077fd0b0b0938"}
{"level":"debug","ts":1633518503.3505151,"logger":"tls.handshake","msg":"matched certificate in cache","subjects":["wikidev.tnonline.net"],"managed":false,"expiration":1640897208,"hash":"01809400870f643468157753337bae8360f7e532f0ada502e82077fd0b0b0938"}

vhost logfile:

{"level":"info","ts":1633517331.7977269,"logger":"http.log.access.log20","msg":"handled request","request":{"remote_addr":"[2604:a880:800:a1::1279:3001]:36030","proto":"HTTP/1.1","method":"GET","host":"wikidev.tnonline.net","uri":"/","headers":{"Accept":["*/*"],"Connection":["keep-alive"],"Accept-Encoding":["gzip, deflate"],"User-Agent":["wget/http3check.net"]},"tls":{"resumed":false,"version":772,"cipher_suite":4865,"proto":"","proto_mutual":true,"server_name":"wikidev.tnonline.net"}},"common_log":"2604:a880:800:a1::1279:3001 - - [06/Oct/2021:12:48:51 +0200] \"GET / HTTP/1.1\" 302 0","user_id":"","duration":0.000074811,"size":0,"status":302,"resp_headers":{"Content-Type":[],"Server":["Caddy"],"Alt-Svc":["h3=\":443\"; ma=2592000,h3-29=\":443\"; ma=2592000"],"Location":["/w/Main_Page"]}}
{"level":"info","ts":1633517337.5447617,"logger":"http.log.access.log20","msg":"handled request","request":{"remote_addr":"[2604:a880:800:a1::1279:3001]:36032","proto":"HTTP/1.1","method":"GET","host":"wikidev.tnonline.net","uri":"/","headers":{"User-Agent":["wget/http3check.net"],"Accept":["*/*"],"Connection":["keep-alive"],"Accept-Encoding":["gzip, deflate"]},"tls":{"resumed":false,"version":772,"cipher_suite":4865,"proto":"","proto_mutual":true,"server_name":"wikidev.tnonline.net"}},"common_log":"2604:a880:800:a1::1279:3001 - - [06/Oct/2021:12:48:57 +0200] \"GET / HTTP/1.1\" 302 0","user_id":"","duration":0.000066115,"size":0,"status":302,"resp_headers":{"Server":["Caddy"],"Alt-Svc":["h3=\":443\"; ma=2592000,h3-29=\":443\"; ma=2592000"],"Location":["/w/Main_Page"],"Content-Type":[]}}
{"level":"info","ts":1633517338.022157,"logger":"http.log.access.log20","msg":"handled request","request":{"remote_addr":"[2604:a880:800:a1::1279:3001]:41267","proto":"HTTP/3","method":"HEAD","host":"wikidev.tnonline.net","uri":"/","headers":{"User-Agent":["http3check.net/lsquic/2.29.0"]},"tls":{"resumed":false,"version":0,"cipher_suite":0,"proto":"","proto_mutual":true,"server_name":""}},"common_log":"2604:a880:800:a1::1279:3001 - - [06/Oct/2021:12:48:58 +0200] \"HEAD / HTTP/3\" 302 0","user_id":"","duration":0.000082446,"size":0,"status":302,"resp_headers":{"Server":["Caddy"],"Alt-Svc":["h3=\":443\"; ma=2592000,h3-29=\":443\"; ma=2592000"],"Location":["/w/Main_Page"],"Content-Type":[]}}
{"level":"info","ts":1633518172.9496891,"logger":"http.log.access.log20","msg":"handled request","request":{"remote_addr":"[2604:a880:800:a1::1279:3001]:36088","proto":"HTTP/1.1","method":"GET","host":"wikidev.tnonline.net","uri":"/","headers":{"Accept-Encoding":["gzip, deflate"],"User-Agent":["wget/http3check.net"],"Accept":["*/*"],"Connection":["keep-alive"]},"tls":{"resumed":false,"version":772,"cipher_suite":4865,"proto":"","proto_mutual":true,"server_name":"wikidev.tnonline.net"}},"common_log":"2604:a880:800:a1::1279:3001 - - [06/Oct/2021:13:02:52 +0200] \"GET / HTTP/1.1\" 302 0","user_id":"","duration":0.000066867,"size":0,"status":302,"resp_headers":{"Server":["Caddy"],"Alt-Svc":["h3=\":443\"; ma=2592000,h3-29=\":443\"; ma=2592000"],"Location":["/w/Main_Page"],"Content-Type":[]}}
{"level":"info","ts":1633518173.4117749,"logger":"http.log.access.log20","msg":"handled request","request":{"remote_addr":"[2604:a880:800:a1::1279:3001]:40752","proto":"HTTP/3","method":"HEAD","host":"wikidev.tnonline.net","uri":"/","headers":{"User-Agent":["http3check.net/lsquic/2.29.0"]},"tls":{"resumed":false,"version":0,"cipher_suite":0,"proto":"","proto_mutual":true,"server_name":""}},"common_log":"2604:a880:800:a1::1279:3001 - - [06/Oct/2021:13:02:53 +0200] \"HEAD / HTTP/3\" 302 0","user_id":"","duration":0.000073248,"size":0,"status":302,"resp_headers":{"Alt-Svc":["h3=\":443\"; ma=2592000,h3-29=\":443\"; ma=2592000"],"Location":["/w/Main_Page"],"Content-Type":[],"Server":["Caddy"]}}
{"level":"info","ts":1633518173.841833,"logger":"http.log.access.log20","msg":"handled request","request":{"remote_addr":"[2604:a880:800:a1::1279:3001]:40752","proto":"HTTP/3","method":"HEAD","host":"wikidev.tnonline.net","uri":"/","headers":{"User-Agent":["http3check.net/lsquic/2.29.0"]},"tls":{"resumed":false,"version":0,"cipher_suite":0,"proto":"","proto_mutual":true,"server_name":""}},"common_log":"2604:a880:800:a1::1279:3001 - - [06/Oct/2021:13:02:53 +0200] \"HEAD / HTTP/3\" 302 0","user_id":"","duration":0.000082767,"size":0,"status":302,"resp_headers":{"Content-Type":[],"Server":["Caddy"],"Alt-Svc":["h3=\":443\"; ma=2592000,h3-29=\":443\"; ma=2592000"],"Location":["/w/Main_Page"]}}

The text was updated successfully, but these errors were encountered:

gedw99 · 2021-10-29T16:45:10Z

On safari clients the headers it expects are here 👍

https://developer.apple.com/forums/thread/682990

Forza-tng · 2021-10-30T12:43:46Z

On safari clients the headers it expects are here 👍

https://developer.apple.com/forums/thread/682990

Not sure what you mean. Did you try safari against https:/wiki.tnonline.net?

awfulcooking · 2021-11-06T19:11:18Z

I don't have a curl handy that supports --http3 :-)

But I am almost sure that the http3checker.net issue is that they refer to h3/h3-29 as HTTP3, and <= h3-27 as QUIC.

Supporting just h3/-29 seems fine. Firefox is using h3 after the first connection to Caddy.

I found that if you do a Ctrl-Shift-r refresh in Firefox, it will use HTTP/2, before learning to use h3 again.

Can play with that behaviour on https://http3.is

bt90 · 2021-12-23T16:15:03Z

Prefer an appropriate DNS entry: https://blog.cloudflare.com/speeding-up-https-and-http-3-negotiation-with-dns/

The alt-svc header is only sent after the connection is already established.

e.g the HTTPS DNS entry for Cloudflare: https://dns.google/query?name=cloudflare.com&rr_type=HTTPS

Forza-tng · 2021-12-25T17:19:52Z

Hi. I've now updated to v2.4.6 h1:HGkGICFGvyrodcqOOclHKfvJC0qTU7vny/7FhYp9hNw= and it seems that http/3 is more reliable.

FireFox now claims h3 on all my sites except for one.

wiki.tnonline.net - HTTP/3 is used
wikidev.tnonline.net - HTTP/3 is used
paste.tnonline.net - HTTP/3 is used
tnonline.net - HTTP/3 is NOT used

I am guessing this is due to some configuration error, but I can't understand what exactly. All vhosts in Caddy have almost identical configurations.

Prefer an appropriate DNS entry

I've added the HTTPS dns entries, but it seems didn't help in this case =(. Seems a little fickle. Perhaps the HTTPS records work best for subdomains only.

paste.tnonline.net.	1	IN	HTTPS	1 paste.tnonline.net. alpn="h3,h3-29,h2" ipv6hint="2001:470:28:704::100" ipv4hint="158.174.254.104"
tnonline.net.		1	IN	HTTPS	1 . alpn="h3,h3-29,h2" ipv6hint="2001:470:28:704::100" ipv4hint="158.174.254.104"
wikidev.tnonline.net.	1	IN	HTTPS	1 wikidev.tnonline.net. alpn="h3,h3-29,h2" ipv6hint="2001:470:28:704::100" ipv4hint="158.174.254.104"
wiki.tnonline.net.	1	IN	HTTPS	1 wiki.tnonline.net. alpn="h3,h3-29,h2" ipv6hint="2001:470:28:704::100" ipv4hint="158.174.254.104"

But I am almost sure that the http3checker.net issue is that they refer to h3/h3-29 as HTTP3, and <= h3-27 as QUIC.

You might be right here. https://gf.dev/http3-test seem a more reliable checker.

bt90 · 2021-12-25T23:08:26Z

The DNS lookup lists two entries for the main domain:

https://dns.google/query?name=tnonline.net&rr_type=HTTPS&ecs=

Forza-tng · 2021-12-31T10:03:47Z

The DNS lookup lists two entries for the main domain:

https://dns.google/query?name=tnonline.net&rr_type=HTTPS&ecs=

Indeed. I added the '.' and 'tnonline.net.' as a way to see if this would make any difference. It unfortonately did not.

According to https://blog.cloudflare.com/speeding-up-https-and-http-3-negotiation-with-dns/ there should only be a need for '.' for the main domain.

bt90 · 2022-04-08T17:59:55Z

@mholt i think we can close here. HTTP3 support works as expected.

francislavoie · 2022-04-08T18:02:29Z

@Forza-tng do you agree? Are your concerns resolved?

I'm not totally following this discussion, so excuse me if I'm missing context.

bt90 · 2022-04-14T15:20:46Z

@francislavoie the concerns expressed by OP are not an issue with caddy itself.

There are basically two things at play here:

results returned by http3check.net look strange at first glance if the server doesn't support older QUIC protocol versions. The HTTP/3 check succeeds as expected though.
the behavior of the alt-svc HTTP header. Browsers will only gradually switch to HTTP3 after the first requests.

mholt added the help wanted 🆘 Extra attention is needed label Oct 6, 2021

francislavoie mentioned this issue Nov 6, 2021

consider enabling QUIC by default #3833

Closed

mholt closed this as completed Apr 14, 2022

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

HTTP/3 does not work properly. Browsers mostly do not choose h3 over h2. #4372

HTTP/3 does not work properly. Browsers mostly do not choose h3 over h2. #4372

Forza-tng commented Oct 6, 2021 •

edited

Loading

gedw99 commented Oct 29, 2021 •

edited

Loading

Forza-tng commented Oct 30, 2021 •

edited

Loading

awfulcooking commented Nov 6, 2021

bt90 commented Dec 23, 2021

Forza-tng commented Dec 25, 2021 •

edited

Loading

bt90 commented Dec 25, 2021

Forza-tng commented Dec 31, 2021 •

edited

Loading

bt90 commented Apr 8, 2022

francislavoie commented Apr 8, 2022 •

edited

Loading

bt90 commented Apr 14, 2022

HTTP/3 does not work properly. Browsers mostly do not choose h3 over h2. #4372

HTTP/3 does not work properly. Browsers mostly do not choose h3 over h2. #4372

Comments

Forza-tng commented Oct 6, 2021 • edited Loading

gedw99 commented Oct 29, 2021 • edited Loading

Forza-tng commented Oct 30, 2021 • edited Loading

awfulcooking commented Nov 6, 2021

bt90 commented Dec 23, 2021

Forza-tng commented Dec 25, 2021 • edited Loading

bt90 commented Dec 25, 2021

Forza-tng commented Dec 31, 2021 • edited Loading

bt90 commented Apr 8, 2022

francislavoie commented Apr 8, 2022 • edited Loading

bt90 commented Apr 14, 2022

Forza-tng commented Oct 6, 2021 •

edited

Loading

gedw99 commented Oct 29, 2021 •

edited

Loading

Forza-tng commented Oct 30, 2021 •

edited

Loading

Forza-tng commented Dec 25, 2021 •

edited

Loading

Forza-tng commented Dec 31, 2021 •

edited

Loading

francislavoie commented Apr 8, 2022 •

edited

Loading