Accept non-conventional ports from addresses resolved through DNS resolver or resolve method

[Nuhvi]

`Warning

Since the DNS protocol has no notion of ports, if you wish to send traffic to a particular port you must include this port in the URL itself, any port in the overridden addr will be ignored and traffic sent to the conventional port for the given scheme (e.g. 80 for http).
`

While this is generally true, the new HTTPS and SVCB records do support ports.

Would it be possible to change the current behavior to accept these ports?

[seanmonstar]

To be sure I understand, you're referring to a custom resolver that asks for those records instead of A or AAAA, and reqwest doesn't know that it's doing that? Then the port could have actually been specified...

Using the port in reqwest probably isn't too hard naively, but so far since it has been ignored, could using it cause surprising behavior if the port returned was junk?

[Nuhvi]

@seanmonstar

To be sure I understand, you're referring to a custom resolver that asks for those records instead of A or AAAA, and reqwest doesn't know that it's doing that? Then the port could have actually been specified...

More or less. Usually recursive resolvers will also get the A and AAAA records that corresponds to the the target of the HTTPS records.

could using it cause surprising behavior if the port returned was junk

I don't imagine why would a resolver set a junk port, and since the default is 0 and since 0 usually means I don't know, it is safe to override 0 port, and respect everything else.

I dived more into the code and I think hyper_util will have to do the same as the h3_client here, so more.

So there is an opportunity here to also extend Resolve trait so it has a resolve_service_binding or resolve_endpoint method, that has a default implementation (hopefully that is enough for backward compatibility):

fn resolve_endpoint(&mut self, name: Name) -> Future<Output = Result<Option<Endpoint>, Self::Error>> {
    None
}

Where Endpoint is a struct that is very similar to HTTPS minus the lifetimes, and converting stuff like . to the actual domain.

Why bother with all of this? well because then reqwest can call resolver.resolve_endpoint(name) earlier in the request life time, and hopefully get early useful informations like alpn, ech, and maybe use ipv4hint and ipv6hint to make early connections that can be dropped later if resolve() returned authoritative addresses from A or AAAA.

I would be happy to work on that if you have some patience for questions.

Disclosure 1: I am not a Rust expert, and I get scared of Send and Sync, but I would like to get over that.

Disclosure 2: My main agenda here is to enable Reqwest to make requests to Pkarr domains, and eventually I will try to extend Rustls to also support certificates signed by Pkarr keys. But since this specific addition to reqwest is useful for more than just my use case, I thought it wouldn't be a waste of your time to suggest it.

[blind-oracle]

A bit sad that this breaking change got into a patch-only hyper-util release which broke some of our workflow.

[blind-oracle]

Also probably the documentation here https://docs.rs/reqwest/latest/reqwest/struct.ClientBuilder.html#method.resolve_to_addrs should be updated since it clearly states that the port from the SocketAddr is not used.

[seanmonstar]

A bit sad that this breaking change got into a patch-only hyper-util release which broke some of our workflow.

Can you say anymore besides it being sad? I'm sorry it caused you problems, but it's also not a very actionable comment... Do you have a custom resolver that was picking a non-zero port that wasn't the real port? Was there a reason that I didn't think of?

[blind-oracle]

Sorry for not being specific.

We had some SocketAddrs passed to .resolve_to_addrs() overrides that contained non-zero ports.

And since for overrides it uses a custom resolver internally - the calls using those started picking ports from SocketAddrs instead of the usual URL schema.

That's why I think it is a breaking change.

[Nuhvi]

@blind-oracle I am curious to know the source of that SocketAddrs and why is the port set to non-zero if not intended to be used?

[blind-oracle]

@Nuhvi well that's another story, they come from different workflow and since the docs state that

Since the DNS protocol has no notion of ports, if you wish to send traffic to a particular port you
must include this port in the URL itself, any port in the overridden addresses will be ignored and
traffic sent to the conventional port for the given scheme (e.g. 80 for http).

we didn't sanitize them to set the ports to zero...