Different DNS resolving based on request header

[aukevanleeuwen]

Hello,

I'm new-ish to Envoy and I'm not 100% this this is the right place to ask, but here goes anyway. I might be juggling the glossary a bit, please forgive me.

I'm using Envoy Gateway and for a certain integration I'm connecting (out of the Kubernetes cluster) with an service behind Akamai. Akamai has the notion of a staging and a production network. To hit the staging network of Akamai (for testing before going live with your Akamai property configuration) you would need to have basically the same Host header as you would send to production, same SNI server name, but connect with a different IP. Basically you would be able to locally resolve the staging network IP (more on that later), put it in your /etc/hosts file and just fire off the request as normal and it would end up on the staging deployment of Akamai.

This staging IP can be resolved as follows: your normal lookup would look something like this:

foo.com.                                  296 IN CNAME foo.com.edgekey.net.
foo.com.edgekey.net.                    21596 IN CNAME somethingsomething.dsca.akamaiedge.net.
somethingsomething.dsca.akamaiedge.net.    16 IN A     1.2.3.4
somethingsomething.dsca.akamaiedge.net.    16 IN A     1.2.3.5

Where if you want to resolve against the staging network, you can resolve the CNAME against edgekey***-staging*** rather than edgekey.net.

foo.com.edgekey-staging.net.                    21599 IN CNAME somethingsomething.dsca.akamaiedge-staging.net.
somethingsomething.dsca.akamaiedge-staging.net.    19 IN A     9.9.9.1
somethingsomething.dsca.akamaiedge-staging.net.    19 IN A     9.9.9.2

Some example in which I've been able to automate some of this away:

In cURL you can 'automate' this a bit like this:

curl --connect-to foo.com:443:foo.com.edgekey-staging.net:443 https://foo.com/

In Java I can override the DNS resolver roughly like this (removed some code for clarity):

public class StagingInetAddressResolver implements InetAddressResolver {
    private final InetAddressResolver builtInResolver;

    // <SNIP> irrelevant ceremony

    @Override
    public Stream<InetAddress> lookupByName(String host, LookupPolicy lookupPolicy) throws UnknownHostException {
        if (resolveOnStagingNetwork(host)) {
            return this.builtInResolver.lookupByName("%s.edgekey-staging.net".formatted(host), lookupPolicy);
        }
        return this.builtInResolver.lookupByName(host, lookupPolicy);
    }

    private static boolean resolveOnStagingNetwork(String host) {
        return Environment.AKAMAI_NETWORK == AkamaiNetwork.STAGING; // <SNIP> some condition on the host
    }
}

I was wondering if it's possible to do something in Envoy like this as well so that traffic which has a certain header which will be send to a cluster connecting to Akamai can be conditionally send to either the 'normal' destination (foo.com) or to the staging destination (foo.edgekey-staging.net)?

[arkodg]

yes this is possible @aukevanleeuwen , what would you like the condition to be ?
you could represent the 2 FQDN endpoints as Backends https://gateway.envoyproxy.io/docs/tasks/traffic/backend/

[aukevanleeuwen]

Ok, but how does that work with having the correct SNI (is not the fqdn of the staging backend in that case) and the Host header? (Also not the fqdn of the 'staging' backend)?

[edit]
I'm reading my question again, and the whole Host header and SNI thing got a bit buried in the rest of the information I realise, sorry for that I should have repeated that more clearly at the end.

[aukevanleeuwen]

Sorry, to answer your question: the condition would be based on the existence of the X-Akamai-Staging header having the value ESSL in this case.

[arkodg]

the SNI would be based on the Host field value in the BackendTLSPolicy https://gateway.envoyproxy.io/docs/tasks/security/backend-tls/

you can instrument this use case by creating 2 rules within a HTTPRoute, one, matching on X-Akamai-Staging and while the other doest, and each rule targeting 2 different Backends

[aukevanleeuwen]

So the Backend fqdn determines the connection endpoint and the BackendTlsPolicy determines both Host header and SNI?

[aukevanleeuwen]

That almost sounds too easy 😆 I'll try tomorrow as it's getting late in this timezone. Thanks a lot for your help so far!

[aukevanleeuwen]

Works. Thanks @arkodg

[arkodg]

glad it worked !