Add conformance test for isolation of HTTP listeners

[pleshakov]

What type of PR is this?
/kind test
/area conformance

What this PR does / why we need it:

Listener isolation was clarified in #2465
This PR adds the corresponding extended feature and the conformance test.

The test includes two subtests:

• the one where hostnames are only configured in listeners
• the one where hostnames are configured both in listeners and HTTPRoutes, where HTTPRoutes try to "steal" requests bound to different listeners, but fail to do so because of the listener isolation.

Note:
I added only tests for HTTP listeners. I believe the fix for this issue #2417 might define the tests for the isolation for HTTPS listeners - those will also involve setting SNI.

Which issue(s) this PR fixes:

Fixes #2416

Does this PR introduce a user-facing change?:

Adds an extended feature GatewayHTTPListenerIsolation with the corresponding conformance test GatewayHTTPListenerIsolation

[k8s-ci-robot]

Hi @pleshakov. Thanks for your PR.

I'm waiting for a kubernetes-sigs member to verify that this patch is reasonable to test. If it is, they should reply with /ok-to-test on its own line. Until that is done, I will not automatically test new commits in this PR, but the usual testing commands by org members will still work. Regular contributors should join the org to skip this step.

Once the patch is verified, the new status will be reflected by the ok-to-test label.

I understand the commands that are listed here.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

[robscott]

Thanks @pleshakov! This is going to require some careful review from multiple people.

/cc @arkodg @mlavacca @sunjayBhatia
/ok-to-test
/hold

[sunjayBhatia]

ran this test real quick against Contour main and it does not pass, so I'll have to dig in further to see whats going on

--- FAIL: TestGatewayConformance (83.36s)
    --- FAIL: TestGatewayConformance/GatewayHTTPListenerIsolation (56.27s)
        --- PASS: TestGatewayConformance/GatewayHTTPListenerIsolation/hostnames_are_configured_only_in_listeners (0.03s)
            --- PASS: TestGatewayConformance/GatewayHTTPListenerIsolation/hostnames_are_configured_only_in_listeners/4_request_to_'bar.example.com/empty-hostname'_should_receive_a_404 (0.01s)
            --- PASS: TestGatewayConformance/GatewayHTTPListenerIsolation/hostnames_are_configured_only_in_listeners/12_request_to_'abc.foo.example.com/empty-hostname'_should_receive_a_404 (0.01s)
            --- PASS: TestGatewayConformance/GatewayHTTPListenerIsolation/hostnames_are_configured_only_in_listeners/3_request_to_'bar.com/abc-foo-example-com'_should_receive_a_404 (0.02s)
            --- PASS: TestGatewayConformance/GatewayHTTPListenerIsolation/hostnames_are_configured_only_in_listeners/2_request_to_'bar.com/wildcard-foo-example-com'_should_receive_a_404 (0.02s)
            --- PASS: TestGatewayConformance/GatewayHTTPListenerIsolation/hostnames_are_configured_only_in_listeners/1_request_to_'bar.com/wildcard-example-com'_should_receive_a_404 (0.03s)
            --- PASS: TestGatewayConformance/GatewayHTTPListenerIsolation/hostnames_are_configured_only_in_listeners/8_request_to_'bar.foo.example.com/empty-hostname'_should_receive_a_404 (0.03s)
            --- PASS: TestGatewayConformance/GatewayHTTPListenerIsolation/hostnames_are_configured_only_in_listeners/11_request_to_'bar.foo.example.com/abc-foo-example-com'_should_receive_a_404 (0.03s)
            --- PASS: TestGatewayConformance/GatewayHTTPListenerIsolation/hostnames_are_configured_only_in_listeners/7_request_to_'bar.example.com/abc-foo-example-com'_should_receive_a_404 (0.03s)
            --- PASS: TestGatewayConformance/GatewayHTTPListenerIsolation/hostnames_are_configured_only_in_listeners/6_request_to_'bar.example.com/wildcard-foo-example-com'_should_receive_a_404 (0.04s)
            --- PASS: TestGatewayConformance/GatewayHTTPListenerIsolation/hostnames_are_configured_only_in_listeners/0_request_to_'bar.com/empty-hostname'_should_go_to_infra-backend-v1 (0.04s)
            --- PASS: TestGatewayConformance/GatewayHTTPListenerIsolation/hostnames_are_configured_only_in_listeners/10_request_to_'bar.foo.example.com/wildcard-foo-example-com'_should_go_to_infra-backend-v1 (0.03s)
            --- PASS: TestGatewayConformance/GatewayHTTPListenerIsolation/hostnames_are_configured_only_in_listeners/9_request_to_'bar.foo.example.com/wildcard-example-com'_should_receive_a_404 (0.03s)
            --- PASS: TestGatewayConformance/GatewayHTTPListenerIsolation/hostnames_are_configured_only_in_listeners/5_request_to_'bar.example.com/wildcard-example-com'_should_go_to_infra-backend-v1 (0.02s)
            --- PASS: TestGatewayConformance/GatewayHTTPListenerIsolation/hostnames_are_configured_only_in_listeners/14_request_to_'abc.foo.example.com/wildcard-foo-example-com'_should_receive_a_404 (0.02s)
            --- PASS: TestGatewayConformance/GatewayHTTPListenerIsolation/hostnames_are_configured_only_in_listeners/13_request_to_'abc.foo.example.com/wildcard-example-com'_should_receive_a_404 (0.02s)
            --- PASS: TestGatewayConformance/GatewayHTTPListenerIsolation/hostnames_are_configured_only_in_listeners/15_request_to_'abc.foo.example.com/abc-foo-example-com'_should_go_to_infra-backend-v1 (0.02s)
        --- FAIL: TestGatewayConformance/GatewayHTTPListenerIsolation/intersecting_hostnames_are_configured_in_listeners_and_HTTPRoutes (0.02s)
            --- PASS: TestGatewayConformance/GatewayHTTPListenerIsolation/intersecting_hostnames_are_configured_in_listeners_and_HTTPRoutes/7_request_to_'bar.example.com/abc-foo-example-com'_should_receive_a_404 (0.01s)
            --- PASS: TestGatewayConformance/GatewayHTTPListenerIsolation/intersecting_hostnames_are_configured_in_listeners_and_HTTPRoutes/0_request_to_'bar.com/empty-hostname'_should_go_to_infra-backend-v1 (0.01s)
            --- PASS: TestGatewayConformance/GatewayHTTPListenerIsolation/intersecting_hostnames_are_configured_in_listeners_and_HTTPRoutes/5_request_to_'bar.example.com/wildcard-example-com'_should_go_to_infra-backend-v1 (0.01s)
            --- PASS: TestGatewayConformance/GatewayHTTPListenerIsolation/intersecting_hostnames_are_configured_in_listeners_and_HTTPRoutes/15_request_to_'abc.foo.example.com/abc-foo-example-com'_should_go_to_infra-backend-v1 (0.02s)
            --- PASS: TestGatewayConformance/GatewayHTTPListenerIsolation/intersecting_hostnames_are_configured_in_listeners_and_HTTPRoutes/6_request_to_'bar.example.com/wildcard-foo-example-com'_should_receive_a_404 (0.02s)
            --- PASS: TestGatewayConformance/GatewayHTTPListenerIsolation/intersecting_hostnames_are_configured_in_listeners_and_HTTPRoutes/10_request_to_'bar.foo.example.com/wildcard-foo-example-com'_should_go_to_infra-backend-v1 (0.02s)
            --- PASS: TestGatewayConformance/GatewayHTTPListenerIsolation/intersecting_hostnames_are_configured_in_listeners_and_HTTPRoutes/11_request_to_'bar.foo.example.com/abc-foo-example-com'_should_receive_a_404 (0.02s)
            --- PASS: TestGatewayConformance/GatewayHTTPListenerIsolation/intersecting_hostnames_are_configured_in_listeners_and_HTTPRoutes/2_request_to_'bar.com/wildcard-foo-example-com'_should_receive_a_404 (0.01s)
            --- PASS: TestGatewayConformance/GatewayHTTPListenerIsolation/intersecting_hostnames_are_configured_in_listeners_and_HTTPRoutes/3_request_to_'bar.com/abc-foo-example-com'_should_receive_a_404 (0.01s)
            --- PASS: TestGatewayConformance/GatewayHTTPListenerIsolation/intersecting_hostnames_are_configured_in_listeners_and_HTTPRoutes/1_request_to_'bar.com/wildcard-example-com'_should_receive_a_404 (0.01s)
            --- FAIL: TestGatewayConformance/GatewayHTTPListenerIsolation/intersecting_hostnames_are_configured_in_listeners_and_HTTPRoutes/14_request_to_'abc.foo.example.com/wildcard-foo-example-com'_should_receive_a_404 (30.00s)
            --- FAIL: TestGatewayConformance/GatewayHTTPListenerIsolation/intersecting_hostnames_are_configured_in_listeners_and_HTTPRoutes/4_request_to_'bar.example.com/empty-hostname'_should_receive_a_404 (30.00s)
            --- FAIL: TestGatewayConformance/GatewayHTTPListenerIsolation/intersecting_hostnames_are_configured_in_listeners_and_HTTPRoutes/13_request_to_'abc.foo.example.com/wildcard-example-com'_should_receive_a_404 (30.00s)
            --- FAIL: TestGatewayConformance/GatewayHTTPListenerIsolation/intersecting_hostnames_are_configured_in_listeners_and_HTTPRoutes/12_request_to_'abc.foo.example.com/empty-hostname'_should_receive_a_404 (30.00s)
            --- FAIL: TestGatewayConformance/GatewayHTTPListenerIsolation/intersecting_hostnames_are_configured_in_listeners_and_HTTPRoutes/8_request_to_'bar.foo.example.com/empty-hostname'_should_receive_a_404 (30.00s)
            --- FAIL: TestGatewayConformance/GatewayHTTPListenerIsolation/intersecting_hostnames_are_configured_in_listeners_and_HTTPRoutes/9_request_to_'bar.foo.example.com/wildcard-example-com'_should_receive_a_404 (30.00s)
FAIL

[arkodg]

these are more tied to hostname intersection b/w listener and route, should this be part of this test or another one ?

[pleshakov]

My interpretation is that listener intersection is about one listener and its HTTPRoutes (so it could be tested separately) --

gateway-api/apis/v1/httproute_types.go

Lines 71 to 73 in 4b1a28d

	// If a hostname is specified by both the Listener and HTTPRoute, there
	// must be at least one intersecting hostname for the HTTPRoute to be
	// attached to the Listener. For example:

while listener isolation is about multiple listeners --

gateway-api/apis/v1/gateway_types.go

Lines 173 to 179 in 4b1a28d

	// Note that requests SHOULD match at most one Listener. For example, if
	// Listeners are defined for "foo.example.com" and "*.example.com", a
	// request to "foo.example.com" SHOULD only be routed using routes attached
	// to the "foo.example.com" Listener (and not the "*.example.com" Listener).
	// This concept is known as "Listener Isolation". Implementations that do
	// not support Listener Isolation MUST clearly document this.
	//

Or to put it differently, correct implementation of listener intersection is required for this test, but ultimately the tests is about listener isolation.

[arkodg]

thanks for adding this @pleshakov, your test approach looks good, added a minor non blocking comment

[arkodg]

/approve

thanks for adding this one !

[arkodg]

@sunjayBhatia do you want to take another look at this PR, or should we get this in ?

[robscott]

I've been meaning to take a look at this too, sorry I lost track of it! Will aim to get some feedback in this week.

/assign

[sunjayBhatia]

@sunjayBhatia do you want to take another look at this PR, or should we get this in ?

yep I'll take another look tmr 👍🏽

[sunjayBhatia]

Contour needs an update to our Listener/Route hostname intersection logic to pass this but this looks like a nice thorough set of tests 👍🏽

[robscott]

Thanks @pleshakov! This is an incredibly detailed test with well-written + explained test cases. For what it's worth, GKE Gateway ran into the same failures that @sunjayBhatia found, but that's on us, thanks for catching our inconsistencies with the spec! A few nits here + a rebase needed but otherwise LGTM.

[robscott]

Do we actually need 2 new Gateways for this test? (Asking because this could significantly increase the time it takes to run conformance tests + the overall resources required).

[pleshakov]

Do we actually need 2 new Gateways for this test? (Asking because this could significantly increase the time it takes to run conformance tests + the overall resources required).

No need for two Gateways, I can make it work with one to decrease time and required resources 👍

[robscott]

That sounds great, thanks! Ping me whenever this is ready for another look

[robscott]

@pleshakov do you have time to merge these Gateways?

[pleshakov]

Hi @robscott

I didn't have enough time to dedicated to this. Apologies!

Is the idea to get the PR merged before the next Gateway API release?

One thing that worries me:

since I already got approvals and confirmations from folks running this against their implementations, although merging should not cause any changes to the test behavior, there is a slight chance for it, since I will be updating code and manifests. So another round reviews and re-runs will be needed.

[mlavacca]

Kong still needs to add logic to isolate HTTPS listeners, but the test looks well written and correct to me 👍

[arkodg]

/approve
/lgtm

[k8s-ci-robot]

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: arkodg, mlavacca, pleshakov, sunjayBhatia

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

• conformance/OWNERS [arkodg,mlavacca,sunjayBhatia]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[k8s-ci-robot]

PR needs rebase.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

[arkodg]

@robscott I'd love to see this one into v1.1, this conformance test will benefit a lot of implementations and not supporting this is lapse in security, to make some progress, can we get this into v1.1 and add a follow up to optimize manifests / resources, this test is opt in, so shouldnt negatively affect anyone by default

[robscott]

@arkodg that makes sense to me, @pleshakov can you rebase this PR? RC2 is scheduled to go out tomorrow, if this PR is ready before that, I'll merge it.

[robscott]

Covered by #3047, closing this one out.