Test different variants of K8s Services in conformance testing

[dprotaso]

What would you like to be added:

We should add conformance tests that ensure Gateway API works well with different variants of K8s Services, Endpoint, EndpointSlices

Short List Variants

• Service - with selector
• Service - with selector, endpoint slice mirror disabled
• Headless Service - with manual endpoints (and their different types)
• Headless Service - with manual endpoint slices (and their different types - ie. FQDN)

Why this is needed:

Different platforms that build on-top of K8s will utilize Services in different ways to suit their needs - it would be great to know Gateway conformance supports these variants. ie. some implementations currently don't track EndpointSlices which is pretty fundamental as they've been stable for numerous years

[Xunzhuo]

/assign
/remove-help

[dprotaso]

Related - #1816

@shaneutt Given ExternalName support is marked as SHOULD NOT - does it still warrant having an opt-in test? The purpose being for implementations (that support this type) having a common way to verify their support.

[shaneutt]

Related - #1816

@shaneutt Given ExternalName support is marked as SHOULD NOT - does it still warrant having an opt-in test? The purpose being for implementations (that support this type) having a common way to verify their support.

Because we consider it Implementation-specific it sorta suggests the opposite of conformance: implementations can choose to implement that functionality in a way they deem appropriate and safe. I'm not saying that we should never have conformance tests that cover Implementation-specific behaviors, but I'm hesitant to say we need one here and wonder if that would make it feel more like "extended". What specifically did you have in mind?

[shaneutt]

@Xunzhuo how are things going here? Need any assistance to move this one forward?

[dprotaso]

@Xunzhuo are you still working on this?

[dprotaso]

/unassign @Xunzhuo

[dprotaso]

/assign @dprotaso

[dprotaso]

I've created a first pass of a test here - #2828

Currently the test exercises the following variants:

1. Service - Headless (spec.clusterIP=None) with Selector
2. Service - Headless no selector with a manual Endpoint (addressType: IPv4)
3. Service - Headless no selector with a manual EndpointSlice (addressType: IPv4)
4. Service - No selector with a manual Endpoint (addressType: IPv4)
5. Service - No selector with a manual EndpointSlice (addressType: IPv4)

[dprotaso]

Ideally we can merge these variants and then discuss further tests to consider
eg.

1. different address types - EndpointSlice:IP6
2. Service - with selector, endpoint slice mirror disabled

[k8s-triage-robot]

The Kubernetes project currently lacks enough contributors to adequately respond to all issues.

This bot triages un-triaged issues according to the following rules:

• After 90d of inactivity, lifecycle/stale is applied
• After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
• After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

• Mark this issue as fresh with /remove-lifecycle stale
• Close this issue with /close
• Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle stale

[dprotaso]

Circling back what's done is

1. Service with Manual EndpointSlices
2. Headless K8s Services with Manual Endpointslices
3. Headless Service

We dropped Endpoint testing (see: #2828 (comment)). New features will be added to EndpointSlice. Also Endpoints get mirrored to EndpointSlices.

The test works with both IP6 and IP4 clusters - it had to be conditional on the environment.

[dprotaso]

What's sorta left to discuss is what do we do with:

1. Service - with selector, endpoint slice mirror disabled See: #1718 (comment)
2. EndpointSlice with FQDN (using CoreDNS this becomes a CNAME - but it's sorta undefined) see: kubernetes/kubernetes#114210

[dprotaso]

re: 1 - @robscott's comment from the PR was

No, there's no reason for controllers to watch both Endpoints and EndpointSlice APIs. Gateway API is newer than the EndpointSlice API so hopefully all implementations are reading from EndpointSlices, but they definitely shouldn't be reading from both APIs.

Confirmed with @arkodg Envoy Gateway only watches endpoint slices
Confirmed with @sunjayBhatia that Contour watches either or but not both. Their default is endpointslices.

Probably best then to not add a test and I don't think it warrants any further API doc changes - watching endpointslices is now sorta implied via the conformance test.

[dprotaso]

tl;dr From Gateway API Meeting Jun 10th

Support FQDN (similar to ExternalName) is too risky and can lead to confused deputy attacks. Hence it's not something the community wants to encourage.

I'm going to close out this issue since there's nothing else actionable here.