Pod IPs removed from EndpointSlices for non-headless services in v0.15

[t0lya]

I have a question regarding Aggregate Service Import enhancement proposal which was implemented in #1144 and shipped in Submariner release 0.15. This change removed pod IP endpoints from EndpointSlice documents for non-headless ClusterIP services.

E.g., Endpoints list contains ClusterIP (address 10.0.0.1 in example). Before 0.15, endpoints contained list of pod IPs.

addressType: IPv4
apiVersion: discovery.k8s.io/v1
endpoints:
- addresses:
  - 10.0.1.1
  conditions:
    ready: true
kind: EndpointSlice
metadata:
  labels:
    endpointslice.kubernetes.io/managed-by: lighthouse-agent.submariner.io
    lighthouse.submariner.io/is-headless: "false"
    lighthouse.submariner.io/sourceNamespace: default
    multicluster.kubernetes.io/service-name: prometheus-node-exporter
    multicluster.kubernetes.io/source-cluster: cluster-a
    submariner-io/clusterID:  cluster-a
    submariner-io/originatingNamespace: default
  name: prometheus-node-exporter-default-cluster-a
  namespace: submariner-k8s-broker
ports:
- name: metrics
  port: 9100
  protocol: TCP

What was the reasoning behind this change and why is it needed? I could not find the reasoning while reading through Aggregate Service Import enhancement proposal. I do not see such a requirement on EndpointSlices in KEP 1645.

My team has a use-case for getting the list of ready endpoints for a given ClusterIP service across all connected clusters. We currently implement this by querying the ready endpoints in the EndpointSlice document on the broker cluster. We are trying to figure out how to preserve this use-case while upgrading Submariner to 0.15.

[tpantelis]

The purpose of the aggregated ServiceImport changes was to align Submariner with the MCS spec (KEP 1645). Prior, each cluster's service IP (for non-headless) was published in its own ServiceImport. Now they're published via a Submariner-specific EndpointSlice. We did previously publish an EndpointSlice but that was only for the purpose of communicating to the CoreDNS plugin load balancer whether or not the service has any backing pod endpoints. We could have just published an EndpointSlice with no endpoint addresses.

The EndpointSlice is really an implementation detail and was/is not intended for consumption outside of Submariner. The fact that previously the EndpointSlice contained the pod endpoints was really just happenstance. While the MCS spec does talk about the use of EndpointSlices to communicate endpoints, it does not require it, in fact a recent discussion in the Multi-Cluster SIG proposes to remove any mention of EndpointSlices in the spec since one implementation doesn't use them at all.