Register IGW service hostname, rather than first IPv4 address.

[krarey]

Opening per conversation with @lkysow.

Changes proposed in this PR:

• Remove the -resolve-hostnames flag from the Ingress Gateway's service-init container. When using a LoadBalancer service, this causes the DNS hostname to be registered in place of the first IPv4 address, bringing several benefits:
  • Consul will return a CNAME, rather than A record for .ingress.consul DNS queries. In the event there are multiple points of ingress (i.e. AWS ELB attached to multiple subnets), the traffic can be distributed in response to load balancer auto-scaling.
  • For load balancer integrations which scale outside of Kubernetes, the Consul service registration will not go stale as a result of this scaling activity.
  • Currently, in the event the service-init container executes before the load balancer resource has finished provisioning, the Consul catalog is updated with an invalid target address for the platform load balancer. Changing this to the LB hostname removes the need to perform a rolling update of the ingress gateway deployment to retrieve the correct address(es).

How I've tested this PR:

• Deployed to an AWS EKS cluster. Used an NLB LoadBalancer k8s service to front Consul ingress gateways, and tested traffic to a destination Vault service. Feel free to ping me on Slack for full environment details.

How I expect reviewers to test this PR:

Checklist:

• Bats tests added
• CHANGELOG entry added (HashiCorp engineers only, community PRs should not add a changelog entry)

[lkysow]

This looks good. The previous assumption was that using a DNS entry for address would break everything but that was an incorrect assumption. This is technically a breaking change so we'll want to add a changelog after merge. @ndhanushkodi ping me if you want some more context here/what to look at.

[ndhanushkodi]

Looks great to me!