Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

docs: Multi-port support for v1.17 GA #19401

Merged
merged 67 commits into from
Nov 7, 2023
Merged
Show file tree
Hide file tree
Changes from 47 commits
Commits
Show all changes
67 commits
Select commit Hold shift + click to select a range
1859dd1
Catalog concept pages creation
boruszak Oct 25, 2023
db661ba
Multi-port services overview - initial update
boruszak Oct 25, 2023
75aacc1
CLI command page creation
boruszak Oct 25, 2023
b6741a9
File location adjustment
boruszak Oct 25, 2023
0966354
nav
boruszak Oct 25, 2023
66c98eb
New resource pages - creation
boruszak Oct 25, 2023
f3002bf
nav fix
boruszak Oct 25, 2023
9c1e4cd
resource info
boruszak Oct 25, 2023
bf25be6
specs start
boruszak Oct 25, 2023
c216908
GRPCRoute specs and structure
boruszak Oct 26, 2023
87a1fc2
GRPCRoute configuration model
boruszak Oct 26, 2023
7290e9d
gRPCRoute models and examples
boruszak Oct 26, 2023
8009aa0
HTTP copy
boruszak Oct 26, 2023
28dd84c
Resource configuration alignment
boruszak Oct 26, 2023
6528f06
Catalogs
boruszak Oct 26, 2023
4303eaf
Deployment error fix
boruszak Oct 26, 2023
cf1201c
HTTPRoute specs
boruszak Oct 27, 2023
42eea7c
TCP Route specifications
boruszak Oct 27, 2023
3c57d16
proxy configuration model
boruszak Oct 27, 2023
fa00cb7
ProxyConfiguration specifications
boruszak Oct 27, 2023
ee7e4a8
Example
boruszak Oct 27, 2023
12c7212
basic traffic permissions info
boruszak Oct 30, 2023
5e8f602
complete config structure
boruszak Oct 30, 2023
556ecda
tab spacing
boruszak Oct 30, 2023
5b24e36
Traffic permission specifications
boruszak Oct 30, 2023
7b1b91a
Proxy config example description
boruszak Oct 30, 2023
ead5752
Apply suggestions from code review
boruszak Oct 31, 2023
0959179
Merge branch 'main' into docs/multi-port-v1-17-ga
boruszak Oct 31, 2023
bb96ad8
v1 concept review updates
boruszak Oct 31, 2023
e95f590
v2 catalog code review updates
boruszak Oct 31, 2023
3c00599
V2 catalog contraints + guidance
boruszak Oct 31, 2023
068703b
Proxyconfiguration code review changes
boruszak Oct 31, 2023
bd268ac
Apply suggestions from code review
boruszak Oct 31, 2023
89addfa
Apply suggestions from code review
boruszak Oct 31, 2023
3113840
Merge branch 'main' into docs/multi-port-v1-17-ga
boruszak Oct 31, 2023
7d438a3
Cross-apply suggestions to reference pages
boruszak Oct 31, 2023
364b49c
More code review suggestions
boruszak Oct 31, 2023
90e9079
comment fix
boruszak Oct 31, 2023
28e487b
Apply suggestions from code review
boruszak Nov 1, 2023
376eeb0
Index + usage updates
boruszak Nov 1, 2023
1b8c675
TCP clarification
boruszak Nov 1, 2023
af1746b
Minor fixes
boruszak Nov 1, 2023
e3ea5f2
remove references to unsupported features
boruszak Nov 1, 2023
4c464e2
comment fix
boruszak Nov 1, 2023
da48d80
Resource command section removed
boruszak Nov 1, 2023
42a6ef0
Tested instructions
boruszak Nov 2, 2023
f7fad9b
More updates based on testing
boruszak Nov 2, 2023
ea54ebb
Apply suggestions from code review
boruszak Nov 2, 2023
c7628de
Merge branch 'main' into docs/multi-port-v1-17-ga
boruszak Nov 2, 2023
8d3e88e
Multi-port traffic permissions example
boruszak Nov 2, 2023
0b0b4c9
parent/child alignment
boruszak Nov 2, 2023
464e4fc
Dataplanes requirement
boruszak Nov 2, 2023
a943e29
Merge branch 'main' into docs/multi-port-v1-17-ga
boruszak Nov 2, 2023
7d679fd
Update version-specific CLI install
boruszak Nov 2, 2023
ac15715
Merge branch 'main' into docs/multi-port-v1-17-ga
boruszak Nov 2, 2023
3255c98
Method 2
boruszak Nov 2, 2023
48bc056
Tab fix
boruszak Nov 2, 2023
fc15a39
Merge branch 'main' into docs/multi-port-v1-17-ga
boruszak Nov 2, 2023
ac82bda
Consul resource updates
boruszak Nov 3, 2023
8ba577b
nav fix
boruszak Nov 3, 2023
4ee651e
Catalog groups
boruszak Nov 3, 2023
b70c39f
Catalog `group` info
boruszak Nov 3, 2023
179cb27
Merge branch 'main' into docs/multi-port-v1-17-ga
boruszak Nov 3, 2023
64b2bb2
time formatting
boruszak Nov 6, 2023
ad99c00
Merge branch 'main' into docs/multi-port-v1-17-ga
boruszak Nov 6, 2023
48f47a0
Merge branch 'main' into docs/multi-port-v1-17-ga
boruszak Nov 6, 2023
4f4315c
Merge branch 'main' into docs/multi-port-v1-17-ga
boruszak Nov 7, 2023
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
37 changes: 37 additions & 0 deletions website/content/docs/architecture/catalog/v1.mdx
Original file line number Diff line number Diff line change
@@ -0,0 +1,37 @@
---
layout: docs
page_title: v1 Catalog API
description: Learn about version 1 of the Consul catalog, including what Consul servers record when they register a service.
---

# v1 Catalog API

This topic provides conceptual information about version 1 (v1) of the Consul catalog API. The catalog tracks registered services and their locations for both service discovery and service mesh use cases.

For more information about the information returned when querying the catalog, including filtering options when querying the catalog for a list of nodes, services, or gateways, refer to the [`/catalog` endpoint reference in the HTTP API documentation](/consul/api-docs/catalog).

## Introduction

Consul tracks information about registered services through its catalog API. This API records user-defined information about the external services, such as their partitions and required health checks. It also records information that Consul assigns for its own operations, such as an ID for each service instance and the [Raft indices](/consul/docs/architecture/consensus) when the instance is registered and modified.

Consul uses v1 of the catalog API by default. Consul v1.17.0 and later ships with version 2 (v2) of the catalog API. V2 is intended for testing and development purposes. V1 and V2 of the catalog APIs cannot run concurrently in a Consul deployment. There is no migration path between catalog versions. For more information, refer to [Consul v2 Catalog API](/consul/docs/architecture/catalog/v2).

## Catalog structure

When Consul registers a service instance using the v1 catalog API, it records the following information about each instance:

| v1 Catalog field | Description | Source |
| :--------------- | :---------- | :----- |
| ID | A unique identifier for a service instance. | Defined by user in [service definition](/consul/docs/services/configuration/services-configuration-reference#id). |
| Node | The connection point where the service is available. | On VMs, defined by user. <br /><br /> On Kubernetes, computed by Consul according to [Kubernetes Nodes](https://kubernetes.io/docs/concepts/architecture/nodes/). |
| Address | The registered address of the service instance. | Defined by user in [service definition](/consul/docs/services/configuration/services-configuration-reference#address). |
| Tagged Addresses | User-defined labels for addresses. | Defined by user in [service definition](/consul/docs/services/configuration/services-configuration-reference#tagged_addresses). |
| NodeMeta | User-defined metadata about the node. | Defined by user |
| Datacenter | The name of the datacenter the service is registered in. | Defined by user |
| Service | The name of the service Consul registers the service instance under. | Defined by user |
| Agent Check | The health checks defined for a service instance managed by a Consul client agent. | Computed by Consul |
| Health Checks | The health checks defined for the service. Refer to [define health checks](/consul/docs/services/usage/checks) for more information. | Defined by user |
boruszak marked this conversation as resolved.
Show resolved Hide resolved
| Partition | The name of the admin partition the service is registered in. Refer to [admin partitions](/consul/docs/enterprise/admin-partitions) for more information. | Defined by user |
| Locality | Region and availability zone of the service. Refer to [`locality`](/consul/docs/agent/config/config-files#locality) for more information. | Defined by user |

Depending on the configuration entries or custom resource definitions you apply to your Consul installation, additional information such as [proxy default behavior](/consul/docs/connect/config-entries/proxy-defaults) is automatically recorded to the catalog for services. You can return this information using the [`/catalog` HTTP API endpoint](/consul/api-docs/catalog).
102 changes: 102 additions & 0 deletions website/content/docs/architecture/catalog/v2.mdx
Original file line number Diff line number Diff line change
@@ -0,0 +1,102 @@
---
layout: docs
page_title: v2 Catalog API
description: Learn about version 2 of the Consul catalog, which uses GAMMA specified resources. Learn how the v2 catalog corresponds to the v1 catalog and Kubernetes resources..
---

# v2 Catalog API

<Warning>
The v2 catalog API is in a beta release for testing and development purposes. Do not use the v2 catalog or multi-port services in secure production environments.
</Warning>
boruszak marked this conversation as resolved.
Show resolved Hide resolved

This topic provides conceptual information about version 2 (v2) of the Consul catalog API. The catalog tracks registered services and their locations for both service discovery and service mesh use cases

Consul supports the v2 catalog for service mesh use cases on Kubernetes deployments only. For more information about Consul’s default catalog, refer to [v1 Catalog API](/consul/docs/architecture/catalog/v1).

## Introduction

When Consul registers services, it records [user-defined and Consul-assigned information](/consul/docs/architecture/catalog/v1#catalog-structure). To determine a service’s identity, v1 of the catalog API records the following information:

- IDs of the specific _service instances_ that are registered
- Locations of the _nodes_ the instances run on
- Names of the _services_ the instances are associated with

This information enables Consul to associate service names with the individual instances and their unique network addresses, and it is essential to Consul’s service discovery and service mesh operations.

The [Consul v1 catalog API](/consul/docs/architecture/catalog/v1) was designed prior to the introduction of Consul’s service mesh features. Communication in Consul’s service mesh is secured through Consul's ACL system, which requires that a Kubernetes ServiceAccount resource match the Service name. As a result, only one service can represent a service instance in the v1 catalog.

The v2 catalog API aligns more closely with the [Kubernetes Gateway API's GAMMA initiative](https://gateway-api.sigs.k8s.io/concepts/gamma/), enabling functionality such as associating Kubernetes Pods with multiple Kubernetes Services and allowing Services and Pods registered with Consul to have multiple ports. For more information about how the differences between the catalog API impacts Consul operations, refer to [changes to Consul's existing architecture](#changes-to-consul-s-existing-architecture).

The v2 catalog API is available alongside the existing v1 catalog API, but the catalogs cannot be used simultaneously. The v2 catalog is disabled by default. This beta release is for testing and development purposes only. We do not recommend implementing v2 in production environments or migrating to v2 until the API is generally available.

## Catalog structure

Consul v1.17 introduces a new version of the catalog API designed to bridge differences between the Consul and Kubernetes data models. The v2 catalog API still tracks services and nodes for Consul, but replaces service instances with _workloads_ and _workload identites_.

The following table describes resources in the v2 catalog and compares them to the v1 catalog and Kubernetes resources. It also states whether these resources are defined by the user or computed by Consul when it registers a service.

| Catalog v2 resource | Description | Catalog v1 resource analogue | Kubernetes resource analogue | Source |
| :------------------ | :---------- | :--------------------------- | :--------------------------- | :----- |
| Service | The name of the service Consul registers a workload under. | Service | None | Defined by user in Kubernetes |
| Node | The address of the node where the workload runs. | Node | [Kubernetes Node](https://kubernetes.io/docs/concepts/architecture/nodes/) | Computed by Consul |
| Workload | An application instance running in a set of one or more Pods scheduled according to a Kubernetes Workload resource such as a Deployment or StatefulSet. | Service instance | [Kubernetes Pod](https://kubernetes.io/docs/concepts/workloads/pods/) | Defined by user in Kubernetes |
| Workload identities | Provides a distinct identity for a workload to assume and is used for Traffic Permissions. | Service instance | [Kubernetes Service Accounts](https://kubernetes.io/docs/concepts/security/service-accounts/) | Defined by user in Kubernetes |
| Service endpoints | Maps services to workload addresses and endpoints. | None | [Kubernetes Endpoints](https://kubernetes.io/docs/reference/kubernetes-api/service-resources/endpoints-v1/) | Computed by Consul |
| Health status | A resource for reporting the health status of a workload. | Service instance health status | None | Computed by Consul |
| Health check | A resource for defining the health checks for a workload. | [Service instance health check](/consul/docs/services/usage/checks) | [Liveness, Readiness, and Startup Probes](https://kubernetes.io/docs/concepts/workloads/pods/pod-lifecycle/#container-probes) | Defined by user in Kubernetes |
| Proxy configuration | Represents a configuration for a sidecar or gateway proxy. | `Proxy` field in service definition | None | Defined by user in Kubernetes or Consul |
| Destinations | Represents explicit service upstreams. | [Proxy Configuration](/consul/docs/connect/proxies/envoy#envoy-proxy-configuration-for-service-mesh) | [Upstream Service annotation](/consul/docs/k8s/annotations-and-labels#consul-hashicorp-com-connect-service-upstreams) | Defined by user in Kubernetes |
| Traffic permissions | Enables L4 traffic authorization according to workload identity instead of service identity. | [Service intentions](/consul/docs/connect/intentions) | None | Defined by user in Consul |

Refer to [`consul resource`](/consul/docs/k8s/multiport/reference/resource-command) for more information about using the Consul CLI to interact with the v2 catalog.

## Changes to Consul’s existing architecture

The change in data models introduced by the v2 Catalog API impacts several aspects of Consul’s operations.

### Traffic permissions resource replaces service intentions

The most significant change to Consul’s architecture and operations when using the v2 catalog structure is the introduction of the traffic permissions resource. This resource replaces the service intentions configuration entry, and enables authorized service-to-service communication for both L4 and L7 applications.

For more information about this resource, including example configurations, refer to [Traffic permissions configuration reference](/consul/docs/k8s/multiport/reference/trafficpermissions).

### HTTPRoute, GRPCRoute, and TCPRoute resources for traffic management

You can configure traffic management behavior such as service splitting in an `HTTPRoute`, `GRPCRoute`, or `TCPRoute` resource. In the v1 catalog, this behavior is defined in dedicated configuration entries. For examples, refer to [service splitter configuration entry reference](/consul/docs/connect/config-entries/service-splitter#examples).

For more information about these resource, including specifications and example configurations, refer to [HTTPRoute resource configuration reference](/consul/docs/k8s/multiport/reference/httproute), [GRPCRoute resource configuration reference](/consul/docs/k8s/multiport/reference/grpcroute), and [TCPRoute resource configuration reference](/consul/docs/k8s/multiport/reference/tcproute).

### New proxy configuration resource

In the v1 catalog, a service’s sidecar proxy and its behavior is [defined in the `Proxy` field of the service definition](/consul/docs/services/usage/define-services). You can also separately [define a service mesh proxy](/consul/docs/connect/proxies/deploy-service-mesh-proxies) and [configure proxy defaults](/consul/docs/connect/config-entries/proxy-defaults).

In the v2 catalog, the `ProxyConfiguration` resource configures a workload's sidecar proxy behavior according to Consul workload identity. Refer to [ProxyConfiguration resource configuration reference](/consul/docs/k8s/multiport/reference/proxyconfiguration) for more information.

## Constraints and limitations

Be aware of the following constraints and technical limitations on the v2 catalog API:

- The v2 catalog API only supports deployments using [Consul dataplanes](/consul/docs/connect/dataplane) instead of client agents. Consul on Kubernetes uses dataplanes by default.
- The v1 and v2 catalog APIs cannot run concurrently.
- The Consul UI does not support the v2 catalog API in this release. You must disable the UI in the Helm chart in order to use the v2 catalog API.
- HCP Consul does not support the v2 catalog API in this release. You cannot [link a self-managed cluster to HCP Consul](/hcp/docs/consul/self-managed) to access its UI or view observability metrics when it uses the v2 catalog.
- We do not recommend updating existing clusters to enable the v2 catalog in this release. Instead, deploy a new Consul cluster and [enable the v2 catalog in the Helm chart](/consul/docs/k8s/multiport/configure#enable-the-v2-catalog).

## Guidance

The following resources are available to help you use the v2 catalog API:

### Usage documentation

- [Multi-port services overview](/consul/docs/k8s/multiport/configure)
boruszak marked this conversation as resolved.
Show resolved Hide resolved
- [Configure multi-port services](/consul/docs/k8s/multiport/configure)
boruszak marked this conversation as resolved.
Show resolved Hide resolved

### Reference documentation

- [`consul resource` CLI command](/consul/docs/k8s/multiport/reference/resource-command)
- [GRPCRoute configuration reference](/consul/docs/k8s/multiport/reference/grpcroute)
- [HTTPRoute configuration reference](/consul/docs/k8s/multiport/reference/httproute)
- [ProxyConfiguration configuration reference](/consul/docs/k8s/multiport/reference/proxyconfiguration)
- [TCPRoute configuration reference](/consul/docs/k8s/multiport/reference/tcproute)
- [TrafficPermissions configuration reference](/consul/docs/k8s/multiport/reference/trafficpermissions)
Loading