Backport of member cli: add -filter expression to flags into release/1.15.x #18275

* perf: Remove expensive reflection from raft/mesh hot path Replaces a reflection-based copy of a struct in the mesh topology with a deep-copy generated implementation. This is in the hot-path of raft FSM updates, and the reflection overhead was a substantial part of mesh registration times (~90%). This could manifest as raft thread saturation, and resulting instability. Co-authored-by: Joel Brandhorst <[email protected]> * add changelog --------- Co-authored-by: Joel Brandhorst <[email protected]> Co-authored-by: John Murret <[email protected]>

…tender` (#17491)

`property-override` is an extension that allows for arbitrarily patching Envoy resources based on resource matching filters. Patch operations resemble a subset of the JSON Patch spec with minor differences to facilitate patching pre-defined (protobuf) schemas. See Envoy Extension product documentation for more details. Co-authored-by: Eric Haberkorn <[email protected]> Co-authored-by: Kyle Havlovitz <[email protected]>

…#17497) * Support Listener in Property Override Add support for patching `Listener` resources via the builtin `property-override` extension. Refactor existing listener patch code in `BasicEnvoyExtender` to simplify addition of resource support. * Support ClusterLoadAssignment in Property Override Add support for patching `ClusterLoadAssignment` resources via the builtin `property-override` extension.

* Move hcp client to subpackage hcpclient (#16800) * [HCP Observability] New MetricsClient (#17100) * Client configured with TLS using HCP config and retry/throttle * Add tests and godoc for metrics client * close body after request * run go mod tidy * Remove one abstraction to use the config from deps * Address PR feedback * remove clone * Extract CloudConfig and mock for future PR * Switch to hclog.FromContext * [HCP Observability] OTELExporter (#17128) * Client configured with TLS using HCP config and retry/throttle * run go mod tidy * Remove one abstraction to use the config from deps * Address PR feedback * Client configured with TLS using HCP config and retry/throttle * run go mod tidy * Create new OTELExporter which uses the MetricsClient Add transform because the conversion is in an /internal package * Fix lint error * early return when there are no metrics * Add NewOTELExporter() function * Downgrade to metrics SDK version: v1.15.0-rc.1 * Fix imports * fix small nits with comments and url.URL * Fix tests by asserting actual error for context cancellation, fix parallel, and make mock more versatile * Cleanup error handling and clarify empty metrics case * Fix input/expected naming in otel_transform_test.go * add comment for metric tracking * Add a general isEmpty method * Add clear error types * update to latest version 1.15.0 of OTEL * [HCP Observability] OTELSink (#17159) * Client configured with TLS using HCP config and retry/throttle * run go mod tidy * Remove one abstraction to use the config from deps * Address PR feedback * Client configured with TLS using HCP config and retry/throttle * run go mod tidy * Create new OTELExporter which uses the MetricsClient Add transform because the conversion is in an /internal package * Fix lint error * early return when there are no metrics * Add NewOTELExporter() function * Downgrade to metrics SDK version: v1.15.0-rc.1 * Fix imports * fix small nits with comments and url.URL * Fix tests by asserting actual error for context cancellation, fix parallel, and make mock more versatile * Cleanup error handling and clarify empty metrics case * Fix input/expected naming in otel_transform_test.go * add comment for metric tracking * Add a general isEmpty method * Add clear error types * update to latest version 1.15.0 of OTEL * Client configured with TLS using HCP config and retry/throttle * run go mod tidy * Remove one abstraction to use the config from deps * Address PR feedback * Initialize OTELSink with sync.Map for all the instrument stores. * Moved PeriodicReader init to NewOtelReader function. This allows us to use a ManualReader for tests. * Switch to mutex instead of sync.Map to avoid type assertion * Add gauge store * Clarify comments * return concrete sink type * Fix lint errors * Move gauge store to be within sink * Use context.TODO,rebase and clenaup opts handling * Rebase onto otl exporter to downgrade metrics API to v1.15.0-rc.1 * Fix imports * Update to latest stable version by rebasing on cc-4933, fix import, remove mutex init, fix opts error messages and use logger from ctx * Add lots of documentation to the OTELSink * Fix gauge store comment and check ok * Add select and ctx.Done() check to gauge callback * use require.Equal for attributes * Fixed import naming * Remove float64 calls and add a NewGaugeStore method * Change name Store to Set in gaugeStore, add concurrency tests in both OTELSink and gauge store * Generate 100 gauge operations * Seperate the labels into goroutines in sink test * Generate kv store for the test case keys to avoid using uuid * Added a race test with 300 samples for OTELSink * Do not pass in waitgroup and use error channel instead. * Using SHA 7dea2225a218872e86d2f580e82c089b321617b0 to avoid build failures in otel * Fix nits * [HCP Observability] Init OTELSink in Telemetry (#17162) * Move hcp client to subpackage hcpclient (#16800) * [HCP Observability] New MetricsClient (#17100) * Client configured with TLS using HCP config and retry/throttle * Add tests and godoc for metrics client * close body after request * run go mod tidy * Remove one abstraction to use the config from deps * Address PR feedback * remove clone * Extract CloudConfig and mock for future PR * Switch to hclog.FromContext * [HCP Observability] New MetricsClient (#17100) * Client configured with TLS using HCP config and retry/throttle * Add tests and godoc for metrics client * close body after request * run go mod tidy * Remove one abstraction to use the config from deps * Address PR feedback * remove clone * Extract CloudConfig and mock for future PR * Switch to hclog.FromContext * [HCP Observability] New MetricsClient (#17100) * Client configured with TLS using HCP config and retry/throttle * Add tests and godoc for metrics client * close body after request * run go mod tidy * Remove one abstraction to use the config from deps * Address PR feedback * remove clone * Extract CloudConfig and mock for future PR * Switch to hclog.FromContext * Client configured with TLS using HCP config and retry/throttle * run go mod tidy * Remove one abstraction to use the config from deps * Address PR feedback * Client configured with TLS using HCP config and retry/throttle * run go mod tidy * Create new OTELExporter which uses the MetricsClient Add transform because the conversion is in an /internal package * Fix lint error * early return when there are no metrics * Add NewOTELExporter() function * Downgrade to metrics SDK version: v1.15.0-rc.1 * Fix imports * fix small nits with comments and url.URL * Fix tests by asserting actual error for context cancellation, fix parallel, and make mock more versatile * Cleanup error handling and clarify empty metrics case * Fix input/expected naming in otel_transform_test.go * add comment for metric tracking * Add a general isEmpty method * Add clear error types * update to latest version 1.15.0 of OTEL * Client configured with TLS using HCP config and retry/throttle * run go mod tidy * Remove one abstraction to use the config from deps * Address PR feedback * Initialize OTELSink with sync.Map for all the instrument stores. * Moved PeriodicReader init to NewOtelReader function. This allows us to use a ManualReader for tests. * Switch to mutex instead of sync.Map to avoid type assertion * Add gauge store * Clarify comments * return concrete sink type * Fix lint errors * Move gauge store to be within sink * Use context.TODO,rebase and clenaup opts handling * Rebase onto otl exporter to downgrade metrics API to v1.15.0-rc.1 * Fix imports * Update to latest stable version by rebasing on cc-4933, fix import, remove mutex init, fix opts error messages and use logger from ctx * Add lots of documentation to the OTELSink * Fix gauge store comment and check ok * Add select and ctx.Done() check to gauge callback * use require.Equal for attributes * Fixed import naming * Remove float64 calls and add a NewGaugeStore method * Change name Store to Set in gaugeStore, add concurrency tests in both OTELSink and gauge store * Generate 100 gauge operations * Seperate the labels into goroutines in sink test * Generate kv store for the test case keys to avoid using uuid * Added a race test with 300 samples for OTELSink * [HCP Observability] OTELExporter (#17128) * Client configured with TLS using HCP config and retry/throttle * run go mod tidy * Remove one abstraction to use the config from deps * Address PR feedback * Client configured with TLS using HCP config and retry/throttle * run go mod tidy * Create new OTELExporter which uses the MetricsClient Add transform because the conversion is in an /internal package * Fix lint error * early return when there are no metrics * Add NewOTELExporter() function * Downgrade to metrics SDK version: v1.15.0-rc.1 * Fix imports * fix small nits with comments and url.URL * Fix tests by asserting actual error for context cancellation, fix parallel, and make mock more versatile * Cleanup error handling and clarify empty metrics case * Fix input/expected naming in otel_transform_test.go * add comment for metric tracking * Add a general isEmpty method * Add clear error types * update to latest version 1.15.0 of OTEL * Do not pass in waitgroup and use error channel instead. * Using SHA 7dea2225a218872e86d2f580e82c089b321617b0 to avoid build failures in otel * Rebase onto otl exporter to downgrade metrics API to v1.15.0-rc.1 * Initialize OTELSink with sync.Map for all the instrument stores. * Added telemetry agent to client and init sink in deps * Fixed client * Initalize sink in deps * init sink in telemetry library * Init deps before telemetry * Use concrete telemetry.OtelSink type * add /v1/metrics * Avoid returning err for telemetry init * move sink init within the IsCloudEnabled() * Use HCPSinkOpts in deps instead * update golden test for configuration file * Switch to using extra sinks in the telemetry library * keep name MetricsConfig * fix log in verifyCCMRegistration * Set logger in context * pass around MetricSink in deps * Fix imports * Rebased onto otel sink pr * Fix URL in test * [HCP Observability] OTELSink (#17159) * Client configured with TLS using HCP config and retry/throttle * run go mod tidy * Remove one abstraction to use the config from deps * Address PR feedback * Client configured with TLS using HCP config and retry/throttle * run go mod tidy * Create new OTELExporter which uses the MetricsClient Add transform because the conversion is in an /internal package * Fix lint error * early return when there are no metrics * Add NewOTELExporter() function * Downgrade to metrics SDK version: v1.15.0-rc.1 * Fix imports * fix small nits with comments and url.URL * Fix tests by asserting actual error for context cancellation, fix parallel, and make mock more versatile * Cleanup error handling and clarify empty metrics case * Fix input/expected naming in otel_transform_test.go * add comment for metric tracking * Add a general isEmpty method * Add clear error types * update to latest version 1.15.0 of OTEL * Client configured with TLS using HCP config and retry/throttle * run go mod tidy * Remove one abstraction to use the config from deps * Address PR feedback * Initialize OTELSink with sync.Map for all the instrument stores. * Moved PeriodicReader init to NewOtelReader function. This allows us to use a ManualReader for tests. * Switch to mutex instead of sync.Map to avoid type assertion * Add gauge store * Clarify comments * return concrete sink type * Fix lint errors * Move gauge store to be within sink * Use context.TODO,rebase and clenaup opts handling * Rebase onto otl exporter to downgrade metrics API to v1.15.0-rc.1 * Fix imports * Update to latest stable version by rebasing on cc-4933, fix import, remove mutex init, fix opts error messages and use logger from ctx * Add lots of documentation to the OTELSink * Fix gauge store comment and check ok * Add select and ctx.Done() check to gauge callback * use require.Equal for attributes * Fixed import naming * Remove float64 calls and add a NewGaugeStore method * Change name Store to Set in gaugeStore, add concurrency tests in both OTELSink and gauge store * Generate 100 gauge operations * Seperate the labels into goroutines in sink test * Generate kv store for the test case keys to avoid using uuid * Added a race test with 300 samples for OTELSink * Do not pass in waitgroup and use error channel instead. * Using SHA 7dea2225a218872e86d2f580e82c089b321617b0 to avoid build failures in otel * Fix nits * pass extraSinks as function param instead * Add default interval as package export * remove verifyCCM func * Add clusterID * Fix import and add t.Parallel() for missing tests * Kick Vercel CI * Remove scheme from endpoint path, and fix error logging * return metrics.MetricSink for sink method * Update SDK * [HCP Observability] Metrics filtering and Labels in Go Metrics sink (#17184) * Move hcp client to subpackage hcpclient (#16800) * [HCP Observability] New MetricsClient (#17100) * Client configured with TLS using HCP config and retry/throttle * Add tests and godoc for metrics client * close body after request * run go mod tidy * Remove one abstraction to use the config from deps * Address PR feedback * remove clone * Extract CloudConfig and mock for future PR * Switch to hclog.FromContext * [HCP Observability] New MetricsClient (#17100) * Client configured with TLS using HCP config and retry/throttle * Add tests and godoc for metrics client * close body after request * run go mod tidy * Remove one abstraction to use the config from deps * Address PR feedback * remove clone * Extract CloudConfig and mock for future PR * Switch to hclog.FromContext * [HCP Observability] New MetricsClient (#17100) * Client configured with TLS using HCP config and retry/throttle * Add tests and godoc for metrics client * close body after request * run go mod tidy * Remove one abstraction to use the config from deps * Address PR feedback * remove clone * Extract CloudConfig and mock for future PR * Switch to hclog.FromContext * Client configured with TLS using HCP config and retry/throttle * run go mod tidy * Remove one abstraction to use the config from deps * Address PR feedback * Client configured with TLS using HCP config and retry/throttle * run go mod tidy * Create new OTELExporter which uses the MetricsClient Add transform because the conversion is in an /internal package * Fix lint error * early return when there are no metrics * Add NewOTELExporter() function * Downgrade to metrics SDK version: v1.15.0-rc.1 * Fix imports * fix small nits with comments and url.URL * Fix tests by asserting actual error for context cancellation, fix parallel, and make mock more versatile * Cleanup error handling and clarify empty metrics case * Fix input/expected naming in otel_transform_test.go * add comment for metric tracking * Add a general isEmpty method * Add clear error types * update to latest version 1.15.0 of OTEL * Client configured with TLS using HCP config and retry/throttle * run go mod tidy * Remove one abstraction to use the config from deps * Address PR feedback * Initialize OTELSink with sync.Map for all the instrument stores. * Moved PeriodicReader init to NewOtelReader function. This allows us to use a ManualReader for tests. * Switch to mutex instead of sync.Map to avoid type assertion * Add gauge store * Clarify comments * return concrete sink type * Fix lint errors * Move gauge store to be within sink * Use context.TODO,rebase and clenaup opts handling * Rebase onto otl exporter to downgrade metrics API to v1.15.0-rc.1 * Fix imports * Update to latest stable version by rebasing on cc-4933, fix import, remove mutex init, fix opts error messages and use logger from ctx * Add lots of documentation to the OTELSink * Fix gauge store comment and check ok * Add select and ctx.Done() check to gauge callback * use require.Equal for attributes * Fixed import naming * Remove float64 calls and add a NewGaugeStore method * Change name Store to Set in gaugeStore, add concurrency tests in both OTELSink and gauge store * Generate 100 gauge operations * Seperate the labels into goroutines in sink test * Generate kv store for the test case keys to avoid using uuid * Added a race test with 300 samples for OTELSink * [HCP Observability] OTELExporter (#17128) * Client configured with TLS using HCP config and retry/throttle * run go mod tidy * Remove one abstraction to use the config from deps * Address PR feedback * Client configured with TLS using HCP config and retry/throttle * run go mod tidy * Create new OTELExporter which uses the MetricsClient Add transform because the conversion is in an /internal package * Fix lint error * early return when there are no metrics * Add NewOTELExporter() function * Downgrade to metrics SDK version: v1.15.0-rc.1 * Fix imports * fix small nits with comments and url.URL * Fix tests by asserting actual error for context cancellation, fix parallel, and make mock more versatile * Cleanup error handling and clarify empty metrics case * Fix input/expected naming in otel_transform_test.go * add comment for metric tracking * Add a general isEmpty method * Add clear error types * update to latest version 1.15.0 of OTEL * Do not pass in waitgroup and use error channel instead. * Using SHA 7dea2225a218872e86d2f580e82c089b321617b0 to avoid build failures in otel * Rebase onto otl exporter to downgrade metrics API to v1.15.0-rc.1 * Initialize OTELSink with sync.Map for all the instrument stores. * Added telemetry agent to client and init sink in deps * Fixed client * Initalize sink in deps * init sink in telemetry library * Init deps before telemetry * Use concrete telemetry.OtelSink type * add /v1/metrics * Avoid returning err for telemetry init * move sink init within the IsCloudEnabled() * Use HCPSinkOpts in deps instead * update golden test for configuration file * Switch to using extra sinks in the telemetry library * keep name MetricsConfig * fix log in verifyCCMRegistration * Set logger in context * pass around MetricSink in deps * Fix imports * Rebased onto otel sink pr * Fix URL in test * [HCP Observability] OTELSink (#17159) * Client configured with TLS using HCP config and retry/throttle * run go mod tidy * Remove one abstraction to use the config from deps * Address PR feedback * Client configured with TLS using HCP config and retry/throttle * run go mod tidy * Create new OTELExporter which uses the MetricsClient Add transform because the conversion is in an /internal package * Fix lint error * early return when there are no metrics * Add NewOTELExporter() function * Downgrade to metrics SDK version: v1.15.0-rc.1 * Fix imports * fix small nits with comments and url.URL * Fix tests by asserting actual error for context cancellation, fix parallel, and make mock more versatile * Cleanup error handling and clarify empty metrics case * Fix input/expected naming in otel_transform_test.go * add comment for metric tracking * Add a general isEmpty method * Add clear error types * update to latest version 1.15.0 of OTEL * Client configured with TLS using HCP config and retry/throttle * run go mod tidy * Remove one abstraction to use the config from deps * Address PR feedback * Initialize OTELSink with sync.Map for all the instrument stores. * Moved PeriodicReader init to NewOtelReader function. This allows us to use a ManualReader for tests. * Switch to mutex instead of sync.Map to avoid type assertion * Add gauge store * Clarify comments * return concrete sink type * Fix lint errors * Move gauge store to be within sink * Use context.TODO,rebase and clenaup opts handling * Rebase onto otl exporter to downgrade metrics API to v1.15.0-rc.1 * Fix imports * Update to latest stable version by rebasing on cc-4933, fix import, remove mutex init, fix opts error messages and use logger from ctx * Add lots of documentation to the OTELSink * Fix gauge store comment and check ok * Add select and ctx.Done() check to gauge callback * use require.Equal for attributes * Fixed import naming * Remove float64 calls and add a NewGaugeStore method * Change name Store to Set in gaugeStore, add concurrency tests in both OTELSink and gauge store * Generate 100 gauge operations * Seperate the labels into goroutines in sink test * Generate kv store for the test case keys to avoid using uuid * Added a race test with 300 samples for OTELSink * Do not pass in waitgroup and use error channel instead. * Using SHA 7dea2225a218872e86d2f580e82c089b321617b0 to avoid build failures in otel * Fix nits * pass extraSinks as function param instead * Add default interval as package export * remove verifyCCM func * Add clusterID * Fix import and add t.Parallel() for missing tests * Kick Vercel CI * Remove scheme from endpoint path, and fix error logging * return metrics.MetricSink for sink method * Update SDK * Added telemetry agent to client and init sink in deps * Add node_id and __replica__ default labels * add function for default labels and set x-hcp-resource-id * Fix labels tests * Commit suggestion for getDefaultLabels Co-authored-by: Joshua Timmons <[email protected]> * Fixed server.id, and t.Parallel() * Make defaultLabels a method on the TelemetryConfig object * Rename FilterList to lowercase filterList * Cleanup filter implemetation by combining regex into a single one, and making the type lowercase * Fix append * use regex directly for filters * Fix x-resource-id test to use mocked value * Fix log.Error formats * Forgot the len(opts.Label) optimization) * Use cfg.NodeID instead --------- Co-authored-by: Joshua Timmons <[email protected]> * remove replic tag (#17484) * [HCP Observability] Add custom metrics for OTEL sink, improve logging, upgrade modules and cleanup metrics client (#17455) * Add custom metrics for Exporter and transform operations * Improve deps logging Run go mod tidy * Upgrade SDK and OTEL * Remove the partial success implemetation and check for HTTP status code in metrics client * Add x-channel * cleanup logs in deps.go based on PR feedback * Change to debug log and lowercase * address test operation feedback * use GetHumanVersion on version * Fix error wrapping * Fix metric names * [HCP Observability] Turn off retries for now until dynamically configurable (#17496) * Remove retries for now until dynamic configuration is possible * Clarify comment * Update changelog * improve changelog --------- Co-authored-by: Joshua Timmons <[email protected]>

* [NET-3092] JWT Verify claims handling

…'s down (#17511)

…emory and disk usage (#17038)

* add upstream service targeting to property override extension * Also add baseline goldens for service specific property override extension. * Refactor the extension framework to put more logic into the templates. * fix up the golden tests

* add FIPS verison info * separate out feature functionality from build identification * split out ent test * add version endpoint

* separate fips gossip * clean up

This commit only contains the OSS PR (datacenter query param support). A separate enterprise PR adds support for ap and namespace query params. Resources in Consul can exists within scopes such as datacenters, cluster peers, admin partitions, and namespaces. You can refer to those resources from interfaces such as the CLI, HTTP API, DNS, and configuration files. Some scope levels have consistent naming: cluster peers are always referred to as "peer". Other scope levels use a short-hand in DNS lookups... - "ns" for namespace - "ap" for admin partition - "dc" for datacenter ...But use long-hand in CLI commands: - "namespace" for namespace - "partition" for admin partition - and "datacenter" However, HTTP API query parameters do not follow a consistent pattern, supporting short-hand for some scopes but long-hand for others: - "ns" for namespace - "partition" for admin partition - and "dc" for datacenter. This inconsistency is confusing, especially for users who have been exposed to providing scope names through another interface such as CLI or DNS queries. This commit improves UX by consistently supporting both short-hand and long-hand forms of the namespace, partition, and datacenter scopes in HTTP API query parameters.

* Sujata's peering-cli branch * Added error message for connecting to cluster * We can export service to peer * export handling multiple peers * export handles multiple peers * export now can handle multiple services * Export after 1st cleanup * Successful export * Added the namespace option * Add .changelog entry * go mod tidy * Stub unit tests for peering export command * added export in peering.go * Adding export_test * Moved the code to services from peers and cleaned the serviceNamespace * Added support for exporting to partitions * Fixed partition bug * Added unit tests for export command * Add multi-tenancy flags * gofmt * Add some helpful comments * Exclude namespace + partition flags when running OSS * cleaned up partition stuff * Validate required flags differently for OSS vs. ENT * Update success output to include only the requested consumers * cleaned up * fixed broken test * gofmt * Include all flags in OSS build * Remove example previously added to peering command * Move stray import into correct block * Update changelog entry to include support for exporting to a partition * Add required-ness label to consumer-peers flag description * Update command/services/export/export.go Co-authored-by: Dan Stough <[email protected]> * Add docs placeholder for new services export command * Moved piece of code to OSS * Break config entry init + update into separate functions * fixed * Vary existing service export comparison for OSS vs. ENT * Move OSS-specific test to export_oss_test.go * Set config entry name based on partition being exported from * Set namespace on added services * Adding namespace * Remove export documentation We will include documentation in a followup PR * Consolidate code from export_oss into export.go * Consolidated export_oss_test.go and export_test.go * Add example of partition export to command synopsis * Allow empty peers flag if partitions flag provided * Add test coverage for -consumer-partitions flag * Update command/services/export/export.go Co-authored-by: Jared Kirschner <[email protected]> * Update command/services/export/export.go Co-authored-by: Jared Kirschner <[email protected]> * Update changelog entry * Use "cluster peers" to clear up any possible confusion * Update test assertions --------- Co-authored-by: 20sr20 <[email protected]> Co-authored-by: Dan Stough <[email protected]> Co-authored-by: Jared Kirschner <[email protected]>

Sameness groups with default-for-failover enabled did not function properly with tproxy whenever all instances of the service disappeared from the local cluster. This occured, because there were no corresponding resolvers (due to the implicit failover policy) which caused VIPs to be deallocated. This ticket expands upon the VIP allocations so that both service-defaults and service-intentions (without destination wildcards) will ensure that the virtual IP exists.

* [API Gateway] Fix use of virtual resolvers in routes * Add changelog entry

When UpstreamEnvoyExtender was introduced, some code was left duplicated between it and BasicEnvoyExtender. One path in that code panics when a TProxy listener patch is attempted due to no upstream data in RuntimeConfig matching the local service (which would only happen in rare cases). Instead, we can remove the special handling of upstream VIPs from BasicEnvoyExtender entirely, greatly simplifying the listener filter patch code and avoiding the panic. UpstreamEnvoyExtender, which needs this code to function, is modified to ensure a panic does not occur. This also fixes a second regression in which the Lua extension was not applied to TProxy outbound listeners.

…tial xDS… (#17317)" (#17540) This reverts commit be7d2a4.

* docs: add note about Nomad support for Consul 1.13.8 * apply code review changes

* updates `consul.service` systemd service unit to use `Type=notify` to resolve issue #16844 * add changelog update to match

…a snapshot restore occurring (#17566) * Fix up case where subscription is terminated due to ACLs changing or a snapshot restore occurring * Add changelog entry * Switch to use errors.Is

* generate docs

TLDR with many modules the versions included in each diverged quite a bit. Attempting to use Go Workspaces produces a bunch of errors. This commit: 1. Fixes envoy-library-references.sh to work again 2. Ensures we are pulling in [email protected] everywhere (previously it was at that version in some modules and others were much older) 3. Remove one usage of golang/protobuf that caused us to have a direct dependency on it. 4. Remove deprecated usage of the Endpoint field in the grpc resolver.Target struct. The current version of grpc (v1.55.0) has removed that field and recommended replacement with URL.Opaque and calls to the Endpoint() func when needing to consume the previous field. 4. `go work init <all the paths to go.mod files>` && `go work sync`. This syncrhonized versions of dependencies from the main workspace/root module to all submodules 5. Updated .gitignore to ignore the go.work and go.work.sum files. This seems to be standard practice at the moment. 6. Update doc comments in protoc-gen-consul-rate-limit to be go fmt compatible 7. Upgraded makefile infra to perform linting, testing and go mod tidy on all modules in a flexible manner. 8. Updated linter rules to prevent usage of golang/protobuf 9. Updated a leader peering test to account for an extra colon in a grpc error message.

* docs: clarify the behavior of prepending hostname to metrics * Update website/content/docs/agent/config/config-files.mdx Co-authored-by: trujillo-adam <[email protected]> --------- Co-authored-by: trujillo-adam <[email protected]>

* Fix subscribing/fetching objects not in the default namespace * add changelog

Ensure that the embedded api struct is properly parsed when deserializing config containing a set ResourceFilter.Services field. Also enhance existing integration test to guard against bugs and exercise this field.

* Add a ReplaceType dep mapper and move them into their own file * Implement the service endpoints controller * Implement a Catalog Controllers Integration Test

…ges (#17599)

Fix some linter warnings before updating the lint-consul-retry code in hashicorp/lint-consul-retry#4

More validation is needed to ensure this behaves as expected; in the meantime, align with docs and disable this proxy type.

* chore: update dev build to 1.17 * chore(ci): add nightly 1.16 test Drop the oldest and add the newest running release branch to nightly builds.

* Add writeAuditRPCEvent to agent_oss * fix the other diffs * backport change log

…sis code (#17609) * [API Gateway] Fix trust domain for external peered services in synthesis code * Add changelog

* backport ent changes to oss * Update .changelog/_5669.txt Co-authored-by: Michael Zalimeni <[email protected]> --------- Co-authored-by: Michael Zalimeni <[email protected]>

Make behaviour of L7 intentions clearer

* [API Gateway] Fix rate limiting for API gateways * Add changelog * Fix failing unit tests * Fix operator usage tests for api package

…8s (#17624) * porting over changes from enterprise repo to oss * applied feedback on service mesh for k8s overview * fixed typo * removed ent-only build script file * Apply suggestions from code review Co-authored-by: Jeff Boruszak <[email protected]> * Apply suggestions from code review Co-authored-by: David Yu <[email protected]> Co-authored-by: Jeff Boruszak <[email protected]> --------- Co-authored-by: Jeff Boruszak <[email protected]> Co-authored-by: David Yu <[email protected]>

* Reference doc updates for permissive mTLS settings * Document config entry filtering * Fix minor doc errors (double slashes in link url paths) --------- Co-authored-by: trujillo-adam <[email protected]>

… resources (#17604) * Add generic experiments configuration and use it to enable catalog v2 resources * Run formatting with -s as CI will validate that this has been done

…rating xDS (#17644) * Add header filter to api-gateway xDS golden test * Stop adding all header filters to virtual host when generating xDS for api-gateway * Regenerate xDS golden file for api-gateway w/ header filter

* Merge pull request #5773 from hashicorp/docs/rate-limiting-from-ip-addresses-1.16 updated docs for rate limiting for IP addresses - 1.16 * Merge pull request #5609 from hashicorp/docs/enterprise-utilization-reporting Add docs for enterprise utilization reporting * Merge pull request #5734 from hashicorp/docs/envoy-ext-1.16 Docs/envoy ext 1.16 * Merge pull request #5773 from hashicorp/docs/rate-limiting-from-ip-addresses-1.16 updated docs for rate limiting for IP addresses - 1.16 * Merge pull request #5609 from hashicorp/docs/enterprise-utilization-reporting Add docs for enterprise utilization reporting * Merge pull request #5734 from hashicorp/docs/envoy-ext-1.16 Docs/envoy ext 1.16 * fix build errors --------- Co-authored-by: trujillo-adam <[email protected]>

- Update changelog to include new entries from release - Update submodule versions to latest published

* port from enterprise branch * Apply suggestions from code review Co-authored-by: shanafarkas <[email protected]> * Update website/content/docs/connect/cluster-peering/usage/create-sameness-groups.mdx * next steps * Update website/content/docs/connect/cluster-peering/usage/create-sameness-groups.mdx Co-authored-by: trujillo-adam <[email protected]> * Update website/content/docs/k8s/connect/cluster-peering/usage/create-sameness-groups.mdx Co-authored-by: trujillo-adam <[email protected]> --------- Co-authored-by: shanafarkas <[email protected]> Co-authored-by: trujillo-adam <[email protected]>

* trimmed CRD step and reqs from installation * updated tech specs * Apply suggestions from code review Co-authored-by: Jeff Boruszak <[email protected]> Co-authored-by: Jeff Apple <[email protected]> * added upgrade instruction * removed tcp port req * described downtime and DT-less upgrades * applied additional review feedback --------- Co-authored-by: Jeff Boruszak <[email protected]> Co-authored-by: Jeff Apple <[email protected]>

* additional feedback * Update website/content/docs/api-gateway/upgrades.mdx Co-authored-by: Jeff Apple <[email protected]> --------- Co-authored-by: Jeff Apple <[email protected]>

* Initial page/nav creation * configuration entry reference page * Usage + fixes * service intentions page * usage * description * config entry updates * formatting fixes * Update website/content/docs/connect/config-entries/service-intentions.mdx Co-authored-by: Paul Glass <[email protected]> * service intentions review fixes * Overview page review fixes * Apply suggestions from code review Co-authored-by: trujillo-adam <[email protected]> --------- Co-authored-by: Paul Glass <[email protected]> Co-authored-by: trujillo-adam <[email protected]>

* Fixes * service intentions fixes

* Merge pull request #5773 from hashicorp/docs/rate-limiting-from-ip-addresses-1.16 updated docs for rate limiting for IP addresses - 1.16 * Merge pull request #5609 from hashicorp/docs/enterprise-utilization-reporting Add docs for enterprise utilization reporting * Merge pull request #5734 from hashicorp/docs/envoy-ext-1.16 Docs/envoy ext 1.16 * Add release notes for 1.16-rc * Add consul-e license utlization reporting * Update with rc absolute links * Update with rc absolute links * fix typo * Apply suggestions from code review Co-authored-by: trujillo-adam <[email protected]> * Update to use callout component * address typo * docs: FIPS 140-2 Compliance (#17668) * Page + nav + formatting * link fix * Update website/content/docs/enterprise/fips.mdx Co-authored-by: trujillo-adam <[email protected]> * Update website/content/docs/enterprise/fips.mdx Co-authored-by: trujillo-adam <[email protected]> * Update website/content/docs/enterprise/fips.mdx Co-authored-by: trujillo-adam <[email protected]> * Update website/content/docs/enterprise/fips.mdx Co-authored-by: trujillo-adam <[email protected]> * Update website/content/docs/enterprise/fips.mdx Co-authored-by: trujillo-adam <[email protected]> * Update website/content/docs/enterprise/fips.mdx Co-authored-by: trujillo-adam <[email protected]> * Update website/content/docs/enterprise/fips.mdx Co-authored-by: trujillo-adam <[email protected]> * Update website/content/docs/enterprise/fips.mdx Co-authored-by: trujillo-adam <[email protected]> * Update website/content/docs/enterprise/fips.mdx Co-authored-by: trujillo-adam <[email protected]> * Update website/content/docs/enterprise/fips.mdx Co-authored-by: trujillo-adam <[email protected]> * link fix * Apply suggestions from code review Co-authored-by: Jeff Apple <[email protected]> * Update website/content/docs/enterprise/fips.mdx --------- Co-authored-by: trujillo-adam <[email protected]> Co-authored-by: Jeff Apple <[email protected]> * fix apigw install values file * fix typos in release notes --------- Co-authored-by: trujillo-adam <[email protected]> Co-authored-by: Jeff Boruszak <[email protected]> Co-authored-by: Jeff Apple <[email protected]>

* adding redirects * Apply suggestions from code review

* fix release notes links * fix typos on fips docs

… 5m for consul-debug (#17596) * changed duration to 5 mins and log level to trace * documentation update * change log

This includes prioritize by localities on disco chain targets rather than resolvers, allowing different targets within the same partition to have different policies.

…te management (#17075) * agent: remove agent cache dependency from service mesh leaf certificate management This extracts the leaf cert management from within the agent cache. This code was produced by the following process: 1. All tests in agent/cache, agent/cache-types, agent/auto-config, agent/consul/servercert were run at each stage. - The tests in agent matching .*Leaf were run at each stage. - The tests in agent/leafcert were run at each stage after they existed. 2. The former leaf cert Fetch implementation was extracted into a new package behind a "fake RPC" endpoint to make it look almost like all other cache type internals. 3. The old cache type was shimmed to use the fake RPC endpoint and generally cleaned up. 4. I selectively duplicated all of Get/Notify/NotifyCallback/Prepopulate from the agent/cache.Cache implementation over into the new package. This was renamed as leafcert.Manager. - Code that was irrelevant to the leaf cert type was deleted (inlining blocking=true, refresh=false) 5. Everything that used the leaf cert cache type (including proxycfg stuff) was shifted to use the leafcert.Manager instead. 6. agent/cache-types tests were moved and gently replumbed to execute as-is against a leafcert.Manager. 7. Inspired by some of the locking changes from derek's branch I split the fat lock into N+1 locks. 8. The waiter chan struct{} was eventually replaced with a singleflight.Group around cache updates, which was likely the biggest net structural change. 9. The awkward two layers or logic produced as a byproduct of marrying the agent cache management code with the leaf cert type code was slowly coalesced and flattened to remove confusion. 10. The .*Leaf tests from the agent package were copied and made to work directly against a leafcert.Manager to increase direct coverage. I have done a best effort attempt to port the previous leaf-cert cache type's tests over in spirit, as well as to take the e2e-ish tests in the agent package with Leaf in the test name and copy those into the agent/leafcert package to get more direct coverage, rather than coverage tangled up in the agent logic. There is no net-new test coverage, just coverage that was pushed around from elsewhere.

* add enterprise notes for IP-based rate limits * Apply suggestions from code review Co-authored-by: Tu Nguyen <[email protected]> Co-authored-by: David Yu <[email protected]> * added bolded 'Enterprise' in list items. --------- Co-authored-by: Tu Nguyen <[email protected]> Co-authored-by: David Yu <[email protected]>

* Update terminating-gateway.mdx * Update exported-services.mdx * Update mesh.mdx

Co-authored-by: trujillo-adam <[email protected]>

* Update Dockerfile * Create 17719.txt

Co-authored-by: trujillo-adam <[email protected]> Co-authored-by: Jared Kirschner <[email protected]>

…consul operator raft list-peers' (#17582) * init * fix tests * added -detailed in docs * added change log * fix doc * checking for entry in map * fix tests * removed detailed flag * removed detailed flag * revert unwanted changes * removed unwanted changes * updated change log * pr review comment changes * pr comment changes single API instead of two * fix change log * fix tests * fix tests * fix test operator raft endpoint test * Update .changelog/17582.txt Co-authored-by: Semir Patel <[email protected]> * nits * updated docs --------- Co-authored-by: Semir Patel <[email protected]>

Bumps [atlassian/gajira-transition](https://github.com/atlassian/gajira-transition) from 3.0.0 to 3.0.1. - [Release notes](https://github.com/atlassian/gajira-transition/releases) - [Commits](atlassian/gajira-transition@4749176...38fc9cd) --- updated-dependencies: - dependency-name: atlassian/gajira-transition dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Incorrect symbol * Clarification * slight edit for clarity

* Update envoy.mdx added more detail around default versus other compatible versions

* init * fix tests * added -detailed in docs * added change log * fix doc * checking for entry in map * fix tests * removed detailed flag * removed detailed flag * revert unwanted changes * removed unwanted changes * updated change log * pr review comment changes * pr comment changes single API instead of two * fix change log * fix tests * fix tests * fix test operator raft endpoint test * Update .changelog/17582.txt Co-authored-by: Semir Patel <[email protected]> * nits * updated docs * explanation added --------- Co-authored-by: Semir Patel <[email protected]>

Fix issue with streaming service health watches. This commit fixes an issue where the health streams were unaware of service export changes. Whenever an exported-services config entry is modified, it is effectively an ACL change. The bug would be triggered by the following situation: - no services are exported - an upstream watch to service X is spawned - the streaming backend filters out data for service X (due to lack of exports) - service X is finally exported In the situation above, the streaming backend does not trigger a refresh of its data. This means that any events that were supposed to have been received prior to the export are NOT backfilled, and the watches never see service X spawning. We currently have decided to not trigger a stream refresh in this situation due to the potential for a thundering herd effect (touching exports would cause a re-fetch of all watches for that partition, potentially). Therefore, a local blocking-query approach was added by this commit for agentless. It's also worth noting that the streaming subscription is currently bypassed most of the time with agentful, because proxycfg has a `req.Source.Node != ""` which prevents the `streamingEnabled` check from passing. This means that while agents should technically have this same issue, they don't experience it with mesh health watches. Note that this is a temporary fix that solves the issue for proxycfg, but not service-discovery use cases.

* Reject inbound Prop Override patch with Services Services filtering is only supported for outbound TrafficDirection patches. * Improve Prop Override unexpected type validation - Guard against additional invalid parent and target types - Add specific error handling for Any fields (unsupported)

This PR is to clarify what happens if the license get command is run on a follower if the leader hasn't been updated with a newer license.

When a patch is found invalid, include its index for easier debugging when multiple patches are provided.

* Stop referenced jwt providers from being deleted

* Implement a Catalog Controllers Lifecycle Integration Test * Prevent triggering the race detector. This allows defining some variables for protobuf constants and using those in comparisons. Without that, something internal in the fmt package ended up looking at the protobuf message size cache and triggering the race detector.

* Implement the Catalog V2 controller integration container tests This now allows the container tests to import things from the root module. However for now we want to be very restrictive about which packages we allow importing. * Add an upgrade test for the new catalog Currently this should be dormant and not executed. However its put in place to detect breaking changes in the future and show an example of how to do an upgrade test with integration tests structured like catalog v2. * Make testutil.Retry capable of performing cleanup operations These cleanup operations are executed after each retry attempt. * Move TestContext to taking an interface instead of a concrete testing.T This allows this to be used on a retry.R or generally anything that meets the interface. * Move to using TestContext instead of background contexts Also this forces all test methods to implement the Cleanup method now instead of that being an optional interface. Co-authored-by: Daniel Upton <[email protected]>

* init * fix tests * added -detailed in docs * added change log * fix doc * checking for entry in map * fix tests * removed detailed flag * removed detailed flag * revert unwanted changes * removed unwanted changes * updated change log * pr review comment changes * pr comment changes single API instead of two * fix change log * fix tests * fix tests * fix test operator raft endpoint test * Update .changelog/17582.txt Co-authored-by: Semir Patel <[email protected]> * nits * updated docs * explanation added * fix doc * fix docs --------- Co-authored-by: Semir Patel <[email protected]>

- Provide more realistics examples for setting properties not already supported natively by Consul - Remove superfluous commas from HCL, correct target service name, and fix service defaults vs. proxy defaults in examples - Align existing integration test to updated docs

…#17747)

* Add documentation for remote debugging of integration tests. * add link from main docs page. * changes related to PR feedback

Explicitly document the limitations of the extension, particularly what kind of fields it is capable of modifying.

* Fix formatting for webhook-certs Consul tutorial * Make a small grammar change to also pick up whitespace changes necessary for formatting --------- Co-authored-by: David Yu <[email protected]>

* [NET-3095] add jwt-authn metrics docs

Consistently set GOPRIVATE to include all hashicorp repos, s.t. private modules are successfully pulled in enterprise CI.

Add missing `-t` flag and fix minor typo.

…ating ACL token (#17739) * AccessorID in request body should be optional when updating ACL token * add a test case * fix test case * add changelog entry for PR #17739

Update CA provider docs Clarify that providers can differ between primary and secondary datacenters Provide a comparison chart for consul vs vault CA providers Loosen Vault CA provider validation for RootPKIPath Update Vault CA provider documentation

…#17821)

* Ensure that git access to private repos uses the ELEVATED_GITHUB_TOKEN * Bump the runner size for the protobuf generation check This has failed previously when the runner process that communicates with GitHub gets starved causing the job to fail.

…vice (#17780) * watch: support -filter for watch checks * Add filter for watch nodes, services, and service - unit test added - Add changelog - update doc

Previously, this only triggered for release/*.*.x branches; however, our release process involves cutting a release/1.16.0 branch, for example, at time of code freeze these days. Any PRs to that branch after code freeze today do not make their way to consul-enterprise. This will make behavior for a .0 branch consistent with current behavior for a .x branch.

Deleted two commas which looks quite like some leftovers.

…ig (#17860) * Add note about path prefix matching behavior for HTTPRoute config * Update website/content/docs/connect/gateways/api-gateway/configuration/http-route.mdx Co-authored-by: trujillo-adam <[email protected]> --------- Co-authored-by: trujillo-adam <[email protected]>

For consistency, resource type names must follow these rules: - `Group` must be snake case, and in most cases a single word. - `GroupVersion` must be lowercase, start with a "v" and end with a number. - `Kind` must be pascal case. These were chosen because they map to our protobuf type naming conventions.

Extends the `proto` make target to generate a protoset file for use with grpcurl etc.

… name (#17160) * Fix a bug that wrongly trims domains when there is an overlap with DC name Before this change, when DC name and domain/alt-domain overlap, the domain name incorrectly trimmed from the query. Example: Given: datacenter = dc-test, alt-domain = test.consul. Querying for "test-node.node.dc-test.consul" will faile, because the code was trimming "test.consul" instead of just ".consul" This change, fixes the issue by adding dot (.) before trimming * trimDomain: ensure domain trimmed without modyfing original domains * update changelog --------- Co-authored-by: Dhia Ayachi <[email protected]>

Signed-off-by: Dan Bond <[email protected]>

…#17874) * add error message * Update website/content/docs/api-gateway/usage/errors.mdx Co-authored-by: Nathan Coleman <[email protected]> * fix formating issues --------- Co-authored-by: Nathan Coleman <[email protected]>

…#17875)

* update doc for config enable_debug * Update website/content/docs/agent/config/config-files.mdx Co-authored-by: trujillo-adam <[email protected]> --------- Co-authored-by: trujillo-adam <[email protected]>

* Allow service identity tokens the ability to read jwt-providers * more tests * service_prefix tests

Co-authored-by: trujillo-adam <[email protected]>

…elemetry Collector (#17888)

* update release notes for 1.16 and 1.2 * update latest consul core release

* Propose new changes to APIgw upgrade instructions * fix build error * update callouts to render correctly * Add hideClipboard to log messages * Added clarification around consul k8s and crds

* adding docker files to verify linux packages. * add verifr-release-linux.yml * updating name * pass inputs directly into jobs * add other linux package platforms * remove on push * fix TARGETARCH on debian and ubuntu so it can check arm64 and amd64 * fixing amazon to use the continue line * add ubuntu i386 * fix comment lines * working * remove commented out workflow jobs * Apply suggestions from code review Co-authored-by: Nathan Coleman <[email protected]> * update fedora and ubuntu to use latest tag --------- Co-authored-by: Nathan Coleman <[email protected]>

* Reference hashicorp/consul instead of consul for Docker image * Update Make targets that pull consul directly

Updating upgrade procedures to encompass expected errors during upgrade process from v1.13.x to v1.14.x.

* Ensure RSA keys are at least 2048 bits in length * Add changelog * update key length check for FIPS compliance * Fix no new variables error and failing to return when error exists from validating * clean up code for better readability * actually return value

* tlsutil: Fix check TLS configuration * Rewording docs. * Update website/content/docs/services/configuration/checks-configuration-reference.mdx Co-authored-by: trujillo-adam <[email protected]> * Fix typos and add changelog entry. --------- Co-authored-by: trujillo-adam <[email protected]>

… APIs (#17937) * Update v1_16_x.mdx * Update connect native golang page --------- Co-authored-by: trujillo-adam <[email protected]>

This reverts commit 3368f14.

This fixes a bug that was identified which resulted in subsequent ConnectCA configuration update not to persist in the cluster.

* Fix streaming backend link * Update health.mdx

…enable_debug (#17565) * # This is a combination of 9 commits. # This is the 1st commit message: init without tests # This is the commit message #2: change log # This is the commit message #3: fix tests # This is the commit message #4: fix tests # This is the commit message #5: added tests # This is the commit message #6: change log breaking change # This is the commit message #7: removed breaking change # This is the commit message #8: fix test # This is the commit message #9: keeping the test behaviour same * # This is a combination of 12 commits. # This is the 1st commit message: init without tests # This is the commit message #2: change log # This is the commit message #3: fix tests # This is the commit message #4: fix tests # This is the commit message #5: added tests # This is the commit message #6: change log breaking change # This is the commit message #7: removed breaking change # This is the commit message #8: fix test # This is the commit message #9: keeping the test behaviour same # This is the commit message #10: made enable debug atomic bool # This is the commit message #11: fix lint # This is the commit message #12: fix test true enable debug * parent 10f500e author absolutelightning <[email protected]> 1687352587 +0530 committer absolutelightning <[email protected]> 1687352592 +0530 init without tests change log fix tests fix tests added tests change log breaking change removed breaking change fix test keeping the test behaviour same made enable debug atomic bool fix lint fix test true enable debug using enable debug in agent as atomic bool test fixes fix tests fix tests added update on correct locaiton fix tests fix reloadable config enable debug fix tests fix init and acl 403 * revert commit

* fix formatting codeblocks * remove unnecessary indents

* update doc * update link

* configuration entry syntax * Example config

servcies => services

* Expose JWKS cluster config through JWTProviderConfigEntry * fix typos, rename trustedCa to trustedCA

This PR fixes a bug that was introduced in: #16021 A user setting a protocol in proxy-defaults would cause tproxy implicit upstreams to not honor the upstream service's protocol set in its `ServiceDefaults.Protocol` field, and would instead always use the proxy-defaults value. Due to the fact that upstreams configured with "tcp" can successfully contact upstream "http" services, this issue was not recognized until recently (a proxy-defaults with "tcp" and a listening service with "http" would make successful requests, but not the opposite). As a temporary work-around, users experiencing this issue can explicitly set the protocol on the `ServiceDefaults.UpstreamConfig.Overrides`, which should take precedence. The fix in this PR removes the proxy-defaults protocol from the wildcard upstream that tproxy uses to configure implicit upstreams. When the protocol was included, it would always overwrite the value during discovery chain compilation, which was not correct. The discovery chain compiler also consumes proxy defaults to determine the protocol, so simply excluding it from the wildcard upstream config map resolves the issue.

…17939) * feat: update operator usage api endpoint to include nodes count * feat: update operator usange cli command to includes nodes count

* fix(cli): remove failing check from 'connect envoy' registration for api gateway * test(integration): add tests to check catalog statsus of gateways on startup * remove extra sleep comment * Update test/integration/consul-container/libs/assert/service.go * changelog

@trujillo-adam

* docs: fix removed service-to-service peering links * docs: extend peering-via-mesh-gateways intro (thanks @trujillo-adam) --------- Co-authored-by: trujillo-adam <[email protected]>

* Warning updates * .x

* updated typo in tab heading * updated tab group typo, too

Static DNS lookups, in addition to explicitly targeting a datacenter, can target a cluster peer. This was added in 95dc0c7 but didn't make the documentation. The driving function for the change is `parseLocality` here: https://github.com/hashicorp/consul/blob/0b1299c28d8127129d61310ee4280055298438e0/agent/dns_oss.go#L25 The biggest change in this is to adjust the standard lookup syntax to tie `.<datacenter>` to `.dc` as required-together, and to append in the similar `.<cluster-peer>.peer` optional argument, both to A record and SRV record lookups. Co-authored-by: David Yu <[email protected]>

…#18032) * Docs for hashicorp/consul-k8s#2293 * remove versions for enterprise features since they are old --------- Co-authored-by: Tu Nguyen <[email protected]>

* docs - add jobs use case for service mesh k8s * add code blocks

* fix(connect): set initial_fetch_time to wait indefinitely * changelog * PR feedback 1

* Fix metrics docs * Add changelog Signed-off-by: josh <[email protected]> --------- Signed-off-by: josh <[email protected]>

* Update route-to-virtual-services.mdx * Update establish-peering.mdx

grammar change

…18075) * Update proxy lifecycle annotations and consul-dataplane flags

* Docs for dataplane upgrade on k8s --------- Co-authored-by: David Yu <[email protected]> Co-authored-by: trujillo-adam <[email protected]>

* update UINodes and UINodeInfo response with consul-version info added as NodeMeta, fetched from serf members * update test cases TestUINodes, TestUINodeInfo * added nil check for map * add consul-version in local agent node metadata * get consul version from serf member and add this as node meta in catalog register request * updated ui mock response to include consul versions as node meta * updated ui trans and added version as query param to node list route * updates in ui templates to display consul version with filter and sorts * updates in ui - model class, serializers,comparators,predicates for consul version feature * added change log for Consul Version Feature * updated to get version from consul service, if for some reason not available from serf * updated changelog text * updated dependent testcases * multiselection version filter * Update agent/consul/state/catalog.go comments updated Co-authored-by: Jared Kirschner <[email protected]> --------- Co-authored-by: Jared Kirschner <[email protected]>

* api gw 1.16 updates * Apply suggestions from code review Co-authored-by: Jeff Boruszak <[email protected]> * update CodeBlockConfig filename * Apply suggestions from code review Co-authored-by: trujillo-adam <[email protected]> Co-authored-by: Jeff Boruszak <[email protected]> * remove non-standard intentions page * Update website/content/docs/api-gateway/configuration/index.mdx Co-authored-by: trujillo-adam <[email protected]> --------- Co-authored-by: Jeff Boruszak <[email protected]> Co-authored-by: trujillo-adam <[email protected]>

* ci: build s390x * ci: test s390x * ci: dev build s390x * no GOOS * ent only * build: publish s390x * fix syntax error * fix syntax error again * fix syntax error again x2 * test branch * Move s390x conditionals to step level * remove test branch --------- Co-authored-by: emilymianeil <[email protected]>

Just a typo in the docs.

…eams (#17991) Configuration that previously was inlined into the Upstreams resource applies to both explicit and implicit upstreams and so it makes sense to split it out into its own resource. It also has other minor changes: - Renames `proxy.proto` proxy_configuration.proto` - Changes the type of `Upstream.destination_ref` from `pbresource.ID` to `pbresource.Reference` - Adds comments to fields that didn't have them

… dependency bugs with go 1.20.6. Pin go to 1.20.5. (#18124) ### Description The following jobs started failing when go 1.20.6 was released: - `go-test-api-1-19` - `go-test-api-1-20` - `compatibility-integration-tests` - `upgrade-integration-tests` `compatibility-integration-tests` and `compatibility-integration-tests` to this testcontainers issue: testcontainers/testcontainers-go#1359. This issue calls for testcontainers to release a new version when one of their dependencies is fixed. When that is done, we will unpin the go versions in `compatibility-integration-tests` and `compatibility-integration-tests`. ### Testing & Reproduction steps See these jobs broken in CI and then see them work with this PR. --------- Co-authored-by: Chris Thain <[email protected]>

### Description This adds notices, that ingress gateway is deprecated, to several places in the product docs where ingress gateway is the topic. ### Testing & Reproduction steps Tested with a local copy of the website. ### Links Deprecation of ingress gateway was announced in the Release Notes for Consul 1.16 and Consul-K8s 1.2. See: [https://developer.hashicorp.com/consul/docs/release-notes/consul/v1_16_x#what-s-deprecated](https://developer.hashicorp.com/consul/docs/release-notes/consul/v1_16_x#what-s-deprecated ) [https://developer.hashicorp.com/consul/docs/release-notes/consul-k8s/v1_2_x#what-s-deprecated](https://developer.hashicorp.com/consul/docs/release-notes/consul-k8s/v1_2_x#what-s-deprecated) ### PR Checklist * [N/A] updated test coverage * [X] external facing docs updated * [X] appropriate backport labels added * [X] not a security concern --------- Co-authored-by: trujillo-adam <[email protected]>

### Description  - Add jwt-provider docs for jwks cluster configuration. The configuration was added here: #17978

### Description  Dan had already started on this [task](#17849) which is needed to start building the HTTP APIs. This just needed some cleanup to get it ready for review. Overview: - Rename `internalResourceServiceClient` to `insecureResourceServiceClient` for name consistency - Configure a `secureResourceServiceClient` with auth enabled ### PR Checklist * [ ] ~updated test coverage~ * [ ] ~external facing docs updated~ * [x] appropriate backport labels added * [ ] ~not a security concern~

Updating RootPKIPath but not IntermediatePKIPath would not update leaf signing certs with the new root. Unsure if this happens in practice but manual testing showed it is a bug that would break mesh and agent connections once the old root is pruned.

… contains socked now error (#18129) ### Description This is related to #18124 where we pinned the go versions in CI to 1.20.5 and 1.19.10. go 1.20.6 and 1.19.11 now validate request host headers for validity, including the hostname cannot be prefixed with slashes. For local communications (npipe://, unix://), the hostname is not used, but we need valid and meaningful hostname. Prior versions go Go would clean the host header, and strip slashes in the process, but go1.20.6 and go1.19.11 no longer do, and reject the host header. Around the community we are seeing that others are intercepting the req.host and if it starts with a slash or ends with .sock, they changing the host to localhost or another dummy value. [client: define a "dummy" hostname to use for local connections by thaJeztah · Pull Request #45942 · moby/moby](moby/moby#45942) ### Testing & Reproduction steps Check CI tests. ### Links * [ ] updated test coverage * [ ] external facing docs updated * [ ] appropriate backport labels added * [ ] not a security concern

…t is valid (#18139) ### Description This is to correct a code problem because this assumes all segments, but when you get to Enterprise, you can be in partition that is not the default partition, in which case specifying all segments does not validate and fails. This is to correct the setting of this filter with `AllSegments` to `true` to only occur when in the the `default` partition. ### Testing & Reproduction steps  ### Links  ### PR Checklist * [ ] updated test coverage * [ ] external facing docs updated * [ ] appropriate backport labels added * [ ] not a security concern

### Description  The mock is used in `http_ent_test` file which caused lint failures. For OSS->ENT parity adding the same change here. ### Links  Identified in OSS->ENT [merge PR](hashicorp/consul-enterprise#6328) ### PR Checklist * [ ] ~updated test coverage~ * [ ] ~external facing docs updated~ * [x] appropriate backport labels added * [ ] ~not a security concern~

…lter (#18062) ### Description  - Currently the jwt-auth filter doesn't take into account the service identity when validating jwt-auth, it only takes into account the path and jwt provider during validation. This causes issues when multiple source intentions restrict access to an endpoint with different JWT providers. - To fix these issues, rather than use the JWT auth filter for validation, we use it in metadata mode and allow it to forward the successful validated JWT token payload to the RBAC filter which will make the decisions. This PR ensures requests with and without JWT tokens successfully go through the jwt-authn filter. The filter however only forwards the data for successful/valid tokens. On the RBAC filter level, we check the payload for claims and token issuer + existing rbac rules. ### Testing & Reproduction steps  - This test covers a multi level jwt requirements (requirements at top level and permissions level). It also assumes you have envoy running, you have a redis and a sidecar proxy service registered, and have a way to generate jwks with jwt. I mostly use: https://www.scottbrady91.com/tools/jwt for this. - first write your proxy defaults ``` Kind = "proxy-defaults" name = "global" config { protocol = "http" } ``` - Create two providers ``` Kind = "jwt-provider" Name = "auth0" Issuer = "https://ronald.local" JSONWebKeySet = { Local = { JWKS = "eyJrZXlzIjog....." } } ``` ``` Kind = "jwt-provider" Name = "okta" Issuer = "https://ronald.local" JSONWebKeySet = { Local = { JWKS = "eyJrZXlzIjogW3...." } } ``` - add a service intention ``` Kind = "service-intentions" Name = "redis" JWT = { Providers = [ { Name = "okta" }, ] } Sources = [ { Name = "*" Permissions = [{ Action = "allow" HTTP = { PathPrefix = "/workspace" } JWT = { Providers = [ { Name = "okta" VerifyClaims = [ { Path = ["aud"] Value = "my_client_app" }, { Path = ["sub"] Value = "5be86359073c434bad2da3932222dabe" } ] }, ] } }, { Action = "allow" HTTP = { PathPrefix = "/" } JWT = { Providers = [ { Name = "auth0" }, ] } }] } ] ``` - generate 3 jwt tokens: 1 from auth0 jwks, 1 from okta jwks with different claims than `/workspace` expects and 1 with correct claims - connect to your envoy (change service and address as needed) to view logs and potential errors. You can add: `-- --log-level debug` to see what data is being forwarded ``` consul connect envoy -sidecar-for redis1 -grpc-addr 127.0.0.1:8502 ``` - Make the following requests: ``` curl -s -H "Authorization: Bearer $Auth0_TOKEN" --insecure --cert leaf.cert --key leaf.key --cacert connect-ca.pem https://localhost:20000/workspace -v RBAC filter denied curl -s -H "Authorization: Bearer $Okta_TOKEN_with_wrong_claims" --insecure --cert leaf.cert --key leaf.key --cacert connect-ca.pem https://localhost:20000/workspace -v RBAC filter denied curl -s -H "Authorization: Bearer $Okta_TOKEN_with_correct_claims" --insecure --cert leaf.cert --key leaf.key --cacert connect-ca.pem https://localhost:20000/workspace -v Successful request ``` ### TODO * [x] Update test coverage * [ ] update integration tests (follow-up PR) * [x] appropriate backport labels added

### Description Add support for consul connect envoy command on windows. This PR fixes the comments of PR - #15114 ### Testing * Built consul.exe from this branch on windows and hosted here - [AWS S3](https://asheshvidyut-bucket.s3.ap-southeast-2.amazonaws.com/consul.zip) * Updated the [tutorial](https://developer.hashicorp.com/consul/tutorials/developer-mesh/consul-windows-workloads) and changed the `consul_url.default` value to [AWS S3](https://asheshvidyut-bucket.s3.ap-southeast-2.amazonaws.com/consul.zip) * Followed the steps in the tutorial and verified that everything is working as described. ### PR Checklist * [x] updated test coverage * [ ] external facing docs updated * [x] appropriate backport labels added * [x] not a security concern --------- Co-authored-by: Franco Bruno Lavayen <[email protected]> Co-authored-by: Jose Ignacio Lorenzo <[email protected]> Co-authored-by: Jose Ignacio Lorenzo <[email protected]> Co-authored-by: Dhia Ayachi <[email protected]>

### Description Addresses #17171 (comment)

test: improve xDS listener code coverage

… tests (#18157) Re-order expected/actual, consul container tests

* group and document make file

Co-authored-by: R.B. Boyer <[email protected]> Co-authored-by: R.B. Boyer <[email protected]> Co-authored-by: Freddy <[email protected]>

* Add FIPS reference to consul enterprise docs * Update website/content/docs/enterprise/index.mdx Co-authored-by: David Yu <[email protected]> * remove support for ecs client (fips) --------- Co-authored-by: David Yu <[email protected]>

Co-authored-by: R.B. Boyer <[email protected]> Co-authored-by: R.B. Boyer <[email protected]> Co-authored-by: Freddy <[email protected]> Co-authored-by: NiniOak <[email protected]>

* Update index.mdx * Update kv.mdx * Update docs-nav-data.json * delete sentinel.mdx * Update redirects.js --------- Co-authored-by: Tu Nguyen <[email protected]>

Bump golang.org/x/net to 0.12.0 While not necessary to directly address CVE-2023-29406 (which should be handled by using a patched version of Go when building), an accompanying change to HTTP/2 error handling does impact agent code. See https://go-review.googlesource.com/c/net/+/506995 for the HTTP/2 change. Bump this dependency across our submodules as well for the sake of potential indirect consumers of `x/net/http`.

Update Go version to 1.20.6 This resolves [CVE-2023-29406] (https://nvd.nist.gov/vuln/detail/CVE-2023-29406) for uses of the `net/http` standard library. Note that until the follow-up to #18124 is done, the version of Go used in those impacted tests will need to remain on 1.20.5.

* Improve XDS test coverage: JWT auth edition more tests * test: xds coverage for jwt listeners --------- Co-authored-by: DanStough <[email protected]>

u[date readme.md

Align all our internal use of submodules on the latest versions.

Result of tsccr-helper -log-level=info -pin-all-workflows . Co-authored-by: hashicorp-tsccr[bot] <hashicorp-tsccr[bot]@users.noreply.github.com>

* Fix Backport Assistant failure PR commenting For general comments on a PR, it looks like you have to use the `/issue` endpoint rather than `/pulls`, which requires commit/other review-specific target details. This matches the endpoint used in `backport-reminder.yml`. * Remove Backport Reminder workflow This is noisy (even when adding multiple labels, individual comments per label are generated), and likely no longer needed: we haven't had this work in a long time due to an expired GH token, and we now have better automation for backport PR assignment.

… [NET-4908] (#18192)

This PR explicitly enables WebSocket upgrades in Envoy's UpgradeConfig for all proxy types. (API Gateway, Ingress, and Sidecar.) Fixes #8283

test: improve xDS cluster code coverage

Net 4222 take config file consul container

Commits on May 25, 2023

Update index.mdx (#17477 )

lkysow authored May 25, 2023

Configuration menu

View commit details

Copy full SHA for 720dda7

Browse repository at this point

Copy the full SHA

720dda7 View commit details

Browse the repository at this point in the history

Commits on Jul 1, 2023

Fix typo (#17198 )

servcies => services

evanphx authored Jul 1, 2023

Configuration menu

View commit details

Copy full SHA for dc6ea1b

Browse repository at this point

Copy the full SHA

dc6ea1b View commit details

Browse the repository at this point in the history

Commits on Jul 15, 2023

re org resource type registry (#18133 )

wangxinyi7 authored Jul 15, 2023

Configuration menu

View commit details

Copy full SHA for e719478

Browse repository at this point

Copy the full SHA

e719478 View commit details

Browse the repository at this point in the history

Backport of member cli: add -filter expression to flags into release/1.15.x #18275

Backport of member cli: add -filter expression to flags into release/1.15.x #18275

Commits on May 25, 2023

Commits on May 26, 2023

Commits on May 29, 2023

Commits on May 30, 2023

Commits on May 31, 2023

Commits on Jun 1, 2023

Commits on Jun 2, 2023

Commits on Jun 5, 2023

Commits on Jun 6, 2023

Commits on Jun 7, 2023

Commits on Jun 8, 2023

Commits on Jun 9, 2023

Commits on Jun 10, 2023

Commits on Jun 12, 2023

Commits on Jun 13, 2023

Commits on Jun 14, 2023

Commits on Jun 15, 2023

Commits on Jun 16, 2023

Commits on Jun 17, 2023

Commits on Jun 20, 2023

Commits on Jun 21, 2023

Commits on Jun 22, 2023

Commits on Jun 23, 2023

Commits on Jun 26, 2023

Commits on Jun 27, 2023

Commits on Jun 28, 2023

Commits on Jun 29, 2023

Commits on Jun 30, 2023

Commits on Jul 1, 2023

Commits on Jul 4, 2023

Commits on Jul 5, 2023

Commits on Jul 6, 2023

Commits on Jul 7, 2023

Commits on Jul 10, 2023

Commits on Jul 11, 2023

Commits on Jul 12, 2023

Commits on Jul 13, 2023

Commits on Jul 14, 2023

Commits on Jul 15, 2023

Commits on Jul 17, 2023

Commits on Jul 18, 2023

Commits on Jul 19, 2023

Commits on Jul 20, 2023

Commits on Jul 21, 2023