docs: K8s secondary DC requirements #18280
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
github-actions
bot
added
the
type/docs
boruszak
added
pr/no-changelog
trujillo-adam
approved these changes
Jul 25, 2023
website/content/docs/k8s/deployment-configurations/multi-cluster/kubernetes.mdx
Outdated
Show resolved
Hide resolved
website/content/docs/k8s/deployment-configurations/multi-cluster/vms-and-kubernetes.mdx
Outdated
Show resolved
Hide resolved
Co-authored-by: trujillo-adam <[email protected]>
4 tasks
|
Backport failed @trujillo-adam. Run: https://github.com/hashicorp/consul/actions/runs/5672531156 |
trujillo-adam
pushed a commit
that referenced
this pull request
Jul 26, 2023
* Requested edit * Apply suggestions from code review Co-authored-by: trujillo-adam <[email protected]> --------- Co-authored-by: trujillo-adam <[email protected]> manual cherry-pick
This was referenced Aug 4, 2023
absolutelightning
added a commit
that referenced
this pull request
Sep 4, 2023
…Version / Datacenter into release/1.15.x (#18639) * Reference hashicorp/consul instead of consul for Docker image (#17914) * Reference hashicorp/consul instead of consul for Docker image * Update Make targets that pull consul directly * Update Consul K8s Upgrade Doc Updates (#17921) Updating upgrade procedures to encompass expected errors during upgrade process from v1.13.x to v1.14.x. * Update sameness-group.mdx (#17915) * Update create-sameness-groups.mdx (#17927) * deps: coredns v1.10.1 (#17912) * Ensure RSA keys are at least 2048 bits in length (#17911) * Ensure RSA keys are at least 2048 bits in length * Add changelog * update key length check for FIPS compliance * Fix no new variables error and failing to return when error exists from validating * clean up code for better readability * actually return value * tlsutil: Fix check TLS configuration (#17481) * tlsutil: Fix check TLS configuration * Rewording docs. * Update website/content/docs/services/configuration/checks-configuration-reference.mdx Co-authored-by: trujillo-adam <[email protected]> * Fix typos and add changelog entry. --------- Co-authored-by: trujillo-adam <[email protected]> * docs: Deprecations for connect-native SDK and specific connect native APIs (#17937) * Update v1_16_x.mdx * Update connect native golang page --------- Co-authored-by: trujillo-adam <[email protected]> * Revert "Add workflow to verify linux release packages (#17904)" (#17942) This reverts commit 3368f14fab500ebe9f6aeab5631dd1d5f5a453e5. * Fixes Secondary ConnectCA update (#17846) This fixes a bug that was identified which resulted in subsequent ConnectCA configuration update not to persist in the cluster. * fixing typo in link to jwt-validations-with-intentions doc (#17955) * Fix streaming backend link (#17958) * Fix streaming backend link * Update health.mdx * Dynamically create jwks clusters for jwt-providers (#17944) * website: remove deprecated agent rpc docs (#17962) * Fix missing BalanceOutboundConnections in v2 catalog. (#17964) * feature - [NET - 4005] - [Supportability] Reloadable Configuration - enable_debug (#17565) * # This is a combination of 9 commits. # This is the 1st commit message: init without tests # This is the commit message #2: change log # This is the commit message #3: fix tests # This is the commit message #4: fix tests # This is the commit message #5: added tests # This is the commit message #6: change log breaking change # This is the commit message #7: removed breaking change # This is the commit message #8: fix test # This is the commit message #9: keeping the test behaviour same * # This is a combination of 12 commits. # This is the 1st commit message: init without tests # This is the commit message #2: change log # This is the commit message #3: fix tests # This is the commit message #4: fix tests # This is the commit message #5: added tests # This is the commit message #6: change log breaking change # This is the commit message #7: removed breaking change # This is the commit message #8: fix test # This is the commit message #9: keeping the test behaviour same # This is the commit message #10: made enable debug atomic bool # This is the commit message #11: fix lint # This is the commit message #12: fix test true enable debug * parent 10f500e895d92cc3691ade7b74a33db755d22039 author absolutelightning <[email protected]> 1687352587 +0530 committer absolutelightning <[email protected]> 1687352592 +0530 init without tests change log fix tests fix tests added tests change log breaking change removed breaking change fix test keeping the test behaviour same made enable debug atomic bool fix lint fix test true enable debug using enable debug in agent as atomic bool test fixes fix tests fix tests added update on correct locaiton fix tests fix reloadable config enable debug fix tests fix init and acl 403 * revert commit * Fix formatting codeblocks on APIgw docs (#17970) * fix formatting codeblocks * remove unnecessary indents * Remove POC code (#17974) * update doc (#17910) * update doc * update link * Remove duplicate and unused newDecodeConfigEntry func (#17979) * docs: samenessGroup YAML examples (#17984) * configuration entry syntax * Example config * Add changelog entry for 1.16.0 (#17987) * Fix typo (#17198) servcies => services * Expose JWKS cluster config through JWTProviderConfigEntry (#17978) * Expose JWKS cluster config through JWTProviderConfigEntry * fix typos, rename trustedCa to trustedCA * Integration test for ext-authz Envoy extension (#17980) * Fix incorrect protocol for transparent proxy upstreams. (#17894) This PR fixes a bug that was introduced in: https://github.com/hashicorp/consul/pull/16021 A user setting a protocol in proxy-defaults would cause tproxy implicit upstreams to not honor the upstream service's protocol set in its `ServiceDefaults.Protocol` field, and would instead always use the proxy-defaults value. Due to the fact that upstreams configured with "tcp" can successfully contact upstream "http" services, this issue was not recognized until recently (a proxy-defaults with "tcp" and a listening service with "http" would make successful requests, but not the opposite). As a temporary work-around, users experiencing this issue can explicitly set the protocol on the `ServiceDefaults.UpstreamConfig.Overrides`, which should take precedence. The fix in this PR removes the proxy-defaults protocol from the wildcard upstream that tproxy uses to configure implicit upstreams. When the protocol was included, it would always overwrite the value during discovery chain compilation, which was not correct. The discovery chain compiler also consumes proxy defaults to determine the protocol, so simply excluding it from the wildcard upstream config map resolves the issue. * feat: include nodes count in operator usage endpoint and cli command (#17939) * feat: update operator usage api endpoint to include nodes count * feat: update operator usange cli command to includes nodes count * [OSS] Improve Gateway Test Coverage of Catalog Health (#18011) * fix(cli): remove failing check from 'connect envoy' registration for api gateway * test(integration): add tests to check catalog statsus of gateways on startup * remove extra sleep comment * Update test/integration/consul-container/libs/assert/service.go * changelog * Fixes Traffic rate limitting docs (#17997) * Fix removed service-to-service peering links (#17221) * docs: fix removed service-to-service peering links * docs: extend peering-via-mesh-gateways intro (thanks @trujillo-adam) --------- Co-authored-by: trujillo-adam <[email protected]> * docs: Sameness "beta" warning (#18017) * Warning updates * .x * updated typo in tab heading (#18022) * updated typo in tab heading * updated tab group typo, too * Document that DNS lookups can target cluster peers (#17990) Static DNS lookups, in addition to explicitly targeting a datacenter, can target a cluster peer. This was added in 95dc0c7b301b70a6b955a8b7c9737c9b86f03df6 but didn't make the documentation. The driving function for the change is `parseLocality` here: https://github.com/hashicorp/consul/blob/0b1299c28d8127129d61310ee4280055298438e0/agent/dns_oss.go#L25 The biggest change in this is to adjust the standard lookup syntax to tie `.<datacenter>` to `.dc` as required-together, and to append in the similar `.<cluster-peer>.peer` optional argument, both to A record and SRV record lookups. Co-authored-by: David Yu <[email protected]> * Add first integration test for jwt auth with intention (#18005) * fix stand-in text for name field (#18030) * removed sameness conf entry from failover nav (#18033) * docs - add service sync annotations and k8s service weight annotation (#18032) * Docs for https://github.com/hashicorp/consul-k8s/pull/2293 * remove versions for enterprise features since they are old --------- Co-authored-by: Tu Nguyen <[email protected]> * docs - add jobs use case for service mesh k8s (#18037) * docs - add jobs use case for service mesh k8s * add code blocks * address feedback (#18045) * Add verify server hostname to tls default (#17155) * [OSS] Fix initial_fetch_timeout to wait for all xDS resources (#18024) * fix(connect): set initial_fetch_time to wait indefinitely * changelog * PR feedback 1 * ui: fix typos for peer service imports (#17999) * test: fix FIPS inline cert test message (#18076) * Fix a couple typos in Agent Telemetry Metrics docs (#18080) * Fix metrics docs * Add changelog Signed-off-by: josh <[email protected]> --------- Signed-off-by: josh <[email protected]> * docs updates - cluster peering and virtual services (#18069) * Update route-to-virtual-services.mdx * Update establish-peering.mdx * Update service-mesh-compare.mdx (#17279) grammar change * Update helm docs on main (#18085) * ci: use gotestsum v1.10.1 [NET-4042] (#18088) * Docs: Update proxy lifecycle annotations and consul-dataplane flags (#18075) * Update proxy lifecycle annotations and consul-dataplane flags * Pass configured role name to Vault for AWS auth in Connect CA (#17885) * Docs for dataplane upgrade on k8s (#18051) * Docs for dataplane upgrade on k8s --------- Co-authored-by: David Yu <[email protected]> Co-authored-by: trujillo-adam <[email protected]> * docs - update upgrade index page to not recommend consul leave. (#18100) * Displays Consul version of each nodes in UI nodes section (#17754) * update UINodes and UINodeInfo response with consul-version info added as NodeMeta, fetched from serf members * update test cases TestUINodes, TestUINodeInfo * added nil check for map * add consul-version in local agent node metadata * get consul version from serf member and add this as node meta in catalog register request * updated ui mock response to include consul versions as node meta * updated ui trans and added version as query param to node list route * updates in ui templates to display consul version with filter and sorts * updates in ui - model class, serializers,comparators,predicates for consul version feature * added change log for Consul Version Feature * updated to get version from consul service, if for some reason not available from serf * updated changelog text * updated dependent testcases * multiselection version filter * Update agent/consul/state/catalog.go comments updated Co-authored-by: Jared Kirschner <[email protected]> --------- Co-authored-by: Jared Kirschner <[email protected]> * api gw 1.16 updates (#18081) * api gw 1.16 updates * Apply suggestions from code review Co-authored-by: Jeff Boruszak <[email protected]> * update CodeBlockConfig filename * Apply suggestions from code review Co-authored-by: trujillo-adam <[email protected]> Co-authored-by: Jeff Boruszak <[email protected]> * remove non-standard intentions page * Update website/content/docs/api-gateway/configuration/index.mdx Co-authored-by: trujillo-adam <[email protected]> --------- Co-authored-by: Jeff Boruszak <[email protected]> Co-authored-by: trujillo-adam <[email protected]> * [NET-4103] ci: build s390x (#18067) * ci: build s390x * ci: test s390x * ci: dev build s390x * no GOOS * ent only * build: publish s390x * fix syntax error * fix syntax error again * fix syntax error again x2 * test branch * Move s390x conditionals to step level * remove test branch --------- Co-authored-by: emilymianeil <[email protected]> * :ermahgerd "Sevice Mesh" -> "Service Mesh" (#18116) Just a typo in the docs. * Split pbmesh.UpstreamsConfiguration as a resource out of pbmesh.Upstreams (#17991) Configuration that previously was inlined into the Upstreams resource applies to both explicit and implicit upstreams and so it makes sense to split it out into its own resource. It also has other minor changes: - Renames `proxy.proto` proxy_configuration.proto` - Changes the type of `Upstream.destination_ref` from `pbresource.ID` to `pbresource.Reference` - Adds comments to fields that didn't have them * [NET-4895] ci - api tests and consul container tests error because of dependency bugs with go 1.20.6. Pin go to 1.20.5. (#18124) ### Description The following jobs started failing when go 1.20.6 was released: - `go-test-api-1-19` - `go-test-api-1-20` - `compatibility-integration-tests` - `upgrade-integration-tests` `compatibility-integration-tests` and `compatibility-integration-tests` to this testcontainers issue: https://github.com/testcontainers/testcontainers-go/issues/1359. This issue calls for testcontainers to release a new version when one of their dependencies is fixed. When that is done, we will unpin the go versions in `compatibility-integration-tests` and `compatibility-integration-tests`. ### Testing & Reproduction steps See these jobs broken in CI and then see them work with this PR. --------- Co-authored-by: Chris Thain <[email protected]> * Add ingress gateway deprecation notices to docs (#18102) ### Description This adds notices, that ingress gateway is deprecated, to several places in the product docs where ingress gateway is the topic. ### Testing & Reproduction steps Tested with a local copy of the website. ### Links Deprecation of ingress gateway was announced in the Release Notes for Consul 1.16 and Consul-K8s 1.2. See: [https://developer.hashicorp.com/consul/docs/release-notes/consul/v1_16_x#what-s-deprecated](https://developer.hashicorp.com/consul/docs/release-notes/consul/v1_16_x#what-s-deprecated ) [https://developer.hashicorp.com/consul/docs/release-notes/consul-k8s/v1_2_x#what-s-deprecated](https://developer.hashicorp.com/consul/docs/release-notes/consul-k8s/v1_2_x#what-s-deprecated) ### PR Checklist * [N/A] updated test coverage * [X] external facing docs updated * [X] appropriate backport labels added * [X] not a security concern --------- Co-authored-by: trujillo-adam <[email protected]> * Add docs for jwt cluster configuration (#18004) ### Description <!-- Please describe why you're making this change, in plain English. --> - Add jwt-provider docs for jwks cluster configuration. The configuration was added here: https://github.com/hashicorp/consul/pull/17978 * Docs: fix unmatched bracket for health checks page (#18134) * NET-4657/add resource service client (#18053) ### Description <!-- Please describe why you're making this change, in plain English. --> Dan had already started on this [task](https://github.com/hashicorp/consul/pull/17849) which is needed to start building the HTTP APIs. This just needed some cleanup to get it ready for review. Overview: - Rename `internalResourceServiceClient` to `insecureResourceServiceClient` for name consistency - Configure a `secureResourceServiceClient` with auth enabled ### PR Checklist * [ ] ~updated test coverage~ * [ ] ~external facing docs updated~ * [x] appropriate backport labels added * [ ] ~not a security concern~ * Fix bug with Vault CA provider (#18112) Updating RootPKIPath but not IntermediatePKIPath would not update leaf signing certs with the new root. Unsure if this happens in practice but manual testing showed it is a bug that would break mesh and agent connections once the old root is pruned. * [NET-4897] net/http host header is now verified and request.host that contains socked now error (#18129) ### Description This is related to https://github.com/hashicorp/consul/pull/18124 where we pinned the go versions in CI to 1.20.5 and 1.19.10. go 1.20.6 and 1.19.11 now validate request host headers for validity, including the hostname cannot be prefixed with slashes. For local communications (npipe://, unix://), the hostname is not used, but we need valid and meaningful hostname. Prior versions go Go would clean the host header, and strip slashes in the process, but go1.20.6 and go1.19.11 no longer do, and reject the host header. Around the community we are seeing that others are intercepting the req.host and if it starts with a slash or ends with .sock, they changing the host to localhost or another dummy value. [client: define a "dummy" hostname to use for local connections by thaJeztah · Pull Request #45942 · moby/moby](https://github.com/moby/moby/pull/45942) ### Testing & Reproduction steps Check CI tests. ### Links * [ ] updated test coverage * [ ] external facing docs updated * [ ] appropriate backport labels added * [ ] not a security concern * add a conditional around setting LANFilter.AllSegments to make sure it is valid (#18139) ### Description This is to correct a code problem because this assumes all segments, but when you get to Enterprise, you can be in partition that is not the default partition, in which case specifying all segments does not validate and fails. This is to correct the setting of this filter with `AllSegments` to `true` to only occur when in the the `default` partition. ### Testing & Reproduction steps <!-- * In the case of bugs, describe how to replicate * If any manual tests were done, document the steps and the conditions to replicate * Call out any important/ relevant unit tests, e2e tests or integration tests you have added or are adding --> ### Links <!-- Include any links here that might be helpful for people reviewing your PR (Tickets, GH issues, API docs, external benchmarks, tools docs, etc). If there are none, feel free to delete this section. Please be mindful not to leak any customer or confidential information. HashiCorp employees may want to use our internal URL shortener to obfuscate links. --> ### PR Checklist * [ ] updated test coverage * [ ] external facing docs updated * [ ] appropriate backport labels added * [ ] not a security concern * chore: bump upgrade integrations tests to 1.15, 116 [NET-4743] (#18130) * re org resource type registry (#18133) * fix: update delegateMock used in ENT (#18149) ### Description <!-- Please describe why you're making this change, in plain English. --> The mock is used in `http_ent_test` file which caused lint failures. For OSS->ENT parity adding the same change here. ### Links <!-- Include any links here that might be helpful for people reviewing your PR (Tickets, GH issues, API docs, external benchmarks, tools docs, etc). If there are none, feel free to delete this section. Please be mindful not to leak any customer or confidential information. HashiCorp employees may want to use our internal URL shortener to obfuscate links. --> Identified in OSS->ENT [merge PR](https://github.com/hashicorp/consul-enterprise/pull/6328) ### PR Checklist * [ ] ~updated test coverage~ * [ ] ~external facing docs updated~ * [x] appropriate backport labels added * [ ] ~not a security concern~ * Use JWT-auth filter in metadata mode & Delegate validation to RBAC filter (#18062) ### Description <!-- Please describe why you're making this change, in plain English. --> - Currently the jwt-auth filter doesn't take into account the service identity when validating jwt-auth, it only takes into account the path and jwt provider during validation. This causes issues when multiple source intentions restrict access to an endpoint with different JWT providers. - To fix these issues, rather than use the JWT auth filter for validation, we use it in metadata mode and allow it to forward the successful validated JWT token payload to the RBAC filter which will make the decisions. This PR ensures requests with and without JWT tokens successfully go through the jwt-authn filter. The filter however only forwards the data for successful/valid tokens. On the RBAC filter level, we check the payload for claims and token issuer + existing rbac rules. ### Testing & Reproduction steps <!-- * In the case of bugs, describe how to replicate * If any manual tests were done, document the steps and the conditions to replicate * Call out any important/ relevant unit tests, e2e tests or integration tests you have added or are adding --> - This test covers a multi level jwt requirements (requirements at top level and permissions level). It also assumes you have envoy running, you have a redis and a sidecar proxy service registered, and have a way to generate jwks with jwt. I mostly use: https://www.scottbrady91.com/tools/jwt for this. - first write your proxy defaults ``` Kind = "proxy-defaults" name = "global" config { protocol = "http" } ``` - Create two providers ``` Kind = "jwt-provider" Name = "auth0" Issuer = "https://ronald.local" JSONWebKeySet = { Local = { JWKS = "eyJrZXlzIjog....." } } ``` ``` Kind = "jwt-provider" Name = "okta" Issuer = "https://ronald.local" JSONWebKeySet = { Local = { JWKS = "eyJrZXlzIjogW3...." } } ``` - add a service intention ``` Kind = "service-intentions" Name = "redis" JWT = { Providers = [ { Name = "okta" }, ] } Sources = [ { Name = "*" Permissions = [{ Action = "allow" HTTP = { PathPrefix = "/workspace" } JWT = { Providers = [ { Name = "okta" VerifyClaims = [ { Path = ["aud"] Value = "my_client_app" }, { Path = ["sub"] Value = "5be86359073c434bad2da3932222dabe" } ] }, ] } }, { Action = "allow" HTTP = { PathPrefix = "/" } JWT = { Providers = [ { Name = "auth0" }, ] } }] } ] ``` - generate 3 jwt tokens: 1 from auth0 jwks, 1 from okta jwks with different claims than `/workspace` expects and 1 with correct claims - connect to your envoy (change service and address as needed) to view logs and potential errors. You can add: `-- --log-level debug` to see what data is being forwarded ``` consul connect envoy -sidecar-for redis1 -grpc-addr 127.0.0.1:8502 ``` - Make the following requests: ``` curl -s -H "Authorization: Bearer $Auth0_TOKEN" --insecure --cert leaf.cert --key leaf.key --cacert connect-ca.pem https://localhost:20000/workspace -v RBAC filter denied curl -s -H "Authorization: Bearer $Okta_TOKEN_with_wrong_claims" --insecure --cert leaf.cert --key leaf.key --cacert connect-ca.pem https://localhost:20000/workspace -v RBAC filter denied curl -s -H "Authorization: Bearer $Okta_TOKEN_with_correct_claims" --insecure --cert leaf.cert --key leaf.key --cacert connect-ca.pem https://localhost:20000/workspace -v Successful request ``` ### TODO * [x] Update test coverage * [ ] update integration tests (follow-up PR) * [x] appropriate backport labels added * Support Consul Connect Envoy Command on Windows (#17694) ### Description Add support for consul connect envoy command on windows. This PR fixes the comments of PR - https://github.com/hashicorp/consul/pull/15114 ### Testing * Built consul.exe from this branch on windows and hosted here - [AWS S3](https://asheshvidyut-bucket.s3.ap-southeast-2.amazonaws.com/consul.zip) * Updated the [tutorial](https://developer.hashicorp.com/consul/tutorials/developer-mesh/consul-windows-workloads) and changed the `consul_url.default` value to [AWS S3](https://asheshvidyut-bucket.s3.ap-southeast-2.amazonaws.com/consul.zip) * Followed the steps in the tutorial and verified that everything is working as described. ### PR Checklist * [x] updated test coverage * [ ] external facing docs updated * [x] appropriate backport labels added * [x] not a security concern --------- Co-authored-by: Franco Bruno Lavayen <[email protected]> Co-authored-by: Jose Ignacio Lorenzo <[email protected]> Co-authored-by: Jose Ignacio Lorenzo <[email protected]> Co-authored-by: Dhia Ayachi <[email protected]> * Change docs to say 168h instead of 7d for server_rejoin_age_max (#18154) ### Description Addresses https://github.com/hashicorp/consul/pull/17171#issuecomment-1636930705 * [OSS] test: improve xDS listener code coverage (#18138) test: improve xDS listener code coverage * Re-order expected/actual for assertContainerState in consul container tests (#18157) Re-order expected/actual, consul container tests * group and document make file (#17943) * group and document make file * Add `testing/deployer` (neé `consul-topology`) [NET-4610] (#17823) Co-authored-by: R.B. Boyer <[email protected]> Co-authored-by: R.B. Boyer <[email protected]> Co-authored-by: Freddy <[email protected]> * [NET-4792] Add integrations tests for jwt-auth (#18169) * Add FIPS reference to consul enterprise docs (#18028) * Add FIPS reference to consul enterprise docs * Update website/content/docs/enterprise/index.mdx Co-authored-by: David Yu <[email protected]> * remove support for ecs client (fips) --------- Co-authored-by: David Yu <[email protected]> * add peering_commontopo tests [NET-3700] (#17951) Co-authored-by: R.B. Boyer <[email protected]> Co-authored-by: R.B. Boyer <[email protected]> Co-authored-by: Freddy <[email protected]> Co-authored-by: NiniOak <[email protected]> * docs - remove Sentinel from enterprise features list (#18176) * Update index.mdx * Update kv.mdx * Update docs-nav-data.json * delete sentinel.mdx * Update redirects.js --------- Co-authored-by: Tu Nguyen <[email protected]> * [NET-4865] Bump golang.org/x/net to 0.12.0 (#18186) Bump golang.org/x/net to 0.12.0 While not necessary to directly address CVE-2023-29406 (which should be handled by using a patched version of Go when building), an accompanying change to HTTP/2 error handling does impact agent code. See https://go-review.googlesource.com/c/net/+/506995 for the HTTP/2 change. Bump this dependency across our submodules as well for the sake of potential indirect consumers of `x/net/http`. * Call resource mutate hook before validate hook (NET-4907) (#18178) * [NET-4865] security: Update Go version to 1.20.6 (#18190) Update Go version to 1.20.6 This resolves [CVE-2023-29406] (https://nvd.nist.gov/vuln/detail/CVE-2023-29406) for uses of the `net/http` standard library. Note that until the follow-up to #18124 is done, the version of Go used in those impacted tests will need to remain on 1.20.5. * Improve XDS test coverage: JWT auth edition (#18183) * Improve XDS test coverage: JWT auth edition more tests * test: xds coverage for jwt listeners --------- Co-authored-by: DanStough <[email protected]> * update readme.md (#18191) u[date readme.md * Update submodules to latest following 1.16.0 (#18197) Align all our internal use of submodules on the latest versions. * SEC-090: Automated trusted workflow pinning (2023-07-18) (#18174) Result of tsccr-helper -log-level=info -pin-all-workflows . Co-authored-by: hashicorp-tsccr[bot] <hashicorp-tsccr[bot]@users.noreply.github.com> * Fix Backport Assistant PR commenting (#18200) * Fix Backport Assistant failure PR commenting For general comments on a PR, it looks like you have to use the `/issue` endpoint rather than `/pulls`, which requires commit/other review-specific target details. This matches the endpoint used in `backport-reminder.yml`. * Remove Backport Reminder workflow This is noisy (even when adding multiple labels, individual comments per label are generated), and likely no longer needed: we haven't had this work in a long time due to an expired GH token, and we now have better automation for backport PR assignment. * resource: Pass resource to Write ACL hook instead of just resource Id [NET-4908] (#18192) * Explicitly enable WebSocket upgrades (#18150) This PR explicitly enables WebSocket upgrades in Envoy's UpgradeConfig for all proxy types. (API Gateway, Ingress, and Sidecar.) Fixes #8283 * docs: fix the description of client rpc (#18206) * NET-4804: Add dashboard for monitoring consul-k8s (#18208) * [OSS] Improve xDS Code Coverage - Clusters (#18165) test: improve xDS cluster code coverage * NET-4222 take config file consul container (#18218) Net 4222 take config file consul container * Envoy Integration Test Windows (#18007) * [CONSUL-395] Update check_hostport and Usage (#40) * [CONSUL-397] Copy envoy binary from Image (#41) * [CONSUL-382] Support openssl in unique test dockerfile (#43) * [CONSUL-405] Add bats to single container (#44) * [CONSUL-414] Run Prometheus Test Cases and Validate Changes (#46) * [CONSUL-410] Run Jaeger in Single container (#45) * [CONSUL-412] Run test-sds-server in single container (#48) * [CONSUL-408] Clean containers (#47) * [CONSUL-384] Rebase and sync fork (#50) * [CONSUL-415] Create Scenarios Troubleshooting Docs (#49) * [CONSUL-417] Update Docs Single Container (#51) * [CONSUL-428] Add Socat to single container (#54) * [CONSUL-424] Replace pkill in kill_envoy function (#52) * [CONSUL-434] Modify Docker run functions in Helper script (#53) * [CONSUL-435] Replace docker run in set_ttl_check_state & wait_for_agent_service_register functions (#55) * [CONSUL-438] Add netcat (nc) in the Single container Dockerfile (#56) * [CONSUL-429] Replace Docker run with Docker exec (#57) * [CONSUL-436] Curl timeout and run tests (#58) * [CONSUL-443] Create dogstatsd Function (#59) * [CONSUL-431] Update Docs Netcat (#60) * [CONSUL-439] Parse nc Command in function (#61) * [CONSUL-463] Review curl Exec and get_ca_root Func (#63) * [CONSUL-453] Docker hostname in Helper functions (#64) * [CONSUL-461] Test wipe volumes without extra cont (#66) * [CONSUL-454] Check ports in the Server and Agent containers (#65) * [CONSUL-441] Update windows dockerfile with version (#62) * [CONSUL-466] Review case-grpc Failing Test (#67) * [CONSUL-494] Review case-cfg-resolver-svc-failover (#68) * [CONSUL-496] Replace docker_wget & docker_curl (#69) * [CONSUL-499] Cleanup Scripts - Remove nanoserver (#70) * [CONSUL-500] Update Troubleshooting Docs (#72) * [CONSUL-502] Pull & Tag Envoy Windows Image (#73) * [CONSUL-504] Replace docker run in docker_consul (#76) * [CONSUL-505] Change admin_bind * [CONSUL-399] Update envoy to 1.23.1 (#78) * [CONSUL-510] Support case-wanfed-gw on Windows (#79) * [CONSUL-506] Update troubleshooting Documentation (#80) * [CONSUL-512] Review debug_dump_volumes Function (#81) * [CONSUL-514] Add zipkin to Docker Image (#82) * [CONSUL-515] Update Documentation (#83) * [CONSUL-529] Support case-consul-exec (#86) * [CONSUL-530] Update Documentation (#87) * [CONSUL-530] Update default consul version 1.13.3 * [CONSUL-539] Cleanup (#91) * [CONSUL-546] Scripts Clean-up (#92) * [CONSUL-491] Support admin_access_log_path value for Windows (#71) * [CONSUL-519] Implement mkfifo Alternative (#84) * [CONSUL-542] Create OS Specific Files for Envoy Package (#88) * [CONSUL-543] Create exec_supported.go (#89) * [CONSUL-544] Test and Build Changes (#90) * Implement os.DevNull * using mmap instead of disk files * fix import in exec-unix * fix nmap open too many arguemtn * go fmt on file * changelog file * fix go mod * Update .changelog/17694.txt Co-authored-by: Dhia Ayachi <[email protected]> * different mmap library * fix bootstrap json * some fixes * chocolatey version fix and image fix * using different library * fix Map funciton call * fix mmap call * fix tcp dump * fix tcp dump * windows tcp dump * Fix docker run * fix tests * fix go mod * fix version 16.0 * fix version * fix version dev * sleep to debug * fix sleep * fix permission issue * fix permission issue * fix permission issue * fix command * fix command * fix funciton * fix assert config entry status command not found * fix command not found assert_cert_has_cn * fix command not found assert_upstream_missing * fix command not found assert_upstream_missing_once * fix command not found get_upstream_endpoint * fix command not found get_envoy_public_listener_once * fix command not found * fix test cases * windows integration test workflow github * made code similar to unix using npipe * fix go.mod * fix dialing of npipe * dont wait * check size of written json * fix undefined n * running * fix dep * fix syntax error * fix workflow file * windows runner * fix runner * fix from json * fix runs on * merge connect envoy * fix cin path * build * fix file name * fix file name * fix dev build * remove unwanted code * fix upload * fix bin name * fix path * checkout current branch * fix path * fix tests * fix shell bash for windows sh files * fix permission of run-test.sh * removed docker dev * added shell bash for tests * fix tag * fix win=true * fix cd * added dev * fix variable undefined * removed failing tests * fix tcp dump image * fix curl * fix curl * tcp dump path * fix tcpdump path * fix curl * fix curl install * stop removing intermediate containers * fix tcpdump docker image * revert -rm * --rm=false * makeing docker image before * fix tcpdump * removed case consul exec * removed terminating gateway simple * comment case wasm * removed data dog * comment out upload coverage * uncomment case-consul-exec * comment case consul exec * if always * logs * using consul 1.17.0 * fix quotes * revert quotes * redirect to dev null * Revert version * revert consul connect * fix version * removed envoy connect * not using function * change log * docker logs * fix logs * restructure bad authz * rmeoved dev null * output * fix file descriptor * fix cacert * fix cacert * fix ca cert * cacert does not work in windows curl * fix func * removed docker logs * added sleep * fix tls * commented case-consul-exec * removed echo * retry docker consul * fix upload bin * uncomment consul exec * copying consul.exe to docker image * copy fix * fix paths * fix path * github workspace path * latest version * Revert "latest version" This reverts commit 5a7d7b82d9e7553bcb01b02557ec8969f9deba1d. * commented consul exec * added ssl revoke best effort * revert best effort * removed unused files * rename var name and change dir * windows runner * permission * needs setup fix * swtich to github runner * fix file path * fix path * fix path * fix path * fix path * fix path * fix build paths * fix tag * nightly runs * added matrix in github workflow, renamed files * fix job * fix matrix * removed brackes * from json * without using job matrix * fix quotes * revert job matrix * fix workflow * fix comment * added comment * nightly runs * removed datadog ci as it is already measured in linux one * running test * Revert "running test" This reverts commit 7013d15a23732179d18ec5d17336e16b26fab5d4. * pr comment fixes * running test now * running subset of test * running subset of test * job matrix * shell bash * removed bash shell * linux machine for job matrix * fix output * added cat to debug * using ubuntu latest * fix job matrix * fix win true * fix go test * revert job matrix --------- Co-authored-by: Jose Ignacio Lorenzo <[email protected]> Co-authored-by: Franco Bruno Lavayen <[email protected]> Co-authored-by: Ivan K Berlot <[email protected]> Co-authored-by: Ezequiel Fernández Ponce <[email protected]> Co-authored-by: joselo85 <[email protected]> Co-authored-by: Ezequiel Fernández Ponce <[email protected]> Co-authored-by: Dhia Ayachi <[email protected]> * fix typos and update ecs compat table (#18215) * fix typos and update ecs compat table * real info for the ecs compat matrix table * Update website/content/docs/ecs/compatibility.mdx Co-authored-by: Chris Thain <[email protected]> --------- Co-authored-by: Chris Thain <[email protected]> * [OSS] proxystate: add proxystate protos (#18216) * proxystate: add proxystate protos to pbmesh and resolve imports and conflicts between message names * ci: don't verify s390x (#18224) * [CC-5718] Remove HCP token requirement during bootstrap (#18140) * [CC-5718] Remove HCP token requirement during bootstrap * Re-add error for loading HCP management token * Remove old comment * Add changelog entry * Remove extra validation line * Apply suggestions from code review Co-authored-by: lornasong <[email protected]> --------- Co-authored-by: lornasong <[email protected]> * [NET-4122] Doc guidance for federation with externalServers (#18207) Doc guidance for federation with externalServers Add guidance for proper configuration when joining to a secondary cluster using WAN fed with external servers also enabled. Also clarify federation requirements and fix formatting for an unrelated value. Update both the Helm chart reference (synced from `consul-k8s`, see hashicorp/consul-k8s#2583) and the docs on using `externalServers`. * [OSS] Improve xDS Code Coverage - Endpoints and Misc (#18222) test: improve xDS endpoints code coverage * Clarify license reporting timing and GDPR compliance (#18237) Add Alicia's edits to clarify log timing and other details * Fix Github Workflow File (#18241) * [CONSUL-382] Support openssl in unique test dockerfile (#43) * [CONSUL-405] Add bats to single container (#44) * [CONSUL-414] Run Prometheus Test Cases and Validate Changes (#46) * [CONSUL-410] Run Jaeger in Single container (#45) * [CONSUL-412] Run test-sds-server in single container (#48) * [CONSUL-408] Clean containers (#47) * [CONSUL-384] Rebase and sync fork (#50) * [CONSUL-415] Create Scenarios Troubleshooting Docs (#49) * [CONSUL-417] Update Docs Single Container (#51) * [CONSUL-428] Add Socat to single container (#54) * [CONSUL-424] Replace pkill in kill_envoy function (#52) * [CONSUL-434] Modify Docker run functions in Helper script (#53) * [CONSUL-435] Replace docker run in set_ttl_check_state & wait_for_agent_service_register functions (#55) * [CONSUL-438] Add netcat (nc) in the Single container Dockerfile (#56) * [CONSUL-429] Replace Docker run with Docker exec (#57) * [CONSUL-436] Curl timeout and run tests (#58) * [CONSUL-443] Create dogstatsd Function (#59) * [CONSUL-431] Update Docs Netcat (#60) * [CONSUL-439] Parse nc Command in function (#61) * [CONSUL-463] Review curl Exec and get_ca_root Func (#63) * [CONSUL-453] Docker hostname in Helper functions (#64) * [CONSUL-461] Test wipe volumes without extra cont (#66) * [CONSUL-454] Check ports in the Server and Agent containers (#65) * [CONSUL-441] Update windows dockerfile with version (#62) * [CONSUL-466] Review case-grpc Failing Test (#67) * [CONSUL-494] Review case-cfg-resolver-svc-failover (#68) * [CONSUL-496] Replace docker_wget & docker_curl (#69) * [CONSUL-499] Cleanup Scripts - Remove nanoserver (#70) * [CONSUL-500] Update Troubleshooting Docs (#72) * [CONSUL-502] Pull & Tag Envoy Windows Image (#73) * [CONSUL-504] Replace docker run in docker_consul (#76) * [CONSUL-505] Change admin_bind * [CONSUL-399] Update envoy to 1.23.1 (#78) * [CONSUL-510] Support case-wanfed-gw on Windows (#79) * [CONSUL-506] Update troubleshooting Documentation (#80) * [CONSUL-512] Review debug_dump_volumes Function (#81) * [CONSUL-514] Add zipkin to Docker Image (#82) * [CONSUL-515] Update Documentation (#83) * [CONSUL-529] Support case-consul-exec (#86) * [CONSUL-530] Update Documentation (#87) * [CONSUL-530] Update default consul version 1.13.3 * [CONSUL-539] Cleanup (#91) * [CONSUL-546] Scripts Clean-up (#92) * [CONSUL-491] Support admin_access_log_path value for Windows (#71) * [CONSUL-519] Implement mkfifo Alternative (#84) * [CONSUL-542] Create OS Specific Files for Envoy Package (#88) * [CONSUL-543] Create exec_supported.go (#89) * [CONSUL-544] Test and Build Changes (#90) * Implement os.DevNull * using mmap instead of disk files * fix import in exec-unix * fix nmap open too many arguemtn * go fmt on file * changelog file * fix go mod * Update .changelog/17694.txt Co-authored-by: Dhia Ayachi <[email protected]> * different mmap library * fix bootstrap json * some fixes * chocolatey version fix and image fix * using different library * fix Map funciton call * fix mmap call * fix tcp dump * fix tcp dump * windows tcp dump * Fix docker run * fix tests * fix go mod * fix version 16.0 * fix version * fix version dev * sleep to debug * fix sleep * fix permission issue * fix permission issue * fix permission issue * fix command * fix command * fix funciton * fix assert config entry status command not found * fix command not found assert_cert_has_cn * fix command not found assert_upstream_missing * fix command not found assert_upstream_missing_once * fix command not found get_upstream_endpoint * fix command not found get_envoy_public_listener_once * fix command not found * fix test cases * windows integration test workflow github * made code similar to unix using npipe * fix go.mod * fix dialing of npipe * dont wait * check size of written json * fix undefined n * running * fix dep * fix syntax error * fix workflow file * windows runner * fix runner * fix from json * fix runs on * merge connect envoy * fix cin path * build * fix file name * fix file name * fix dev build * remove unwanted code * fix upload * fix bin name * fix path * checkout current branch * fix path * fix tests * fix shell bash for windows sh files * fix permission of run-test.sh * removed docker dev * added shell bash for tests * fix tag * fix win=true * fix cd * added dev * fix variable undefined * removed failing tests * fix tcp dump image * fix curl * fix curl * tcp dump path * fix tcpdump path * fix curl * fix curl install * stop removing intermediate containers * fix tcpdump docker image * revert -rm * --rm=false * makeing docker image before * fix tcpdump * removed case consul exec * removed terminating gateway simple * comment case wasm * removed data dog * comment out upload coverage * uncomment case-consul-exec * comment case consul exec * if always * logs * using consul 1.17.0 * fix quotes * revert quotes * redirect to dev null * Revert version * revert consul connect * fix version * removed envoy connect * not using function * change log * docker logs * fix logs * restructure bad authz * rmeoved dev null * output * fix file descriptor * fix cacert * fix cacert * fix ca cert * cacert does not work in windows curl * fix func * removed docker logs * added sleep * fix tls * commented case-consul-exec * removed echo * retry docker consul * fix upload bin * uncomment consul exec * copying consul.exe to docker image * copy fix * fix paths * fix path * github workspace path * latest version * Revert "latest version" This reverts commit 5a7d7b82d9e7553bcb01b02557ec8969f9deba1d. * commented consul exec * added ssl revoke best effort * revert best effort * removed unused files * rename var name and change dir * windows runner * permission * needs setup fix * swtich to github runner * fix file path * fix path * fix path * fix path * fix path * fix path * fix build paths * fix tag * nightly runs * added matrix in github workflow, renamed files * fix job * fix matrix * removed brackes * from json * without using job matrix * fix quotes * revert job matrix * fix workflow * fix comment * added comment * nightly runs * removed datadog ci as it is already measured in linux one * running test * Revert "running test" This reverts commit 7013d15a23732179d18ec5d17336e16b26fab5d4. * pr comment fixes * running test now * running subset of test * running subset of test * job matrix * shell bash * removed bash shell * linux machine for job matrix * fix output * added cat to debug * using ubuntu latest * fix job matrix * fix win true * fix go test * revert job matrix * Fix tests --------- Co-authored-by: Ivan K Berlot <[email protected]> Co-authored-by: Jose Ignacio Lorenzo <[email protected]> Co-authored-by: Franco Bruno Lavayen <[email protected]> Co-authored-by: Ezequiel Fernández Ponce <[email protected]> Co-authored-by: joselo85 <[email protected]> Co-authored-by: Ezequiel Fernández Ponce <[email protected]> Co-authored-by: Dhia Ayachi <[email protected]> * NET-4996 - filter go-tests and test-integration workflows from running on docs only and ui only changes (#18236) * Align build arch matrix with enterprise (#18235) Ensure that OSS remains in sync w/ Enterprise by aligning the format of arch matrix args for various build jobs. * Revert "NET-4996 - filter go-tests and test-integration workflows from running on docs only and ui only changes" (#18248) Revert "NET-4996 - filter go-tests and test-integration workflows from running on docs only and ui only changes (#18236)" This reverts commit a11dba710e6ce6f172c0fa6c9b61567cc1efffc8. * resource: Add scope to resource type registration [NET-4976] (#18214) Enables querying a resource type's registration to determine if a resource is cluster, partition, or partition and namespace scoped. * Fix some inconsistencies in jwt docs (#18234) * NET-1825: More new ACL token creation docs (#18063) Co-authored-by: trujillo-adam <[email protected]> * grafana: add the panel resource usage of connect injector (#18247) * [NET-3700] Backfill changelog entry for c2bbe67 and 7402d06 (#18259) Backfill changelog entry for c2bbe67 and 7402d06 Add a changelog entry for the follow-up PR since it was specific to the fix and references the original change. * NET-4897 - update comment to include the current issue url from the go team. (#18263) * fix typos, style, and improper links (#18269) * member cli: add -filter expression to flags (#18223) * member cli: add -filter expression to flags * changelog * update doc * Add test cases * use quote * go-tests: disable s390x (#18273) * docs: Update ext-authz documentation for kubernetes (#18281) Update ext-authz documentation for kubernetes * docs: Consul on Kubernetes specific upgrade info (#18230) * Compatibility page - dataplanes mention * Upgrading higher-level dataplane mention * `exec=` string callout * More visible for upgrade page * Apply suggestions from code review Co-authored-by: Tu Nguyen <[email protected]> --------- Co-authored-by: Tu Nguyen <[email protected]> Co-authored-by: John Murret <[email protected]> * Fix typo in Envoy extensions doc (#18284) * docs: K8s secondary DC requirements (#18280) * Requested edit * Apply suggestions from code review Co-authored-by: trujillo-adam <[email protected]> --------- Co-authored-by: trujillo-adam <[email protected]> * api-gateway: subscribe to bound-api-gateway only after receiving api-gateway (#18291) * api-gateway: subscribe to bound-api-gateway only after receiving api-gateway This fixes a race condition due to our dependency on having the listener(s) from the api-gateway config entry in order to fully and properly process the resources on the bound-api-gateway config entry. * Apply suggestions from code review * Add changelog entry * Update K8s changelog to address cloud auto-join change in 1.0.0 (#18293) * Update list of Envoy versions (#18300) Update supported envoy versions to 1.24.10, 1.25.9, 1.26.4, 1.27.0. * [NET-4904] Update list of Envoy versions in docs (#18306) Update list of Envoy versions in docs Update supported Envoy versions across Consul release versions. * Update actions for TSCCR (#18317) Update action versions before deadline * Fix topology view when displaying mixed connect-native/normal services. (#13023) * Fix topoloy intention with mixed connect-native/normal services. If a service is registered twice, once with connect-native and once without, the topology views would prune the existing intentions. This change brings the code more in line with the transparent proxy behavior. * Dedupe nodes in the ServiceTopology ui endpoint (like done with tags). * Consider a service connect-native as soon as one instance is. * Stop JWT provider from being written in non default namespace (#18325) * docs: Fix some comments (#17118) Signed-off-by: cui fliter <[email protected]> * [NET-4703] Prevent partial application of Envoy extensions (#18068) Prevent partial application of Envoy extensions Ensure that non-required extensions do not change xDS resources before exiting on failure by cloning proto messages prior to applying each extension. To support this change, also move `CanApply` checks up a layer and make them prior to attempting extension application, s.t. we avoid unnecessary copies where extensions can't be applied. Last, ensure that we do not allow panics from `CanApply` or `Extend` checks to escape the attempted extension application. * docs: Simplify example jq commands by removing pipes (#18327) Simplify jq command examples by removing pipes to other commands. * fix typo in create a mesh token docs (#18337) * [CC-5719] Add support for builtin global-read-only policy (#18319) * [CC-5719] Add support for builtin global-read-only policy * Add changelog * Add read-only to docs * Fix some minor issues. * Change from ReplaceAll to Sprintf * Change IsValidPolicy name to return an error instead of bool * Fix PolicyList test * Fix other tests * Apply suggestions from code review Co-authored-by: Paul Glass <[email protected]> * Fix state store test for policy list. * Fix naming issues * Update acl/validation.go Co-authored-by: Chris Thain <[email protected]> * Update agent/consul/acl_endpoint.go --------- Co-authored-by: Paul Glass <[email protected]> Co-authored-by: Chris Thain <[email protected]> * [NET-5121] proxystate: move protos to subdirectory to avoid conflicts (#18335) * also makes a few protos updates * resource: adding various helpers for working with resources (#18342) This is a bit of a grab bag of helpers that I found useful for working with them when authoring substantial Controllers. Subsequent PRs will make use of them. * Wasm integration tests for local and remote wasm files (#17756) * wasm integration tests for local and remote wasm files refactoring and cleanup for wasm testing remove wasm debug logging PR feedback, wasm build lock correct path pattern for wasm build files Add new helper function to minimize changes to existing test code Remove extra param mod tidy add custom service setup to test lib add wait until static server sidecar can reach nginx sidecar Doc comments PR feedback Update workflows to compile wasm for integration tests Fix docker build path Fix package name for linter Update makefile, fix redeclared function Update expected wasm filename Debug test ls in workflow remove pwd in favor of relative path more debugging Build wasm in compatability tests as well Build wasm directly in ci rather than in container Debug tinygo and llvm version Change wasm file extension Remove tinygo debugging Remove extra comments * Add compiled wasm and build instructions * [HCP Telemetry] Periodic Refresh for Dynamic Telemetry Configuration (#18168) * OTElExporter now uses an EndpointProvider to discover the endpoint * OTELSink uses a ConfigProvider to obtain filters and labels configuration * improve tests for otel_sink * Regex logic is moved into client for a method on the TelemetryConfig object * Create a telemetry_config_provider and update deps to use it * Fix conversion * fix import newline * Add logger to hcp client and move telemetry_config out of the client.go file * Add a telemetry_config.go to refactor client.go * Update deps * update hcp deps test * Modify telemetry_config_providers * Check for nil filters * PR review updates * Fix comments and move around pieces * Fix comments * Remove context from client struct * Moved ctx out of sink struct and fixed filters, added a test * Remove named imports, use errors.New if not fformatting * Remove HCP dependencies in telemetry package * Add success metric and move lock only to grab the t.cfgHahs * Update hash * fix nits * Create an equals method and add tests * Improve telemetry_config_provider.go tests * Add race test * Add missing godoc * Remove mock for MetricsClient * Avoid goroutine test panics * trying to kick CI lint issues by upgrading mod * imprve test code and add hasher for testing * Use structure logging for filters, fix error constants, and default to allow all regex * removed hashin and modify logic to simplify * Improve race test and fix PR feedback by removing hash equals and avoid testing the timer.Ticker logic, and instead unit test * Ran make go-mod-tidy * Use errtypes in the test * Add changelog * add safety check for exporter endpoint * remove require.Contains by using error types, fix structure logging, and fix success metric typo in exporter * Fixed race test to have changing config values * Send success metric before modifying config * Avoid the defer and move the success metric under * NET-4240 - Snapshots are failing on Windows (#18302) * fix go mod * fix go sum * added change log * ran make go mod tidy * Register ProxyStateTemplate Resource (#18316) Also, change the ProxyState.id to identity. This is because we already have the id of this proxy from the resource, and this id should be name-aligned with the workload it represents. It should also have the owner ref set to the workload ID if we need that. And so the id field seems unnecessary. We do, however, need a reference to workload identity so that we can authorize the proxy when it initially connects to the xDS server. * [NET-5146] security: Update Go version to 1.20.7 and `x/net` to 0.13.0 (#18358) * Update Go version to 1.20.7 This resolves [CVE-2023-29409] (https://nvd.nist.gov/vuln/detail/CVE-2023-29409)(`crypto/tls`). * Bump golang.org/x/net to 0.13.0 Addresses [CVE-2023-3978](https://nvd.nist.gov/vuln/detail/CVE-2023-3978) for security scans (non-impacting). * mesh: adding the protobuf types and resources backing mesh config v2 (#18351) * [OSS] test: xds coverage for routes (#18369) test: xds coverage for routes * Fix policy lookup to allow for slashes (#18347) * Fix policy lookup to allow for slashes * Fix suggestions * Fix other test * Revert some lines * add some initial CODEOWNERS (#18346) * Add redirects for mesh-gateway docs (#18377) * expose grpc as http endpoint (#18221) expose resource grpc endpoints as http endpoints * bimapper: fix a bug and add some more test coverage (#18387) * [docs] Fix ServiceDefaults example in distributed tracing (#17212) Fix ServiceDefaults example in distributed tracing. * Fix #17730 - Dev mode has new line (#18367) * adding new line only in case of pretty in url not in dev mode * change log added * chore: make go-mod-tidy (#18388) * resource: Make resource read tenancy aware (#18397) * Not using chmod - fixed integration test for Enterprise (#18401) * Not using chmod * checking icacls * Removed push * replaced ordered list of rate limit ops with flow diagram (#18398) * replaced ordered list of rate limit ops with flow diagram * made diagram font bigger * updated colors for dark mode img * fixed typo * recentered dark mode image * Apply suggestions from code review Co-authored-by: Jeff Boruszak <[email protected]> --------- Co-authored-by: Jeff Boruszak <[email protected]> * Upgrade test: remove outdated test and disable log due to verbosity (#18403) * remove outdated test * disable log since we have too many parallel tests * bump testcontainers-go from 0.22.0 and remove pinned go version in in… (#18395) * bump testcontainers-go from 0.22.0 and remove pinned go version in integ test * go mod tidy * Replace deprecated target.Authority with target.URL.Host * Add ServiceEndpoints Mutation hook tests (#18404) * Add ServiceEndpoints Mutation hook tests * Move endpoint owner validation into the validation hook Also there were some minor changes to error validation to account for go-cmp not liking to peer through an errors.errorstring type that get created by errors.New * catalog: add FailoverPolicy mutation and validation hooks (#18390) Add most of the validation and mutation hooks for the FailoverPolicy resource. * NET-5115 Add retry + timeout filters for api-gateway (#18324) * squash, implement retry/timeout in consul core * update tests * catalog: adding a controller to reconcile FailoverPolicy resources (#18399) Add most of the semantic cross-resource validation for FailoverPolicy resources using a new controller. * fix the error in ent repo (#18421) fix the error in ent repo * build: updates for 1.16.1 release (#18415) * build: updates for 1.16.1 release * build: fix missing replace directive for envoyextensions * build: go mod tidy testing/deployer * update ECS links (#18419) * docs: fix incorrect proxy-defaults config in Lua Envoy extension (#18418) fix incorrect proxy-defaults config * resource: Make resource write tenancy aware (#18423) * [NET-5217] [OSS] Derive sidecar proxy locality from parent service (#18437) * Add logging to locality policy application In OSS, this is currently a no-op. * Inherit locality when registering sidecars When sidecar locality is not explicitly configured, inherit locality from the proxied service. * NET-4984: Update APIGW Config Entries for JWT Auth (#18366) * Added oss config entries for Policy and JWT on APIGW * Updated structs for config entry * Updated comments, ran deep-copy * Move JWT configuration into OSS file * Add in the config entry OSS file for jwts * Added changelog * fixing proto spacing * Moved to using manually written deep copy method * Use pointers for override/default fields in apigw config entries * Run gen scripts for changed types * k8s compat - Openshift versions (#18307) * Update compatibility.mdx Co-authored-by: trujillo-adam <[email protected]> * APIGW: Update HTTPRouteConfigEntry for JWT Auth (#18422) * Updated httproute config entry for JWT Filters * Added manual deepcopy method for httproute jwt filter * Fix test * Update JWTFilter to be in oss file * Add changelog * Add build tags for deepcopy oss file * [COMPLIANCE] License changes (#18443) * Adding explicit MPL license for sub-package This directory and its subdirectories (packages) contain files licensed with the MPLv2 `LICENSE` file in this directory and are intentionally licensed separately from the BSL `LICENSE` file at the root of this repository. * Adding explicit MPL license for sub-package This directory and its subdirectories (packages) contain files licensed with the MPLv2 `LICENSE` file in this directory and are intentionally licensed separately from the BSL `LICENSE` file at the root of this repository. * Updating the license from MPL to Business Source License Going forward, this project will be licensed under the Business Source License v1.1. Please see our blog post for more details at <Blog URL>, FAQ at www.hashicorp.com/licensing-faq, and details of the license at www.hashicorp.com/bsl. * add missing license headers * Update copyright file headers to BUSL-1.1 * Update copyright file headers to BUSL-1.1 * Update copyright file headers to BUSL-1.1 * Update copyright file headers to BUSL-1.1 * Update copyright file headers to BUSL-1.1 * Update copyright file headers to BUSL-1.1 * Update copyright file headers to BUSL-1.1 * Update copyright file headers to BUSL-1.1 * Update copyright file headers to BUSL-1.1 * Update copyright file headers to BUSL-1.1 * Update copyright file headers to BUSL-1.1 * Update copyright file headers to BUSL-1.1 * Update copyright file headers to BUSL-1.1 * Update copyright file headers to BUSL-1.1 * Update copyright file headers to BUSL-1.1 --------- Co-authored-by: hashicorp-copywrite[bot] <110428419+hashicorp-copywrite[bot]@users.noreply.github.com> * Net-2708/delete resource endpoint (#18420) * feat: add http delete endpoint for resource service * refactor: clean up * Disable deep-copy codegen verification for now. (#18446) Once we figure out what to do with license headers for these files we should re-enable this check. * build: upgrade to latest buf v1.26.0 (#18426) * NET-4952 Add docs for export command (#18425) * Init file for services export command docs * Add link for export command to nav * Add export to list of subcommands for services * Add usage and basic command options * Add API options using includes * Add section for enterprise options * Include argument value in flag format * Add examples * Apply suggestions from code review Co-authored-by: Jeff Boruszak <[email protected]> --------- Co-authored-by: Jeff Boruszak <[email protected]> * Net-2712/resource hcl parsing (#18250) * Initial protohcl implementation Co-authored-by: Matt Keeler <[email protected]> Co-authored-by: Daniel Upton <[email protected]> * resourcehcl: implement resource decoding on top of protohcl Co-authored-by: Daniel Upton <[email protected]> * fix: resolve ci failures * test: add additional unmarshalling tests * refactor: update function test to clean protohcl package imports --------- Co-authored-by: Matt Keeler <[email protected]> Co-authored-by: Daniel Upton <[email protected]> * read endpoint (#18268) implement http read endpoint to expose resource grpc service read method * Net-2707/list resource endpoint (#18444) feat: list resources endpoint * Fix incorrect yaml in examples (#18463) * NET-5187: Upgrade test timeout due to log producer errors (#18461) Stop log producer before restarting container * [NET-4799] [OSS] xdsv2: listeners L4 support for connect proxies (#18436) * refactor to avoid future import cycles * resource: Make resource list …
absolutelightning
added a commit
that referenced
this pull request
Sep 4, 2023
…1.14.x (#18628) * api-gateway: add operation cannot be fulfilled error to common errors (#17874) * add error message * Update website/content/docs/api-gateway/usage/errors.mdx Co-authored-by: Nathan Coleman <[email protected]> * fix formating issues --------- Co-authored-by: Nathan Coleman <[email protected]> * api-gateway: add step to upgrade instructions for creating intentions (#17875) * Changelog - add 1.13.9, 1.14.8, and 1.15.4 (#17889) * docs: update config enable_debug (#17866) * update doc for config enable_debug * Update website/content/docs/agent/config/config-files.mdx Co-authored-by: trujillo-adam <[email protected]> --------- Co-authored-by: trujillo-adam <[email protected]> * Update wording on WAN fed and intermediate_pki_path (#17850) * Allow service identity tokens the ability to read jwt-providers (#17893) * Allow service identity tokens the ability to read jwt-providers * more tests * service_prefix tests * Update docs (#17476) Co-authored-by: trujillo-adam <[email protected]> * Add emit_tags_as_labels to envoy bootstrap config when using Consul Telemetry Collector (#17888) * Fix command from kg to kubectl get (#17903) * Create and update release notes for 1.16 and 1.2 (#17895) * update release notes for 1.16 and 1.2 * update latest consul core release * Propose new changes to APIgw upgrade instructions (#17693) * Propose new changes to APIgw upgrade instructions * fix build error * update callouts to render correctly * Add hideClipboard to log messages * Added clarification around consul k8s and crds * Add workflow to verify linux release packages (#17904) * adding docker files to verify linux packages. * add verifr-release-linux.yml * updating name * pass inputs directly into jobs * add other linux package platforms * remove on push * fix TARGETARCH on debian and ubuntu so it can check arm64 and amd64 * fixing amazon to use the continue line * add ubuntu i386 * fix comment lines * working * remove commented out workflow jobs * Apply suggestions from code review Co-authored-by: Nathan Coleman <[email protected]> * update fedora and ubuntu to use latest tag --------- Co-authored-by: Nathan Coleman <[email protected]> * Reference hashicorp/consul instead of consul for Docker image (#17914) * Reference hashicorp/consul instead of consul for Docker image * Update Make targets that pull consul directly * Update Consul K8s Upgrade Doc Updates (#17921) Updating upgrade procedures to encompass expected errors during upgrade process from v1.13.x to v1.14.x. * Update sameness-group.mdx (#17915) * Update create-sameness-groups.mdx (#17927) * deps: coredns v1.10.1 (#17912) * Ensure RSA keys are at least 2048 bits in length (#17911) * Ensure RSA keys are at least 2048 bits in length * Add changelog * update key length check for FIPS compliance * Fix no new variables error and failing to return when error exists from validating * clean up code for better readability * actually return value * tlsutil: Fix check TLS configuration (#17481) * tlsutil: Fix check TLS configuration * Rewording docs. * Update website/content/docs/services/configuration/checks-configuration-reference.mdx Co-authored-by: trujillo-adam <[email protected]> * Fix typos and add changelog entry. --------- Co-authored-by: trujillo-adam <[email protected]> * docs: Deprecations for connect-native SDK and specific connect native APIs (#17937) * Update v1_16_x.mdx * Update connect native golang page --------- Co-authored-by: trujillo-adam <[email protected]> * Revert "Add workflow to verify linux release packages (#17904)" (#17942) This reverts commit 3368f14fab500ebe9f6aeab5631dd1d5f5a453e5. * Fixes Secondary ConnectCA update (#17846) This fixes a bug that was identified which resulted in subsequent ConnectCA configuration update not to persist in the cluster. * fixing typo in link to jwt-validations-with-intentions doc (#17955) * Fix streaming backend link (#17958) * Fix streaming backend link * Update health.mdx * Dynamically create jwks clusters for jwt-providers (#17944) * website: remove deprecated agent rpc docs (#17962) * Fix missing BalanceOutboundConnections in v2 catalog. (#17964) * feature - [NET - 4005] - [Supportability] Reloadable Configuration - enable_debug (#17565) * # This is a combination of 9 commits. # This is the 1st commit message: init without tests # This is the commit message #2: change log # This is the commit message #3: fix tests # This is the commit message #4: fix tests # This is the commit message #5: added tests # This is the commit message #6: change log breaking change # This is the commit message #7: removed breaking change # This is the commit message #8: fix test # This is the commit message #9: keeping the test behaviour same * # This is a combination of 12 commits. # This is the 1st commit message: init without tests # This is the commit message #2: change log # This is the commit message #3: fix tests # This is the commit message #4: fix tests # This is the commit message #5: added tests # This is the commit message #6: change log breaking change # This is the commit message #7: removed breaking change # This is the commit message #8: fix test # This is the commit message #9: keeping the test behaviour same # This is the commit message #10: made enable debug atomic bool # This is the commit message #11: fix lint # This is the commit message #12: fix test true enable debug * parent 10f500e895d92cc3691ade7b74a33db755d22039 author absolutelightning <[email protected]> 1687352587 +0530 committer absolutelightning <[email protected]> 1687352592 +0530 init without tests change log fix tests fix tests added tests change log breaking change removed breaking change fix test keeping the test behaviour same made enable debug atomic bool fix lint fix test true enable debug using enable debug in agent as atomic bool test fixes fix tests fix tests added update on correct locaiton fix tests fix reloadable config enable debug fix tests fix init and acl 403 * revert commit * Fix formatting codeblocks on APIgw docs (#17970) * fix formatting codeblocks * remove unnecessary indents * Remove POC code (#17974) * update doc (#17910) * update doc * update link * Remove duplicate and unused newDecodeConfigEntry func (#17979) * docs: samenessGroup YAML examples (#17984) * configuration entry syntax * Example config * Add changelog entry for 1.16.0 (#17987) * Fix typo (#17198) servcies => services * Expose JWKS cluster config through JWTProviderConfigEntry (#17978) * Expose JWKS cluster config through JWTProviderConfigEntry * fix typos, rename trustedCa to trustedCA * Integration test for ext-authz Envoy extension (#17980) * Fix incorrect protocol for transparent proxy upstreams. (#17894) This PR fixes a bug that was introduced in: https://github.com/hashicorp/consul/pull/16021 A user setting a protocol in proxy-defaults would cause tproxy implicit upstreams to not honor the upstream service's protocol set in its `ServiceDefaults.Protocol` field, and would instead always use the proxy-defaults value. Due to the fact that upstreams configured with "tcp" can successfully contact upstream "http" services, this issue was not recognized until recently (a proxy-defaults with "tcp" and a listening service with "http" would make successful requests, but not the opposite). As a temporary work-around, users experiencing this issue can explicitly set the protocol on the `ServiceDefaults.UpstreamConfig.Overrides`, which should take precedence. The fix in this PR removes the proxy-defaults protocol from the wildcard upstream that tproxy uses to configure implicit upstreams. When the protocol was included, it would always overwrite the value during discovery chain compilation, which was not correct. The discovery chain compiler also consumes proxy defaults to determine the protocol, so simply excluding it from the wildcard upstream config map resolves the issue. * feat: include nodes count in operator usage endpoint and cli command (#17939) * feat: update operator usage api endpoint to include nodes count * feat: update operator usange cli command to includes nodes count * [OSS] Improve Gateway Test Coverage of Catalog Health (#18011) * fix(cli): remove failing check from 'connect envoy' registration for api gateway * test(integration): add tests to check catalog statsus of gateways on startup * remove extra sleep comment * Update test/integration/consul-container/libs/assert/service.go * changelog * Fixes Traffic rate limitting docs (#17997) * Fix removed service-to-service peering links (#17221) * docs: fix removed service-to-service peering links * docs: extend peering-via-mesh-gateways intro (thanks @trujillo-adam) --------- Co-authored-by: trujillo-adam <[email protected]> * docs: Sameness "beta" warning (#18017) * Warning updates * .x * updated typo in tab heading (#18022) * updated typo in tab heading * updated tab group typo, too * Document that DNS lookups can target cluster peers (#17990) Static DNS lookups, in addition to explicitly targeting a datacenter, can target a cluster peer. This was added in 95dc0c7b301b70a6b955a8b7c9737c9b86f03df6 but didn't make the documentation. The driving function for the change is `parseLocality` here: https://github.com/hashicorp/consul/blob/0b1299c28d8127129d61310ee4280055298438e0/agent/dns_oss.go#L25 The biggest change in this is to adjust the standard lookup syntax to tie `.<datacenter>` to `.dc` as required-together, and to append in the similar `.<cluster-peer>.peer` optional argument, both to A record and SRV record lookups. Co-authored-by: David Yu <[email protected]> * Add first integration test for jwt auth with intention (#18005) * fix stand-in text for name field (#18030) * removed sameness conf entry from failover nav (#18033) * docs - add service sync annotations and k8s service weight annotation (#18032) * Docs for https://github.com/hashicorp/consul-k8s/pull/2293 * remove versions for enterprise features since they are old --------- Co-authored-by: Tu Nguyen <[email protected]> * docs - add jobs use case for service mesh k8s (#18037) * docs - add jobs use case for service mesh k8s * add code blocks * address feedback (#18045) * Add verify server hostname to tls default (#17155) * [OSS] Fix initial_fetch_timeout to wait for all xDS resources (#18024) * fix(connect): set initial_fetch_time to wait indefinitely * changelog * PR feedback 1 * ui: fix typos for peer service imports (#17999) * test: fix FIPS inline cert test message (#18076) * Fix a couple typos in Agent Telemetry Metrics docs (#18080) * Fix metrics docs * Add changelog Signed-off-by: josh <[email protected]> --------- Signed-off-by: josh <[email protected]> * docs updates - cluster peering and virtual services (#18069) * Update route-to-virtual-services.mdx * Update establish-peering.mdx * Update service-mesh-compare.mdx (#17279) grammar change * Update helm docs on main (#18085) * ci: use gotestsum v1.10.1 [NET-4042] (#18088) * Docs: Update proxy lifecycle annotations and consul-dataplane flags (#18075) * Update proxy lifecycle annotations and consul-dataplane flags * Pass configured role name to Vault for AWS auth in Connect CA (#17885) * Docs for dataplane upgrade on k8s (#18051) * Docs for dataplane upgrade on k8s --------- Co-authored-by: David Yu <[email protected]> Co-authored-by: trujillo-adam <[email protected]> * docs - update upgrade index page to not recommend consul leave. (#18100) * Displays Consul version of each nodes in UI nodes section (#17754) * update UINodes and UINodeInfo response with consul-version info added as NodeMeta, fetched from serf members * update test cases TestUINodes, TestUINodeInfo * added nil check for map * add consul-version in local agent node metadata * get consul version from serf member and add this as node meta in catalog register request * updated ui mock response to include consul versions as node meta * updated ui trans and added version as query param to node list route * updates in ui templates to display consul version with filter and sorts * updates in ui - model class, serializers,comparators,predicates for consul version feature * added change log for Consul Version Feature * updated to get version from consul service, if for some reason not available from serf * updated changelog text * updated dependent testcases * multiselection version filter * Update agent/consul/state/catalog.go comments updated Co-authored-by: Jared Kirschner <[email protected]> --------- Co-authored-by: Jared Kirschner <[email protected]> * api gw 1.16 updates (#18081) * api gw 1.16 updates * Apply suggestions from code review Co-authored-by: Jeff Boruszak <[email protected]> * update CodeBlockConfig filename * Apply suggestions from code review Co-authored-by: trujillo-adam <[email protected]> Co-authored-by: Jeff Boruszak <[email protected]> * remove non-standard intentions page * Update website/content/docs/api-gateway/configuration/index.mdx Co-authored-by: trujillo-adam <[email protected]> --------- Co-authored-by: Jeff Boruszak <[email protected]> Co-authored-by: trujillo-adam <[email protected]> * [NET-4103] ci: build s390x (#18067) * ci: build s390x * ci: test s390x * ci: dev build s390x * no GOOS * ent only * build: publish s390x * fix syntax error * fix syntax error again * fix syntax error again x2 * test branch * Move s390x conditionals to step level * remove test branch --------- Co-authored-by: emilymianeil <[email protected]> * :ermahgerd "Sevice Mesh" -> "Service Mesh" (#18116) Just a typo in the docs. * Split pbmesh.UpstreamsConfiguration as a resource out of pbmesh.Upstreams (#17991) Configuration that previously was inlined into the Upstreams resource applies to both explicit and implicit upstreams and so it makes sense to split it out into its own resource. It also has other minor changes: - Renames `proxy.proto` proxy_configuration.proto` - Changes the type of `Upstream.destination_ref` from `pbresource.ID` to `pbresource.Reference` - Adds comments to fields that didn't have them * [NET-4895] ci - api tests and consul container tests error because of dependency bugs with go 1.20.6. Pin go to 1.20.5. (#18124) ### Description The following jobs started failing when go 1.20.6 was released: - `go-test-api-1-19` - `go-test-api-1-20` - `compatibility-integration-tests` - `upgrade-integration-tests` `compatibility-integration-tests` and `compatibility-integration-tests` to this testcontainers issue: https://github.com/testcontainers/testcontainers-go/issues/1359. This issue calls for testcontainers to release a new version when one of their dependencies is fixed. When that is done, we will unpin the go versions in `compatibility-integration-tests` and `compatibility-integration-tests`. ### Testing & Reproduction steps See these jobs broken in CI and then see them work with this PR. --------- Co-authored-by: Chris Thain <[email protected]> * Add ingress gateway deprecation notices to docs (#18102) ### Description This adds notices, that ingress gateway is deprecated, to several places in the product docs where ingress gateway is the topic. ### Testing & Reproduction steps Tested with a local copy of the website. ### Links Deprecation of ingress gateway was announced in the Release Notes for Consul 1.16 and Consul-K8s 1.2. See: [https://developer.hashicorp.com/consul/docs/release-notes/consul/v1_16_x#what-s-deprecated](https://developer.hashicorp.com/consul/docs/release-notes/consul/v1_16_x#what-s-deprecated ) [https://developer.hashicorp.com/consul/docs/release-notes/consul-k8s/v1_2_x#what-s-deprecated](https://developer.hashicorp.com/consul/docs/release-notes/consul-k8s/v1_2_x#what-s-deprecated) ### PR Checklist * [N/A] updated test coverage * [X] external facing docs updated * [X] appropriate backport labels added * [X] not a security concern --------- Co-authored-by: trujillo-adam <[email protected]> * Add docs for jwt cluster configuration (#18004) ### Description <!-- Please describe why you're making this change, in plain English. --> - Add jwt-provider docs for jwks cluster configuration. The configuration was added here: https://github.com/hashicorp/consul/pull/17978 * Docs: fix unmatched bracket for health checks page (#18134) * NET-4657/add resource service client (#18053) ### Description <!-- Please describe why you're making this change, in plain English. --> Dan had already started on this [task](https://github.com/hashicorp/consul/pull/17849) which is needed to start building the HTTP APIs. This just needed some cleanup to get it ready for review. Overview: - Rename `internalResourceServiceClient` to `insecureResourceServiceClient` for name consistency - Configure a `secureResourceServiceClient` with auth enabled ### PR Checklist * [ ] ~updated test coverage~ * [ ] ~external facing docs updated~ * [x] appropriate backport labels added * [ ] ~not a security concern~ * Fix bug with Vault CA provider (#18112) Updating RootPKIPath but not IntermediatePKIPath would not update leaf signing certs with the new root. Unsure if this happens in practice but manual testing showed it is a bug that would break mesh and agent connections once the old root is pruned. * [NET-4897] net/http host header is now verified and request.host that contains socked now error (#18129) ### Description This is related to https://github.com/hashicorp/consul/pull/18124 where we pinned the go versions in CI to 1.20.5 and 1.19.10. go 1.20.6 and 1.19.11 now validate request host headers for validity, including the hostname cannot be prefixed with slashes. For local communications (npipe://, unix://), the hostname is not used, but we need valid and meaningful hostname. Prior versions go Go would clean the host header, and strip slashes in the process, but go1.20.6 and go1.19.11 no longer do, and reject the host header. Around the community we are seeing that others are intercepting the req.host and if it starts with a slash or ends with .sock, they changing the host to localhost or another dummy value. [client: define a "dummy" hostname to use for local connections by thaJeztah · Pull Request #45942 · moby/moby](https://github.com/moby/moby/pull/45942) ### Testing & Reproduction steps Check CI tests. ### Links * [ ] updated test coverage * [ ] external facing docs updated * [ ] appropriate backport labels added * [ ] not a security concern * add a conditional around setting LANFilter.AllSegments to make sure it is valid (#18139) ### Description This is to correct a code problem because this assumes all segments, but when you get to Enterprise, you can be in partition that is not the default partition, in which case specifying all segments does not validate and fails. This is to correct the setting of this filter with `AllSegments` to `true` to only occur when in the the `default` partition. ### Testing & Reproduction steps <!-- * In the case of bugs, describe how to replicate * If any manual tests were done, document the steps and the conditions to replicate * Call out any important/ relevant unit tests, e2e tests or integration tests you have added or are adding --> ### Links <!-- Include any links here that might be helpful for people reviewing your PR (Tickets, GH issues, API docs, external benchmarks, tools docs, etc). If there are none, feel free to delete this section. Please be mindful not to leak any customer or confidential information. HashiCorp employees may want to use our internal URL shortener to obfuscate links. --> ### PR Checklist * [ ] updated test coverage * [ ] external facing docs updated * [ ] appropriate backport labels added * [ ] not a security concern * chore: bump upgrade integrations tests to 1.15, 116 [NET-4743] (#18130) * re org resource type registry (#18133) * fix: update delegateMock used in ENT (#18149) ### Description <!-- Please describe why you're making this change, in plain English. --> The mock is used in `http_ent_test` file which caused lint failures. For OSS->ENT parity adding the same change here. ### Links <!-- Include any links here that might be helpful for people reviewing your PR (Tickets, GH issues, API docs, external benchmarks, tools docs, etc). If there are none, feel free to delete this section. Please be mindful not to leak any customer or confidential information. HashiCorp employees may want to use our internal URL shortener to obfuscate links. --> Identified in OSS->ENT [merge PR](https://github.com/hashicorp/consul-enterprise/pull/6328) ### PR Checklist * [ ] ~updated test coverage~ * [ ] ~external facing docs updated~ * [x] appropriate backport labels added * [ ] ~not a security concern~ * Use JWT-auth filter in metadata mode & Delegate validation to RBAC filter (#18062) ### Description <!-- Please describe why you're making this change, in plain English. --> - Currently the jwt-auth filter doesn't take into account the service identity when validating jwt-auth, it only takes into account the path and jwt provider during validation. This causes issues when multiple source intentions restrict access to an endpoint with different JWT providers. - To fix these issues, rather than use the JWT auth filter for validation, we use it in metadata mode and allow it to forward the successful validated JWT token payload to the RBAC filter which will make the decisions. This PR ensures requests with and without JWT tokens successfully go through the jwt-authn filter. The filter however only forwards the data for successful/valid tokens. On the RBAC filter level, we check the payload for claims and token issuer + existing rbac rules. ### Testing & Reproduction steps <!-- * In the case of bugs, describe how to replicate * If any manual tests were done, document the steps and the conditions to replicate * Call out any important/ relevant unit tests, e2e tests or integration tests you have added or are adding --> - This test covers a multi level jwt requirements (requirements at top level and permissions level). It also assumes you have envoy running, you have a redis and a sidecar proxy service registered, and have a way to generate jwks with jwt. I mostly use: https://www.scottbrady91.com/tools/jwt for this. - first write your proxy defaults ``` Kind = "proxy-defaults" name = "global" config { protocol = "http" } ``` - Create two providers ``` Kind = "jwt-provider" Name = "auth0" Issuer = "https://ronald.local" JSONWebKeySet = { Local = { JWKS = "eyJrZXlzIjog....." } } ``` ``` Kind = "jwt-provider" Name = "okta" Issuer = "https://ronald.local" JSONWebKeySet = { Local = { JWKS = "eyJrZXlzIjogW3...." } } ``` - add a service intention ``` Kind = "service-intentions" Name = "redis" JWT = { Providers = [ { Name = "okta" }, ] } Sources = [ { Name = "*" Permissions = [{ Action = "allow" HTTP = { PathPrefix = "/workspace" } JWT = { Providers = [ { Name = "okta" VerifyClaims = [ { Path = ["aud"] Value = "my_client_app" }, { Path = ["sub"] Value = "5be86359073c434bad2da3932222dabe" } ] }, ] } }, { Action = "allow" HTTP = { PathPrefix = "/" } JWT = { Providers = [ { Name = "auth0" }, ] } }] } ] ``` - generate 3 jwt tokens: 1 from auth0 jwks, 1 from okta jwks with different claims than `/workspace` expects and 1 with correct claims - connect to your envoy (change service and address as needed) to view logs and potential errors. You can add: `-- --log-level debug` to see what data is being forwarded ``` consul connect envoy -sidecar-for redis1 -grpc-addr 127.0.0.1:8502 ``` - Make the following requests: ``` curl -s -H "Authorization: Bearer $Auth0_TOKEN" --insecure --cert leaf.cert --key leaf.key --cacert connect-ca.pem https://localhost:20000/workspace -v RBAC filter denied curl -s -H "Authorization: Bearer $Okta_TOKEN_with_wrong_claims" --insecure --cert leaf.cert --key leaf.key --cacert connect-ca.pem https://localhost:20000/workspace -v RBAC filter denied curl -s -H "Authorization: Bearer $Okta_TOKEN_with_correct_claims" --insecure --cert leaf.cert --key leaf.key --cacert connect-ca.pem https://localhost:20000/workspace -v Successful request ``` ### TODO * [x] Update test coverage * [ ] update integration tests (follow-up PR) * [x] appropriate backport labels added * Support Consul Connect Envoy Command on Windows (#17694) ### Description Add support for consul connect envoy command on windows. This PR fixes the comments of PR - https://github.com/hashicorp/consul/pull/15114 ### Testing * Built consul.exe from this branch on windows and hosted here - [AWS S3](https://asheshvidyut-bucket.s3.ap-southeast-2.amazonaws.com/consul.zip) * Updated the [tutorial](https://developer.hashicorp.com/consul/tutorials/developer-mesh/consul-windows-workloads) and changed the `consul_url.default` value to [AWS S3](https://asheshvidyut-bucket.s3.ap-southeast-2.amazonaws.com/consul.zip) * Followed the steps in the tutorial and verified that everything is working as described. ### PR Checklist * [x] updated test coverage * [ ] external facing docs updated * [x] appropriate backport labels added * [x] not a security concern --------- Co-authored-by: Franco Bruno Lavayen <[email protected]> Co-authored-by: Jose Ignacio Lorenzo <[email protected]> Co-authored-by: Jose Ignacio Lorenzo <[email protected]> Co-authored-by: Dhia Ayachi <[email protected]> * Change docs to say 168h instead of 7d for server_rejoin_age_max (#18154) ### Description Addresses https://github.com/hashicorp/consul/pull/17171#issuecomment-1636930705 * [OSS] test: improve xDS listener code coverage (#18138) test: improve xDS listener code coverage * Re-order expected/actual for assertContainerState in consul container tests (#18157) Re-order expected/actual, consul container tests * group and document make file (#17943) * group and document make file * Add `testing/deployer` (neé `consul-topology`) [NET-4610] (#17823) Co-authored-by: R.B. Boyer <[email protected]> Co-authored-by: R.B. Boyer <[email protected]> Co-authored-by: Freddy <[email protected]> * [NET-4792] Add integrations tests for jwt-auth (#18169) * Add FIPS reference to consul enterprise docs (#18028) * Add FIPS reference to consul enterprise docs * Update website/content/docs/enterprise/index.mdx Co-authored-by: David Yu <[email protected]> * remove support for ecs client (fips) --------- Co-authored-by: David Yu <[email protected]> * add peering_commontopo tests [NET-3700] (#17951) Co-authored-by: R.B. Boyer <[email protected]> Co-authored-by: R.B. Boyer <[email protected]> Co-authored-by: Freddy <[email protected]> Co-authored-by: NiniOak <[email protected]> * docs - remove Sentinel from enterprise features list (#18176) * Update index.mdx * Update kv.mdx * Update docs-nav-data.json * delete sentinel.mdx * Update redirects.js --------- Co-authored-by: Tu Nguyen <[email protected]> * [NET-4865] Bump golang.org/x/net to 0.12.0 (#18186) Bump golang.org/x/net to 0.12.0 While not necessary to directly address CVE-2023-29406 (which should be handled by using a patched version of Go when building), an accompanying change to HTTP/2 error handling does impact agent code. See https://go-review.googlesource.com/c/net/+/506995 for the HTTP/2 change. Bump this dependency across our submodules as well for the sake of potential indirect consumers of `x/net/http`. * Call resource mutate hook before validate hook (NET-4907) (#18178) * [NET-4865] security: Update Go version to 1.20.6 (#18190) Update Go version to 1.20.6 This resolves [CVE-2023-29406] (https://nvd.nist.gov/vuln/detail/CVE-2023-29406) for uses of the `net/http` standard library. Note that until the follow-up to #18124 is done, the version of Go used in those impacted tests will need to remain on 1.20.5. * Improve XDS test coverage: JWT auth edition (#18183) * Improve XDS test coverage: JWT auth edition more tests * test: xds coverage for jwt listeners --------- Co-authored-by: DanStough <[email protected]> * update readme.md (#18191) u[date readme.md * Update submodules to latest following 1.16.0 (#18197) Align all our internal use of submodules on the latest versions. * SEC-090: Automated trusted workflow pinning (2023-07-18) (#18174) Result of tsccr-helper -log-level=info -pin-all-workflows . Co-authored-by: hashicorp-tsccr[bot] <hashicorp-tsccr[bot]@users.noreply.github.com> * Fix Backport Assistant PR commenting (#18200) * Fix Backport Assistant failure PR commenting For general comments on a PR, it looks like you have to use the `/issue` endpoint rather than `/pulls`, which requires commit/other review-specific target details. This matches the endpoint used in `backport-reminder.yml`. * Remove Backport Reminder workflow This is noisy (even when adding multiple labels, individual comments per label are generated), and likely no longer needed: we haven't had this work in a long time due to an expired GH token, and we now have better automation for backport PR assignment. * resource: Pass resource to Write ACL hook instead of just resource Id [NET-4908] (#18192) * Explicitly enable WebSocket upgrades (#18150) This PR explicitly enables WebSocket upgrades in Envoy's UpgradeConfig for all proxy types. (API Gateway, Ingress, and Sidecar.) Fixes #8283 * docs: fix the description of client rpc (#18206) * NET-4804: Add dashboard for monitoring consul-k8s (#18208) * [OSS] Improve xDS Code Coverage - Clusters (#18165) test: improve xDS cluster code coverage * NET-4222 take config file consul container (#18218) Net 4222 take config file consul container * Envoy Integration Test Windows (#18007) * [CONSUL-395] Update check_hostport and Usage (#40) * [CONSUL-397] Copy envoy binary from Image (#41) * [CONSUL-382] Support openssl in unique test dockerfile (#43) * [CONSUL-405] Add bats to single container (#44) * [CONSUL-414] Run Prometheus Test Cases and Validate Changes (#46) * [CONSUL-410] Run Jaeger in Single container (#45) * [CONSUL-412] Run test-sds-server in single container (#48) * [CONSUL-408] Clean containers (#47) * [CONSUL-384] Rebase and sync fork (#50) * [CONSUL-415] Create Scenarios Troubleshooting Docs (#49) * [CONSUL-417] Update Docs Single Container (#51) * [CONSUL-428] Add Socat to single container (#54) * [CONSUL-424] Replace pkill in kill_envoy function (#52) * [CONSUL-434] Modify Docker run functions in Helper script (#53) * [CONSUL-435] Replace docker run in set_ttl_check_state & wait_for_agent_service_register functions (#55) * [CONSUL-438] Add netcat (nc) in the Single container Dockerfile (#56) * [CONSUL-429] Replace Docker run with Docker exec (#57) * [CONSUL-436] Curl timeout and run tests (#58) * [CONSUL-443] Create dogstatsd Function (#59) * [CONSUL-431] Update Docs Netcat (#60) * [CONSUL-439] Parse nc Command in function (#61) * [CONSUL-463] Review curl Exec and get_ca_root Func (#63) * [CONSUL-453] Docker hostname in Helper functions (#64) * [CONSUL-461] Test wipe volumes without extra cont (#66) * [CONSUL-454] Check ports in the Server and Agent containers (#65) * [CONSUL-441] Update windows dockerfile with version (#62) * [CONSUL-466] Review case-grpc Failing Test (#67) * [CONSUL-494] Review case-cfg-resolver-svc-failover (#68) * [CONSUL-496] Replace docker_wget & docker_curl (#69) * [CONSUL-499] Cleanup Scripts - Remove nanoserver (#70) * [CONSUL-500] Update Troubleshooting Docs (#72) * [CONSUL-502] Pull & Tag Envoy Windows Image (#73) * [CONSUL-504] Replace docker run in docker_consul (#76) * [CONSUL-505] Change admin_bind * [CONSUL-399] Update envoy to 1.23.1 (#78) * [CONSUL-510] Support case-wanfed-gw on Windows (#79) * [CONSUL-506] Update troubleshooting Documentation (#80) * [CONSUL-512] Review debug_dump_volumes Function (#81) * [CONSUL-514] Add zipkin to Docker Image (#82) * [CONSUL-515] Update Documentation (#83) * [CONSUL-529] Support case-consul-exec (#86) * [CONSUL-530] Update Documentation (#87) * [CONSUL-530] Update default consul version 1.13.3 * [CONSUL-539] Cleanup (#91) * [CONSUL-546] Scripts Clean-up (#92) * [CONSUL-491] Support admin_access_log_path value for Windows (#71) * [CONSUL-519] Implement mkfifo Alternative (#84) * [CONSUL-542] Create OS Specific Files for Envoy Package (#88) * [CONSUL-543] Create exec_supported.go (#89) * [CONSUL-544] Test and Build Changes (#90) * Implement os.DevNull * using mmap instead of disk files * fix import in exec-unix * fix nmap open too many arguemtn * go fmt on file * changelog file * fix go mod * Update .changelog/17694.txt Co-authored-by: Dhia Ayachi <[email protected]> * different mmap library * fix bootstrap json * some fixes * chocolatey version fix and image fix * using different library * fix Map funciton call * fix mmap call * fix tcp dump * fix tcp dump * windows tcp dump * Fix docker run * fix tests * fix go mod * fix version 16.0 * fix version * fix version dev * sleep to debug * fix sleep * fix permission issue * fix permission issue * fix permission issue * fix command * fix command * fix funciton * fix assert config entry status command not found * fix command not found assert_cert_has_cn * fix command not found assert_upstream_missing * fix command not found assert_upstream_missing_once * fix command not found get_upstream_endpoint * fix command not found get_envoy_public_listener_once * fix command not found * fix test cases * windows integration test workflow github * made code similar to unix using npipe * fix go.mod * fix dialing of npipe * dont wait * check size of written json * fix undefined n * running * fix dep * fix syntax error * fix workflow file * windows runner * fix runner * fix from json * fix runs on * merge connect envoy * fix cin path * build * fix file name * fix file name * fix dev build * remove unwanted code * fix upload * fix bin name * fix path * checkout current branch * fix path * fix tests * fix shell bash for windows sh files * fix permission of run-test.sh * removed docker dev * added shell bash for tests * fix tag * fix win=true * fix cd * added dev * fix variable undefined * removed failing tests * fix tcp dump image * fix curl * fix curl * tcp dump path * fix tcpdump path * fix curl * fix curl install * stop removing intermediate containers * fix tcpdump docker image * revert -rm * --rm=false * makeing docker image before * fix tcpdump * removed case consul exec * removed terminating gateway simple * comment case wasm * removed data dog * comment out upload coverage * uncomment case-consul-exec * comment case consul exec * if always * logs * using consul 1.17.0 * fix quotes * revert quotes * redirect to dev null * Revert version * revert consul connect * fix version * removed envoy connect * not using function * change log * docker logs * fix logs * restructure bad authz * rmeoved dev null * output * fix file descriptor * fix cacert * fix cacert * fix ca cert * cacert does not work in windows curl * fix func * removed docker logs * added sleep * fix tls * commented case-consul-exec * removed echo * retry docker consul * fix upload bin * uncomment consul exec * copying consul.exe to docker image * copy fix * fix paths * fix path * github workspace path * latest version * Revert "latest version" This reverts commit 5a7d7b82d9e7553bcb01b02557ec8969f9deba1d. * commented consul exec * added ssl revoke best effort * revert best effort * removed unused files * rename var name and change dir * windows runner * permission * needs setup fix * swtich to github runner * fix file path * fix path * fix path * fix path * fix path * fix path * fix build paths * fix tag * nightly runs * added matrix in github workflow, renamed files * fix job * fix matrix * removed brackes * from json * without using job matrix * fix quotes * revert job matrix * fix workflow * fix comment * added comment * nightly runs * removed datadog ci as it is already measured in linux one * running test * Revert "running test" This reverts commit 7013d15a23732179d18ec5d17336e16b26fab5d4. * pr comment fixes * running test now * running subset of test * running subset of test * job matrix * shell bash * removed bash shell * linux machine for job matrix * fix output * added cat to debug * using ubuntu latest * fix job matrix * fix win true * fix go test * revert job matrix --------- Co-authored-by: Jose Ignacio Lorenzo <[email protected]> Co-authored-by: Franco Bruno Lavayen <[email protected]> Co-authored-by: Ivan K Berlot <[email protected]> Co-authored-by: Ezequiel Fernández Ponce <[email protected]> Co-authored-by: joselo85 <[email protected]> Co-authored-by: Ezequiel Fernández Ponce <[email protected]> Co-authored-by: Dhia Ayachi <[email protected]> * fix typos and update ecs compat table (#18215) * fix typos and update ecs compat table * real info for the ecs compat matrix table * Update website/content/docs/ecs/compatibility.mdx Co-authored-by: Chris Thain <[email protected]> --------- Co-authored-by: Chris Thain <[email protected]> * [OSS] proxystate: add proxystate protos (#18216) * proxystate: add proxystate protos to pbmesh and resolve imports and conflicts between message names * ci: don't verify s390x (#18224) * [CC-5718] Remove HCP token requirement during bootstrap (#18140) * [CC-5718] Remove HCP token requirement during bootstrap * Re-add error for loading HCP management token * Remove old comment * Add changelog entry * Remove extra validation line * Apply suggestions from code review Co-authored-by: lornasong <[email protected]> --------- Co-authored-by: lornasong <[email protected]> * [NET-4122] Doc guidance for federation with externalServers (#18207) Doc guidance for federation with externalServers Add guidance for proper configuration when joining to a secondary cluster using WAN fed with external servers also enabled. Also clarify federation requirements and fix formatting for an unrelated value. Update both the Helm chart reference (synced from `consul-k8s`, see hashicorp/consul-k8s#2583) and the docs on using `externalServers`. * [OSS] Improve xDS Code Coverage - Endpoints and Misc (#18222) test: improve xDS endpoints code coverage * Clarify license reporting timing and GDPR compliance (#18237) Add Alicia's edits to clarify log timing and other details * Fix Github Workflow File (#18241) * [CONSUL-382] Support openssl in unique test dockerfile (#43) * [CONSUL-405] Add bats to single container (#44) * [CONSUL-414] Run Prometheus Test Cases and Validate Changes (#46) * [CONSUL-410] Run Jaeger in Single container (#45) * [CONSUL-412] Run test-sds-server in single container (#48) * [CONSUL-408] Clean containers (#47) * [CONSUL-384] Rebase and sync fork (#50) * [CONSUL-415] Create Scenarios Troubleshooting Docs (#49) * [CONSUL-417] Update Docs Single Container (#51) * [CONSUL-428] Add Socat to single container (#54) * [CONSUL-424] Replace pkill in kill_envoy function (#52) * [CONSUL-434] Modify Docker run functions in Helper script (#53) * [CONSUL-435] Replace docker run in set_ttl_check_state & wait_for_agent_service_register functions (#55) * [CONSUL-438] Add netcat (nc) in the Single container Dockerfile (#56) * [CONSUL-429] Replace Docker run with Docker exec (#57) * [CONSUL-436] Curl timeout and run tests (#58) * [CONSUL-443] Create dogstatsd Function (#59) * [CONSUL-431] Update Docs Netcat (#60) * [CONSUL-439] Parse nc Command in function (#61) * [CONSUL-463] Review curl Exec and get_ca_root Func (#63) * [CONSUL-453] Docker hostname in Helper functions (#64) * [CONSUL-461] Test wipe volumes without extra cont (#66) * [CONSUL-454] Check ports in the Server and Agent containers (#65) * [CONSUL-441] Update windows dockerfile with version (#62) * [CONSUL-466] Review case-grpc Failing Test (#67) * [CONSUL-494] Review case-cfg-resolver-svc-failover (#68) * [CONSUL-496] Replace docker_wget & docker_curl (#69) * [CONSUL-499] Cleanup Scripts - Remove nanoserver (#70) * [CONSUL-500] Update Troubleshooting Docs (#72) * [CONSUL-502] Pull & Tag Envoy Windows Image (#73) * [CONSUL-504] Replace docker run in docker_consul (#76) * [CONSUL-505] Change admin_bind * [CONSUL-399] Update envoy to 1.23.1 (#78) * [CONSUL-510] Support case-wanfed-gw on Windows (#79) * [CONSUL-506] Update troubleshooting Documentation (#80) * [CONSUL-512] Review debug_dump_volumes Function (#81) * [CONSUL-514] Add zipkin to Docker Image (#82) * [CONSUL-515] Update Documentation (#83) * [CONSUL-529] Support case-consul-exec (#86) * [CONSUL-530] Update Documentation (#87) * [CONSUL-530] Update default consul version 1.13.3 * [CONSUL-539] Cleanup (#91) * [CONSUL-546] Scripts Clean-up (#92) * [CONSUL-491] Support admin_access_log_path value for Windows (#71) * [CONSUL-519] Implement mkfifo Alternative (#84) * [CONSUL-542] Create OS Specific Files for Envoy Package (#88) * [CONSUL-543] Create exec_supported.go (#89) * [CONSUL-544] Test and Build Changes (#90) * Implement os.DevNull * using mmap instead of disk files * fix import in exec-unix * fix nmap open too many arguemtn * go fmt on file * changelog file * fix go mod * Update .changelog/17694.txt Co-authored-by: Dhia Ayachi <[email protected]> * different mmap library * fix bootstrap json * some fixes * chocolatey version fix and image fix * using different library * fix Map funciton call * fix mmap call * fix tcp dump * fix tcp dump * windows tcp dump * Fix docker run * fix tests * fix go mod * fix version 16.0 * fix version * fix version dev * sleep to debug * fix sleep * fix permission issue * fix permission issue * fix permission issue * fix command * fix command * fix funciton * fix assert config entry status command not found * fix command not found assert_cert_has_cn * fix command not found assert_upstream_missing * fix command not found assert_upstream_missing_once * fix command not found get_upstream_endpoint * fix command not found get_envoy_public_listener_once * fix command not found * fix test cases * windows integration test workflow github * made code similar to unix using npipe * fix go.mod * fix dialing of npipe * dont wait * check size of written json * fix undefined n * running * fix dep * fix syntax error * fix workflow file * windows runner * fix runner * fix from json * fix runs on * merge connect envoy * fix cin path * build * fix file name * fix file name * fix dev build * remove unwanted code * fix upload * fix bin name * fix path * checkout current branch * fix path * fix tests * fix shell bash for windows sh files * fix permission of run-test.sh * removed docker dev * added shell bash for tests * fix tag * fix win=true * fix cd * added dev * fix variable undefined * removed failing tests * fix tcp dump image * fix curl * fix curl * tcp dump path * fix tcpdump path * fix curl * fix curl install * stop removing intermediate containers * fix tcpdump docker image * revert -rm * --rm=false * makeing docker image before * fix tcpdump * removed case consul exec * removed terminating gateway simple * comment case wasm * removed data dog * comment out upload coverage * uncomment case-consul-exec * comment case consul exec * if always * logs * using consul 1.17.0 * fix quotes * revert quotes * redirect to dev null * Revert version * revert consul connect * fix version * removed envoy connect * not using function * change log * docker logs * fix logs * restructure bad authz * rmeoved dev null * output * fix file descriptor * fix cacert * fix cacert * fix ca cert * cacert does not work in windows curl * fix func * removed docker logs * added sleep * fix tls * commented case-consul-exec * removed echo * retry docker consul * fix upload bin * uncomment consul exec * copying consul.exe to docker image * copy fix * fix paths * fix path * github workspace path * latest version * Revert "latest version" This reverts commit 5a7d7b82d9e7553bcb01b02557ec8969f9deba1d. * commented consul exec * added ssl revoke best effort * revert best effort * removed unused files * rename var name and change dir * windows runner * permission * needs setup fix * swtich to github runner * fix file path * fix path * fix path * fix path * fix path * fix path * fix build paths * fix tag * nightly runs * added matrix in github workflow, renamed files * fix job * fix matrix * removed brackes * from json * without using job matrix * fix quotes * revert job matrix * fix workflow * fix comment * added comment * nightly runs * removed datadog ci as it is already measured in linux one * running test * Revert "running test" This reverts commit 7013d15a23732179d18ec5d17336e16b26fab5d4. * pr comment fixes * running test now * running subset of test * running subset of test * job matrix * shell bash * removed bash shell * linux machine for job matrix * fix output * added cat to debug * using ubuntu latest * fix job matrix * fix win true * fix go test * revert job matrix * Fix tests --------- Co-authored-by: Ivan K Berlot <[email protected]> Co-authored-by: Jose Ignacio Lorenzo <[email protected]> Co-authored-by: Franco Bruno Lavayen <[email protected]> Co-authored-by: Ezequiel Fernández Ponce <[email protected]> Co-authored-by: joselo85 <[email protected]> Co-authored-by: Ezequiel Fernández Ponce <[email protected]> Co-authored-by: Dhia Ayachi <[email protected]> * NET-4996 - filter go-tests and test-integration workflows from running on docs only and ui only changes (#18236) * Align build arch matrix with enterprise (#18235) Ensure that OSS remains in sync w/ Enterprise by aligning the format of arch matrix args for various build jobs. * Revert "NET-4996 - filter go-tests and test-integration workflows from running on docs only and ui only changes" (#18248) Revert "NET-4996 - filter go-tests and test-integration workflows from running on docs only and ui only changes (#18236)" This reverts commit a11dba710e6ce6f172c0fa6c9b61567cc1efffc8. * resource: Add scope to resource type registration [NET-4976] (#18214) Enables querying a resource type's registration to determine if a resource is cluster, partition, or partition and namespace scoped. * Fix some inconsistencies in jwt docs (#18234) * NET-1825: More new ACL token creation docs (#18063) Co-authored-by: trujillo-adam <[email protected]> * grafana: add the panel resource usage of connect injector (#18247) * [NET-3700] Backfill changelog entry for c2bbe67 and 7402d06 (#18259) Backfill changelog entry for c2bbe67 and 7402d06 Add a changelog entry for the follow-up PR since it was specific to the fix and references the original change. * NET-4897 - update comment to include the current issue url from the go team. (#18263) * fix typos, style, and improper links (#18269) * member cli: add -filter expression to flags (#18223) * member cli: add -filter expression to flags * changelog * update doc * Add test cases * use quote * go-tests: disable s390x (#18273) * docs: Update ext-authz documentation for kubernetes (#18281) Update ext-authz documentation for kubernetes * docs: Consul on Kubernetes specific upgrade info (#18230) * Compatibility page - dataplanes mention * Upgrading higher-level dataplane mention * `exec=` string callout * More visible for upgrade page * Apply suggestions from code review Co-authored-by: Tu Nguyen <[email protected]> --------- Co-authored-by: Tu Nguyen <[email protected]> Co-authored-by: John Murret <[email protected]> * Fix typo in Envoy extensions doc (#18284) * docs: K8s secondary DC requirements (#18280) * Requested edit * Apply suggestions from code review Co-authored-by: trujillo-adam <[email protected]> --------- Co-authored-by: trujillo-adam <[email protected]> * api-gateway: subscribe to bound-api-gateway only after receiving api-gateway (#18291) * api-gateway: subscribe to bound-api-gateway only after receiving api-gateway This fixes a race condition due to our dependency on having the listener(s) from the api-gateway config entry in order to fully and properly process the resources on the bound-api-gateway config entry. * Apply suggestions from code review * Add changelog entry * Update K8s changelog to address cloud auto-join change in 1.0.0 (#18293) * Update list of Envoy versions (#18300) Update supported envoy versions to 1.24.10, 1.25.9, 1.26.4, 1.27.0. * [NET-4904] Update list of Envoy versions in docs (#18306) Update list of Envoy versions in docs Update supported Envoy versions across Consul release versions. * Update actions for TSCCR (#18317) Update action versions before deadline * Fix topology view when displaying mixed connect-native/normal services. (#13023) * Fix topoloy intention with mixed connect-native/normal services. If a service is registered twice, once with connect-native and once without, the topology views would prune the existing intentions. This change brings the code more in line with the transparent proxy behavior. * Dedupe nodes in the ServiceTopology ui endpoint (like done with tags). * Consider a service connect-native as soon as one instance is. * Stop JWT provider from being written in non default namespace (#18325) * docs: Fix some comments (#17118) Signed-off-by: cui fliter <[email protected]> * [NET-4703] Prevent partial application of Envoy extensions (#18068) Prevent partial application of Envoy extensions Ensure that non-required extensions do not change xDS resources before exiting on failure by cloning proto messages prior to applying each extension. To support this change, also move `CanApply` checks up a layer and make them prior to attempting extension application, s.t. we avoid unnecessary copies where extensions can't be applied. Last, ensure that we do not allow panics from `CanApply` or `Extend` checks to escape the attempted extension application. * docs: Simplify example jq commands by removing pipes (#18327) Simplify jq command examples by removing pipes to other commands. * fix typo in create a mesh token docs (#18337) * [CC-5719] Add support for builtin global-read-only policy (#18319) * [CC-5719] Add support for builtin global-read-only policy * Add changelog * Add read-only to docs * Fix some minor issues. * Change from ReplaceAll to Sprintf * Change IsValidPolicy name to return an error instead of bool * Fix PolicyList test * Fix other tests * Apply suggestions from code review Co-authored-by: Paul Glass <[email protected]> * Fix state store test for policy list. * Fix naming issues * Update acl/validation.go Co-authored-by: Chris Thain <[email protected]> * Update agent/consul/acl_endpoint.go --------- Co-authored-by: Paul Glass <[email protected]> Co-authored-by: Chris Thain <[email protected]> * [NET-5121] proxystate: move protos to subdirectory to avoid conflicts (#18335) * also makes a few protos updates * resource: adding various helpers for working with resources (#18342) This is a bit of a grab bag of helpers that I found useful for working with them when authoring substantial Controllers. Subsequent PRs will make use of them. * Wasm integration tests for local and remote wasm files (#17756) * wasm integration tests for local and remote wasm files refactoring and cleanup for wasm testing remove wasm debug logging PR feedback, wasm build lock correct path pattern for wasm build files Add new helper function to minimize changes to existing test code Remove extra param mod tidy add custom service setup to test lib add wait until static server sidecar can reach nginx sidecar Doc comments PR feedback Update workflows to compile wasm for integration tests Fix docker build path Fix package name for linter Update makefile, fix redeclared function Update expected wasm filename Debug test ls in workflow remove pwd in favor of relative path more debugging Build wasm in compatability tests as well Build wasm directly in ci rather than in container Debug tinygo and llvm version Change wasm file extension Remove tinygo debugging Remove extra comments * Add compiled wasm and build instructions * [HCP Telemetry] Periodic Refresh for Dynamic Telemetry Configuration (#18168) * OTElExporter now uses an EndpointProvider to discover the endpoint * OTELSink uses a ConfigProvider to obtain filters and labels configuration * improve tests for otel_sink * Regex logic is moved into client for a method on the TelemetryConfig object * Create a telemetry_config_provider and update deps to use it * Fix conversion * fix import newline * Add logger to hcp client and move telemetry_config out of the client.go file * Add a telemetry_config.go to refactor client.go * Update deps * update hcp deps test * Modify telemetry_config_providers * Check for nil filters * PR review updates * Fix comments and move around pieces * Fix comments * Remove context from client struct * Moved ctx out of sink struct and fixed filters, added a test * Remove named imports, use errors.New if not fformatting * Remove HCP dependencies in telemetry package * Add success metric and move lock only to grab the t.cfgHahs * Update hash * fix nits * Create an equals method and add tests * Improve telemetry_config_provider.go tests * Add race test * Add missing godoc * Remove mock for MetricsClient * Avoid goroutine test panics * trying to kick CI lint issues by upgrading mod * imprve test code and add hasher for testing * Use structure logging for filters, fix error constants, and default to allow all regex * removed hashin and modify logic to simplify * Improve race test and fix PR feedback by removing hash equals and avoid testing the timer.Ticker logic, and instead unit test * Ran make go-mod-tidy * Use errtypes in the test * Add changelog * add safety check for exporter endpoint * remove require.Contains by using error types, fix structure logging, and fix success metric typo in exporter * Fixed race test to have changing config values * Send success metric before modifying config * Avoid the defer and move the success metric under * NET-4240 - Snapshots are failing on Windows (#18302) * fix go mod * fix go sum * added change log * ran make go mod tidy * Register ProxyStateTemplate Resource (#18316) Also, change the ProxyState.id to identity. This is because we already have the id of this proxy from the resource, and this id should be name-aligned with the workload it represents. It should also have the owner ref set to the workload ID if we need that. And so the id field seems unnecessary. We do, however, need a reference to workload identity so that we can authorize the proxy when it initially connects to the xDS server. * [NET-5146] security: Update Go version to 1.20.7 and `x/net` to 0.13.0 (#18358) * Update Go version to 1.20.7 This resolves [CVE-2023-29409] (https://nvd.nist.gov/vuln/detail/CVE-2023-29409)(`crypto/tls`). * Bump golang.org/x/net to 0.13.0 Addresses [CVE-2023-3978](https://nvd.nist.gov/vuln/detail/CVE-2023-3978) for security scans (non-impacting). * mesh: adding the protobuf types and resources backing mesh config v2 (#18351) * [OSS] test: xds coverage for routes (#18369) test: xds coverage for routes * Fix policy lookup to allow for slashes (#18347) * Fix policy lookup to allow for slashes * Fix suggestions * Fix other test * Revert some lines * add some initial CODEOWNERS (#18346) * Add redirects for mesh-gateway docs (#18377) * expose grpc as http endpoint (#18221) expose resource grpc endpoints as http endpoints * bimapper: fix a bug and add some more test coverage (#18387) * [docs] Fix ServiceDefaults example in distributed tracing (#17212) Fix ServiceDefaults example in distributed tracing. * Fix #17730 - Dev mode has new line (#18367) * adding new line only in case of pretty in url not in dev mode * change log added * chore: make go-mod-tidy (#18388) * resource: Make resource read tenancy aware (#18397) * Not using chmod - fixed integration test for Enterprise (#18401) * Not using chmod * checking icacls * Removed push * replaced ordered list of rate limit ops with flow diagram (#18398) * replaced ordered list of rate limit ops with flow diagram * made diagram font bigger * updated colors for dark mode img * fixed typo * recentered dark mode image * Apply suggestions from code review Co-authored-by: Jeff Boruszak <[email protected]> --------- Co-authored-by: Jeff Boruszak <[email protected]> * Upgrade test: remove outdated test and disable log due to verbosity (#18403) * remove outdated test * disable log since we have too many parallel tests * bump testcontainers-go from 0.22.0 and remove pinned go version in in… (#18395) * bump testcontainers-go from 0.22.0 and remove pinned go version in integ test * go mod tidy * Replace deprecated target.Authority with target.URL.Host * Add ServiceEndpoints Mutation hook tests (#18404) * Add ServiceEndpoints Mutation hook tests * Move endpoint owner validation into the validation hook Also there were some minor changes to error validation to account for go-cmp not liking to peer through an errors.errorstring type that get created by errors.New * catalog: add FailoverPolicy mutation and validation hooks (#18390) Add most of the validation and mutation hooks for the FailoverPolicy resource. * NET-5115 Add retry + timeout filters for api-gateway (#18324) * squash, implement retry/timeout in consul core * update tests * catalog: adding a controller to reconcile FailoverPolicy resources (#18399) Add most of the semantic cross-resource validation for FailoverPolicy resources using a new controller. * fix the error in ent repo (#18421) fix the error in ent repo * build: updates for 1.16.1 release (#18415) * build: updates for 1.16.1 release * build: fix missing replace directive for envoyextensions * build: go mod tidy testing/deployer * update ECS links (#18419) * docs: fix incorrect proxy-defaults config in Lua Envoy extension (#18418) fix incorrect proxy-defaults config * resource: Make resource write tenancy aware (#18423) * [NET-5217] [OSS] Derive sidecar proxy locality from parent service (#18437) * Add logging to locality policy application In OSS, this is currently a no-op. * Inherit locality when registering sidecars When sidecar locality is not explicitly configured, inherit locality from the proxied service. * NET-4984: Update APIGW Config Entries for JWT Auth (#18366) * Added oss config entries for Policy and JWT on APIGW * Updated structs for config entry * Updated comments, ran deep-copy * Move JWT configuration into OSS file * Add in the config entry OSS file for jwts * Added changelog * fixing proto spacing * Moved to using manually written deep copy method * Use pointers for override/default fields in apigw config entries * Run gen scripts for changed types * k8s compat - Openshift versions (#18307) * Update compatibility.mdx Co-authored-by: trujillo-adam <[email protected]> * APIGW: Update HTTPRouteConfigEntry for JWT Auth (#18422) * Updated httproute config entry for JWT Filters * Added manual deepcopy method for httproute jwt filter * Fix test * Update JWTFilter to be in oss file * Add changelog * Add build tags for deepcopy oss file * [COMPLIANCE] License changes (#18443) * Adding explicit MPL license for sub-package This directory and its subdirectories (packages) contain files licensed with the MPLv2 `LICENSE` file in this directory and are intentionally licensed separately from the BSL `LICENSE` file at the root of this repository. * Adding explicit MPL license for sub-package This directory and its subdirectories (packages) contain files licensed with the MPLv2 `LICENSE` file in this directory and are intentionally licensed separately from the BSL `LICENSE` file at the root of this repository. * Updating the license from MPL to Business Source License Going forward, this project will be licensed under the Business Source License v1.1. Please see our blog post for more details at <Blog URL>, FAQ at www.hashicorp.com/licensing-faq, and details of the license at www.hashicorp.com/bsl. * add missing license headers * Update copyright file headers to BUSL-1.1 * Update copyright file headers to BUSL-1.1 * Update copyright file headers to BUSL-1.1 * Update copyright file headers to BUSL-1.1 * Update copyright file headers to BUSL-1.1 * Update copyright file headers to BUSL-1.1 * Update copyright file headers to BUSL-1.1 * Update copyright file headers to BUSL-1.1 * Update copyright file headers to BUSL-1.1 * Up…
absolutelightning
added a commit
that referenced
this pull request
Sep 4, 2023
…1.16.x (#18630) * api-gateway: add step to upgrade instructions for creating intentions (#17875) * Changelog - add 1.13.9, 1.14.8, and 1.15.4 (#17889) * docs: update config enable_debug (#17866) * update doc for config enable_debug * Update website/content/docs/agent/config/config-files.mdx Co-authored-by: trujillo-adam <[email protected]> --------- Co-authored-by: trujillo-adam <[email protected]> * Update wording on WAN fed and intermediate_pki_path (#17850) * Allow service identity tokens the ability to read jwt-providers (#17893) * Allow service identity tokens the ability to read jwt-providers * more tests * service_prefix tests * Update docs (#17476) Co-authored-by: trujillo-adam <[email protected]> * Add emit_tags_as_labels to envoy bootstrap config when using Consul Telemetry Collector (#17888) * Fix command from kg to kubectl get (#17903) * Create and update release notes for 1.16 and 1.2 (#17895) * update release notes for 1.16 and 1.2 * update latest consul core release * Propose new changes to APIgw upgrade instructions (#17693) * Propose new changes to APIgw upgrade instructions * fix build error * update callouts to render correctly * Add hideClipboard to log messages * Added clarification around consul k8s and crds * Add workflow to verify linux release packages (#17904) * adding docker files to verify linux packages. * add verifr-release-linux.yml * updating name * pass inputs directly into jobs * add other linux package platforms * remove on push * fix TARGETARCH on debian and ubuntu so it can check arm64 and amd64 * fixing amazon to use the continue line * add ubuntu i386 * fix comment lines * working * remove commented out workflow jobs * Apply suggestions from code review Co-authored-by: Nathan Coleman <[email protected]> * update fedora and ubuntu to use latest tag --------- Co-authored-by: Nathan Coleman <[email protected]> * Reference hashicorp/consul instead of consul for Docker image (#17914) * Reference hashicorp/consul instead of consul for Docker image * Update Make targets that pull consul directly * Update Consul K8s Upgrade Doc Updates (#17921) Updating upgrade procedures to encompass expected errors during upgrade process from v1.13.x to v1.14.x. * Update sameness-group.mdx (#17915) * Update create-sameness-groups.mdx (#17927) * deps: coredns v1.10.1 (#17912) * Ensure RSA keys are at least 2048 bits in length (#17911) * Ensure RSA keys are at least 2048 bits in length * Add changelog * update key length check for FIPS compliance * Fix no new variables error and failing to return when error exists from validating * clean up code for better readability * actually return value * tlsutil: Fix check TLS configuration (#17481) * tlsutil: Fix check TLS configuration * Rewording docs. * Update website/content/docs/services/configuration/checks-configuration-reference.mdx Co-authored-by: trujillo-adam <[email protected]> * Fix typos and add changelog entry. --------- Co-authored-by: trujillo-adam <[email protected]> * docs: Deprecations for connect-native SDK and specific connect native APIs (#17937) * Update v1_16_x.mdx * Update connect native golang page --------- Co-authored-by: trujillo-adam <[email protected]> * Revert "Add workflow to verify linux release packages (#17904)" (#17942) This reverts commit 3368f14fab500ebe9f6aeab5631dd1d5f5a453e5. * Fixes Secondary ConnectCA update (#17846) This fixes a bug that was identified which resulted in subsequent ConnectCA configuration update not to persist in the cluster. * fixing typo in link to jwt-validations-with-intentions doc (#17955) * Fix streaming backend link (#17958) * Fix streaming backend link * Update health.mdx * Dynamically create jwks clusters for jwt-providers (#17944) * website: remove deprecated agent rpc docs (#17962) * Fix missing BalanceOutboundConnections in v2 catalog. (#17964) * feature - [NET - 4005] - [Supportability] Reloadable Configuration - enable_debug (#17565) * # This is a combination of 9 commits. # This is the 1st commit message: init without tests # This is the commit message #2: change log # This is the commit message #3: fix tests # This is the commit message #4: fix tests # This is the commit message #5: added tests # This is the commit message #6: change log breaking change # This is the commit message #7: removed breaking change # This is the commit message #8: fix test # This is the commit message #9: keeping the test behaviour same * # This is a combination of 12 commits. # This is the 1st commit message: init without tests # This is the commit message #2: change log # This is the commit message #3: fix tests # This is the commit message #4: fix tests # This is the commit message #5: added tests # This is the commit message #6: change log breaking change # This is the commit message #7: removed breaking change # This is the commit message #8: fix test # This is the commit message #9: keeping the test behaviour same # This is the commit message #10: made enable debug atomic bool # This is the commit message #11: fix lint # This is the commit message #12: fix test true enable debug * parent 10f500e895d92cc3691ade7b74a33db755d22039 author absolutelightning <[email protected]> 1687352587 +0530 committer absolutelightning <[email protected]> 1687352592 +0530 init without tests change log fix tests fix tests added tests change log breaking change removed breaking change fix test keeping the test behaviour same made enable debug atomic bool fix lint fix test true enable debug using enable debug in agent as atomic bool test fixes fix tests fix tests added update on correct locaiton fix tests fix reloadable config enable debug fix tests fix init and acl 403 * revert commit * Fix formatting codeblocks on APIgw docs (#17970) * fix formatting codeblocks * remove unnecessary indents * Remove POC code (#17974) * update doc (#17910) * update doc * update link * Remove duplicate and unused newDecodeConfigEntry func (#17979) * docs: samenessGroup YAML examples (#17984) * configuration entry syntax * Example config * Add changelog entry for 1.16.0 (#17987) * Fix typo (#17198) servcies => services * Expose JWKS cluster config through JWTProviderConfigEntry (#17978) * Expose JWKS cluster config through JWTProviderConfigEntry * fix typos, rename trustedCa to trustedCA * Integration test for ext-authz Envoy extension (#17980) * Fix incorrect protocol for transparent proxy upstreams. (#17894) This PR fixes a bug that was introduced in: https://github.com/hashicorp/consul/pull/16021 A user setting a protocol in proxy-defaults would cause tproxy implicit upstreams to not honor the upstream service's protocol set in its `ServiceDefaults.Protocol` field, and would instead always use the proxy-defaults value. Due to the fact that upstreams configured with "tcp" can successfully contact upstream "http" services, this issue was not recognized until recently (a proxy-defaults with "tcp" and a listening service with "http" would make successful requests, but not the opposite). As a temporary work-around, users experiencing this issue can explicitly set the protocol on the `ServiceDefaults.UpstreamConfig.Overrides`, which should take precedence. The fix in this PR removes the proxy-defaults protocol from the wildcard upstream that tproxy uses to configure implicit upstreams. When the protocol was included, it would always overwrite the value during discovery chain compilation, which was not correct. The discovery chain compiler also consumes proxy defaults to determine the protocol, so simply excluding it from the wildcard upstream config map resolves the issue. * feat: include nodes count in operator usage endpoint and cli command (#17939) * feat: update operator usage api endpoint to include nodes count * feat: update operator usange cli command to includes nodes count * [OSS] Improve Gateway Test Coverage of Catalog Health (#18011) * fix(cli): remove failing check from 'connect envoy' registration for api gateway * test(integration): add tests to check catalog statsus of gateways on startup * remove extra sleep comment * Update test/integration/consul-container/libs/assert/service.go * changelog * Fixes Traffic rate limitting docs (#17997) * Fix removed service-to-service peering links (#17221) * docs: fix removed service-to-service peering links * docs: extend peering-via-mesh-gateways intro (thanks @trujillo-adam) --------- Co-authored-by: trujillo-adam <[email protected]> * docs: Sameness "beta" warning (#18017) * Warning updates * .x * updated typo in tab heading (#18022) * updated typo in tab heading * updated tab group typo, too * Document that DNS lookups can target cluster peers (#17990) Static DNS lookups, in addition to explicitly targeting a datacenter, can target a cluster peer. This was added in 95dc0c7b301b70a6b955a8b7c9737c9b86f03df6 but didn't make the documentation. The driving function for the change is `parseLocality` here: https://github.com/hashicorp/consul/blob/0b1299c28d8127129d61310ee4280055298438e0/agent/dns_oss.go#L25 The biggest change in this is to adjust the standard lookup syntax to tie `.<datacenter>` to `.dc` as required-together, and to append in the similar `.<cluster-peer>.peer` optional argument, both to A record and SRV record lookups. Co-authored-by: David Yu <[email protected]> * Add first integration test for jwt auth with intention (#18005) * fix stand-in text for name field (#18030) * removed sameness conf entry from failover nav (#18033) * docs - add service sync annotations and k8s service weight annotation (#18032) * Docs for https://github.com/hashicorp/consul-k8s/pull/2293 * remove versions for enterprise features since they are old --------- Co-authored-by: Tu Nguyen <[email protected]> * docs - add jobs use case for service mesh k8s (#18037) * docs - add jobs use case for service mesh k8s * add code blocks * address feedback (#18045) * Add verify server hostname to tls default (#17155) * [OSS] Fix initial_fetch_timeout to wait for all xDS resources (#18024) * fix(connect): set initial_fetch_time to wait indefinitely * changelog * PR feedback 1 * ui: fix typos for peer service imports (#17999) * test: fix FIPS inline cert test message (#18076) * Fix a couple typos in Agent Telemetry Metrics docs (#18080) * Fix metrics docs * Add changelog Signed-off-by: josh <[email protected]> --------- Signed-off-by: josh <[email protected]> * docs updates - cluster peering and virtual services (#18069) * Update route-to-virtual-services.mdx * Update establish-peering.mdx * Update service-mesh-compare.mdx (#17279) grammar change * Update helm docs on main (#18085) * ci: use gotestsum v1.10.1 [NET-4042] (#18088) * Docs: Update proxy lifecycle annotations and consul-dataplane flags (#18075) * Update proxy lifecycle annotations and consul-dataplane flags * Pass configured role name to Vault for AWS auth in Connect CA (#17885) * Docs for dataplane upgrade on k8s (#18051) * Docs for dataplane upgrade on k8s --------- Co-authored-by: David Yu <[email protected]> Co-authored-by: trujillo-adam <[email protected]> * docs - update upgrade index page to not recommend consul leave. (#18100) * Displays Consul version of each nodes in UI nodes section (#17754) * update UINodes and UINodeInfo response with consul-version info added as NodeMeta, fetched from serf members * update test cases TestUINodes, TestUINodeInfo * added nil check for map * add consul-version in local agent node metadata * get consul version from serf member and add this as node meta in catalog register request * updated ui mock response to include consul versions as node meta * updated ui trans and added version as query param to node list route * updates in ui templates to display consul version with filter and sorts * updates in ui - model class, serializers,comparators,predicates for consul version feature * added change log for Consul Version Feature * updated to get version from consul service, if for some reason not available from serf * updated changelog text * updated dependent testcases * multiselection version filter * Update agent/consul/state/catalog.go comments updated Co-authored-by: Jared Kirschner <[email protected]> --------- Co-authored-by: Jared Kirschner <[email protected]> * api gw 1.16 updates (#18081) * api gw 1.16 updates * Apply suggestions from code review Co-authored-by: Jeff Boruszak <[email protected]> * update CodeBlockConfig filename * Apply suggestions from code review Co-authored-by: trujillo-adam <[email protected]> Co-authored-by: Jeff Boruszak <[email protected]> * remove non-standard intentions page * Update website/content/docs/api-gateway/configuration/index.mdx Co-authored-by: trujillo-adam <[email protected]> --------- Co-authored-by: Jeff Boruszak <[email protected]> Co-authored-by: trujillo-adam <[email protected]> * [NET-4103] ci: build s390x (#18067) * ci: build s390x * ci: test s390x * ci: dev build s390x * no GOOS * ent only * build: publish s390x * fix syntax error * fix syntax error again * fix syntax error again x2 * test branch * Move s390x conditionals to step level * remove test branch --------- Co-authored-by: emilymianeil <[email protected]> * :ermahgerd "Sevice Mesh" -> "Service Mesh" (#18116) Just a typo in the docs. * Split pbmesh.UpstreamsConfiguration as a resource out of pbmesh.Upstreams (#17991) Configuration that previously was inlined into the Upstreams resource applies to both explicit and implicit upstreams and so it makes sense to split it out into its own resource. It also has other minor changes: - Renames `proxy.proto` proxy_configuration.proto` - Changes the type of `Upstream.destination_ref` from `pbresource.ID` to `pbresource.Reference` - Adds comments to fields that didn't have them * [NET-4895] ci - api tests and consul container tests error because of dependency bugs with go 1.20.6. Pin go to 1.20.5. (#18124) ### Description The following jobs started failing when go 1.20.6 was released: - `go-test-api-1-19` - `go-test-api-1-20` - `compatibility-integration-tests` - `upgrade-integration-tests` `compatibility-integration-tests` and `compatibility-integration-tests` to this testcontainers issue: https://github.com/testcontainers/testcontainers-go/issues/1359. This issue calls for testcontainers to release a new version when one of their dependencies is fixed. When that is done, we will unpin the go versions in `compatibility-integration-tests` and `compatibility-integration-tests`. ### Testing & Reproduction steps See these jobs broken in CI and then see them work with this PR. --------- Co-authored-by: Chris Thain <[email protected]> * Add ingress gateway deprecation notices to docs (#18102) ### Description This adds notices, that ingress gateway is deprecated, to several places in the product docs where ingress gateway is the topic. ### Testing & Reproduction steps Tested with a local copy of the website. ### Links Deprecation of ingress gateway was announced in the Release Notes for Consul 1.16 and Consul-K8s 1.2. See: [https://developer.hashicorp.com/consul/docs/release-notes/consul/v1_16_x#what-s-deprecated](https://developer.hashicorp.com/consul/docs/release-notes/consul/v1_16_x#what-s-deprecated ) [https://developer.hashicorp.com/consul/docs/release-notes/consul-k8s/v1_2_x#what-s-deprecated](https://developer.hashicorp.com/consul/docs/release-notes/consul-k8s/v1_2_x#what-s-deprecated) ### PR Checklist * [N/A] updated test coverage * [X] external facing docs updated * [X] appropriate backport labels added * [X] not a security concern --------- Co-authored-by: trujillo-adam <[email protected]> * Add docs for jwt cluster configuration (#18004) ### Description <!-- Please describe why you're making this change, in plain English. --> - Add jwt-provider docs for jwks cluster configuration. The configuration was added here: https://github.com/hashicorp/consul/pull/17978 * Docs: fix unmatched bracket for health checks page (#18134) * NET-4657/add resource service client (#18053) ### Description <!-- Please describe why you're making this change, in plain English. --> Dan had already started on this [task](https://github.com/hashicorp/consul/pull/17849) which is needed to start building the HTTP APIs. This just needed some cleanup to get it ready for review. Overview: - Rename `internalResourceServiceClient` to `insecureResourceServiceClient` for name consistency - Configure a `secureResourceServiceClient` with auth enabled ### PR Checklist * [ ] ~updated test coverage~ * [ ] ~external facing docs updated~ * [x] appropriate backport labels added * [ ] ~not a security concern~ * Fix bug with Vault CA provider (#18112) Updating RootPKIPath but not IntermediatePKIPath would not update leaf signing certs with the new root. Unsure if this happens in practice but manual testing showed it is a bug that would break mesh and agent connections once the old root is pruned. * [NET-4897] net/http host header is now verified and request.host that contains socked now error (#18129) ### Description This is related to https://github.com/hashicorp/consul/pull/18124 where we pinned the go versions in CI to 1.20.5 and 1.19.10. go 1.20.6 and 1.19.11 now validate request host headers for validity, including the hostname cannot be prefixed with slashes. For local communications (npipe://, unix://), the hostname is not used, but we need valid and meaningful hostname. Prior versions go Go would clean the host header, and strip slashes in the process, but go1.20.6 and go1.19.11 no longer do, and reject the host header. Around the community we are seeing that others are intercepting the req.host and if it starts with a slash or ends with .sock, they changing the host to localhost or another dummy value. [client: define a "dummy" hostname to use for local connections by thaJeztah · Pull Request #45942 · moby/moby](https://github.com/moby/moby/pull/45942) ### Testing & Reproduction steps Check CI tests. ### Links * [ ] updated test coverage * [ ] external facing docs updated * [ ] appropriate backport labels added * [ ] not a security concern * add a conditional around setting LANFilter.AllSegments to make sure it is valid (#18139) ### Description This is to correct a code problem because this assumes all segments, but when you get to Enterprise, you can be in partition that is not the default partition, in which case specifying all segments does not validate and fails. This is to correct the setting of this filter with `AllSegments` to `true` to only occur when in the the `default` partition. ### Testing & Reproduction steps <!-- * In the case of bugs, describe how to replicate * If any manual tests were done, document the steps and the conditions to replicate * Call out any important/ relevant unit tests, e2e tests or integration tests you have added or are adding --> ### Links <!-- Include any links here that might be helpful for people reviewing your PR (Tickets, GH issues, API docs, external benchmarks, tools docs, etc). If there are none, feel free to delete this section. Please be mindful not to leak any customer or confidential information. HashiCorp employees may want to use our internal URL shortener to obfuscate links. --> ### PR Checklist * [ ] updated test coverage * [ ] external facing docs updated * [ ] appropriate backport labels added * [ ] not a security concern * chore: bump upgrade integrations tests to 1.15, 116 [NET-4743] (#18130) * re org resource type registry (#18133) * fix: update delegateMock used in ENT (#18149) ### Description <!-- Please describe why you're making this change, in plain English. --> The mock is used in `http_ent_test` file which caused lint failures. For OSS->ENT parity adding the same change here. ### Links <!-- Include any links here that might be helpful for people reviewing your PR (Tickets, GH issues, API docs, external benchmarks, tools docs, etc). If there are none, feel free to delete this section. Please be mindful not to leak any customer or confidential information. HashiCorp employees may want to use our internal URL shortener to obfuscate links. --> Identified in OSS->ENT [merge PR](https://github.com/hashicorp/consul-enterprise/pull/6328) ### PR Checklist * [ ] ~updated test coverage~ * [ ] ~external facing docs updated~ * [x] appropriate backport labels added * [ ] ~not a security concern~ * Use JWT-auth filter in metadata mode & Delegate validation to RBAC filter (#18062) ### Description <!-- Please describe why you're making this change, in plain English. --> - Currently the jwt-auth filter doesn't take into account the service identity when validating jwt-auth, it only takes into account the path and jwt provider during validation. This causes issues when multiple source intentions restrict access to an endpoint with different JWT providers. - To fix these issues, rather than use the JWT auth filter for validation, we use it in metadata mode and allow it to forward the successful validated JWT token payload to the RBAC filter which will make the decisions. This PR ensures requests with and without JWT tokens successfully go through the jwt-authn filter. The filter however only forwards the data for successful/valid tokens. On the RBAC filter level, we check the payload for claims and token issuer + existing rbac rules. ### Testing & Reproduction steps <!-- * In the case of bugs, describe how to replicate * If any manual tests were done, document the steps and the conditions to replicate * Call out any important/ relevant unit tests, e2e tests or integration tests you have added or are adding --> - This test covers a multi level jwt requirements (requirements at top level and permissions level). It also assumes you have envoy running, you have a redis and a sidecar proxy service registered, and have a way to generate jwks with jwt. I mostly use: https://www.scottbrady91.com/tools/jwt for this. - first write your proxy defaults ``` Kind = "proxy-defaults" name = "global" config { protocol = "http" } ``` - Create two providers ``` Kind = "jwt-provider" Name = "auth0" Issuer = "https://ronald.local" JSONWebKeySet = { Local = { JWKS = "eyJrZXlzIjog....." } } ``` ``` Kind = "jwt-provider" Name = "okta" Issuer = "https://ronald.local" JSONWebKeySet = { Local = { JWKS = "eyJrZXlzIjogW3...." } } ``` - add a service intention ``` Kind = "service-intentions" Name = "redis" JWT = { Providers = [ { Name = "okta" }, ] } Sources = [ { Name = "*" Permissions = [{ Action = "allow" HTTP = { PathPrefix = "/workspace" } JWT = { Providers = [ { Name = "okta" VerifyClaims = [ { Path = ["aud"] Value = "my_client_app" }, { Path = ["sub"] Value = "5be86359073c434bad2da3932222dabe" } ] }, ] } }, { Action = "allow" HTTP = { PathPrefix = "/" } JWT = { Providers = [ { Name = "auth0" }, ] } }] } ] ``` - generate 3 jwt tokens: 1 from auth0 jwks, 1 from okta jwks with different claims than `/workspace` expects and 1 with correct claims - connect to your envoy (change service and address as needed) to view logs and potential errors. You can add: `-- --log-level debug` to see what data is being forwarded ``` consul connect envoy -sidecar-for redis1 -grpc-addr 127.0.0.1:8502 ``` - Make the following requests: ``` curl -s -H "Authorization: Bearer $Auth0_TOKEN" --insecure --cert leaf.cert --key leaf.key --cacert connect-ca.pem https://localhost:20000/workspace -v RBAC filter denied curl -s -H "Authorization: Bearer $Okta_TOKEN_with_wrong_claims" --insecure --cert leaf.cert --key leaf.key --cacert connect-ca.pem https://localhost:20000/workspace -v RBAC filter denied curl -s -H "Authorization: Bearer $Okta_TOKEN_with_correct_claims" --insecure --cert leaf.cert --key leaf.key --cacert connect-ca.pem https://localhost:20000/workspace -v Successful request ``` ### TODO * [x] Update test coverage * [ ] update integration tests (follow-up PR) * [x] appropriate backport labels added * Support Consul Connect Envoy Command on Windows (#17694) ### Description Add support for consul connect envoy command on windows. This PR fixes the comments of PR - https://github.com/hashicorp/consul/pull/15114 ### Testing * Built consul.exe from this branch on windows and hosted here - [AWS S3](https://asheshvidyut-bucket.s3.ap-southeast-2.amazonaws.com/consul.zip) * Updated the [tutorial](https://developer.hashicorp.com/consul/tutorials/developer-mesh/consul-windows-workloads) and changed the `consul_url.default` value to [AWS S3](https://asheshvidyut-bucket.s3.ap-southeast-2.amazonaws.com/consul.zip) * Followed the steps in the tutorial and verified that everything is working as described. ### PR Checklist * [x] updated test coverage * [ ] external facing docs updated * [x] appropriate backport labels added * [x] not a security concern --------- Co-authored-by: Franco Bruno Lavayen <[email protected]> Co-authored-by: Jose Ignacio Lorenzo <[email protected]> Co-authored-by: Jose Ignacio Lorenzo <[email protected]> Co-authored-by: Dhia Ayachi <[email protected]> * Change docs to say 168h instead of 7d for server_rejoin_age_max (#18154) ### Description Addresses https://github.com/hashicorp/consul/pull/17171#issuecomment-1636930705 * [OSS] test: improve xDS listener code coverage (#18138) test: improve xDS listener code coverage * Re-order expected/actual for assertContainerState in consul container tests (#18157) Re-order expected/actual, consul container tests * group and document make file (#17943) * group and document make file * Add `testing/deployer` (neé `consul-topology`) [NET-4610] (#17823) Co-authored-by: R.B. Boyer <[email protected]> Co-authored-by: R.B. Boyer <[email protected]> Co-authored-by: Freddy <[email protected]> * [NET-4792] Add integrations tests for jwt-auth (#18169) * Add FIPS reference to consul enterprise docs (#18028) * Add FIPS reference to consul enterprise docs * Update website/content/docs/enterprise/index.mdx Co-authored-by: David Yu <[email protected]> * remove support for ecs client (fips) --------- Co-authored-by: David Yu <[email protected]> * add peering_commontopo tests [NET-3700] (#17951) Co-authored-by: R.B. Boyer <[email protected]> Co-authored-by: R.B. Boyer <[email protected]> Co-authored-by: Freddy <[email protected]> Co-authored-by: NiniOak <[email protected]> * docs - remove Sentinel from enterprise features list (#18176) * Update index.mdx * Update kv.mdx * Update docs-nav-data.json * delete sentinel.mdx * Update redirects.js --------- Co-authored-by: Tu Nguyen <[email protected]> * [NET-4865] Bump golang.org/x/net to 0.12.0 (#18186) Bump golang.org/x/net to 0.12.0 While not necessary to directly address CVE-2023-29406 (which should be handled by using a patched version of Go when building), an accompanying change to HTTP/2 error handling does impact agent code. See https://go-review.googlesource.com/c/net/+/506995 for the HTTP/2 change. Bump this dependency across our submodules as well for the sake of potential indirect consumers of `x/net/http`. * Call resource mutate hook before validate hook (NET-4907) (#18178) * [NET-4865] security: Update Go version to 1.20.6 (#18190) Update Go version to 1.20.6 This resolves [CVE-2023-29406] (https://nvd.nist.gov/vuln/detail/CVE-2023-29406) for uses of the `net/http` standard library. Note that until the follow-up to #18124 is done, the version of Go used in those impacted tests will need to remain on 1.20.5. * Improve XDS test coverage: JWT auth edition (#18183) * Improve XDS test coverage: JWT auth edition more tests * test: xds coverage for jwt listeners --------- Co-authored-by: DanStough <[email protected]> * update readme.md (#18191) u[date readme.md * Update submodules to latest following 1.16.0 (#18197) Align all our internal use of submodules on the latest versions. * SEC-090: Automated trusted workflow pinning (2023-07-18) (#18174) Result of tsccr-helper -log-level=info -pin-all-workflows . Co-authored-by: hashicorp-tsccr[bot] <hashicorp-tsccr[bot]@users.noreply.github.com> * Fix Backport Assistant PR commenting (#18200) * Fix Backport Assistant failure PR commenting For general comments on a PR, it looks like you have to use the `/issue` endpoint rather than `/pulls`, which requires commit/other review-specific target details. This matches the endpoint used in `backport-reminder.yml`. * Remove Backport Reminder workflow This is noisy (even when adding multiple labels, individual comments per label are generated), and likely no longer needed: we haven't had this work in a long time due to an expired GH token, and we now have better automation for backport PR assignment. * resource: Pass resource to Write ACL hook instead of just resource Id [NET-4908] (#18192) * Explicitly enable WebSocket upgrades (#18150) This PR explicitly enables WebSocket upgrades in Envoy's UpgradeConfig for all proxy types. (API Gateway, Ingress, and Sidecar.) Fixes #8283 * docs: fix the description of client rpc (#18206) * NET-4804: Add dashboard for monitoring consul-k8s (#18208) * [OSS] Improve xDS Code Coverage - Clusters (#18165) test: improve xDS cluster code coverage * NET-4222 take config file consul container (#18218) Net 4222 take config file consul container * Envoy Integration Test Windows (#18007) * [CONSUL-395] Update check_hostport and Usage (#40) * [CONSUL-397] Copy envoy binary from Image (#41) * [CONSUL-382] Support openssl in unique test dockerfile (#43) * [CONSUL-405] Add bats to single container (#44) * [CONSUL-414] Run Prometheus Test Cases and Validate Changes (#46) * [CONSUL-410] Run Jaeger in Single container (#45) * [CONSUL-412] Run test-sds-server in single container (#48) * [CONSUL-408] Clean containers (#47) * [CONSUL-384] Rebase and sync fork (#50) * [CONSUL-415] Create Scenarios Troubleshooting Docs (#49) * [CONSUL-417] Update Docs Single Container (#51) * [CONSUL-428] Add Socat to single container (#54) * [CONSUL-424] Replace pkill in kill_envoy function (#52) * [CONSUL-434] Modify Docker run functions in Helper script (#53) * [CONSUL-435] Replace docker run in set_ttl_check_state & wait_for_agent_service_register functions (#55) * [CONSUL-438] Add netcat (nc) in the Single container Dockerfile (#56) * [CONSUL-429] Replace Docker run with Docker exec (#57) * [CONSUL-436] Curl timeout and run tests (#58) * [CONSUL-443] Create dogstatsd Function (#59) * [CONSUL-431] Update Docs Netcat (#60) * [CONSUL-439] Parse nc Command in function (#61) * [CONSUL-463] Review curl Exec and get_ca_root Func (#63) * [CONSUL-453] Docker hostname in Helper functions (#64) * [CONSUL-461] Test wipe volumes without extra cont (#66) * [CONSUL-454] Check ports in the Server and Agent containers (#65) * [CONSUL-441] Update windows dockerfile with version (#62) * [CONSUL-466] Review case-grpc Failing Test (#67) * [CONSUL-494] Review case-cfg-resolver-svc-failover (#68) * [CONSUL-496] Replace docker_wget & docker_curl (#69) * [CONSUL-499] Cleanup Scripts - Remove nanoserver (#70) * [CONSUL-500] Update Troubleshooting Docs (#72) * [CONSUL-502] Pull & Tag Envoy Windows Image (#73) * [CONSUL-504] Replace docker run in docker_consul (#76) * [CONSUL-505] Change admin_bind * [CONSUL-399] Update envoy to 1.23.1 (#78) * [CONSUL-510] Support case-wanfed-gw on Windows (#79) * [CONSUL-506] Update troubleshooting Documentation (#80) * [CONSUL-512] Review debug_dump_volumes Function (#81) * [CONSUL-514] Add zipkin to Docker Image (#82) * [CONSUL-515] Update Documentation (#83) * [CONSUL-529] Support case-consul-exec (#86) * [CONSUL-530] Update Documentation (#87) * [CONSUL-530] Update default consul version 1.13.3 * [CONSUL-539] Cleanup (#91) * [CONSUL-546] Scripts Clean-up (#92) * [CONSUL-491] Support admin_access_log_path value for Windows (#71) * [CONSUL-519] Implement mkfifo Alternative (#84) * [CONSUL-542] Create OS Specific Files for Envoy Package (#88) * [CONSUL-543] Create exec_supported.go (#89) * [CONSUL-544] Test and Build Changes (#90) * Implement os.DevNull * using mmap instead of disk files * fix import in exec-unix * fix nmap open too many arguemtn * go fmt on file * changelog file * fix go mod * Update .changelog/17694.txt Co-authored-by: Dhia Ayachi <[email protected]> * different mmap library * fix bootstrap json * some fixes * chocolatey version fix and image fix * using different library * fix Map funciton call * fix mmap call * fix tcp dump * fix tcp dump * windows tcp dump * Fix docker run * fix tests * fix go mod * fix version 16.0 * fix version * fix version dev * sleep to debug * fix sleep * fix permission issue * fix permission issue * fix permission issue * fix command * fix command * fix funciton * fix assert config entry status command not found * fix command not found assert_cert_has_cn * fix command not found assert_upstream_missing * fix command not found assert_upstream_missing_once * fix command not found get_upstream_endpoint * fix command not found get_envoy_public_listener_once * fix command not found * fix test cases * windows integration test workflow github * made code similar to unix using npipe * fix go.mod * fix dialing of npipe * dont wait * check size of written json * fix undefined n * running * fix dep * fix syntax error * fix workflow file * windows runner * fix runner * fix from json * fix runs on * merge connect envoy * fix cin path * build * fix file name * fix file name * fix dev build * remove unwanted code * fix upload * fix bin name * fix path * checkout current branch * fix path * fix tests * fix shell bash for windows sh files * fix permission of run-test.sh * removed docker dev * added shell bash for tests * fix tag * fix win=true * fix cd * added dev * fix variable undefined * removed failing tests * fix tcp dump image * fix curl * fix curl * tcp dump path * fix tcpdump path * fix curl * fix curl install * stop removing intermediate containers * fix tcpdump docker image * revert -rm * --rm=false * makeing docker image before * fix tcpdump * removed case consul exec * removed terminating gateway simple * comment case wasm * removed data dog * comment out upload coverage * uncomment case-consul-exec * comment case consul exec * if always * logs * using consul 1.17.0 * fix quotes * revert quotes * redirect to dev null * Revert version * revert consul connect * fix version * removed envoy connect * not using function * change log * docker logs * fix logs * restructure bad authz * rmeoved dev null * output * fix file descriptor * fix cacert * fix cacert * fix ca cert * cacert does not work in windows curl * fix func * removed docker logs * added sleep * fix tls * commented case-consul-exec * removed echo * retry docker consul * fix upload bin * uncomment consul exec * copying consul.exe to docker image * copy fix * fix paths * fix path * github workspace path * latest version * Revert "latest version" This reverts commit 5a7d7b82d9e7553bcb01b02557ec8969f9deba1d. * commented consul exec * added ssl revoke best effort * revert best effort * removed unused files * rename var name and change dir * windows runner * permission * needs setup fix * swtich to github runner * fix file path * fix path * fix path * fix path * fix path * fix path * fix build paths * fix tag * nightly runs * added matrix in github workflow, renamed files * fix job * fix matrix * removed brackes * from json * without using job matrix * fix quotes * revert job matrix * fix workflow * fix comment * added comment * nightly runs * removed datadog ci as it is already measured in linux one * running test * Revert "running test" This reverts commit 7013d15a23732179d18ec5d17336e16b26fab5d4. * pr comment fixes * running test now * running subset of test * running subset of test * job matrix * shell bash * removed bash shell * linux machine for job matrix * fix output * added cat to debug * using ubuntu latest * fix job matrix * fix win true * fix go test * revert job matrix --------- Co-authored-by: Jose Ignacio Lorenzo <[email protected]> Co-authored-by: Franco Bruno Lavayen <[email protected]> Co-authored-by: Ivan K Berlot <[email protected]> Co-authored-by: Ezequiel Fernández Ponce <[email protected]> Co-authored-by: joselo85 <[email protected]> Co-authored-by: Ezequiel Fernández Ponce <[email protected]> Co-authored-by: Dhia Ayachi <[email protected]> * fix typos and update ecs compat table (#18215) * fix typos and update ecs compat table * real info for the ecs compat matrix table * Update website/content/docs/ecs/compatibility.mdx Co-authored-by: Chris Thain <[email protected]> --------- Co-authored-by: Chris Thain <[email protected]> * [OSS] proxystate: add proxystate protos (#18216) * proxystate: add proxystate protos to pbmesh and resolve imports and conflicts between message names * ci: don't verify s390x (#18224) * [CC-5718] Remove HCP token requirement during bootstrap (#18140) * [CC-5718] Remove HCP token requirement during bootstrap * Re-add error for loading HCP management token * Remove old comment * Add changelog entry * Remove extra validation line * Apply suggestions from code review Co-authored-by: lornasong <[email protected]> --------- Co-authored-by: lornasong <[email protected]> * [NET-4122] Doc guidance for federation with externalServers (#18207) Doc guidance for federation with externalServers Add guidance for proper configuration when joining to a secondary cluster using WAN fed with external servers also enabled. Also clarify federation requirements and fix formatting for an unrelated value. Update both the Helm chart reference (synced from `consul-k8s`, see hashicorp/consul-k8s#2583) and the docs on using `externalServers`. * [OSS] Improve xDS Code Coverage - Endpoints and Misc (#18222) test: improve xDS endpoints code coverage * Clarify license reporting timing and GDPR compliance (#18237) Add Alicia's edits to clarify log timing and other details * Fix Github Workflow File (#18241) * [CONSUL-382] Support openssl in unique test dockerfile (#43) * [CONSUL-405] Add bats to single container (#44) * [CONSUL-414] Run Prometheus Test Cases and Validate Changes (#46) * [CONSUL-410] Run Jaeger in Single container (#45) * [CONSUL-412] Run test-sds-server in single container (#48) * [CONSUL-408] Clean containers (#47) * [CONSUL-384] Rebase and sync fork (#50) * [CONSUL-415] Create Scenarios Troubleshooting Docs (#49) * [CONSUL-417] Update Docs Single Container (#51) * [CONSUL-428] Add Socat to single container (#54) * [CONSUL-424] Replace pkill in kill_envoy function (#52) * [CONSUL-434] Modify Docker run functions in Helper script (#53) * [CONSUL-435] Replace docker run in set_ttl_check_state & wait_for_agent_service_register functions (#55) * [CONSUL-438] Add netcat (nc) in the Single container Dockerfile (#56) * [CONSUL-429] Replace Docker run with Docker exec (#57) * [CONSUL-436] Curl timeout and run tests (#58) * [CONSUL-443] Create dogstatsd Function (#59) * [CONSUL-431] Update Docs Netcat (#60) * [CONSUL-439] Parse nc Command in function (#61) * [CONSUL-463] Review curl Exec and get_ca_root Func (#63) * [CONSUL-453] Docker hostname in Helper functions (#64) * [CONSUL-461] Test wipe volumes without extra cont (#66) * [CONSUL-454] Check ports in the Server and Agent containers (#65) * [CONSUL-441] Update windows dockerfile with version (#62) * [CONSUL-466] Review case-grpc Failing Test (#67) * [CONSUL-494] Review case-cfg-resolver-svc-failover (#68) * [CONSUL-496] Replace docker_wget & docker_curl (#69) * [CONSUL-499] Cleanup Scripts - Remove nanoserver (#70) * [CONSUL-500] Update Troubleshooting Docs (#72) * [CONSUL-502] Pull & Tag Envoy Windows Image (#73) * [CONSUL-504] Replace docker run in docker_consul (#76) * [CONSUL-505] Change admin_bind * [CONSUL-399] Update envoy to 1.23.1 (#78) * [CONSUL-510] Support case-wanfed-gw on Windows (#79) * [CONSUL-506] Update troubleshooting Documentation (#80) * [CONSUL-512] Review debug_dump_volumes Function (#81) * [CONSUL-514] Add zipkin to Docker Image (#82) * [CONSUL-515] Update Documentation (#83) * [CONSUL-529] Support case-consul-exec (#86) * [CONSUL-530] Update Documentation (#87) * [CONSUL-530] Update default consul version 1.13.3 * [CONSUL-539] Cleanup (#91) * [CONSUL-546] Scripts Clean-up (#92) * [CONSUL-491] Support admin_access_log_path value for Windows (#71) * [CONSUL-519] Implement mkfifo Alternative (#84) * [CONSUL-542] Create OS Specific Files for Envoy Package (#88) * [CONSUL-543] Create exec_supported.go (#89) * [CONSUL-544] Test and Build Changes (#90) * Implement os.DevNull * using mmap instead of disk files * fix import in exec-unix * fix nmap open too many arguemtn * go fmt on file * changelog file * fix go mod * Update .changelog/17694.txt Co-authored-by: Dhia Ayachi <[email protected]> * different mmap library * fix bootstrap json * some fixes * chocolatey version fix and image fix * using different library * fix Map funciton call * fix mmap call * fix tcp dump * fix tcp dump * windows tcp dump * Fix docker run * fix tests * fix go mod * fix version 16.0 * fix version * fix version dev * sleep to debug * fix sleep * fix permission issue * fix permission issue * fix permission issue * fix command * fix command * fix funciton * fix assert config entry status command not found * fix command not found assert_cert_has_cn * fix command not found assert_upstream_missing * fix command not found assert_upstream_missing_once * fix command not found get_upstream_endpoint * fix command not found get_envoy_public_listener_once * fix command not found * fix test cases * windows integration test workflow github * made code similar to unix using npipe * fix go.mod * fix dialing of npipe * dont wait * check size of written json * fix undefined n * running * fix dep * fix syntax error * fix workflow file * windows runner * fix runner * fix from json * fix runs on * merge connect envoy * fix cin path * build * fix file name * fix file name * fix dev build * remove unwanted code * fix upload * fix bin name * fix path * checkout current branch * fix path * fix tests * fix shell bash for windows sh files * fix permission of run-test.sh * removed docker dev * added shell bash for tests * fix tag * fix win=true * fix cd * added dev * fix variable undefined * removed failing tests * fix tcp dump image * fix curl * fix curl * tcp dump path * fix tcpdump path * fix curl * fix curl install * stop removing intermediate containers * fix tcpdump docker image * revert -rm * --rm=false * makeing docker image before * fix tcpdump * removed case consul exec * removed terminating gateway simple * comment case wasm * removed data dog * comment out upload coverage * uncomment case-consul-exec * comment case consul exec * if always * logs * using consul 1.17.0 * fix quotes * revert quotes * redirect to dev null * Revert version * revert consul connect * fix version * removed envoy connect * not using function * change log * docker logs * fix logs * restructure bad authz * rmeoved dev null * output * fix file descriptor * fix cacert * fix cacert * fix ca cert * cacert does not work in windows curl * fix func * removed docker logs * added sleep * fix tls * commented case-consul-exec * removed echo * retry docker consul * fix upload bin * uncomment consul exec * copying consul.exe to docker image * copy fix * fix paths * fix path * github workspace path * latest version * Revert "latest version" This reverts commit 5a7d7b82d9e7553bcb01b02557ec8969f9deba1d. * commented consul exec * added ssl revoke best effort * revert best effort * removed unused files * rename var name and change dir * windows runner * permission * needs setup fix * swtich to github runner * fix file path * fix path * fix path * fix path * fix path * fix path * fix build paths * fix tag * nightly runs * added matrix in github workflow, renamed files * fix job * fix matrix * removed brackes * from json * without using job matrix * fix quotes * revert job matrix * fix workflow * fix comment * added comment * nightly runs * removed datadog ci as it is already measured in linux one * running test * Revert "running test" This reverts commit 7013d15a23732179d18ec5d17336e16b26fab5d4. * pr comment fixes * running test now * running subset of test * running subset of test * job matrix * shell bash * removed bash shell * linux machine for job matrix * fix output * added cat to debug * using ubuntu latest * fix job matrix * fix win true * fix go test * revert job matrix * Fix tests --------- Co-authored-by: Ivan K Berlot <[email protected]> Co-authored-by: Jose Ignacio Lorenzo <[email protected]> Co-authored-by: Franco Bruno Lavayen <[email protected]> Co-authored-by: Ezequiel Fernández Ponce <[email protected]> Co-authored-by: joselo85 <[email protected]> Co-authored-by: Ezequiel Fernández Ponce <[email protected]> Co-authored-by: Dhia Ayachi <[email protected]> * NET-4996 - filter go-tests and test-integration workflows from running on docs only and ui only changes (#18236) * Align build arch matrix with enterprise (#18235) Ensure that OSS remains in sync w/ Enterprise by aligning the format of arch matrix args for various build jobs. * Revert "NET-4996 - filter go-tests and test-integration workflows from running on docs only and ui only changes" (#18248) Revert "NET-4996 - filter go-tests and test-integration workflows from running on docs only and ui only changes (#18236)" This reverts commit a11dba710e6ce6f172c0fa6c9b61567cc1efffc8. * resource: Add scope to resource type registration [NET-4976] (#18214) Enables querying a resource type's registration to determine if a resource is cluster, partition, or partition and namespace scoped. * Fix some inconsistencies in jwt docs (#18234) * NET-1825: More new ACL token creation docs (#18063) Co-authored-by: trujillo-adam <[email protected]> * grafana: add the panel resource usage of connect injector (#18247) * [NET-3700] Backfill changelog entry for c2bbe67 and 7402d06 (#18259) Backfill changelog entry for c2bbe67 and 7402d06 Add a changelog entry for the follow-up PR since it was specific to the fix and references the original change. * NET-4897 - update comment to include the current issue url from the go team. (#18263) * fix typos, style, and improper links (#18269) * member cli: add -filter expression to flags (#18223) * member cli: add -filter expression to flags * changelog * update doc * Add test cases * use quote * go-tests: disable s390x (#18273) * docs: Update ext-authz documentation for kubernetes (#18281) Update ext-authz documentation for kubernetes * docs: Consul on Kubernetes specific upgrade info (#18230) * Compatibility page - dataplanes mention * Upgrading higher-level dataplane mention * `exec=` string callout * More visible for upgrade page * Apply suggestions from code review Co-authored-by: Tu Nguyen <[email protected]> --------- Co-authored-by: Tu Nguyen <[email protected]> Co-authored-by: John Murret <[email protected]> * Fix typo in Envoy extensions doc (#18284) * docs: K8s secondary DC requirements (#18280) * Requested edit * Apply suggestions from code review Co-authored-by: trujillo-adam <[email protected]> --------- Co-authored-by: trujillo-adam <[email protected]> * api-gateway: subscribe to bound-api-gateway only after receiving api-gateway (#18291) * api-gateway: subscribe to bound-api-gateway only after receiving api-gateway This fixes a race condition due to our dependency on having the listener(s) from the api-gateway config entry in order to fully and properly process the resources on the bound-api-gateway config entry. * Apply suggestions from code review * Add changelog entry * Update K8s changelog to address cloud auto-join change in 1.0.0 (#18293) * Update list of Envoy versions (#18300) Update supported envoy versions to 1.24.10, 1.25.9, 1.26.4, 1.27.0. * [NET-4904] Update list of Envoy versions in docs (#18306) Update list of Envoy versions in docs Update supported Envoy versions across Consul release versions. * Update actions for TSCCR (#18317) Update action versions before deadline * Fix topology view when displaying mixed connect-native/normal services. (#13023) * Fix topoloy intention with mixed connect-native/normal services. If a service is registered twice, once with connect-native and once without, the topology views would prune the existing intentions. This change brings the code more in line with the transparent proxy behavior. * Dedupe nodes in the ServiceTopology ui endpoint (like done with tags). * Consider a service connect-native as soon as one instance is. * Stop JWT provider from being written in non default namespace (#18325) * docs: Fix some comments (#17118) Signed-off-by: cui fliter <[email protected]> * [NET-4703] Prevent partial application of Envoy extensions (#18068) Prevent partial application of Envoy extensions Ensure that non-required extensions do not change xDS resources before exiting on failure by cloning proto messages prior to applying each extension. To support this change, also move `CanApply` checks up a layer and make them prior to attempting extension application, s.t. we avoid unnecessary copies where extensions can't be applied. Last, ensure that we do not allow panics from `CanApply` or `Extend` checks to escape the attempted extension application. * docs: Simplify example jq commands by removing pipes (#18327) Simplify jq command examples by removing pipes to other commands. * fix typo in create a mesh token docs (#18337) * [CC-5719] Add support for builtin global-read-only policy (#18319) * [CC-5719] Add support for builtin global-read-only policy * Add changelog * Add read-only to docs * Fix some minor issues. * Change from ReplaceAll to Sprintf * Change IsValidPolicy name to return an error instead of bool * Fix PolicyList test * Fix other tests * Apply suggestions from code review Co-authored-by: Paul Glass <[email protected]> * Fix state store test for policy list. * Fix naming issues * Update acl/validation.go Co-authored-by: Chris Thain <[email protected]> * Update agent/consul/acl_endpoint.go --------- Co-authored-by: Paul Glass <[email protected]> Co-authored-by: Chris Thain <[email protected]> * [NET-5121] proxystate: move protos to subdirectory to avoid conflicts (#18335) * also makes a few protos updates * resource: adding various helpers for working with resources (#18342) This is a bit of a grab bag of helpers that I found useful for working with them when authoring substantial Controllers. Subsequent PRs will make use of them. * Wasm integration tests for local and remote wasm files (#17756) * wasm integration tests for local and remote wasm files refactoring and cleanup for wasm testing remove wasm debug logging PR feedback, wasm build lock correct path pattern for wasm build files Add new helper function to minimize changes to existing test code Remove extra param mod tidy add custom service setup to test lib add wait until static server sidecar can reach nginx sidecar Doc comments PR feedback Update workflows to compile wasm for integration tests Fix docker build path Fix package name for linter Update makefile, fix redeclared function Update expected wasm filename Debug test ls in workflow remove pwd in favor of relative path more debugging Build wasm in compatability tests as well Build wasm directly in ci rather than in container Debug tinygo and llvm version Change wasm file extension Remove tinygo debugging Remove extra comments * Add compiled wasm and build instructions * [HCP Telemetry] Periodic Refresh for Dynamic Telemetry Configuration (#18168) * OTElExporter now uses an EndpointProvider to discover the endpoint * OTELSink uses a ConfigProvider to obtain filters and labels configuration * improve tests for otel_sink * Regex logic is moved into client for a method on the TelemetryConfig object * Create a telemetry_config_provider and update deps to use it * Fix conversion * fix import newline * Add logger to hcp client and move telemetry_config out of the client.go file * Add a telemetry_config.go to refactor client.go * Update deps * update hcp deps test * Modify telemetry_config_providers * Check for nil filters * PR review updates * Fix comments and move around pieces * Fix comments * Remove context from client struct * Moved ctx out of sink struct and fixed filters, added a test * Remove named imports, use errors.New if not fformatting * Remove HCP dependencies in telemetry package * Add success metric and move lock only to grab the t.cfgHahs * Update hash * fix nits * Create an equals method and add tests * Improve telemetry_config_provider.go tests * Add race test * Add missing godoc * Remove mock for MetricsClient * Avoid goroutine test panics * trying to kick CI lint issues by upgrading mod * imprve test code and add hasher for testing * Use structure logging for filters, fix error constants, and default to allow all regex * removed hashin and modify logic to simplify * Improve race test and fix PR feedback by removing hash equals and avoid testing the timer.Ticker logic, and instead unit test * Ran make go-mod-tidy * Use errtypes in the test * Add changelog * add safety check for exporter endpoint * remove require.Contains by using error types, fix structure logging, and fix success metric typo in exporter * Fixed race test to have changing config values * Send success metric before modifying config * Avoid the defer and move the success metric under * NET-4240 - Snapshots are failing on Windows (#18302) * fix go mod * fix go sum * added change log * ran make go mod tidy * Register ProxyStateTemplate Resource (#18316) Also, change the ProxyState.id to identity. This is because we already have the id of this proxy from the resource, and this id should be name-aligned with the workload it represents. It should also have the owner ref set to the workload ID if we need that. And so the id field seems unnecessary. We do, however, need a reference to workload identity so that we can authorize the proxy when it initially connects to the xDS server. * [NET-5146] security: Update Go version to 1.20.7 and `x/net` to 0.13.0 (#18358) * Update Go version to 1.20.7 This resolves [CVE-2023-29409] (https://nvd.nist.gov/vuln/detail/CVE-2023-29409)(`crypto/tls`). * Bump golang.org/x/net to 0.13.0 Addresses [CVE-2023-3978](https://nvd.nist.gov/vuln/detail/CVE-2023-3978) for security scans (non-impacting). * mesh: adding the protobuf types and resources backing mesh config v2 (#18351) * [OSS] test: xds coverage for routes (#18369) test: xds coverage for routes * Fix policy lookup to allow for slashes (#18347) * Fix policy lookup to allow for slashes * Fix suggestions * Fix other test * Revert some lines * add some initial CODEOWNERS (#18346) * Add redirects for mesh-gateway docs (#18377) * expose grpc as http endpoint (#18221) expose resource grpc endpoints as http endpoints * bimapper: fix a bug and add some more test coverage (#18387) * [docs] Fix ServiceDefaults example in distributed tracing (#17212) Fix ServiceDefaults example in distributed tracing. * Fix #17730 - Dev mode has new line (#18367) * adding new line only in case of pretty in url not in dev mode * change log added * chore: make go-mod-tidy (#18388) * resource: Make resource read tenancy aware (#18397) * Not using chmod - fixed integration test for Enterprise (#18401) * Not using chmod * checking icacls * Removed push * replaced ordered list of rate limit ops with flow diagram (#18398) * replaced ordered list of rate limit ops with flow diagram * made diagram font bigger * updated colors for dark mode img * fixed typo * recentered dark mode image * Apply suggestions from code review Co-authored-by: Jeff Boruszak <[email protected]> --------- Co-authored-by: Jeff Boruszak <[email protected]> * Upgrade test: remove outdated test and disable log due to verbosity (#18403) * remove outdated test * disable log since we have too many parallel tests * bump testcontainers-go from 0.22.0 and remove pinned go version in in… (#18395) * bump testcontainers-go from 0.22.0 and remove pinned go version in integ test * go mod tidy * Replace deprecated target.Authority with target.URL.Host * Add ServiceEndpoints Mutation hook tests (#18404) * Add ServiceEndpoints Mutation hook tests * Move endpoint owner validation into the validation hook Also there were some minor changes to error validation to account for go-cmp not liking to peer through an errors.errorstring type that get created by errors.New * catalog: add FailoverPolicy mutation and validation hooks (#18390) Add most of the validation and mutation hooks for the FailoverPolicy resource. * NET-5115 Add retry + timeout filters for api-gateway (#18324) * squash, implement retry/timeout in consul core * update tests * catalog: adding a controller to reconcile FailoverPolicy resources (#18399) Add most of the semantic cross-resource validation for FailoverPolicy resources using a new controller. * fix the error in ent repo (#18421) fix the error in ent repo * build: updates for 1.16.1 release (#18415) * build: updates for 1.16.1 release * build: fix missing replace directive for envoyextensions * build: go mod tidy testing/deployer * update ECS links (#18419) * docs: fix incorrect proxy-defaults config in Lua Envoy extension (#18418) fix incorrect proxy-defaults config * resource: Make resource write tenancy aware (#18423) * [NET-5217] [OSS] Derive sidecar proxy locality from parent service (#18437) * Add logging to locality policy application In OSS, this is currently a no-op. * Inherit locality when registering sidecars When sidecar locality is not explicitly configured, inherit locality from the proxied service. * NET-4984: Update APIGW Config Entries for JWT Auth (#18366) * Added oss config entries for Policy and JWT on APIGW * Updated structs for config entry * Updated comments, ran deep-copy * Move JWT configuration into OSS file * Add in the config entry OSS file for jwts * Added changelog * fixing proto spacing * Moved to using manually written deep copy method * Use pointers for override/default fields in apigw config entries * Run gen scripts for changed types * k8s compat - Openshift versions (#18307) * Update compatibility.mdx Co-authored-by: trujillo-adam <[email protected]> * APIGW: Update HTTPRouteConfigEntry for JWT Auth (#18422) * Updated httproute config entry for JWT Filters * Added manual deepcopy method for httproute jwt filter * Fix test * Update JWTFilter to be in oss file * Add changelog * Add build tags for deepcopy oss file * [COMPLIANCE] License changes (#18443) * Adding explicit MPL license for sub-package This directory and its subdirectories (packages) contain files licensed with the MPLv2 `LICENSE` file in this directory and are intentionally licensed separately from the BSL `LICENSE` file at the root of this repository. * Adding explicit MPL license for sub-package This directory and its subdirectories (packages) contain files licensed with the MPLv2 `LICENSE` file in this directory and are intentionally licensed separately from the BSL `LICENSE` file at the root of this repository. * Updating the license from MPL to Business Source License Going forward, this project will be licensed under the Business Source License v1.1. Please see our blog post for more details at <Blog URL>, FAQ at www.hashicorp.com/licensing-faq, and details of the license at www.hashicorp.com/bsl. * add missing license headers * Update copyright file headers to BUSL-1.1 * Update copyright file headers to BUSL-1.1 * Update copyright file headers to BUSL-1.1 * Update copyright file headers to BUSL-1.1 * Update copyright file headers to BUSL-1.1 * Update copyright file headers to BUSL-1.1 * Update copyright file headers to BUSL-1.1 * Update copyright file headers to BUSL-1.1 * Update copyright file headers to BUSL-1.1 * Update copyright file headers to BUSL-1.1 * Update copyright file headers to BUSL-1.1 * Update copyright file headers to BUSL-1.1 * Update copyright file headers to BUSL-1.1 * Update copyright file headers to BUSL-1.1 * Update copyright file headers to BUSL-1.1 --------- Co-authored-by: hashicorp-copywrite[bot] <11042…
absolutelightning
added a commit
that referenced
this pull request
Sep 4, 2023
… nomad - fix bug into release/1.14.x (#18640) * api-gateway: add operation cannot be fulfilled error to common errors (#17874) * add error message * Update website/content/docs/api-gateway/usage/errors.mdx Co-authored-by: Nathan Coleman <[email protected]> * fix formating issues --------- Co-authored-by: Nathan Coleman <[email protected]> * api-gateway: add step to upgrade instructions for creating intentions (#17875) * Changelog - add 1.13.9, 1.14.8, and 1.15.4 (#17889) * docs: update config enable_debug (#17866) * update doc for config enable_debug * Update website/content/docs/agent/config/config-files.mdx Co-authored-by: trujillo-adam <[email protected]> --------- Co-authored-by: trujillo-adam <[email protected]> * Update wording on WAN fed and intermediate_pki_path (#17850) * Allow service identity tokens the ability to read jwt-providers (#17893) * Allow service identity tokens the ability to read jwt-providers * more tests * service_prefix tests * Update docs (#17476) Co-authored-by: trujillo-adam <[email protected]> * Add emit_tags_as_labels to envoy bootstrap config when using Consul Telemetry Collector (#17888) * Fix command from kg to kubectl get (#17903) * Create and update release notes for 1.16 and 1.2 (#17895) * update release notes for 1.16 and 1.2 * update latest consul core release * Propose new changes to APIgw upgrade instructions (#17693) * Propose new changes to APIgw upgrade instructions * fix build error * update callouts to render correctly * Add hideClipboard to log messages * Added clarification around consul k8s and crds * Add workflow to verify linux release packages (#17904) * adding docker files to verify linux packages. * add verifr-release-linux.yml * updating name * pass inputs directly into jobs * add other linux package platforms * remove on push * fix TARGETARCH on debian and ubuntu so it can check arm64 and amd64 * fixing amazon to use the continue line * add ubuntu i386 * fix comment lines * working * remove commented out workflow jobs * Apply suggestions from code review Co-authored-by: Nathan Coleman <[email protected]> * update fedora and ubuntu to use latest tag --------- Co-authored-by: Nathan Coleman <[email protected]> * Reference hashicorp/consul instead of consul for Docker image (#17914) * Reference hashicorp/consul instead of consul for Docker image * Update Make targets that pull consul directly * Update Consul K8s Upgrade Doc Updates (#17921) Updating upgrade procedures to encompass expected errors during upgrade process from v1.13.x to v1.14.x. * Update sameness-group.mdx (#17915) * Update create-sameness-groups.mdx (#17927) * deps: coredns v1.10.1 (#17912) * Ensure RSA keys are at least 2048 bits in length (#17911) * Ensure RSA keys are at least 2048 bits in length * Add changelog * update key length check for FIPS compliance * Fix no new variables error and failing to return when error exists from validating * clean up code for better readability * actually return value * tlsutil: Fix check TLS configuration (#17481) * tlsutil: Fix check TLS configuration * Rewording docs. * Update website/content/docs/services/configuration/checks-configuration-reference.mdx Co-authored-by: trujillo-adam <[email protected]> * Fix typos and add changelog entry. --------- Co-authored-by: trujillo-adam <[email protected]> * docs: Deprecations for connect-native SDK and specific connect native APIs (#17937) * Update v1_16_x.mdx * Update connect native golang page --------- Co-authored-by: trujillo-adam <[email protected]> * Revert "Add workflow to verify linux release packages (#17904)" (#17942) This reverts commit 3368f14fab500ebe9f6aeab5631dd1d5f5a453e5. * Fixes Secondary ConnectCA update (#17846) This fixes a bug that was identified which resulted in subsequent ConnectCA configuration update not to persist in the cluster. * fixing typo in link to jwt-validations-with-intentions doc (#17955) * Fix streaming backend link (#17958) * Fix streaming backend link * Update health.mdx * Dynamically create jwks clusters for jwt-providers (#17944) * website: remove deprecated agent rpc docs (#17962) * Fix missing BalanceOutboundConnections in v2 catalog. (#17964) * feature - [NET - 4005] - [Supportability] Reloadable Configuration - enable_debug (#17565) * # This is a combination of 9 commits. # This is the 1st commit message: init without tests # This is the commit message #2: change log # This is the commit message #3: fix tests # This is the commit message #4: fix tests # This is the commit message #5: added tests # This is the commit message #6: change log breaking change # This is the commit message #7: removed breaking change # This is the commit message #8: fix test # This is the commit message #9: keeping the test behaviour same * # This is a combination of 12 commits. # This is the 1st commit message: init without tests # This is the commit message #2: change log # This is the commit message #3: fix tests # This is the commit message #4: fix tests # This is the commit message #5: added tests # This is the commit message #6: change log breaking change # This is the commit message #7: removed breaking change # This is the commit message #8: fix test # This is the commit message #9: keeping the test behaviour same # This is the commit message #10: made enable debug atomic bool # This is the commit message #11: fix lint # This is the commit message #12: fix test true enable debug * parent 10f500e895d92cc3691ade7b74a33db755d22039 author absolutelightning <[email protected]> 1687352587 +0530 committer absolutelightning <[email protected]> 1687352592 +0530 init without tests change log fix tests fix tests added tests change log breaking change removed breaking change fix test keeping the test behaviour same made enable debug atomic bool fix lint fix test true enable debug using enable debug in agent as atomic bool test fixes fix tests fix tests added update on correct locaiton fix tests fix reloadable config enable debug fix tests fix init and acl 403 * revert commit * Fix formatting codeblocks on APIgw docs (#17970) * fix formatting codeblocks * remove unnecessary indents * Remove POC code (#17974) * update doc (#17910) * update doc * update link * Remove duplicate and unused newDecodeConfigEntry func (#17979) * docs: samenessGroup YAML examples (#17984) * configuration entry syntax * Example config * Add changelog entry for 1.16.0 (#17987) * Fix typo (#17198) servcies => services * Expose JWKS cluster config through JWTProviderConfigEntry (#17978) * Expose JWKS cluster config through JWTProviderConfigEntry * fix typos, rename trustedCa to trustedCA * Integration test for ext-authz Envoy extension (#17980) * Fix incorrect protocol for transparent proxy upstreams. (#17894) This PR fixes a bug that was introduced in: https://github.com/hashicorp/consul/pull/16021 A user setting a protocol in proxy-defaults would cause tproxy implicit upstreams to not honor the upstream service's protocol set in its `ServiceDefaults.Protocol` field, and would instead always use the proxy-defaults value. Due to the fact that upstreams configured with "tcp" can successfully contact upstream "http" services, this issue was not recognized until recently (a proxy-defaults with "tcp" and a listening service with "http" would make successful requests, but not the opposite). As a temporary work-around, users experiencing this issue can explicitly set the protocol on the `ServiceDefaults.UpstreamConfig.Overrides`, which should take precedence. The fix in this PR removes the proxy-defaults protocol from the wildcard upstream that tproxy uses to configure implicit upstreams. When the protocol was included, it would always overwrite the value during discovery chain compilation, which was not correct. The discovery chain compiler also consumes proxy defaults to determine the protocol, so simply excluding it from the wildcard upstream config map resolves the issue. * feat: include nodes count in operator usage endpoint and cli command (#17939) * feat: update operator usage api endpoint to include nodes count * feat: update operator usange cli command to includes nodes count * [OSS] Improve Gateway Test Coverage of Catalog Health (#18011) * fix(cli): remove failing check from 'connect envoy' registration for api gateway * test(integration): add tests to check catalog statsus of gateways on startup * remove extra sleep comment * Update test/integration/consul-container/libs/assert/service.go * changelog * Fixes Traffic rate limitting docs (#17997) * Fix removed service-to-service peering links (#17221) * docs: fix removed service-to-service peering links * docs: extend peering-via-mesh-gateways intro (thanks @trujillo-adam) --------- Co-authored-by: trujillo-adam <[email protected]> * docs: Sameness "beta" warning (#18017) * Warning updates * .x * updated typo in tab heading (#18022) * updated typo in tab heading * updated tab group typo, too * Document that DNS lookups can target cluster peers (#17990) Static DNS lookups, in addition to explicitly targeting a datacenter, can target a cluster peer. This was added in 95dc0c7b301b70a6b955a8b7c9737c9b86f03df6 but didn't make the documentation. The driving function for the change is `parseLocality` here: https://github.com/hashicorp/consul/blob/0b1299c28d8127129d61310ee4280055298438e0/agent/dns_oss.go#L25 The biggest change in this is to adjust the standard lookup syntax to tie `.<datacenter>` to `.dc` as required-together, and to append in the similar `.<cluster-peer>.peer` optional argument, both to A record and SRV record lookups. Co-authored-by: David Yu <[email protected]> * Add first integration test for jwt auth with intention (#18005) * fix stand-in text for name field (#18030) * removed sameness conf entry from failover nav (#18033) * docs - add service sync annotations and k8s service weight annotation (#18032) * Docs for https://github.com/hashicorp/consul-k8s/pull/2293 * remove versions for enterprise features since they are old --------- Co-authored-by: Tu Nguyen <[email protected]> * docs - add jobs use case for service mesh k8s (#18037) * docs - add jobs use case for service mesh k8s * add code blocks * address feedback (#18045) * Add verify server hostname to tls default (#17155) * [OSS] Fix initial_fetch_timeout to wait for all xDS resources (#18024) * fix(connect): set initial_fetch_time to wait indefinitely * changelog * PR feedback 1 * ui: fix typos for peer service imports (#17999) * test: fix FIPS inline cert test message (#18076) * Fix a couple typos in Agent Telemetry Metrics docs (#18080) * Fix metrics docs * Add changelog Signed-off-by: josh <[email protected]> --------- Signed-off-by: josh <[email protected]> * docs updates - cluster peering and virtual services (#18069) * Update route-to-virtual-services.mdx * Update establish-peering.mdx * Update service-mesh-compare.mdx (#17279) grammar change * Update helm docs on main (#18085) * ci: use gotestsum v1.10.1 [NET-4042] (#18088) * Docs: Update proxy lifecycle annotations and consul-dataplane flags (#18075) * Update proxy lifecycle annotations and consul-dataplane flags * Pass configured role name to Vault for AWS auth in Connect CA (#17885) * Docs for dataplane upgrade on k8s (#18051) * Docs for dataplane upgrade on k8s --------- Co-authored-by: David Yu <[email protected]> Co-authored-by: trujillo-adam <[email protected]> * docs - update upgrade index page to not recommend consul leave. (#18100) * Displays Consul version of each nodes in UI nodes section (#17754) * update UINodes and UINodeInfo response with consul-version info added as NodeMeta, fetched from serf members * update test cases TestUINodes, TestUINodeInfo * added nil check for map * add consul-version in local agent node metadata * get consul version from serf member and add this as node meta in catalog register request * updated ui mock response to include consul versions as node meta * updated ui trans and added version as query param to node list route * updates in ui templates to display consul version with filter and sorts * updates in ui - model class, serializers,comparators,predicates for consul version feature * added change log for Consul Version Feature * updated to get version from consul service, if for some reason not available from serf * updated changelog text * updated dependent testcases * multiselection version filter * Update agent/consul/state/catalog.go comments updated Co-authored-by: Jared Kirschner <[email protected]> --------- Co-authored-by: Jared Kirschner <[email protected]> * api gw 1.16 updates (#18081) * api gw 1.16 updates * Apply suggestions from code review Co-authored-by: Jeff Boruszak <[email protected]> * update CodeBlockConfig filename * Apply suggestions from code review Co-authored-by: trujillo-adam <[email protected]> Co-authored-by: Jeff Boruszak <[email protected]> * remove non-standard intentions page * Update website/content/docs/api-gateway/configuration/index.mdx Co-authored-by: trujillo-adam <[email protected]> --------- Co-authored-by: Jeff Boruszak <[email protected]> Co-authored-by: trujillo-adam <[email protected]> * [NET-4103] ci: build s390x (#18067) * ci: build s390x * ci: test s390x * ci: dev build s390x * no GOOS * ent only * build: publish s390x * fix syntax error * fix syntax error again * fix syntax error again x2 * test branch * Move s390x conditionals to step level * remove test branch --------- Co-authored-by: emilymianeil <[email protected]> * :ermahgerd "Sevice Mesh" -> "Service Mesh" (#18116) Just a typo in the docs. * Split pbmesh.UpstreamsConfiguration as a resource out of pbmesh.Upstreams (#17991) Configuration that previously was inlined into the Upstreams resource applies to both explicit and implicit upstreams and so it makes sense to split it out into its own resource. It also has other minor changes: - Renames `proxy.proto` proxy_configuration.proto` - Changes the type of `Upstream.destination_ref` from `pbresource.ID` to `pbresource.Reference` - Adds comments to fields that didn't have them * [NET-4895] ci - api tests and consul container tests error because of dependency bugs with go 1.20.6. Pin go to 1.20.5. (#18124) ### Description The following jobs started failing when go 1.20.6 was released: - `go-test-api-1-19` - `go-test-api-1-20` - `compatibility-integration-tests` - `upgrade-integration-tests` `compatibility-integration-tests` and `compatibility-integration-tests` to this testcontainers issue: https://github.com/testcontainers/testcontainers-go/issues/1359. This issue calls for testcontainers to release a new version when one of their dependencies is fixed. When that is done, we will unpin the go versions in `compatibility-integration-tests` and `compatibility-integration-tests`. ### Testing & Reproduction steps See these jobs broken in CI and then see them work with this PR. --------- Co-authored-by: Chris Thain <[email protected]> * Add ingress gateway deprecation notices to docs (#18102) ### Description This adds notices, that ingress gateway is deprecated, to several places in the product docs where ingress gateway is the topic. ### Testing & Reproduction steps Tested with a local copy of the website. ### Links Deprecation of ingress gateway was announced in the Release Notes for Consul 1.16 and Consul-K8s 1.2. See: [https://developer.hashicorp.com/consul/docs/release-notes/consul/v1_16_x#what-s-deprecated](https://developer.hashicorp.com/consul/docs/release-notes/consul/v1_16_x#what-s-deprecated ) [https://developer.hashicorp.com/consul/docs/release-notes/consul-k8s/v1_2_x#what-s-deprecated](https://developer.hashicorp.com/consul/docs/release-notes/consul-k8s/v1_2_x#what-s-deprecated) ### PR Checklist * [N/A] updated test coverage * [X] external facing docs updated * [X] appropriate backport labels added * [X] not a security concern --------- Co-authored-by: trujillo-adam <[email protected]> * Add docs for jwt cluster configuration (#18004) ### Description <!-- Please describe why you're making this change, in plain English. --> - Add jwt-provider docs for jwks cluster configuration. The configuration was added here: https://github.com/hashicorp/consul/pull/17978 * Docs: fix unmatched bracket for health checks page (#18134) * NET-4657/add resource service client (#18053) ### Description <!-- Please describe why you're making this change, in plain English. --> Dan had already started on this [task](https://github.com/hashicorp/consul/pull/17849) which is needed to start building the HTTP APIs. This just needed some cleanup to get it ready for review. Overview: - Rename `internalResourceServiceClient` to `insecureResourceServiceClient` for name consistency - Configure a `secureResourceServiceClient` with auth enabled ### PR Checklist * [ ] ~updated test coverage~ * [ ] ~external facing docs updated~ * [x] appropriate backport labels added * [ ] ~not a security concern~ * Fix bug with Vault CA provider (#18112) Updating RootPKIPath but not IntermediatePKIPath would not update leaf signing certs with the new root. Unsure if this happens in practice but manual testing showed it is a bug that would break mesh and agent connections once the old root is pruned. * [NET-4897] net/http host header is now verified and request.host that contains socked now error (#18129) ### Description This is related to https://github.com/hashicorp/consul/pull/18124 where we pinned the go versions in CI to 1.20.5 and 1.19.10. go 1.20.6 and 1.19.11 now validate request host headers for validity, including the hostname cannot be prefixed with slashes. For local communications (npipe://, unix://), the hostname is not used, but we need valid and meaningful hostname. Prior versions go Go would clean the host header, and strip slashes in the process, but go1.20.6 and go1.19.11 no longer do, and reject the host header. Around the community we are seeing that others are intercepting the req.host and if it starts with a slash or ends with .sock, they changing the host to localhost or another dummy value. [client: define a "dummy" hostname to use for local connections by thaJeztah · Pull Request #45942 · moby/moby](https://github.com/moby/moby/pull/45942) ### Testing & Reproduction steps Check CI tests. ### Links * [ ] updated test coverage * [ ] external facing docs updated * [ ] appropriate backport labels added * [ ] not a security concern * add a conditional around setting LANFilter.AllSegments to make sure it is valid (#18139) ### Description This is to correct a code problem because this assumes all segments, but when you get to Enterprise, you can be in partition that is not the default partition, in which case specifying all segments does not validate and fails. This is to correct the setting of this filter with `AllSegments` to `true` to only occur when in the the `default` partition. ### Testing & Reproduction steps <!-- * In the case of bugs, describe how to replicate * If any manual tests were done, document the steps and the conditions to replicate * Call out any important/ relevant unit tests, e2e tests or integration tests you have added or are adding --> ### Links <!-- Include any links here that might be helpful for people reviewing your PR (Tickets, GH issues, API docs, external benchmarks, tools docs, etc). If there are none, feel free to delete this section. Please be mindful not to leak any customer or confidential information. HashiCorp employees may want to use our internal URL shortener to obfuscate links. --> ### PR Checklist * [ ] updated test coverage * [ ] external facing docs updated * [ ] appropriate backport labels added * [ ] not a security concern * chore: bump upgrade integrations tests to 1.15, 116 [NET-4743] (#18130) * re org resource type registry (#18133) * fix: update delegateMock used in ENT (#18149) ### Description <!-- Please describe why you're making this change, in plain English. --> The mock is used in `http_ent_test` file which caused lint failures. For OSS->ENT parity adding the same change here. ### Links <!-- Include any links here that might be helpful for people reviewing your PR (Tickets, GH issues, API docs, external benchmarks, tools docs, etc). If there are none, feel free to delete this section. Please be mindful not to leak any customer or confidential information. HashiCorp employees may want to use our internal URL shortener to obfuscate links. --> Identified in OSS->ENT [merge PR](https://github.com/hashicorp/consul-enterprise/pull/6328) ### PR Checklist * [ ] ~updated test coverage~ * [ ] ~external facing docs updated~ * [x] appropriate backport labels added * [ ] ~not a security concern~ * Use JWT-auth filter in metadata mode & Delegate validation to RBAC filter (#18062) ### Description <!-- Please describe why you're making this change, in plain English. --> - Currently the jwt-auth filter doesn't take into account the service identity when validating jwt-auth, it only takes into account the path and jwt provider during validation. This causes issues when multiple source intentions restrict access to an endpoint with different JWT providers. - To fix these issues, rather than use the JWT auth filter for validation, we use it in metadata mode and allow it to forward the successful validated JWT token payload to the RBAC filter which will make the decisions. This PR ensures requests with and without JWT tokens successfully go through the jwt-authn filter. The filter however only forwards the data for successful/valid tokens. On the RBAC filter level, we check the payload for claims and token issuer + existing rbac rules. ### Testing & Reproduction steps <!-- * In the case of bugs, describe how to replicate * If any manual tests were done, document the steps and the conditions to replicate * Call out any important/ relevant unit tests, e2e tests or integration tests you have added or are adding --> - This test covers a multi level jwt requirements (requirements at top level and permissions level). It also assumes you have envoy running, you have a redis and a sidecar proxy service registered, and have a way to generate jwks with jwt. I mostly use: https://www.scottbrady91.com/tools/jwt for this. - first write your proxy defaults ``` Kind = "proxy-defaults" name = "global" config { protocol = "http" } ``` - Create two providers ``` Kind = "jwt-provider" Name = "auth0" Issuer = "https://ronald.local" JSONWebKeySet = { Local = { JWKS = "eyJrZXlzIjog....." } } ``` ``` Kind = "jwt-provider" Name = "okta" Issuer = "https://ronald.local" JSONWebKeySet = { Local = { JWKS = "eyJrZXlzIjogW3...." } } ``` - add a service intention ``` Kind = "service-intentions" Name = "redis" JWT = { Providers = [ { Name = "okta" }, ] } Sources = [ { Name = "*" Permissions = [{ Action = "allow" HTTP = { PathPrefix = "/workspace" } JWT = { Providers = [ { Name = "okta" VerifyClaims = [ { Path = ["aud"] Value = "my_client_app" }, { Path = ["sub"] Value = "5be86359073c434bad2da3932222dabe" } ] }, ] } }, { Action = "allow" HTTP = { PathPrefix = "/" } JWT = { Providers = [ { Name = "auth0" }, ] } }] } ] ``` - generate 3 jwt tokens: 1 from auth0 jwks, 1 from okta jwks with different claims than `/workspace` expects and 1 with correct claims - connect to your envoy (change service and address as needed) to view logs and potential errors. You can add: `-- --log-level debug` to see what data is being forwarded ``` consul connect envoy -sidecar-for redis1 -grpc-addr 127.0.0.1:8502 ``` - Make the following requests: ``` curl -s -H "Authorization: Bearer $Auth0_TOKEN" --insecure --cert leaf.cert --key leaf.key --cacert connect-ca.pem https://localhost:20000/workspace -v RBAC filter denied curl -s -H "Authorization: Bearer $Okta_TOKEN_with_wrong_claims" --insecure --cert leaf.cert --key leaf.key --cacert connect-ca.pem https://localhost:20000/workspace -v RBAC filter denied curl -s -H "Authorization: Bearer $Okta_TOKEN_with_correct_claims" --insecure --cert leaf.cert --key leaf.key --cacert connect-ca.pem https://localhost:20000/workspace -v Successful request ``` ### TODO * [x] Update test coverage * [ ] update integration tests (follow-up PR) * [x] appropriate backport labels added * Support Consul Connect Envoy Command on Windows (#17694) ### Description Add support for consul connect envoy command on windows. This PR fixes the comments of PR - https://github.com/hashicorp/consul/pull/15114 ### Testing * Built consul.exe from this branch on windows and hosted here - [AWS S3](https://asheshvidyut-bucket.s3.ap-southeast-2.amazonaws.com/consul.zip) * Updated the [tutorial](https://developer.hashicorp.com/consul/tutorials/developer-mesh/consul-windows-workloads) and changed the `consul_url.default` value to [AWS S3](https://asheshvidyut-bucket.s3.ap-southeast-2.amazonaws.com/consul.zip) * Followed the steps in the tutorial and verified that everything is working as described. ### PR Checklist * [x] updated test coverage * [ ] external facing docs updated * [x] appropriate backport labels added * [x] not a security concern --------- Co-authored-by: Franco Bruno Lavayen <[email protected]> Co-authored-by: Jose Ignacio Lorenzo <[email protected]> Co-authored-by: Jose Ignacio Lorenzo <[email protected]> Co-authored-by: Dhia Ayachi <[email protected]> * Change docs to say 168h instead of 7d for server_rejoin_age_max (#18154) ### Description Addresses https://github.com/hashicorp/consul/pull/17171#issuecomment-1636930705 * [OSS] test: improve xDS listener code coverage (#18138) test: improve xDS listener code coverage * Re-order expected/actual for assertContainerState in consul container tests (#18157) Re-order expected/actual, consul container tests * group and document make file (#17943) * group and document make file * Add `testing/deployer` (neé `consul-topology`) [NET-4610] (#17823) Co-authored-by: R.B. Boyer <[email protected]> Co-authored-by: R.B. Boyer <[email protected]> Co-authored-by: Freddy <[email protected]> * [NET-4792] Add integrations tests for jwt-auth (#18169) * Add FIPS reference to consul enterprise docs (#18028) * Add FIPS reference to consul enterprise docs * Update website/content/docs/enterprise/index.mdx Co-authored-by: David Yu <[email protected]> * remove support for ecs client (fips) --------- Co-authored-by: David Yu <[email protected]> * add peering_commontopo tests [NET-3700] (#17951) Co-authored-by: R.B. Boyer <[email protected]> Co-authored-by: R.B. Boyer <[email protected]> Co-authored-by: Freddy <[email protected]> Co-authored-by: NiniOak <[email protected]> * docs - remove Sentinel from enterprise features list (#18176) * Update index.mdx * Update kv.mdx * Update docs-nav-data.json * delete sentinel.mdx * Update redirects.js --------- Co-authored-by: Tu Nguyen <[email protected]> * [NET-4865] Bump golang.org/x/net to 0.12.0 (#18186) Bump golang.org/x/net to 0.12.0 While not necessary to directly address CVE-2023-29406 (which should be handled by using a patched version of Go when building), an accompanying change to HTTP/2 error handling does impact agent code. See https://go-review.googlesource.com/c/net/+/506995 for the HTTP/2 change. Bump this dependency across our submodules as well for the sake of potential indirect consumers of `x/net/http`. * Call resource mutate hook before validate hook (NET-4907) (#18178) * [NET-4865] security: Update Go version to 1.20.6 (#18190) Update Go version to 1.20.6 This resolves [CVE-2023-29406] (https://nvd.nist.gov/vuln/detail/CVE-2023-29406) for uses of the `net/http` standard library. Note that until the follow-up to #18124 is done, the version of Go used in those impacted tests will need to remain on 1.20.5. * Improve XDS test coverage: JWT auth edition (#18183) * Improve XDS test coverage: JWT auth edition more tests * test: xds coverage for jwt listeners --------- Co-authored-by: DanStough <[email protected]> * update readme.md (#18191) u[date readme.md * Update submodules to latest following 1.16.0 (#18197) Align all our internal use of submodules on the latest versions. * SEC-090: Automated trusted workflow pinning (2023-07-18) (#18174) Result of tsccr-helper -log-level=info -pin-all-workflows . Co-authored-by: hashicorp-tsccr[bot] <hashicorp-tsccr[bot]@users.noreply.github.com> * Fix Backport Assistant PR commenting (#18200) * Fix Backport Assistant failure PR commenting For general comments on a PR, it looks like you have to use the `/issue` endpoint rather than `/pulls`, which requires commit/other review-specific target details. This matches the endpoint used in `backport-reminder.yml`. * Remove Backport Reminder workflow This is noisy (even when adding multiple labels, individual comments per label are generated), and likely no longer needed: we haven't had this work in a long time due to an expired GH token, and we now have better automation for backport PR assignment. * resource: Pass resource to Write ACL hook instead of just resource Id [NET-4908] (#18192) * Explicitly enable WebSocket upgrades (#18150) This PR explicitly enables WebSocket upgrades in Envoy's UpgradeConfig for all proxy types. (API Gateway, Ingress, and Sidecar.) Fixes #8283 * docs: fix the description of client rpc (#18206) * NET-4804: Add dashboard for monitoring consul-k8s (#18208) * [OSS] Improve xDS Code Coverage - Clusters (#18165) test: improve xDS cluster code coverage * NET-4222 take config file consul container (#18218) Net 4222 take config file consul container * Envoy Integration Test Windows (#18007) * [CONSUL-395] Update check_hostport and Usage (#40) * [CONSUL-397] Copy envoy binary from Image (#41) * [CONSUL-382] Support openssl in unique test dockerfile (#43) * [CONSUL-405] Add bats to single container (#44) * [CONSUL-414] Run Prometheus Test Cases and Validate Changes (#46) * [CONSUL-410] Run Jaeger in Single container (#45) * [CONSUL-412] Run test-sds-server in single container (#48) * [CONSUL-408] Clean containers (#47) * [CONSUL-384] Rebase and sync fork (#50) * [CONSUL-415] Create Scenarios Troubleshooting Docs (#49) * [CONSUL-417] Update Docs Single Container (#51) * [CONSUL-428] Add Socat to single container (#54) * [CONSUL-424] Replace pkill in kill_envoy function (#52) * [CONSUL-434] Modify Docker run functions in Helper script (#53) * [CONSUL-435] Replace docker run in set_ttl_check_state & wait_for_agent_service_register functions (#55) * [CONSUL-438] Add netcat (nc) in the Single container Dockerfile (#56) * [CONSUL-429] Replace Docker run with Docker exec (#57) * [CONSUL-436] Curl timeout and run tests (#58) * [CONSUL-443] Create dogstatsd Function (#59) * [CONSUL-431] Update Docs Netcat (#60) * [CONSUL-439] Parse nc Command in function (#61) * [CONSUL-463] Review curl Exec and get_ca_root Func (#63) * [CONSUL-453] Docker hostname in Helper functions (#64) * [CONSUL-461] Test wipe volumes without extra cont (#66) * [CONSUL-454] Check ports in the Server and Agent containers (#65) * [CONSUL-441] Update windows dockerfile with version (#62) * [CONSUL-466] Review case-grpc Failing Test (#67) * [CONSUL-494] Review case-cfg-resolver-svc-failover (#68) * [CONSUL-496] Replace docker_wget & docker_curl (#69) * [CONSUL-499] Cleanup Scripts - Remove nanoserver (#70) * [CONSUL-500] Update Troubleshooting Docs (#72) * [CONSUL-502] Pull & Tag Envoy Windows Image (#73) * [CONSUL-504] Replace docker run in docker_consul (#76) * [CONSUL-505] Change admin_bind * [CONSUL-399] Update envoy to 1.23.1 (#78) * [CONSUL-510] Support case-wanfed-gw on Windows (#79) * [CONSUL-506] Update troubleshooting Documentation (#80) * [CONSUL-512] Review debug_dump_volumes Function (#81) * [CONSUL-514] Add zipkin to Docker Image (#82) * [CONSUL-515] Update Documentation (#83) * [CONSUL-529] Support case-consul-exec (#86) * [CONSUL-530] Update Documentation (#87) * [CONSUL-530] Update default consul version 1.13.3 * [CONSUL-539] Cleanup (#91) * [CONSUL-546] Scripts Clean-up (#92) * [CONSUL-491] Support admin_access_log_path value for Windows (#71) * [CONSUL-519] Implement mkfifo Alternative (#84) * [CONSUL-542] Create OS Specific Files for Envoy Package (#88) * [CONSUL-543] Create exec_supported.go (#89) * [CONSUL-544] Test and Build Changes (#90) * Implement os.DevNull * using mmap instead of disk files * fix import in exec-unix * fix nmap open too many arguemtn * go fmt on file * changelog file * fix go mod * Update .changelog/17694.txt Co-authored-by: Dhia Ayachi <[email protected]> * different mmap library * fix bootstrap json * some fixes * chocolatey version fix and image fix * using different library * fix Map funciton call * fix mmap call * fix tcp dump * fix tcp dump * windows tcp dump * Fix docker run * fix tests * fix go mod * fix version 16.0 * fix version * fix version dev * sleep to debug * fix sleep * fix permission issue * fix permission issue * fix permission issue * fix command * fix command * fix funciton * fix assert config entry status command not found * fix command not found assert_cert_has_cn * fix command not found assert_upstream_missing * fix command not found assert_upstream_missing_once * fix command not found get_upstream_endpoint * fix command not found get_envoy_public_listener_once * fix command not found * fix test cases * windows integration test workflow github * made code similar to unix using npipe * fix go.mod * fix dialing of npipe * dont wait * check size of written json * fix undefined n * running * fix dep * fix syntax error * fix workflow file * windows runner * fix runner * fix from json * fix runs on * merge connect envoy * fix cin path * build * fix file name * fix file name * fix dev build * remove unwanted code * fix upload * fix bin name * fix path * checkout current branch * fix path * fix tests * fix shell bash for windows sh files * fix permission of run-test.sh * removed docker dev * added shell bash for tests * fix tag * fix win=true * fix cd * added dev * fix variable undefined * removed failing tests * fix tcp dump image * fix curl * fix curl * tcp dump path * fix tcpdump path * fix curl * fix curl install * stop removing intermediate containers * fix tcpdump docker image * revert -rm * --rm=false * makeing docker image before * fix tcpdump * removed case consul exec * removed terminating gateway simple * comment case wasm * removed data dog * comment out upload coverage * uncomment case-consul-exec * comment case consul exec * if always * logs * using consul 1.17.0 * fix quotes * revert quotes * redirect to dev null * Revert version * revert consul connect * fix version * removed envoy connect * not using function * change log * docker logs * fix logs * restructure bad authz * rmeoved dev null * output * fix file descriptor * fix cacert * fix cacert * fix ca cert * cacert does not work in windows curl * fix func * removed docker logs * added sleep * fix tls * commented case-consul-exec * removed echo * retry docker consul * fix upload bin * uncomment consul exec * copying consul.exe to docker image * copy fix * fix paths * fix path * github workspace path * latest version * Revert "latest version" This reverts commit 5a7d7b82d9e7553bcb01b02557ec8969f9deba1d. * commented consul exec * added ssl revoke best effort * revert best effort * removed unused files * rename var name and change dir * windows runner * permission * needs setup fix * swtich to github runner * fix file path * fix path * fix path * fix path * fix path * fix path * fix build paths * fix tag * nightly runs * added matrix in github workflow, renamed files * fix job * fix matrix * removed brackes * from json * without using job matrix * fix quotes * revert job matrix * fix workflow * fix comment * added comment * nightly runs * removed datadog ci as it is already measured in linux one * running test * Revert "running test" This reverts commit 7013d15a23732179d18ec5d17336e16b26fab5d4. * pr comment fixes * running test now * running subset of test * running subset of test * job matrix * shell bash * removed bash shell * linux machine for job matrix * fix output * added cat to debug * using ubuntu latest * fix job matrix * fix win true * fix go test * revert job matrix --------- Co-authored-by: Jose Ignacio Lorenzo <[email protected]> Co-authored-by: Franco Bruno Lavayen <[email protected]> Co-authored-by: Ivan K Berlot <[email protected]> Co-authored-by: Ezequiel Fernández Ponce <[email protected]> Co-authored-by: joselo85 <[email protected]> Co-authored-by: Ezequiel Fernández Ponce <[email protected]> Co-authored-by: Dhia Ayachi <[email protected]> * fix typos and update ecs compat table (#18215) * fix typos and update ecs compat table * real info for the ecs compat matrix table * Update website/content/docs/ecs/compatibility.mdx Co-authored-by: Chris Thain <[email protected]> --------- Co-authored-by: Chris Thain <[email protected]> * [OSS] proxystate: add proxystate protos (#18216) * proxystate: add proxystate protos to pbmesh and resolve imports and conflicts between message names * ci: don't verify s390x (#18224) * [CC-5718] Remove HCP token requirement during bootstrap (#18140) * [CC-5718] Remove HCP token requirement during bootstrap * Re-add error for loading HCP management token * Remove old comment * Add changelog entry * Remove extra validation line * Apply suggestions from code review Co-authored-by: lornasong <[email protected]> --------- Co-authored-by: lornasong <[email protected]> * [NET-4122] Doc guidance for federation with externalServers (#18207) Doc guidance for federation with externalServers Add guidance for proper configuration when joining to a secondary cluster using WAN fed with external servers also enabled. Also clarify federation requirements and fix formatting for an unrelated value. Update both the Helm chart reference (synced from `consul-k8s`, see hashicorp/consul-k8s#2583) and the docs on using `externalServers`. * [OSS] Improve xDS Code Coverage - Endpoints and Misc (#18222) test: improve xDS endpoints code coverage * Clarify license reporting timing and GDPR compliance (#18237) Add Alicia's edits to clarify log timing and other details * Fix Github Workflow File (#18241) * [CONSUL-382] Support openssl in unique test dockerfile (#43) * [CONSUL-405] Add bats to single container (#44) * [CONSUL-414] Run Prometheus Test Cases and Validate Changes (#46) * [CONSUL-410] Run Jaeger in Single container (#45) * [CONSUL-412] Run test-sds-server in single container (#48) * [CONSUL-408] Clean containers (#47) * [CONSUL-384] Rebase and sync fork (#50) * [CONSUL-415] Create Scenarios Troubleshooting Docs (#49) * [CONSUL-417] Update Docs Single Container (#51) * [CONSUL-428] Add Socat to single container (#54) * [CONSUL-424] Replace pkill in kill_envoy function (#52) * [CONSUL-434] Modify Docker run functions in Helper script (#53) * [CONSUL-435] Replace docker run in set_ttl_check_state & wait_for_agent_service_register functions (#55) * [CONSUL-438] Add netcat (nc) in the Single container Dockerfile (#56) * [CONSUL-429] Replace Docker run with Docker exec (#57) * [CONSUL-436] Curl timeout and run tests (#58) * [CONSUL-443] Create dogstatsd Function (#59) * [CONSUL-431] Update Docs Netcat (#60) * [CONSUL-439] Parse nc Command in function (#61) * [CONSUL-463] Review curl Exec and get_ca_root Func (#63) * [CONSUL-453] Docker hostname in Helper functions (#64) * [CONSUL-461] Test wipe volumes without extra cont (#66) * [CONSUL-454] Check ports in the Server and Agent containers (#65) * [CONSUL-441] Update windows dockerfile with version (#62) * [CONSUL-466] Review case-grpc Failing Test (#67) * [CONSUL-494] Review case-cfg-resolver-svc-failover (#68) * [CONSUL-496] Replace docker_wget & docker_curl (#69) * [CONSUL-499] Cleanup Scripts - Remove nanoserver (#70) * [CONSUL-500] Update Troubleshooting Docs (#72) * [CONSUL-502] Pull & Tag Envoy Windows Image (#73) * [CONSUL-504] Replace docker run in docker_consul (#76) * [CONSUL-505] Change admin_bind * [CONSUL-399] Update envoy to 1.23.1 (#78) * [CONSUL-510] Support case-wanfed-gw on Windows (#79) * [CONSUL-506] Update troubleshooting Documentation (#80) * [CONSUL-512] Review debug_dump_volumes Function (#81) * [CONSUL-514] Add zipkin to Docker Image (#82) * [CONSUL-515] Update Documentation (#83) * [CONSUL-529] Support case-consul-exec (#86) * [CONSUL-530] Update Documentation (#87) * [CONSUL-530] Update default consul version 1.13.3 * [CONSUL-539] Cleanup (#91) * [CONSUL-546] Scripts Clean-up (#92) * [CONSUL-491] Support admin_access_log_path value for Windows (#71) * [CONSUL-519] Implement mkfifo Alternative (#84) * [CONSUL-542] Create OS Specific Files for Envoy Package (#88) * [CONSUL-543] Create exec_supported.go (#89) * [CONSUL-544] Test and Build Changes (#90) * Implement os.DevNull * using mmap instead of disk files * fix import in exec-unix * fix nmap open too many arguemtn * go fmt on file * changelog file * fix go mod * Update .changelog/17694.txt Co-authored-by: Dhia Ayachi <[email protected]> * different mmap library * fix bootstrap json * some fixes * chocolatey version fix and image fix * using different library * fix Map funciton call * fix mmap call * fix tcp dump * fix tcp dump * windows tcp dump * Fix docker run * fix tests * fix go mod * fix version 16.0 * fix version * fix version dev * sleep to debug * fix sleep * fix permission issue * fix permission issue * fix permission issue * fix command * fix command * fix funciton * fix assert config entry status command not found * fix command not found assert_cert_has_cn * fix command not found assert_upstream_missing * fix command not found assert_upstream_missing_once * fix command not found get_upstream_endpoint * fix command not found get_envoy_public_listener_once * fix command not found * fix test cases * windows integration test workflow github * made code similar to unix using npipe * fix go.mod * fix dialing of npipe * dont wait * check size of written json * fix undefined n * running * fix dep * fix syntax error * fix workflow file * windows runner * fix runner * fix from json * fix runs on * merge connect envoy * fix cin path * build * fix file name * fix file name * fix dev build * remove unwanted code * fix upload * fix bin name * fix path * checkout current branch * fix path * fix tests * fix shell bash for windows sh files * fix permission of run-test.sh * removed docker dev * added shell bash for tests * fix tag * fix win=true * fix cd * added dev * fix variable undefined * removed failing tests * fix tcp dump image * fix curl * fix curl * tcp dump path * fix tcpdump path * fix curl * fix curl install * stop removing intermediate containers * fix tcpdump docker image * revert -rm * --rm=false * makeing docker image before * fix tcpdump * removed case consul exec * removed terminating gateway simple * comment case wasm * removed data dog * comment out upload coverage * uncomment case-consul-exec * comment case consul exec * if always * logs * using consul 1.17.0 * fix quotes * revert quotes * redirect to dev null * Revert version * revert consul connect * fix version * removed envoy connect * not using function * change log * docker logs * fix logs * restructure bad authz * rmeoved dev null * output * fix file descriptor * fix cacert * fix cacert * fix ca cert * cacert does not work in windows curl * fix func * removed docker logs * added sleep * fix tls * commented case-consul-exec * removed echo * retry docker consul * fix upload bin * uncomment consul exec * copying consul.exe to docker image * copy fix * fix paths * fix path * github workspace path * latest version * Revert "latest version" This reverts commit 5a7d7b82d9e7553bcb01b02557ec8969f9deba1d. * commented consul exec * added ssl revoke best effort * revert best effort * removed unused files * rename var name and change dir * windows runner * permission * needs setup fix * swtich to github runner * fix file path * fix path * fix path * fix path * fix path * fix path * fix build paths * fix tag * nightly runs * added matrix in github workflow, renamed files * fix job * fix matrix * removed brackes * from json * without using job matrix * fix quotes * revert job matrix * fix workflow * fix comment * added comment * nightly runs * removed datadog ci as it is already measured in linux one * running test * Revert "running test" This reverts commit 7013d15a23732179d18ec5d17336e16b26fab5d4. * pr comment fixes * running test now * running subset of test * running subset of test * job matrix * shell bash * removed bash shell * linux machine for job matrix * fix output * added cat to debug * using ubuntu latest * fix job matrix * fix win true * fix go test * revert job matrix * Fix tests --------- Co-authored-by: Ivan K Berlot <[email protected]> Co-authored-by: Jose Ignacio Lorenzo <[email protected]> Co-authored-by: Franco Bruno Lavayen <[email protected]> Co-authored-by: Ezequiel Fernández Ponce <[email protected]> Co-authored-by: joselo85 <[email protected]> Co-authored-by: Ezequiel Fernández Ponce <[email protected]> Co-authored-by: Dhia Ayachi <[email protected]> * NET-4996 - filter go-tests and test-integration workflows from running on docs only and ui only changes (#18236) * Align build arch matrix with enterprise (#18235) Ensure that OSS remains in sync w/ Enterprise by aligning the format of arch matrix args for various build jobs. * Revert "NET-4996 - filter go-tests and test-integration workflows from running on docs only and ui only changes" (#18248) Revert "NET-4996 - filter go-tests and test-integration workflows from running on docs only and ui only changes (#18236)" This reverts commit a11dba710e6ce6f172c0fa6c9b61567cc1efffc8. * resource: Add scope to resource type registration [NET-4976] (#18214) Enables querying a resource type's registration to determine if a resource is cluster, partition, or partition and namespace scoped. * Fix some inconsistencies in jwt docs (#18234) * NET-1825: More new ACL token creation docs (#18063) Co-authored-by: trujillo-adam <[email protected]> * grafana: add the panel resource usage of connect injector (#18247) * [NET-3700] Backfill changelog entry for c2bbe67 and 7402d06 (#18259) Backfill changelog entry for c2bbe67 and 7402d06 Add a changelog entry for the follow-up PR since it was specific to the fix and references the original change. * NET-4897 - update comment to include the current issue url from the go team. (#18263) * fix typos, style, and improper links (#18269) * member cli: add -filter expression to flags (#18223) * member cli: add -filter expression to flags * changelog * update doc * Add test cases * use quote * go-tests: disable s390x (#18273) * docs: Update ext-authz documentation for kubernetes (#18281) Update ext-authz documentation for kubernetes * docs: Consul on Kubernetes specific upgrade info (#18230) * Compatibility page - dataplanes mention * Upgrading higher-level dataplane mention * `exec=` string callout * More visible for upgrade page * Apply suggestions from code review Co-authored-by: Tu Nguyen <[email protected]> --------- Co-authored-by: Tu Nguyen <[email protected]> Co-authored-by: John Murret <[email protected]> * Fix typo in Envoy extensions doc (#18284) * docs: K8s secondary DC requirements (#18280) * Requested edit * Apply suggestions from code review Co-authored-by: trujillo-adam <[email protected]> --------- Co-authored-by: trujillo-adam <[email protected]> * api-gateway: subscribe to bound-api-gateway only after receiving api-gateway (#18291) * api-gateway: subscribe to bound-api-gateway only after receiving api-gateway This fixes a race condition due to our dependency on having the listener(s) from the api-gateway config entry in order to fully and properly process the resources on the bound-api-gateway config entry. * Apply suggestions from code review * Add changelog entry * Update K8s changelog to address cloud auto-join change in 1.0.0 (#18293) * Update list of Envoy versions (#18300) Update supported envoy versions to 1.24.10, 1.25.9, 1.26.4, 1.27.0. * [NET-4904] Update list of Envoy versions in docs (#18306) Update list of Envoy versions in docs Update supported Envoy versions across Consul release versions. * Update actions for TSCCR (#18317) Update action versions before deadline * Fix topology view when displaying mixed connect-native/normal services. (#13023) * Fix topoloy intention with mixed connect-native/normal services. If a service is registered twice, once with connect-native and once without, the topology views would prune the existing intentions. This change brings the code more in line with the transparent proxy behavior. * Dedupe nodes in the ServiceTopology ui endpoint (like done with tags). * Consider a service connect-native as soon as one instance is. * Stop JWT provider from being written in non default namespace (#18325) * docs: Fix some comments (#17118) Signed-off-by: cui fliter <[email protected]> * [NET-4703] Prevent partial application of Envoy extensions (#18068) Prevent partial application of Envoy extensions Ensure that non-required extensions do not change xDS resources before exiting on failure by cloning proto messages prior to applying each extension. To support this change, also move `CanApply` checks up a layer and make them prior to attempting extension application, s.t. we avoid unnecessary copies where extensions can't be applied. Last, ensure that we do not allow panics from `CanApply` or `Extend` checks to escape the attempted extension application. * docs: Simplify example jq commands by removing pipes (#18327) Simplify jq command examples by removing pipes to other commands. * fix typo in create a mesh token docs (#18337) * [CC-5719] Add support for builtin global-read-only policy (#18319) * [CC-5719] Add support for builtin global-read-only policy * Add changelog * Add read-only to docs * Fix some minor issues. * Change from ReplaceAll to Sprintf * Change IsValidPolicy name to return an error instead of bool * Fix PolicyList test * Fix other tests * Apply suggestions from code review Co-authored-by: Paul Glass <[email protected]> * Fix state store test for policy list. * Fix naming issues * Update acl/validation.go Co-authored-by: Chris Thain <[email protected]> * Update agent/consul/acl_endpoint.go --------- Co-authored-by: Paul Glass <[email protected]> Co-authored-by: Chris Thain <[email protected]> * [NET-5121] proxystate: move protos to subdirectory to avoid conflicts (#18335) * also makes a few protos updates * resource: adding various helpers for working with resources (#18342) This is a bit of a grab bag of helpers that I found useful for working with them when authoring substantial Controllers. Subsequent PRs will make use of them. * Wasm integration tests for local and remote wasm files (#17756) * wasm integration tests for local and remote wasm files refactoring and cleanup for wasm testing remove wasm debug logging PR feedback, wasm build lock correct path pattern for wasm build files Add new helper function to minimize changes to existing test code Remove extra param mod tidy add custom service setup to test lib add wait until static server sidecar can reach nginx sidecar Doc comments PR feedback Update workflows to compile wasm for integration tests Fix docker build path Fix package name for linter Update makefile, fix redeclared function Update expected wasm filename Debug test ls in workflow remove pwd in favor of relative path more debugging Build wasm in compatability tests as well Build wasm directly in ci rather than in container Debug tinygo and llvm version Change wasm file extension Remove tinygo debugging Remove extra comments * Add compiled wasm and build instructions * [HCP Telemetry] Periodic Refresh for Dynamic Telemetry Configuration (#18168) * OTElExporter now uses an EndpointProvider to discover the endpoint * OTELSink uses a ConfigProvider to obtain filters and labels configuration * improve tests for otel_sink * Regex logic is moved into client for a method on the TelemetryConfig object * Create a telemetry_config_provider and update deps to use it * Fix conversion * fix import newline * Add logger to hcp client and move telemetry_config out of the client.go file * Add a telemetry_config.go to refactor client.go * Update deps * update hcp deps test * Modify telemetry_config_providers * Check for nil filters * PR review updates * Fix comments and move around pieces * Fix comments * Remove context from client struct * Moved ctx out of sink struct and fixed filters, added a test * Remove named imports, use errors.New if not fformatting * Remove HCP dependencies in telemetry package * Add success metric and move lock only to grab the t.cfgHahs * Update hash * fix nits * Create an equals method and add tests * Improve telemetry_config_provider.go tests * Add race test * Add missing godoc * Remove mock for MetricsClient * Avoid goroutine test panics * trying to kick CI lint issues by upgrading mod * imprve test code and add hasher for testing * Use structure logging for filters, fix error constants, and default to allow all regex * removed hashin and modify logic to simplify * Improve race test and fix PR feedback by removing hash equals and avoid testing the timer.Ticker logic, and instead unit test * Ran make go-mod-tidy * Use errtypes in the test * Add changelog * add safety check for exporter endpoint * remove require.Contains by using error types, fix structure logging, and fix success metric typo in exporter * Fixed race test to have changing config values * Send success metric before modifying config * Avoid the defer and move the success metric under * NET-4240 - Snapshots are failing on Windows (#18302) * fix go mod * fix go sum * added change log * ran make go mod tidy * Register ProxyStateTemplate Resource (#18316) Also, change the ProxyState.id to identity. This is because we already have the id of this proxy from the resource, and this id should be name-aligned with the workload it represents. It should also have the owner ref set to the workload ID if we need that. And so the id field seems unnecessary. We do, however, need a reference to workload identity so that we can authorize the proxy when it initially connects to the xDS server. * [NET-5146] security: Update Go version to 1.20.7 and `x/net` to 0.13.0 (#18358) * Update Go version to 1.20.7 This resolves [CVE-2023-29409] (https://nvd.nist.gov/vuln/detail/CVE-2023-29409)(`crypto/tls`). * Bump golang.org/x/net to 0.13.0 Addresses [CVE-2023-3978](https://nvd.nist.gov/vuln/detail/CVE-2023-3978) for security scans (non-impacting). * mesh: adding the protobuf types and resources backing mesh config v2 (#18351) * [OSS] test: xds coverage for routes (#18369) test: xds coverage for routes * Fix policy lookup to allow for slashes (#18347) * Fix policy lookup to allow for slashes * Fix suggestions * Fix other test * Revert some lines * add some initial CODEOWNERS (#18346) * Add redirects for mesh-gateway docs (#18377) * expose grpc as http endpoint (#18221) expose resource grpc endpoints as http endpoints * bimapper: fix a bug and add some more test coverage (#18387) * [docs] Fix ServiceDefaults example in distributed tracing (#17212) Fix ServiceDefaults example in distributed tracing. * Fix #17730 - Dev mode has new line (#18367) * adding new line only in case of pretty in url not in dev mode * change log added * chore: make go-mod-tidy (#18388) * resource: Make resource read tenancy aware (#18397) * Not using chmod - fixed integration test for Enterprise (#18401) * Not using chmod * checking icacls * Removed push * replaced ordered list of rate limit ops with flow diagram (#18398) * replaced ordered list of rate limit ops with flow diagram * made diagram font bigger * updated colors for dark mode img * fixed typo * recentered dark mode image * Apply suggestions from code review Co-authored-by: Jeff Boruszak <[email protected]> --------- Co-authored-by: Jeff Boruszak <[email protected]> * Upgrade test: remove outdated test and disable log due to verbosity (#18403) * remove outdated test * disable log since we have too many parallel tests * bump testcontainers-go from 0.22.0 and remove pinned go version in in… (#18395) * bump testcontainers-go from 0.22.0 and remove pinned go version in integ test * go mod tidy * Replace deprecated target.Authority with target.URL.Host * Add ServiceEndpoints Mutation hook tests (#18404) * Add ServiceEndpoints Mutation hook tests * Move endpoint owner validation into the validation hook Also there were some minor changes to error validation to account for go-cmp not liking to peer through an errors.errorstring type that get created by errors.New * catalog: add FailoverPolicy mutation and validation hooks (#18390) Add most of the validation and mutation hooks for the FailoverPolicy resource. * NET-5115 Add retry + timeout filters for api-gateway (#18324) * squash, implement retry/timeout in consul core * update tests * catalog: adding a controller to reconcile FailoverPolicy resources (#18399) Add most of the semantic cross-resource validation for FailoverPolicy resources using a new controller. * fix the error in ent repo (#18421) fix the error in ent repo * build: updates for 1.16.1 release (#18415) * build: updates for 1.16.1 release * build: fix missing replace directive for envoyextensions * build: go mod tidy testing/deployer * update ECS links (#18419) * docs: fix incorrect proxy-defaults config in Lua Envoy extension (#18418) fix incorrect proxy-defaults config * resource: Make resource write tenancy aware (#18423) * [NET-5217] [OSS] Derive sidecar proxy locality from parent service (#18437) * Add logging to locality policy application In OSS, this is currently a no-op. * Inherit locality when registering sidecars When sidecar locality is not explicitly configured, inherit locality from the proxied service. * NET-4984: Update APIGW Config Entries for JWT Auth (#18366) * Added oss config entries for Policy and JWT on APIGW * Updated structs for config entry * Updated comments, ran deep-copy * Move JWT configuration into OSS file * Add in the config entry OSS file for jwts * Added changelog * fixing proto spacing * Moved to using manually written deep copy method * Use pointers for override/default fields in apigw config entries * Run gen scripts for changed types * k8s compat - Openshift versions (#18307) * Update compatibility.mdx Co-authored-by: trujillo-adam <[email protected]> * APIGW: Update HTTPRouteConfigEntry for JWT Auth (#18422) * Updated httproute config entry for JWT Filters * Added manual deepcopy method for httproute jwt filter * Fix test * Update JWTFilter to be in oss file * Add changelog * Add build tags for deepcopy oss file * [COMPLIANCE] License changes (#18443) * Adding explicit MPL license for sub-package This directory and its subdirectories (packages) contain files licensed with the MPLv2 `LICENSE` file in this directory and are intentionally licensed separately from the BSL `LICENSE` file at the root of this repository. * Adding explicit MPL license for sub-package This directory and its subdirectories (packages) contain files licensed with the MPLv2 `LICENSE` file in this directory and are intentionally licensed separately from the BSL `LICENSE` file at the root of this repository. * Updating the license from MPL to Business Source License Going forward, this project will be licensed under the Business Source License v1.1. Please see our blog post for more details at <Blog URL>, FAQ at www.hashicorp.com/licensing-faq, and details of the license at www.hashicorp.com/bsl. * add missing license headers * Update copyright file headers to BUSL-1.1 * Update copyright file headers to BUSL-1.1 * Update copyright file headers to BUSL-1.1 * Update copyright file headers to BUSL-1.1 * Update copyright file headers to BUSL-1.1 * Update copyright file headers to BUSL-1.1 * Update copyright file headers to BUSL-1.1 * Update copyright file headers to BUSL-1.1 * Update copyright fi…
absolutelightning
added a commit
that referenced
this pull request
Sep 4, 2023
… nomad - fix bug into release/1.16.x (#18642) * Changelog - add 1.13.9, 1.14.8, and 1.15.4 (#17889) * docs: update config enable_debug (#17866) * update doc for config enable_debug * Update website/content/docs/agent/config/config-files.mdx Co-authored-by: trujillo-adam <[email protected]> --------- Co-authored-by: trujillo-adam <[email protected]> * Update wording on WAN fed and intermediate_pki_path (#17850) * Allow service identity tokens the ability to read jwt-providers (#17893) * Allow service identity tokens the ability to read jwt-providers * more tests * service_prefix tests * Update docs (#17476) Co-authored-by: trujillo-adam <[email protected]> * Add emit_tags_as_labels to envoy bootstrap config when using Consul Telemetry Collector (#17888) * Fix command from kg to kubectl get (#17903) * Create and update release notes for 1.16 and 1.2 (#17895) * update release notes for 1.16 and 1.2 * update latest consul core release * Propose new changes to APIgw upgrade instructions (#17693) * Propose new changes to APIgw upgrade instructions * fix build error * update callouts to render correctly * Add hideClipboard to log messages * Added clarification around consul k8s and crds * Add workflow to verify linux release packages (#17904) * adding docker files to verify linux packages. * add verifr-release-linux.yml * updating name * pass inputs directly into jobs * add other linux package platforms * remove on push * fix TARGETARCH on debian and ubuntu so it can check arm64 and amd64 * fixing amazon to use the continue line * add ubuntu i386 * fix comment lines * working * remove commented out workflow jobs * Apply suggestions from code review Co-authored-by: Nathan Coleman <[email protected]> * update fedora and ubuntu to use latest tag --------- Co-authored-by: Nathan Coleman <[email protected]> * Reference hashicorp/consul instead of consul for Docker image (#17914) * Reference hashicorp/consul instead of consul for Docker image * Update Make targets that pull consul directly * Update Consul K8s Upgrade Doc Updates (#17921) Updating upgrade procedures to encompass expected errors during upgrade process from v1.13.x to v1.14.x. * Update sameness-group.mdx (#17915) * Update create-sameness-groups.mdx (#17927) * deps: coredns v1.10.1 (#17912) * Ensure RSA keys are at least 2048 bits in length (#17911) * Ensure RSA keys are at least 2048 bits in length * Add changelog * update key length check for FIPS compliance * Fix no new variables error and failing to return when error exists from validating * clean up code for better readability * actually return value * tlsutil: Fix check TLS configuration (#17481) * tlsutil: Fix check TLS configuration * Rewording docs. * Update website/content/docs/services/configuration/checks-configuration-reference.mdx Co-authored-by: trujillo-adam <[email protected]> * Fix typos and add changelog entry. --------- Co-authored-by: trujillo-adam <[email protected]> * docs: Deprecations for connect-native SDK and specific connect native APIs (#17937) * Update v1_16_x.mdx * Update connect native golang page --------- Co-authored-by: trujillo-adam <[email protected]> * Revert "Add workflow to verify linux release packages (#17904)" (#17942) This reverts commit 3368f14fab500ebe9f6aeab5631dd1d5f5a453e5. * Fixes Secondary ConnectCA update (#17846) This fixes a bug that was identified which resulted in subsequent ConnectCA configuration update not to persist in the cluster. * fixing typo in link to jwt-validations-with-intentions doc (#17955) * Fix streaming backend link (#17958) * Fix streaming backend link * Update health.mdx * Dynamically create jwks clusters for jwt-providers (#17944) * website: remove deprecated agent rpc docs (#17962) * Fix missing BalanceOutboundConnections in v2 catalog. (#17964) * feature - [NET - 4005] - [Supportability] Reloadable Configuration - enable_debug (#17565) * # This is a combination of 9 commits. # This is the 1st commit message: init without tests # This is the commit message #2: change log # This is the commit message #3: fix tests # This is the commit message #4: fix tests # This is the commit message #5: added tests # This is the commit message #6: change log breaking change # This is the commit message #7: removed breaking change # This is the commit message #8: fix test # This is the commit message #9: keeping the test behaviour same * # This is a combination of 12 commits. # This is the 1st commit message: init without tests # This is the commit message #2: change log # This is the commit message #3: fix tests # This is the commit message #4: fix tests # This is the commit message #5: added tests # This is the commit message #6: change log breaking change # This is the commit message #7: removed breaking change # This is the commit message #8: fix test # This is the commit message #9: keeping the test behaviour same # This is the commit message #10: made enable debug atomic bool # This is the commit message #11: fix lint # This is the commit message #12: fix test true enable debug * parent 10f500e895d92cc3691ade7b74a33db755d22039 author absolutelightning <[email protected]> 1687352587 +0530 committer absolutelightning <[email protected]> 1687352592 +0530 init without tests change log fix tests fix tests added tests change log breaking change removed breaking change fix test keeping the test behaviour same made enable debug atomic bool fix lint fix test true enable debug using enable debug in agent as atomic bool test fixes fix tests fix tests added update on correct locaiton fix tests fix reloadable config enable debug fix tests fix init and acl 403 * revert commit * Fix formatting codeblocks on APIgw docs (#17970) * fix formatting codeblocks * remove unnecessary indents * Remove POC code (#17974) * update doc (#17910) * update doc * update link * Remove duplicate and unused newDecodeConfigEntry func (#17979) * docs: samenessGroup YAML examples (#17984) * configuration entry syntax * Example config * Add changelog entry for 1.16.0 (#17987) * Fix typo (#17198) servcies => services * Expose JWKS cluster config through JWTProviderConfigEntry (#17978) * Expose JWKS cluster config through JWTProviderConfigEntry * fix typos, rename trustedCa to trustedCA * Integration test for ext-authz Envoy extension (#17980) * Fix incorrect protocol for transparent proxy upstreams. (#17894) This PR fixes a bug that was introduced in: https://github.com/hashicorp/consul/pull/16021 A user setting a protocol in proxy-defaults would cause tproxy implicit upstreams to not honor the upstream service's protocol set in its `ServiceDefaults.Protocol` field, and would instead always use the proxy-defaults value. Due to the fact that upstreams configured with "tcp" can successfully contact upstream "http" services, this issue was not recognized until recently (a proxy-defaults with "tcp" and a listening service with "http" would make successful requests, but not the opposite). As a temporary work-around, users experiencing this issue can explicitly set the protocol on the `ServiceDefaults.UpstreamConfig.Overrides`, which should take precedence. The fix in this PR removes the proxy-defaults protocol from the wildcard upstream that tproxy uses to configure implicit upstreams. When the protocol was included, it would always overwrite the value during discovery chain compilation, which was not correct. The discovery chain compiler also consumes proxy defaults to determine the protocol, so simply excluding it from the wildcard upstream config map resolves the issue. * feat: include nodes count in operator usage endpoint and cli command (#17939) * feat: update operator usage api endpoint to include nodes count * feat: update operator usange cli command to includes nodes count * [OSS] Improve Gateway Test Coverage of Catalog Health (#18011) * fix(cli): remove failing check from 'connect envoy' registration for api gateway * test(integration): add tests to check catalog statsus of gateways on startup * remove extra sleep comment * Update test/integration/consul-container/libs/assert/service.go * changelog * Fixes Traffic rate limitting docs (#17997) * Fix removed service-to-service peering links (#17221) * docs: fix removed service-to-service peering links * docs: extend peering-via-mesh-gateways intro (thanks @trujillo-adam) --------- Co-authored-by: trujillo-adam <[email protected]> * docs: Sameness "beta" warning (#18017) * Warning updates * .x * updated typo in tab heading (#18022) * updated typo in tab heading * updated tab group typo, too * Document that DNS lookups can target cluster peers (#17990) Static DNS lookups, in addition to explicitly targeting a datacenter, can target a cluster peer. This was added in 95dc0c7b301b70a6b955a8b7c9737c9b86f03df6 but didn't make the documentation. The driving function for the change is `parseLocality` here: https://github.com/hashicorp/consul/blob/0b1299c28d8127129d61310ee4280055298438e0/agent/dns_oss.go#L25 The biggest change in this is to adjust the standard lookup syntax to tie `.<datacenter>` to `.dc` as required-together, and to append in the similar `.<cluster-peer>.peer` optional argument, both to A record and SRV record lookups. Co-authored-by: David Yu <[email protected]> * Add first integration test for jwt auth with intention (#18005) * fix stand-in text for name field (#18030) * removed sameness conf entry from failover nav (#18033) * docs - add service sync annotations and k8s service weight annotation (#18032) * Docs for https://github.com/hashicorp/consul-k8s/pull/2293 * remove versions for enterprise features since they are old --------- Co-authored-by: Tu Nguyen <[email protected]> * docs - add jobs use case for service mesh k8s (#18037) * docs - add jobs use case for service mesh k8s * add code blocks * address feedback (#18045) * Add verify server hostname to tls default (#17155) * [OSS] Fix initial_fetch_timeout to wait for all xDS resources (#18024) * fix(connect): set initial_fetch_time to wait indefinitely * changelog * PR feedback 1 * ui: fix typos for peer service imports (#17999) * test: fix FIPS inline cert test message (#18076) * Fix a couple typos in Agent Telemetry Metrics docs (#18080) * Fix metrics docs * Add changelog Signed-off-by: josh <[email protected]> --------- Signed-off-by: josh <[email protected]> * docs updates - cluster peering and virtual services (#18069) * Update route-to-virtual-services.mdx * Update establish-peering.mdx * Update service-mesh-compare.mdx (#17279) grammar change * Update helm docs on main (#18085) * ci: use gotestsum v1.10.1 [NET-4042] (#18088) * Docs: Update proxy lifecycle annotations and consul-dataplane flags (#18075) * Update proxy lifecycle annotations and consul-dataplane flags * Pass configured role name to Vault for AWS auth in Connect CA (#17885) * Docs for dataplane upgrade on k8s (#18051) * Docs for dataplane upgrade on k8s --------- Co-authored-by: David Yu <[email protected]> Co-authored-by: trujillo-adam <[email protected]> * docs - update upgrade index page to not recommend consul leave. (#18100) * Displays Consul version of each nodes in UI nodes section (#17754) * update UINodes and UINodeInfo response with consul-version info added as NodeMeta, fetched from serf members * update test cases TestUINodes, TestUINodeInfo * added nil check for map * add consul-version in local agent node metadata * get consul version from serf member and add this as node meta in catalog register request * updated ui mock response to include consul versions as node meta * updated ui trans and added version as query param to node list route * updates in ui templates to display consul version with filter and sorts * updates in ui - model class, serializers,comparators,predicates for consul version feature * added change log for Consul Version Feature * updated to get version from consul service, if for some reason not available from serf * updated changelog text * updated dependent testcases * multiselection version filter * Update agent/consul/state/catalog.go comments updated Co-authored-by: Jared Kirschner <[email protected]> --------- Co-authored-by: Jared Kirschner <[email protected]> * api gw 1.16 updates (#18081) * api gw 1.16 updates * Apply suggestions from code review Co-authored-by: Jeff Boruszak <[email protected]> * update CodeBlockConfig filename * Apply suggestions from code review Co-authored-by: trujillo-adam <[email protected]> Co-authored-by: Jeff Boruszak <[email protected]> * remove non-standard intentions page * Update website/content/docs/api-gateway/configuration/index.mdx Co-authored-by: trujillo-adam <[email protected]> --------- Co-authored-by: Jeff Boruszak <[email protected]> Co-authored-by: trujillo-adam <[email protected]> * [NET-4103] ci: build s390x (#18067) * ci: build s390x * ci: test s390x * ci: dev build s390x * no GOOS * ent only * build: publish s390x * fix syntax error * fix syntax error again * fix syntax error again x2 * test branch * Move s390x conditionals to step level * remove test branch --------- Co-authored-by: emilymianeil <[email protected]> * :ermahgerd "Sevice Mesh" -> "Service Mesh" (#18116) Just a typo in the docs. * Split pbmesh.UpstreamsConfiguration as a resource out of pbmesh.Upstreams (#17991) Configuration that previously was inlined into the Upstreams resource applies to both explicit and implicit upstreams and so it makes sense to split it out into its own resource. It also has other minor changes: - Renames `proxy.proto` proxy_configuration.proto` - Changes the type of `Upstream.destination_ref` from `pbresource.ID` to `pbresource.Reference` - Adds comments to fields that didn't have them * [NET-4895] ci - api tests and consul container tests error because of dependency bugs with go 1.20.6. Pin go to 1.20.5. (#18124) ### Description The following jobs started failing when go 1.20.6 was released: - `go-test-api-1-19` - `go-test-api-1-20` - `compatibility-integration-tests` - `upgrade-integration-tests` `compatibility-integration-tests` and `compatibility-integration-tests` to this testcontainers issue: https://github.com/testcontainers/testcontainers-go/issues/1359. This issue calls for testcontainers to release a new version when one of their dependencies is fixed. When that is done, we will unpin the go versions in `compatibility-integration-tests` and `compatibility-integration-tests`. ### Testing & Reproduction steps See these jobs broken in CI and then see them work with this PR. --------- Co-authored-by: Chris Thain <[email protected]> * Add ingress gateway deprecation notices to docs (#18102) ### Description This adds notices, that ingress gateway is deprecated, to several places in the product docs where ingress gateway is the topic. ### Testing & Reproduction steps Tested with a local copy of the website. ### Links Deprecation of ingress gateway was announced in the Release Notes for Consul 1.16 and Consul-K8s 1.2. See: [https://developer.hashicorp.com/consul/docs/release-notes/consul/v1_16_x#what-s-deprecated](https://developer.hashicorp.com/consul/docs/release-notes/consul/v1_16_x#what-s-deprecated ) [https://developer.hashicorp.com/consul/docs/release-notes/consul-k8s/v1_2_x#what-s-deprecated](https://developer.hashicorp.com/consul/docs/release-notes/consul-k8s/v1_2_x#what-s-deprecated) ### PR Checklist * [N/A] updated test coverage * [X] external facing docs updated * [X] appropriate backport labels added * [X] not a security concern --------- Co-authored-by: trujillo-adam <[email protected]> * Add docs for jwt cluster configuration (#18004) ### Description <!-- Please describe why you're making this change, in plain English. --> - Add jwt-provider docs for jwks cluster configuration. The configuration was added here: https://github.com/hashicorp/consul/pull/17978 * Docs: fix unmatched bracket for health checks page (#18134) * NET-4657/add resource service client (#18053) ### Description <!-- Please describe why you're making this change, in plain English. --> Dan had already started on this [task](https://github.com/hashicorp/consul/pull/17849) which is needed to start building the HTTP APIs. This just needed some cleanup to get it ready for review. Overview: - Rename `internalResourceServiceClient` to `insecureResourceServiceClient` for name consistency - Configure a `secureResourceServiceClient` with auth enabled ### PR Checklist * [ ] ~updated test coverage~ * [ ] ~external facing docs updated~ * [x] appropriate backport labels added * [ ] ~not a security concern~ * Fix bug with Vault CA provider (#18112) Updating RootPKIPath but not IntermediatePKIPath would not update leaf signing certs with the new root. Unsure if this happens in practice but manual testing showed it is a bug that would break mesh and agent connections once the old root is pruned. * [NET-4897] net/http host header is now verified and request.host that contains socked now error (#18129) ### Description This is related to https://github.com/hashicorp/consul/pull/18124 where we pinned the go versions in CI to 1.20.5 and 1.19.10. go 1.20.6 and 1.19.11 now validate request host headers for validity, including the hostname cannot be prefixed with slashes. For local communications (npipe://, unix://), the hostname is not used, but we need valid and meaningful hostname. Prior versions go Go would clean the host header, and strip slashes in the process, but go1.20.6 and go1.19.11 no longer do, and reject the host header. Around the community we are seeing that others are intercepting the req.host and if it starts with a slash or ends with .sock, they changing the host to localhost or another dummy value. [client: define a "dummy" hostname to use for local connections by thaJeztah · Pull Request #45942 · moby/moby](https://github.com/moby/moby/pull/45942) ### Testing & Reproduction steps Check CI tests. ### Links * [ ] updated test coverage * [ ] external facing docs updated * [ ] appropriate backport labels added * [ ] not a security concern * add a conditional around setting LANFilter.AllSegments to make sure it is valid (#18139) ### Description This is to correct a code problem because this assumes all segments, but when you get to Enterprise, you can be in partition that is not the default partition, in which case specifying all segments does not validate and fails. This is to correct the setting of this filter with `AllSegments` to `true` to only occur when in the the `default` partition. ### Testing & Reproduction steps <!-- * In the case of bugs, describe how to replicate * If any manual tests were done, document the steps and the conditions to replicate * Call out any important/ relevant unit tests, e2e tests or integration tests you have added or are adding --> ### Links <!-- Include any links here that might be helpful for people reviewing your PR (Tickets, GH issues, API docs, external benchmarks, tools docs, etc). If there are none, feel free to delete this section. Please be mindful not to leak any customer or confidential information. HashiCorp employees may want to use our internal URL shortener to obfuscate links. --> ### PR Checklist * [ ] updated test coverage * [ ] external facing docs updated * [ ] appropriate backport labels added * [ ] not a security concern * chore: bump upgrade integrations tests to 1.15, 116 [NET-4743] (#18130) * re org resource type registry (#18133) * fix: update delegateMock used in ENT (#18149) ### Description <!-- Please describe why you're making this change, in plain English. --> The mock is used in `http_ent_test` file which caused lint failures. For OSS->ENT parity adding the same change here. ### Links <!-- Include any links here that might be helpful for people reviewing your PR (Tickets, GH issues, API docs, external benchmarks, tools docs, etc). If there are none, feel free to delete this section. Please be mindful not to leak any customer or confidential information. HashiCorp employees may want to use our internal URL shortener to obfuscate links. --> Identified in OSS->ENT [merge PR](https://github.com/hashicorp/consul-enterprise/pull/6328) ### PR Checklist * [ ] ~updated test coverage~ * [ ] ~external facing docs updated~ * [x] appropriate backport labels added * [ ] ~not a security concern~ * Use JWT-auth filter in metadata mode & Delegate validation to RBAC filter (#18062) ### Description <!-- Please describe why you're making this change, in plain English. --> - Currently the jwt-auth filter doesn't take into account the service identity when validating jwt-auth, it only takes into account the path and jwt provider during validation. This causes issues when multiple source intentions restrict access to an endpoint with different JWT providers. - To fix these issues, rather than use the JWT auth filter for validation, we use it in metadata mode and allow it to forward the successful validated JWT token payload to the RBAC filter which will make the decisions. This PR ensures requests with and without JWT tokens successfully go through the jwt-authn filter. The filter however only forwards the data for successful/valid tokens. On the RBAC filter level, we check the payload for claims and token issuer + existing rbac rules. ### Testing & Reproduction steps <!-- * In the case of bugs, describe how to replicate * If any manual tests were done, document the steps and the conditions to replicate * Call out any important/ relevant unit tests, e2e tests or integration tests you have added or are adding --> - This test covers a multi level jwt requirements (requirements at top level and permissions level). It also assumes you have envoy running, you have a redis and a sidecar proxy service registered, and have a way to generate jwks with jwt. I mostly use: https://www.scottbrady91.com/tools/jwt for this. - first write your proxy defaults ``` Kind = "proxy-defaults" name = "global" config { protocol = "http" } ``` - Create two providers ``` Kind = "jwt-provider" Name = "auth0" Issuer = "https://ronald.local" JSONWebKeySet = { Local = { JWKS = "eyJrZXlzIjog....." } } ``` ``` Kind = "jwt-provider" Name = "okta" Issuer = "https://ronald.local" JSONWebKeySet = { Local = { JWKS = "eyJrZXlzIjogW3...." } } ``` - add a service intention ``` Kind = "service-intentions" Name = "redis" JWT = { Providers = [ { Name = "okta" }, ] } Sources = [ { Name = "*" Permissions = [{ Action = "allow" HTTP = { PathPrefix = "/workspace" } JWT = { Providers = [ { Name = "okta" VerifyClaims = [ { Path = ["aud"] Value = "my_client_app" }, { Path = ["sub"] Value = "5be86359073c434bad2da3932222dabe" } ] }, ] } }, { Action = "allow" HTTP = { PathPrefix = "/" } JWT = { Providers = [ { Name = "auth0" }, ] } }] } ] ``` - generate 3 jwt tokens: 1 from auth0 jwks, 1 from okta jwks with different claims than `/workspace` expects and 1 with correct claims - connect to your envoy (change service and address as needed) to view logs and potential errors. You can add: `-- --log-level debug` to see what data is being forwarded ``` consul connect envoy -sidecar-for redis1 -grpc-addr 127.0.0.1:8502 ``` - Make the following requests: ``` curl -s -H "Authorization: Bearer $Auth0_TOKEN" --insecure --cert leaf.cert --key leaf.key --cacert connect-ca.pem https://localhost:20000/workspace -v RBAC filter denied curl -s -H "Authorization: Bearer $Okta_TOKEN_with_wrong_claims" --insecure --cert leaf.cert --key leaf.key --cacert connect-ca.pem https://localhost:20000/workspace -v RBAC filter denied curl -s -H "Authorization: Bearer $Okta_TOKEN_with_correct_claims" --insecure --cert leaf.cert --key leaf.key --cacert connect-ca.pem https://localhost:20000/workspace -v Successful request ``` ### TODO * [x] Update test coverage * [ ] update integration tests (follow-up PR) * [x] appropriate backport labels added * Support Consul Connect Envoy Command on Windows (#17694) ### Description Add support for consul connect envoy command on windows. This PR fixes the comments of PR - https://github.com/hashicorp/consul/pull/15114 ### Testing * Built consul.exe from this branch on windows and hosted here - [AWS S3](https://asheshvidyut-bucket.s3.ap-southeast-2.amazonaws.com/consul.zip) * Updated the [tutorial](https://developer.hashicorp.com/consul/tutorials/developer-mesh/consul-windows-workloads) and changed the `consul_url.default` value to [AWS S3](https://asheshvidyut-bucket.s3.ap-southeast-2.amazonaws.com/consul.zip) * Followed the steps in the tutorial and verified that everything is working as described. ### PR Checklist * [x] updated test coverage * [ ] external facing docs updated * [x] appropriate backport labels added * [x] not a security concern --------- Co-authored-by: Franco Bruno Lavayen <[email protected]> Co-authored-by: Jose Ignacio Lorenzo <[email protected]> Co-authored-by: Jose Ignacio Lorenzo <[email protected]> Co-authored-by: Dhia Ayachi <[email protected]> * Change docs to say 168h instead of 7d for server_rejoin_age_max (#18154) ### Description Addresses https://github.com/hashicorp/consul/pull/17171#issuecomment-1636930705 * [OSS] test: improve xDS listener code coverage (#18138) test: improve xDS listener code coverage * Re-order expected/actual for assertContainerState in consul container tests (#18157) Re-order expected/actual, consul container tests * group and document make file (#17943) * group and document make file * Add `testing/deployer` (neé `consul-topology`) [NET-4610] (#17823) Co-authored-by: R.B. Boyer <[email protected]> Co-authored-by: R.B. Boyer <[email protected]> Co-authored-by: Freddy <[email protected]> * [NET-4792] Add integrations tests for jwt-auth (#18169) * Add FIPS reference to consul enterprise docs (#18028) * Add FIPS reference to consul enterprise docs * Update website/content/docs/enterprise/index.mdx Co-authored-by: David Yu <[email protected]> * remove support for ecs client (fips) --------- Co-authored-by: David Yu <[email protected]> * add peering_commontopo tests [NET-3700] (#17951) Co-authored-by: R.B. Boyer <[email protected]> Co-authored-by: R.B. Boyer <[email protected]> Co-authored-by: Freddy <[email protected]> Co-authored-by: NiniOak <[email protected]> * docs - remove Sentinel from enterprise features list (#18176) * Update index.mdx * Update kv.mdx * Update docs-nav-data.json * delete sentinel.mdx * Update redirects.js --------- Co-authored-by: Tu Nguyen <[email protected]> * [NET-4865] Bump golang.org/x/net to 0.12.0 (#18186) Bump golang.org/x/net to 0.12.0 While not necessary to directly address CVE-2023-29406 (which should be handled by using a patched version of Go when building), an accompanying change to HTTP/2 error handling does impact agent code. See https://go-review.googlesource.com/c/net/+/506995 for the HTTP/2 change. Bump this dependency across our submodules as well for the sake of potential indirect consumers of `x/net/http`. * Call resource mutate hook before validate hook (NET-4907) (#18178) * [NET-4865] security: Update Go version to 1.20.6 (#18190) Update Go version to 1.20.6 This resolves [CVE-2023-29406] (https://nvd.nist.gov/vuln/detail/CVE-2023-29406) for uses of the `net/http` standard library. Note that until the follow-up to #18124 is done, the version of Go used in those impacted tests will need to remain on 1.20.5. * Improve XDS test coverage: JWT auth edition (#18183) * Improve XDS test coverage: JWT auth edition more tests * test: xds coverage for jwt listeners --------- Co-authored-by: DanStough <[email protected]> * update readme.md (#18191) u[date readme.md * Update submodules to latest following 1.16.0 (#18197) Align all our internal use of submodules on the latest versions. * SEC-090: Automated trusted workflow pinning (2023-07-18) (#18174) Result of tsccr-helper -log-level=info -pin-all-workflows . Co-authored-by: hashicorp-tsccr[bot] <hashicorp-tsccr[bot]@users.noreply.github.com> * Fix Backport Assistant PR commenting (#18200) * Fix Backport Assistant failure PR commenting For general comments on a PR, it looks like you have to use the `/issue` endpoint rather than `/pulls`, which requires commit/other review-specific target details. This matches the endpoint used in `backport-reminder.yml`. * Remove Backport Reminder workflow This is noisy (even when adding multiple labels, individual comments per label are generated), and likely no longer needed: we haven't had this work in a long time due to an expired GH token, and we now have better automation for backport PR assignment. * resource: Pass resource to Write ACL hook instead of just resource Id [NET-4908] (#18192) * Explicitly enable WebSocket upgrades (#18150) This PR explicitly enables WebSocket upgrades in Envoy's UpgradeConfig for all proxy types. (API Gateway, Ingress, and Sidecar.) Fixes #8283 * docs: fix the description of client rpc (#18206) * NET-4804: Add dashboard for monitoring consul-k8s (#18208) * [OSS] Improve xDS Code Coverage - Clusters (#18165) test: improve xDS cluster code coverage * NET-4222 take config file consul container (#18218) Net 4222 take config file consul container * Envoy Integration Test Windows (#18007) * [CONSUL-395] Update check_hostport and Usage (#40) * [CONSUL-397] Copy envoy binary from Image (#41) * [CONSUL-382] Support openssl in unique test dockerfile (#43) * [CONSUL-405] Add bats to single container (#44) * [CONSUL-414] Run Prometheus Test Cases and Validate Changes (#46) * [CONSUL-410] Run Jaeger in Single container (#45) * [CONSUL-412] Run test-sds-server in single container (#48) * [CONSUL-408] Clean containers (#47) * [CONSUL-384] Rebase and sync fork (#50) * [CONSUL-415] Create Scenarios Troubleshooting Docs (#49) * [CONSUL-417] Update Docs Single Container (#51) * [CONSUL-428] Add Socat to single container (#54) * [CONSUL-424] Replace pkill in kill_envoy function (#52) * [CONSUL-434] Modify Docker run functions in Helper script (#53) * [CONSUL-435] Replace docker run in set_ttl_check_state & wait_for_agent_service_register functions (#55) * [CONSUL-438] Add netcat (nc) in the Single container Dockerfile (#56) * [CONSUL-429] Replace Docker run with Docker exec (#57) * [CONSUL-436] Curl timeout and run tests (#58) * [CONSUL-443] Create dogstatsd Function (#59) * [CONSUL-431] Update Docs Netcat (#60) * [CONSUL-439] Parse nc Command in function (#61) * [CONSUL-463] Review curl Exec and get_ca_root Func (#63) * [CONSUL-453] Docker hostname in Helper functions (#64) * [CONSUL-461] Test wipe volumes without extra cont (#66) * [CONSUL-454] Check ports in the Server and Agent containers (#65) * [CONSUL-441] Update windows dockerfile with version (#62) * [CONSUL-466] Review case-grpc Failing Test (#67) * [CONSUL-494] Review case-cfg-resolver-svc-failover (#68) * [CONSUL-496] Replace docker_wget & docker_curl (#69) * [CONSUL-499] Cleanup Scripts - Remove nanoserver (#70) * [CONSUL-500] Update Troubleshooting Docs (#72) * [CONSUL-502] Pull & Tag Envoy Windows Image (#73) * [CONSUL-504] Replace docker run in docker_consul (#76) * [CONSUL-505] Change admin_bind * [CONSUL-399] Update envoy to 1.23.1 (#78) * [CONSUL-510] Support case-wanfed-gw on Windows (#79) * [CONSUL-506] Update troubleshooting Documentation (#80) * [CONSUL-512] Review debug_dump_volumes Function (#81) * [CONSUL-514] Add zipkin to Docker Image (#82) * [CONSUL-515] Update Documentation (#83) * [CONSUL-529] Support case-consul-exec (#86) * [CONSUL-530] Update Documentation (#87) * [CONSUL-530] Update default consul version 1.13.3 * [CONSUL-539] Cleanup (#91) * [CONSUL-546] Scripts Clean-up (#92) * [CONSUL-491] Support admin_access_log_path value for Windows (#71) * [CONSUL-519] Implement mkfifo Alternative (#84) * [CONSUL-542] Create OS Specific Files for Envoy Package (#88) * [CONSUL-543] Create exec_supported.go (#89) * [CONSUL-544] Test and Build Changes (#90) * Implement os.DevNull * using mmap instead of disk files * fix import in exec-unix * fix nmap open too many arguemtn * go fmt on file * changelog file * fix go mod * Update .changelog/17694.txt Co-authored-by: Dhia Ayachi <[email protected]> * different mmap library * fix bootstrap json * some fixes * chocolatey version fix and image fix * using different library * fix Map funciton call * fix mmap call * fix tcp dump * fix tcp dump * windows tcp dump * Fix docker run * fix tests * fix go mod * fix version 16.0 * fix version * fix version dev * sleep to debug * fix sleep * fix permission issue * fix permission issue * fix permission issue * fix command * fix command * fix funciton * fix assert config entry status command not found * fix command not found assert_cert_has_cn * fix command not found assert_upstream_missing * fix command not found assert_upstream_missing_once * fix command not found get_upstream_endpoint * fix command not found get_envoy_public_listener_once * fix command not found * fix test cases * windows integration test workflow github * made code similar to unix using npipe * fix go.mod * fix dialing of npipe * dont wait * check size of written json * fix undefined n * running * fix dep * fix syntax error * fix workflow file * windows runner * fix runner * fix from json * fix runs on * merge connect envoy * fix cin path * build * fix file name * fix file name * fix dev build * remove unwanted code * fix upload * fix bin name * fix path * checkout current branch * fix path * fix tests * fix shell bash for windows sh files * fix permission of run-test.sh * removed docker dev * added shell bash for tests * fix tag * fix win=true * fix cd * added dev * fix variable undefined * removed failing tests * fix tcp dump image * fix curl * fix curl * tcp dump path * fix tcpdump path * fix curl * fix curl install * stop removing intermediate containers * fix tcpdump docker image * revert -rm * --rm=false * makeing docker image before * fix tcpdump * removed case consul exec * removed terminating gateway simple * comment case wasm * removed data dog * comment out upload coverage * uncomment case-consul-exec * comment case consul exec * if always * logs * using consul 1.17.0 * fix quotes * revert quotes * redirect to dev null * Revert version * revert consul connect * fix version * removed envoy connect * not using function * change log * docker logs * fix logs * restructure bad authz * rmeoved dev null * output * fix file descriptor * fix cacert * fix cacert * fix ca cert * cacert does not work in windows curl * fix func * removed docker logs * added sleep * fix tls * commented case-consul-exec * removed echo * retry docker consul * fix upload bin * uncomment consul exec * copying consul.exe to docker image * copy fix * fix paths * fix path * github workspace path * latest version * Revert "latest version" This reverts commit 5a7d7b82d9e7553bcb01b02557ec8969f9deba1d. * commented consul exec * added ssl revoke best effort * revert best effort * removed unused files * rename var name and change dir * windows runner * permission * needs setup fix * swtich to github runner * fix file path * fix path * fix path * fix path * fix path * fix path * fix build paths * fix tag * nightly runs * added matrix in github workflow, renamed files * fix job * fix matrix * removed brackes * from json * without using job matrix * fix quotes * revert job matrix * fix workflow * fix comment * added comment * nightly runs * removed datadog ci as it is already measured in linux one * running test * Revert "running test" This reverts commit 7013d15a23732179d18ec5d17336e16b26fab5d4. * pr comment fixes * running test now * running subset of test * running subset of test * job matrix * shell bash * removed bash shell * linux machine for job matrix * fix output * added cat to debug * using ubuntu latest * fix job matrix * fix win true * fix go test * revert job matrix --------- Co-authored-by: Jose Ignacio Lorenzo <[email protected]> Co-authored-by: Franco Bruno Lavayen <[email protected]> Co-authored-by: Ivan K Berlot <[email protected]> Co-authored-by: Ezequiel Fernández Ponce <[email protected]> Co-authored-by: joselo85 <[email protected]> Co-authored-by: Ezequiel Fernández Ponce <[email protected]> Co-authored-by: Dhia Ayachi <[email protected]> * fix typos and update ecs compat table (#18215) * fix typos and update ecs compat table * real info for the ecs compat matrix table * Update website/content/docs/ecs/compatibility.mdx Co-authored-by: Chris Thain <[email protected]> --------- Co-authored-by: Chris Thain <[email protected]> * [OSS] proxystate: add proxystate protos (#18216) * proxystate: add proxystate protos to pbmesh and resolve imports and conflicts between message names * ci: don't verify s390x (#18224) * [CC-5718] Remove HCP token requirement during bootstrap (#18140) * [CC-5718] Remove HCP token requirement during bootstrap * Re-add error for loading HCP management token * Remove old comment * Add changelog entry * Remove extra validation line * Apply suggestions from code review Co-authored-by: lornasong <[email protected]> --------- Co-authored-by: lornasong <[email protected]> * [NET-4122] Doc guidance for federation with externalServers (#18207) Doc guidance for federation with externalServers Add guidance for proper configuration when joining to a secondary cluster using WAN fed with external servers also enabled. Also clarify federation requirements and fix formatting for an unrelated value. Update both the Helm chart reference (synced from `consul-k8s`, see hashicorp/consul-k8s#2583) and the docs on using `externalServers`. * [OSS] Improve xDS Code Coverage - Endpoints and Misc (#18222) test: improve xDS endpoints code coverage * Clarify license reporting timing and GDPR compliance (#18237) Add Alicia's edits to clarify log timing and other details * Fix Github Workflow File (#18241) * [CONSUL-382] Support openssl in unique test dockerfile (#43) * [CONSUL-405] Add bats to single container (#44) * [CONSUL-414] Run Prometheus Test Cases and Validate Changes (#46) * [CONSUL-410] Run Jaeger in Single container (#45) * [CONSUL-412] Run test-sds-server in single container (#48) * [CONSUL-408] Clean containers (#47) * [CONSUL-384] Rebase and sync fork (#50) * [CONSUL-415] Create Scenarios Troubleshooting Docs (#49) * [CONSUL-417] Update Docs Single Container (#51) * [CONSUL-428] Add Socat to single container (#54) * [CONSUL-424] Replace pkill in kill_envoy function (#52) * [CONSUL-434] Modify Docker run functions in Helper script (#53) * [CONSUL-435] Replace docker run in set_ttl_check_state & wait_for_agent_service_register functions (#55) * [CONSUL-438] Add netcat (nc) in the Single container Dockerfile (#56) * [CONSUL-429] Replace Docker run with Docker exec (#57) * [CONSUL-436] Curl timeout and run tests (#58) * [CONSUL-443] Create dogstatsd Function (#59) * [CONSUL-431] Update Docs Netcat (#60) * [CONSUL-439] Parse nc Command in function (#61) * [CONSUL-463] Review curl Exec and get_ca_root Func (#63) * [CONSUL-453] Docker hostname in Helper functions (#64) * [CONSUL-461] Test wipe volumes without extra cont (#66) * [CONSUL-454] Check ports in the Server and Agent containers (#65) * [CONSUL-441] Update windows dockerfile with version (#62) * [CONSUL-466] Review case-grpc Failing Test (#67) * [CONSUL-494] Review case-cfg-resolver-svc-failover (#68) * [CONSUL-496] Replace docker_wget & docker_curl (#69) * [CONSUL-499] Cleanup Scripts - Remove nanoserver (#70) * [CONSUL-500] Update Troubleshooting Docs (#72) * [CONSUL-502] Pull & Tag Envoy Windows Image (#73) * [CONSUL-504] Replace docker run in docker_consul (#76) * [CONSUL-505] Change admin_bind * [CONSUL-399] Update envoy to 1.23.1 (#78) * [CONSUL-510] Support case-wanfed-gw on Windows (#79) * [CONSUL-506] Update troubleshooting Documentation (#80) * [CONSUL-512] Review debug_dump_volumes Function (#81) * [CONSUL-514] Add zipkin to Docker Image (#82) * [CONSUL-515] Update Documentation (#83) * [CONSUL-529] Support case-consul-exec (#86) * [CONSUL-530] Update Documentation (#87) * [CONSUL-530] Update default consul version 1.13.3 * [CONSUL-539] Cleanup (#91) * [CONSUL-546] Scripts Clean-up (#92) * [CONSUL-491] Support admin_access_log_path value for Windows (#71) * [CONSUL-519] Implement mkfifo Alternative (#84) * [CONSUL-542] Create OS Specific Files for Envoy Package (#88) * [CONSUL-543] Create exec_supported.go (#89) * [CONSUL-544] Test and Build Changes (#90) * Implement os.DevNull * using mmap instead of disk files * fix import in exec-unix * fix nmap open too many arguemtn * go fmt on file * changelog file * fix go mod * Update .changelog/17694.txt Co-authored-by: Dhia Ayachi <[email protected]> * different mmap library * fix bootstrap json * some fixes * chocolatey version fix and image fix * using different library * fix Map funciton call * fix mmap call * fix tcp dump * fix tcp dump * windows tcp dump * Fix docker run * fix tests * fix go mod * fix version 16.0 * fix version * fix version dev * sleep to debug * fix sleep * fix permission issue * fix permission issue * fix permission issue * fix command * fix command * fix funciton * fix assert config entry status command not found * fix command not found assert_cert_has_cn * fix command not found assert_upstream_missing * fix command not found assert_upstream_missing_once * fix command not found get_upstream_endpoint * fix command not found get_envoy_public_listener_once * fix command not found * fix test cases * windows integration test workflow github * made code similar to unix using npipe * fix go.mod * fix dialing of npipe * dont wait * check size of written json * fix undefined n * running * fix dep * fix syntax error * fix workflow file * windows runner * fix runner * fix from json * fix runs on * merge connect envoy * fix cin path * build * fix file name * fix file name * fix dev build * remove unwanted code * fix upload * fix bin name * fix path * checkout current branch * fix path * fix tests * fix shell bash for windows sh files * fix permission of run-test.sh * removed docker dev * added shell bash for tests * fix tag * fix win=true * fix cd * added dev * fix variable undefined * removed failing tests * fix tcp dump image * fix curl * fix curl * tcp dump path * fix tcpdump path * fix curl * fix curl install * stop removing intermediate containers * fix tcpdump docker image * revert -rm * --rm=false * makeing docker image before * fix tcpdump * removed case consul exec * removed terminating gateway simple * comment case wasm * removed data dog * comment out upload coverage * uncomment case-consul-exec * comment case consul exec * if always * logs * using consul 1.17.0 * fix quotes * revert quotes * redirect to dev null * Revert version * revert consul connect * fix version * removed envoy connect * not using function * change log * docker logs * fix logs * restructure bad authz * rmeoved dev null * output * fix file descriptor * fix cacert * fix cacert * fix ca cert * cacert does not work in windows curl * fix func * removed docker logs * added sleep * fix tls * commented case-consul-exec * removed echo * retry docker consul * fix upload bin * uncomment consul exec * copying consul.exe to docker image * copy fix * fix paths * fix path * github workspace path * latest version * Revert "latest version" This reverts commit 5a7d7b82d9e7553bcb01b02557ec8969f9deba1d. * commented consul exec * added ssl revoke best effort * revert best effort * removed unused files * rename var name and change dir * windows runner * permission * needs setup fix * swtich to github runner * fix file path * fix path * fix path * fix path * fix path * fix path * fix build paths * fix tag * nightly runs * added matrix in github workflow, renamed files * fix job * fix matrix * removed brackes * from json * without using job matrix * fix quotes * revert job matrix * fix workflow * fix comment * added comment * nightly runs * removed datadog ci as it is already measured in linux one * running test * Revert "running test" This reverts commit 7013d15a23732179d18ec5d17336e16b26fab5d4. * pr comment fixes * running test now * running subset of test * running subset of test * job matrix * shell bash * removed bash shell * linux machine for job matrix * fix output * added cat to debug * using ubuntu latest * fix job matrix * fix win true * fix go test * revert job matrix * Fix tests --------- Co-authored-by: Ivan K Berlot <[email protected]> Co-authored-by: Jose Ignacio Lorenzo <[email protected]> Co-authored-by: Franco Bruno Lavayen <[email protected]> Co-authored-by: Ezequiel Fernández Ponce <[email protected]> Co-authored-by: joselo85 <[email protected]> Co-authored-by: Ezequiel Fernández Ponce <[email protected]> Co-authored-by: Dhia Ayachi <[email protected]> * NET-4996 - filter go-tests and test-integration workflows from running on docs only and ui only changes (#18236) * Align build arch matrix with enterprise (#18235) Ensure that OSS remains in sync w/ Enterprise by aligning the format of arch matrix args for various build jobs. * Revert "NET-4996 - filter go-tests and test-integration workflows from running on docs only and ui only changes" (#18248) Revert "NET-4996 - filter go-tests and test-integration workflows from running on docs only and ui only changes (#18236)" This reverts commit a11dba710e6ce6f172c0fa6c9b61567cc1efffc8. * resource: Add scope to resource type registration [NET-4976] (#18214) Enables querying a resource type's registration to determine if a resource is cluster, partition, or partition and namespace scoped. * Fix some inconsistencies in jwt docs (#18234) * NET-1825: More new ACL token creation docs (#18063) Co-authored-by: trujillo-adam <[email protected]> * grafana: add the panel resource usage of connect injector (#18247) * [NET-3700] Backfill changelog entry for c2bbe67 and 7402d06 (#18259) Backfill changelog entry for c2bbe67 and 7402d06 Add a changelog entry for the follow-up PR since it was specific to the fix and references the original change. * NET-4897 - update comment to include the current issue url from the go team. (#18263) * fix typos, style, and improper links (#18269) * member cli: add -filter expression to flags (#18223) * member cli: add -filter expression to flags * changelog * update doc * Add test cases * use quote * go-tests: disable s390x (#18273) * docs: Update ext-authz documentation for kubernetes (#18281) Update ext-authz documentation for kubernetes * docs: Consul on Kubernetes specific upgrade info (#18230) * Compatibility page - dataplanes mention * Upgrading higher-level dataplane mention * `exec=` string callout * More visible for upgrade page * Apply suggestions from code review Co-authored-by: Tu Nguyen <[email protected]> --------- Co-authored-by: Tu Nguyen <[email protected]> Co-authored-by: John Murret <[email protected]> * Fix typo in Envoy extensions doc (#18284) * docs: K8s secondary DC requirements (#18280) * Requested edit * Apply suggestions from code review Co-authored-by: trujillo-adam <[email protected]> --------- Co-authored-by: trujillo-adam <[email protected]> * api-gateway: subscribe to bound-api-gateway only after receiving api-gateway (#18291) * api-gateway: subscribe to bound-api-gateway only after receiving api-gateway This fixes a race condition due to our dependency on having the listener(s) from the api-gateway config entry in order to fully and properly process the resources on the bound-api-gateway config entry. * Apply suggestions from code review * Add changelog entry * Update K8s changelog to address cloud auto-join change in 1.0.0 (#18293) * Update list of Envoy versions (#18300) Update supported envoy versions to 1.24.10, 1.25.9, 1.26.4, 1.27.0. * [NET-4904] Update list of Envoy versions in docs (#18306) Update list of Envoy versions in docs Update supported Envoy versions across Consul release versions. * Update actions for TSCCR (#18317) Update action versions before deadline * Fix topology view when displaying mixed connect-native/normal services. (#13023) * Fix topoloy intention with mixed connect-native/normal services. If a service is registered twice, once with connect-native and once without, the topology views would prune the existing intentions. This change brings the code more in line with the transparent proxy behavior. * Dedupe nodes in the ServiceTopology ui endpoint (like done with tags). * Consider a service connect-native as soon as one instance is. * Stop JWT provider from being written in non default namespace (#18325) * docs: Fix some comments (#17118) Signed-off-by: cui fliter <[email protected]> * [NET-4703] Prevent partial application of Envoy extensions (#18068) Prevent partial application of Envoy extensions Ensure that non-required extensions do not change xDS resources before exiting on failure by cloning proto messages prior to applying each extension. To support this change, also move `CanApply` checks up a layer and make them prior to attempting extension application, s.t. we avoid unnecessary copies where extensions can't be applied. Last, ensure that we do not allow panics from `CanApply` or `Extend` checks to escape the attempted extension application. * docs: Simplify example jq commands by removing pipes (#18327) Simplify jq command examples by removing pipes to other commands. * fix typo in create a mesh token docs (#18337) * [CC-5719] Add support for builtin global-read-only policy (#18319) * [CC-5719] Add support for builtin global-read-only policy * Add changelog * Add read-only to docs * Fix some minor issues. * Change from ReplaceAll to Sprintf * Change IsValidPolicy name to return an error instead of bool * Fix PolicyList test * Fix other tests * Apply suggestions from code review Co-authored-by: Paul Glass <[email protected]> * Fix state store test for policy list. * Fix naming issues * Update acl/validation.go Co-authored-by: Chris Thain <[email protected]> * Update agent/consul/acl_endpoint.go --------- Co-authored-by: Paul Glass <[email protected]> Co-authored-by: Chris Thain <[email protected]> * [NET-5121] proxystate: move protos to subdirectory to avoid conflicts (#18335) * also makes a few protos updates * resource: adding various helpers for working with resources (#18342) This is a bit of a grab bag of helpers that I found useful for working with them when authoring substantial Controllers. Subsequent PRs will make use of them. * Wasm integration tests for local and remote wasm files (#17756) * wasm integration tests for local and remote wasm files refactoring and cleanup for wasm testing remove wasm debug logging PR feedback, wasm build lock correct path pattern for wasm build files Add new helper function to minimize changes to existing test code Remove extra param mod tidy add custom service setup to test lib add wait until static server sidecar can reach nginx sidecar Doc comments PR feedback Update workflows to compile wasm for integration tests Fix docker build path Fix package name for linter Update makefile, fix redeclared function Update expected wasm filename Debug test ls in workflow remove pwd in favor of relative path more debugging Build wasm in compatability tests as well Build wasm directly in ci rather than in container Debug tinygo and llvm version Change wasm file extension Remove tinygo debugging Remove extra comments * Add compiled wasm and build instructions * [HCP Telemetry] Periodic Refresh for Dynamic Telemetry Configuration (#18168) * OTElExporter now uses an EndpointProvider to discover the endpoint * OTELSink uses a ConfigProvider to obtain filters and labels configuration * improve tests for otel_sink * Regex logic is moved into client for a method on the TelemetryConfig object * Create a telemetry_config_provider and update deps to use it * Fix conversion * fix import newline * Add logger to hcp client and move telemetry_config out of the client.go file * Add a telemetry_config.go to refactor client.go * Update deps * update hcp deps test * Modify telemetry_config_providers * Check for nil filters * PR review updates * Fix comments and move around pieces * Fix comments * Remove context from client struct * Moved ctx out of sink struct and fixed filters, added a test * Remove named imports, use errors.New if not fformatting * Remove HCP dependencies in telemetry package * Add success metric and move lock only to grab the t.cfgHahs * Update hash * fix nits * Create an equals method and add tests * Improve telemetry_config_provider.go tests * Add race test * Add missing godoc * Remove mock for MetricsClient * Avoid goroutine test panics * trying to kick CI lint issues by upgrading mod * imprve test code and add hasher for testing * Use structure logging for filters, fix error constants, and default to allow all regex * removed hashin and modify logic to simplify * Improve race test and fix PR feedback by removing hash equals and avoid testing the timer.Ticker logic, and instead unit test * Ran make go-mod-tidy * Use errtypes in the test * Add changelog * add safety check for exporter endpoint * remove require.Contains by using error types, fix structure logging, and fix success metric typo in exporter * Fixed race test to have changing config values * Send success metric before modifying config * Avoid the defer and move the success metric under * NET-4240 - Snapshots are failing on Windows (#18302) * fix go mod * fix go sum * added change log * ran make go mod tidy * Register ProxyStateTemplate Resource (#18316) Also, change the ProxyState.id to identity. This is because we already have the id of this proxy from the resource, and this id should be name-aligned with the workload it represents. It should also have the owner ref set to the workload ID if we need that. And so the id field seems unnecessary. We do, however, need a reference to workload identity so that we can authorize the proxy when it initially connects to the xDS server. * [NET-5146] security: Update Go version to 1.20.7 and `x/net` to 0.13.0 (#18358) * Update Go version to 1.20.7 This resolves [CVE-2023-29409] (https://nvd.nist.gov/vuln/detail/CVE-2023-29409)(`crypto/tls`). * Bump golang.org/x/net to 0.13.0 Addresses [CVE-2023-3978](https://nvd.nist.gov/vuln/detail/CVE-2023-3978) for security scans (non-impacting). * mesh: adding the protobuf types and resources backing mesh config v2 (#18351) * [OSS] test: xds coverage for routes (#18369) test: xds coverage for routes * Fix policy lookup to allow for slashes (#18347) * Fix policy lookup to allow for slashes * Fix suggestions * Fix other test * Revert some lines * add some initial CODEOWNERS (#18346) * Add redirects for mesh-gateway docs (#18377) * expose grpc as http endpoint (#18221) expose resource grpc endpoints as http endpoints * bimapper: fix a bug and add some more test coverage (#18387) * [docs] Fix ServiceDefaults example in distributed tracing (#17212) Fix ServiceDefaults example in distributed tracing. * Fix #17730 - Dev mode has new line (#18367) * adding new line only in case of pretty in url not in dev mode * change log added * chore: make go-mod-tidy (#18388) * resource: Make resource read tenancy aware (#18397) * Not using chmod - fixed integration test for Enterprise (#18401) * Not using chmod * checking icacls * Removed push * replaced ordered list of rate limit ops with flow diagram (#18398) * replaced ordered list of rate limit ops with flow diagram * made diagram font bigger * updated colors for dark mode img * fixed typo * recentered dark mode image * Apply suggestions from code review Co-authored-by: Jeff Boruszak <[email protected]> --------- Co-authored-by: Jeff Boruszak <[email protected]> * Upgrade test: remove outdated test and disable log due to verbosity (#18403) * remove outdated test * disable log since we have too many parallel tests * bump testcontainers-go from 0.22.0 and remove pinned go version in in… (#18395) * bump testcontainers-go from 0.22.0 and remove pinned go version in integ test * go mod tidy * Replace deprecated target.Authority with target.URL.Host * Add ServiceEndpoints Mutation hook tests (#18404) * Add ServiceEndpoints Mutation hook tests * Move endpoint owner validation into the validation hook Also there were some minor changes to error validation to account for go-cmp not liking to peer through an errors.errorstring type that get created by errors.New * catalog: add FailoverPolicy mutation and validation hooks (#18390) Add most of the validation and mutation hooks for the FailoverPolicy resource. * NET-5115 Add retry + timeout filters for api-gateway (#18324) * squash, implement retry/timeout in consul core * update tests * catalog: adding a controller to reconcile FailoverPolicy resources (#18399) Add most of the semantic cross-resource validation for FailoverPolicy resources using a new controller. * fix the error in ent repo (#18421) fix the error in ent repo * build: updates for 1.16.1 release (#18415) * build: updates for 1.16.1 release * build: fix missing replace directive for envoyextensions * build: go mod tidy testing/deployer * update ECS links (#18419) * docs: fix incorrect proxy-defaults config in Lua Envoy extension (#18418) fix incorrect proxy-defaults config * resource: Make resource write tenancy aware (#18423) * [NET-5217] [OSS] Derive sidecar proxy locality from parent service (#18437) * Add logging to locality policy application In OSS, this is currently a no-op. * Inherit locality when registering sidecars When sidecar locality is not explicitly configured, inherit locality from the proxied service. * NET-4984: Update APIGW Config Entries for JWT Auth (#18366) * Added oss config entries for Policy and JWT on APIGW * Updated structs for config entry * Updated comments, ran deep-copy * Move JWT configuration into OSS file * Add in the config entry OSS file for jwts * Added changelog * fixing proto spacing * Moved to using manually written deep copy method * Use pointers for override/default fields in apigw config entries * Run gen scripts for changed types * k8s compat - Openshift versions (#18307) * Update compatibility.mdx Co-authored-by: trujillo-adam <[email protected]> * APIGW: Update HTTPRouteConfigEntry for JWT Auth (#18422) * Updated httproute config entry for JWT Filters * Added manual deepcopy method for httproute jwt filter * Fix test * Update JWTFilter to be in oss file * Add changelog * Add build tags for deepcopy oss file * [COMPLIANCE] License changes (#18443) * Adding explicit MPL license for sub-package This directory and its subdirectories (packages) contain files licensed with the MPLv2 `LICENSE` file in this directory and are intentionally licensed separately from the BSL `LICENSE` file at the root of this repository. * Adding explicit MPL license for sub-package This directory and its subdirectories (packages) contain files licensed with the MPLv2 `LICENSE` file in this directory and are intentionally licensed separately from the BSL `LICENSE` file at the root of this repository. * Updating the license from MPL to Business Source License Going forward, this project will be licensed under the Business Source License v1.1. Please see our blog post for more details at <Blog URL>, FAQ at www.hashicorp.com/licensing-faq, and details of the license at www.hashicorp.com/bsl. * add missing license headers * Update copyright file headers to BUSL-1.1 * Update copyright file headers to BUSL-1.1 * Update copyright file headers to BUSL-1.1 * Update copyright file headers to BUSL-1.1 * Update copyright file headers to BUSL-1.1 * Update copyright file headers to BUSL-1.1 * Update copyright file headers to BUSL-1.1 * Update copyright file headers to BUSL-1.1 * Update copyright file headers to BUSL-1.1 * Update copyright file headers to BUSL-1.1 * Update copyright file headers to BUSL-1.1 * Update copyright file headers to BUSL-1.1 * Update copyright file headers to BUSL-1.1 * Update copyright file headers to BUSL-1.1 * Update copyright file headers to BUSL-1.1 --------- Co-authored-by: hashicorp-copywrite[bot] <110428419+hashicorp-copywrite[bot]@users.noreply.github.com…
trujillo-adam
added a commit
that referenced
this pull request
Sep 7, 2023
* Update Consul K8s Upgrade Doc Updates (#17921) Updating upgrade procedures to encompass expected errors during upgrade process from v1.13.x to v1.14.x. * Update sameness-group.mdx (#17915) * Update create-sameness-groups.mdx (#17927) * deps: coredns v1.10.1 (#17912) * Ensure RSA keys are at least 2048 bits in length (#17911) * Ensure RSA keys are at least 2048 bits in length * Add changelog * update key length check for FIPS compliance * Fix no new variables error and failing to return when error exists from validating * clean up code for better readability * actually return value * tlsutil: Fix check TLS configuration (#17481) * tlsutil: Fix check TLS configuration * Rewording docs. * Update website/content/docs/services/configuration/checks-configuration-reference.mdx Co-authored-by: trujillo-adam <[email protected]> * Fix typos and add changelog entry. --------- Co-authored-by: trujillo-adam <[email protected]> * docs: Deprecations for connect-native SDK and specific connect native APIs (#17937) * Update v1_16_x.mdx * Update connect native golang page --------- Co-authored-by: trujillo-adam <[email protected]> * Revert "Add workflow to verify linux release packages (#17904)" (#17942) This reverts commit 3368f14fab500ebe9f6aeab5631dd1d5f5a453e5. * Fixes Secondary ConnectCA update (#17846) This fixes a bug that was identified which resulted in subsequent ConnectCA configuration update not to persist in the cluster. * fixing typo in link to jwt-validations-with-intentions doc (#17955) * Fix streaming backend link (#17958) * Fix streaming backend link * Update health.mdx * Dynamically create jwks clusters for jwt-providers (#17944) * website: remove deprecated agent rpc docs (#17962) * Fix missing BalanceOutboundConnections in v2 catalog. (#17964) * feature - [NET - 4005] - [Supportability] Reloadable Configuration - enable_debug (#17565) * # This is a combination of 9 commits. # This is the 1st commit message: init without tests # This is the commit message #2: change log # This is the commit message #3: fix tests # This is the commit message #4: fix tests # This is the commit message #5: added tests # This is the commit message #6: change log breaking change # This is the commit message #7: removed breaking change # This is the commit message #8: fix test # This is the commit message #9: keeping the test behaviour same * # This is a combination of 12 commits. # This is the 1st commit message: init without tests # This is the commit message #2: change log # This is the commit message #3: fix tests # This is the commit message #4: fix tests # This is the commit message #5: added tests # This is the commit message #6: change log breaking change # This is the commit message #7: removed breaking change # This is the commit message #8: fix test # This is the commit message #9: keeping the test behaviour same # This is the commit message #10: made enable debug atomic bool # This is the commit message #11: fix lint # This is the commit message #12: fix test true enable debug * parent 10f500e895d92cc3691ade7b74a33db755d22039 author absolutelightning <[email protected]> 1687352587 +0530 committer absolutelightning <[email protected]> 1687352592 +0530 init without tests change log fix tests fix tests added tests change log breaking change removed breaking change fix test keeping the test behaviour same made enable debug atomic bool fix lint fix test true enable debug using enable debug in agent as atomic bool test fixes fix tests fix tests added update on correct locaiton fix tests fix reloadable config enable debug fix tests fix init and acl 403 * revert commit * Fix formatting codeblocks on APIgw docs (#17970) * fix formatting codeblocks * remove unnecessary indents * Remove POC code (#17974) * update doc (#17910) * update doc * update link * Remove duplicate and unused newDecodeConfigEntry func (#17979) * docs: samenessGroup YAML examples (#17984) * configuration entry syntax * Example config * Add changelog entry for 1.16.0 (#17987) * Fix typo (#17198) servcies => services * Expose JWKS cluster config through JWTProviderConfigEntry (#17978) * Expose JWKS cluster config through JWTProviderConfigEntry * fix typos, rename trustedCa to trustedCA * Integration test for ext-authz Envoy extension (#17980) * Fix incorrect protocol for transparent proxy upstreams. (#17894) This PR fixes a bug that was introduced in: https://github.com/hashicorp/consul/pull/16021 A user setting a protocol in proxy-defaults would cause tproxy implicit upstreams to not honor the upstream service's protocol set in its `ServiceDefaults.Protocol` field, and would instead always use the proxy-defaults value. Due to the fact that upstreams configured with "tcp" can successfully contact upstream "http" services, this issue was not recognized until recently (a proxy-defaults with "tcp" and a listening service with "http" would make successful requests, but not the opposite). As a temporary work-around, users experiencing this issue can explicitly set the protocol on the `ServiceDefaults.UpstreamConfig.Overrides`, which should take precedence. The fix in this PR removes the proxy-defaults protocol from the wildcard upstream that tproxy uses to configure implicit upstreams. When the protocol was included, it would always overwrite the value during discovery chain compilation, which was not correct. The discovery chain compiler also consumes proxy defaults to determine the protocol, so simply excluding it from the wildcard upstream config map resolves the issue. * feat: include nodes count in operator usage endpoint and cli command (#17939) * feat: update operator usage api endpoint to include nodes count * feat: update operator usange cli command to includes nodes count * [OSS] Improve Gateway Test Coverage of Catalog Health (#18011) * fix(cli): remove failing check from 'connect envoy' registration for api gateway * test(integration): add tests to check catalog statsus of gateways on startup * remove extra sleep comment * Update test/integration/consul-container/libs/assert/service.go * changelog * Fixes Traffic rate limitting docs (#17997) * Fix removed service-to-service peering links (#17221) * docs: fix removed service-to-service peering links * docs: extend peering-via-mesh-gateways intro (thanks @trujillo-adam) --------- Co-authored-by: trujillo-adam <[email protected]> * docs: Sameness "beta" warning (#18017) * Warning updates * .x * updated typo in tab heading (#18022) * updated typo in tab heading * updated tab group typo, too * Document that DNS lookups can target cluster peers (#17990) Static DNS lookups, in addition to explicitly targeting a datacenter, can target a cluster peer. This was added in 95dc0c7b301b70a6b955a8b7c9737c9b86f03df6 but didn't make the documentation. The driving function for the change is `parseLocality` here: https://github.com/hashicorp/consul/blob/0b1299c28d8127129d61310ee4280055298438e0/agent/dns_oss.go#L25 The biggest change in this is to adjust the standard lookup syntax to tie `.<datacenter>` to `.dc` as required-together, and to append in the similar `.<cluster-peer>.peer` optional argument, both to A record and SRV record lookups. Co-authored-by: David Yu <[email protected]> * Add first integration test for jwt auth with intention (#18005) * fix stand-in text for name field (#18030) * removed sameness conf entry from failover nav (#18033) * docs - add service sync annotations and k8s service weight annotation (#18032) * Docs for https://github.com/hashicorp/consul-k8s/pull/2293 * remove versions for enterprise features since they are old --------- Co-authored-by: Tu Nguyen <[email protected]> * docs - add jobs use case for service mesh k8s (#18037) * docs - add jobs use case for service mesh k8s * add code blocks * address feedback (#18045) * Add verify server hostname to tls default (#17155) * [OSS] Fix initial_fetch_timeout to wait for all xDS resources (#18024) * fix(connect): set initial_fetch_time to wait indefinitely * changelog * PR feedback 1 * ui: fix typos for peer service imports (#17999) * test: fix FIPS inline cert test message (#18076) * Fix a couple typos in Agent Telemetry Metrics docs (#18080) * Fix metrics docs * Add changelog Signed-off-by: josh <[email protected]> --------- Signed-off-by: josh <[email protected]> * docs updates - cluster peering and virtual services (#18069) * Update route-to-virtual-services.mdx * Update establish-peering.mdx * Update service-mesh-compare.mdx (#17279) grammar change * Update helm docs on main (#18085) * ci: use gotestsum v1.10.1 [NET-4042] (#18088) * Docs: Update proxy lifecycle annotations and consul-dataplane flags (#18075) * Update proxy lifecycle annotations and consul-dataplane flags * Pass configured role name to Vault for AWS auth in Connect CA (#17885) * Docs for dataplane upgrade on k8s (#18051) * Docs for dataplane upgrade on k8s --------- Co-authored-by: David Yu <[email protected]> Co-authored-by: trujillo-adam <[email protected]> * docs - update upgrade index page to not recommend consul leave. (#18100) * Displays Consul version of each nodes in UI nodes section (#17754) * update UINodes and UINodeInfo response with consul-version info added as NodeMeta, fetched from serf members * update test cases TestUINodes, TestUINodeInfo * added nil check for map * add consul-version in local agent node metadata * get consul version from serf member and add this as node meta in catalog register request * updated ui mock response to include consul versions as node meta * updated ui trans and added version as query param to node list route * updates in ui templates to display consul version with filter and sorts * updates in ui - model class, serializers,comparators,predicates for consul version feature * added change log for Consul Version Feature * updated to get version from consul service, if for some reason not available from serf * updated changelog text * updated dependent testcases * multiselection version filter * Update agent/consul/state/catalog.go comments updated Co-authored-by: Jared Kirschner <[email protected]> --------- Co-authored-by: Jared Kirschner <[email protected]> * api gw 1.16 updates (#18081) * api gw 1.16 updates * Apply suggestions from code review Co-authored-by: Jeff Boruszak <[email protected]> * update CodeBlockConfig filename * Apply suggestions from code review Co-authored-by: trujillo-adam <[email protected]> Co-authored-by: Jeff Boruszak <[email protected]> * remove non-standard intentions page * Update website/content/docs/api-gateway/configuration/index.mdx Co-authored-by: trujillo-adam <[email protected]> --------- Co-authored-by: Jeff Boruszak <[email protected]> Co-authored-by: trujillo-adam <[email protected]> * [NET-4103] ci: build s390x (#18067) * ci: build s390x * ci: test s390x * ci: dev build s390x * no GOOS * ent only * build: publish s390x * fix syntax error * fix syntax error again * fix syntax error again x2 * test branch * Move s390x conditionals to step level * remove test branch --------- Co-authored-by: emilymianeil <[email protected]> * :ermahgerd "Sevice Mesh" -> "Service Mesh" (#18116) Just a typo in the docs. * Split pbmesh.UpstreamsConfiguration as a resource out of pbmesh.Upstreams (#17991) Configuration that previously was inlined into the Upstreams resource applies to both explicit and implicit upstreams and so it makes sense to split it out into its own resource. It also has other minor changes: - Renames `proxy.proto` proxy_configuration.proto` - Changes the type of `Upstream.destination_ref` from `pbresource.ID` to `pbresource.Reference` - Adds comments to fields that didn't have them * [NET-4895] ci - api tests and consul container tests error because of dependency bugs with go 1.20.6. Pin go to 1.20.5. (#18124) ### Description The following jobs started failing when go 1.20.6 was released: - `go-test-api-1-19` - `go-test-api-1-20` - `compatibility-integration-tests` - `upgrade-integration-tests` `compatibility-integration-tests` and `compatibility-integration-tests` to this testcontainers issue: https://github.com/testcontainers/testcontainers-go/issues/1359. This issue calls for testcontainers to release a new version when one of their dependencies is fixed. When that is done, we will unpin the go versions in `compatibility-integration-tests` and `compatibility-integration-tests`. ### Testing & Reproduction steps See these jobs broken in CI and then see them work with this PR. --------- Co-authored-by: Chris Thain <[email protected]> * Add ingress gateway deprecation notices to docs (#18102) ### Description This adds notices, that ingress gateway is deprecated, to several places in the product docs where ingress gateway is the topic. ### Testing & Reproduction steps Tested with a local copy of the website. ### Links Deprecation of ingress gateway was announced in the Release Notes for Consul 1.16 and Consul-K8s 1.2. See: [https://developer.hashicorp.com/consul/docs/release-notes/consul/v1_16_x#what-s-deprecated](https://developer.hashicorp.com/consul/docs/release-notes/consul/v1_16_x#what-s-deprecated ) [https://developer.hashicorp.com/consul/docs/release-notes/consul-k8s/v1_2_x#what-s-deprecated](https://developer.hashicorp.com/consul/docs/release-notes/consul-k8s/v1_2_x#what-s-deprecated) ### PR Checklist * [N/A] updated test coverage * [X] external facing docs updated * [X] appropriate backport labels added * [X] not a security concern --------- Co-authored-by: trujillo-adam <[email protected]> * Add docs for jwt cluster configuration (#18004) ### Description <!-- Please describe why you're making this change, in plain English. --> - Add jwt-provider docs for jwks cluster configuration. The configuration was added here: https://github.com/hashicorp/consul/pull/17978 * Docs: fix unmatched bracket for health checks page (#18134) * NET-4657/add resource service client (#18053) ### Description <!-- Please describe why you're making this change, in plain English. --> Dan had already started on this [task](https://github.com/hashicorp/consul/pull/17849) which is needed to start building the HTTP APIs. This just needed some cleanup to get it ready for review. Overview: - Rename `internalResourceServiceClient` to `insecureResourceServiceClient` for name consistency - Configure a `secureResourceServiceClient` with auth enabled ### PR Checklist * [ ] ~updated test coverage~ * [ ] ~external facing docs updated~ * [x] appropriate backport labels added * [ ] ~not a security concern~ * Fix bug with Vault CA provider (#18112) Updating RootPKIPath but not IntermediatePKIPath would not update leaf signing certs with the new root. Unsure if this happens in practice but manual testing showed it is a bug that would break mesh and agent connections once the old root is pruned. * [NET-4897] net/http host header is now verified and request.host that contains socked now error (#18129) ### Description This is related to https://github.com/hashicorp/consul/pull/18124 where we pinned the go versions in CI to 1.20.5 and 1.19.10. go 1.20.6 and 1.19.11 now validate request host headers for validity, including the hostname cannot be prefixed with slashes. For local communications (npipe://, unix://), the hostname is not used, but we need valid and meaningful hostname. Prior versions go Go would clean the host header, and strip slashes in the process, but go1.20.6 and go1.19.11 no longer do, and reject the host header. Around the community we are seeing that others are intercepting the req.host and if it starts with a slash or ends with .sock, they changing the host to localhost or another dummy value. [client: define a "dummy" hostname to use for local connections by thaJeztah · Pull Request #45942 · moby/moby](https://github.com/moby/moby/pull/45942) ### Testing & Reproduction steps Check CI tests. ### Links * [ ] updated test coverage * [ ] external facing docs updated * [ ] appropriate backport labels added * [ ] not a security concern * add a conditional around setting LANFilter.AllSegments to make sure it is valid (#18139) ### Description This is to correct a code problem because this assumes all segments, but when you get to Enterprise, you can be in partition that is not the default partition, in which case specifying all segments does not validate and fails. This is to correct the setting of this filter with `AllSegments` to `true` to only occur when in the the `default` partition. ### Testing & Reproduction steps <!-- * In the case of bugs, describe how to replicate * If any manual tests were done, document the steps and the conditions to replicate * Call out any important/ relevant unit tests, e2e tests or integration tests you have added or are adding --> ### Links <!-- Include any links here that might be helpful for people reviewing your PR (Tickets, GH issues, API docs, external benchmarks, tools docs, etc). If there are none, feel free to delete this section. Please be mindful not to leak any customer or confidential information. HashiCorp employees may want to use our internal URL shortener to obfuscate links. --> ### PR Checklist * [ ] updated test coverage * [ ] external facing docs updated * [ ] appropriate backport labels added * [ ] not a security concern * chore: bump upgrade integrations tests to 1.15, 116 [NET-4743] (#18130) * re org resource type registry (#18133) * fix: update delegateMock used in ENT (#18149) ### Description <!-- Please describe why you're making this change, in plain English. --> The mock is used in `http_ent_test` file which caused lint failures. For OSS->ENT parity adding the same change here. ### Links <!-- Include any links here that might be helpful for people reviewing your PR (Tickets, GH issues, API docs, external benchmarks, tools docs, etc). If there are none, feel free to delete this section. Please be mindful not to leak any customer or confidential information. HashiCorp employees may want to use our internal URL shortener to obfuscate links. --> Identified in OSS->ENT [merge PR](https://github.com/hashicorp/consul-enterprise/pull/6328) ### PR Checklist * [ ] ~updated test coverage~ * [ ] ~external facing docs updated~ * [x] appropriate backport labels added * [ ] ~not a security concern~ * Use JWT-auth filter in metadata mode & Delegate validation to RBAC filter (#18062) ### Description <!-- Please describe why you're making this change, in plain English. --> - Currently the jwt-auth filter doesn't take into account the service identity when validating jwt-auth, it only takes into account the path and jwt provider during validation. This causes issues when multiple source intentions restrict access to an endpoint with different JWT providers. - To fix these issues, rather than use the JWT auth filter for validation, we use it in metadata mode and allow it to forward the successful validated JWT token payload to the RBAC filter which will make the decisions. This PR ensures requests with and without JWT tokens successfully go through the jwt-authn filter. The filter however only forwards the data for successful/valid tokens. On the RBAC filter level, we check the payload for claims and token issuer + existing rbac rules. ### Testing & Reproduction steps <!-- * In the case of bugs, describe how to replicate * If any manual tests were done, document the steps and the conditions to replicate * Call out any important/ relevant unit tests, e2e tests or integration tests you have added or are adding --> - This test covers a multi level jwt requirements (requirements at top level and permissions level). It also assumes you have envoy running, you have a redis and a sidecar proxy service registered, and have a way to generate jwks with jwt. I mostly use: https://www.scottbrady91.com/tools/jwt for this. - first write your proxy defaults ``` Kind = "proxy-defaults" name = "global" config { protocol = "http" } ``` - Create two providers ``` Kind = "jwt-provider" Name = "auth0" Issuer = "https://ronald.local" JSONWebKeySet = { Local = { JWKS = "eyJrZXlzIjog....." } } ``` ``` Kind = "jwt-provider" Name = "okta" Issuer = "https://ronald.local" JSONWebKeySet = { Local = { JWKS = "eyJrZXlzIjogW3...." } } ``` - add a service intention ``` Kind = "service-intentions" Name = "redis" JWT = { Providers = [ { Name = "okta" }, ] } Sources = [ { Name = "*" Permissions = [{ Action = "allow" HTTP = { PathPrefix = "/workspace" } JWT = { Providers = [ { Name = "okta" VerifyClaims = [ { Path = ["aud"] Value = "my_client_app" }, { Path = ["sub"] Value = "5be86359073c434bad2da3932222dabe" } ] }, ] } }, { Action = "allow" HTTP = { PathPrefix = "/" } JWT = { Providers = [ { Name = "auth0" }, ] } }] } ] ``` - generate 3 jwt tokens: 1 from auth0 jwks, 1 from okta jwks with different claims than `/workspace` expects and 1 with correct claims - connect to your envoy (change service and address as needed) to view logs and potential errors. You can add: `-- --log-level debug` to see what data is being forwarded ``` consul connect envoy -sidecar-for redis1 -grpc-addr 127.0.0.1:8502 ``` - Make the following requests: ``` curl -s -H "Authorization: Bearer $Auth0_TOKEN" --insecure --cert leaf.cert --key leaf.key --cacert connect-ca.pem https://localhost:20000/workspace -v RBAC filter denied curl -s -H "Authorization: Bearer $Okta_TOKEN_with_wrong_claims" --insecure --cert leaf.cert --key leaf.key --cacert connect-ca.pem https://localhost:20000/workspace -v RBAC filter denied curl -s -H "Authorization: Bearer $Okta_TOKEN_with_correct_claims" --insecure --cert leaf.cert --key leaf.key --cacert connect-ca.pem https://localhost:20000/workspace -v Successful request ``` ### TODO * [x] Update test coverage * [ ] update integration tests (follow-up PR) * [x] appropriate backport labels added * Support Consul Connect Envoy Command on Windows (#17694) ### Description Add support for consul connect envoy command on windows. This PR fixes the comments of PR - https://github.com/hashicorp/consul/pull/15114 ### Testing * Built consul.exe from this branch on windows and hosted here - [AWS S3](https://asheshvidyut-bucket.s3.ap-southeast-2.amazonaws.com/consul.zip) * Updated the [tutorial](https://developer.hashicorp.com/consul/tutorials/developer-mesh/consul-windows-workloads) and changed the `consul_url.default` value to [AWS S3](https://asheshvidyut-bucket.s3.ap-southeast-2.amazonaws.com/consul.zip) * Followed the steps in the tutorial and verified that everything is working as described. ### PR Checklist * [x] updated test coverage * [ ] external facing docs updated * [x] appropriate backport labels added * [x] not a security concern --------- Co-authored-by: Franco Bruno Lavayen <[email protected]> Co-authored-by: Jose Ignacio Lorenzo <[email protected]> Co-authored-by: Jose Ignacio Lorenzo <[email protected]> Co-authored-by: Dhia Ayachi <[email protected]> * Change docs to say 168h instead of 7d for server_rejoin_age_max (#18154) ### Description Addresses https://github.com/hashicorp/consul/pull/17171#issuecomment-1636930705 * [OSS] test: improve xDS listener code coverage (#18138) test: improve xDS listener code coverage * Re-order expected/actual for assertContainerState in consul container tests (#18157) Re-order expected/actual, consul container tests * group and document make file (#17943) * group and document make file * Add `testing/deployer` (neé `consul-topology`) [NET-4610] (#17823) Co-authored-by: R.B. Boyer <[email protected]> Co-authored-by: R.B. Boyer <[email protected]> Co-authored-by: Freddy <[email protected]> * [NET-4792] Add integrations tests for jwt-auth (#18169) * Add FIPS reference to consul enterprise docs (#18028) * Add FIPS reference to consul enterprise docs * Update website/content/docs/enterprise/index.mdx Co-authored-by: David Yu <[email protected]> * remove support for ecs client (fips) --------- Co-authored-by: David Yu <[email protected]> * add peering_commontopo tests [NET-3700] (#17951) Co-authored-by: R.B. Boyer <[email protected]> Co-authored-by: R.B. Boyer <[email protected]> Co-authored-by: Freddy <[email protected]> Co-authored-by: NiniOak <[email protected]> * docs - remove Sentinel from enterprise features list (#18176) * Update index.mdx * Update kv.mdx * Update docs-nav-data.json * delete sentinel.mdx * Update redirects.js --------- Co-authored-by: Tu Nguyen <[email protected]> * [NET-4865] Bump golang.org/x/net to 0.12.0 (#18186) Bump golang.org/x/net to 0.12.0 While not necessary to directly address CVE-2023-29406 (which should be handled by using a patched version of Go when building), an accompanying change to HTTP/2 error handling does impact agent code. See https://go-review.googlesource.com/c/net/+/506995 for the HTTP/2 change. Bump this dependency across our submodules as well for the sake of potential indirect consumers of `x/net/http`. * Call resource mutate hook before validate hook (NET-4907) (#18178) * [NET-4865] security: Update Go version to 1.20.6 (#18190) Update Go version to 1.20.6 This resolves [CVE-2023-29406] (https://nvd.nist.gov/vuln/detail/CVE-2023-29406) for uses of the `net/http` standard library. Note that until the follow-up to #18124 is done, the version of Go used in those impacted tests will need to remain on 1.20.5. * Improve XDS test coverage: JWT auth edition (#18183) * Improve XDS test coverage: JWT auth edition more tests * test: xds coverage for jwt listeners --------- Co-authored-by: DanStough <[email protected]> * update readme.md (#18191) u[date readme.md * Update submodules to latest following 1.16.0 (#18197) Align all our internal use of submodules on the latest versions. * SEC-090: Automated trusted workflow pinning (2023-07-18) (#18174) Result of tsccr-helper -log-level=info -pin-all-workflows . Co-authored-by: hashicorp-tsccr[bot] <hashicorp-tsccr[bot]@users.noreply.github.com> * Fix Backport Assistant PR commenting (#18200) * Fix Backport Assistant failure PR commenting For general comments on a PR, it looks like you have to use the `/issue` endpoint rather than `/pulls`, which requires commit/other review-specific target details. This matches the endpoint used in `backport-reminder.yml`. * Remove Backport Reminder workflow This is noisy (even when adding multiple labels, individual comments per label are generated), and likely no longer needed: we haven't had this work in a long time due to an expired GH token, and we now have better automation for backport PR assignment. * resource: Pass resource to Write ACL hook instead of just resource Id [NET-4908] (#18192) * Explicitly enable WebSocket upgrades (#18150) This PR explicitly enables WebSocket upgrades in Envoy's UpgradeConfig for all proxy types. (API Gateway, Ingress, and Sidecar.) Fixes #8283 * docs: fix the description of client rpc (#18206) * NET-4804: Add dashboard for monitoring consul-k8s (#18208) * [OSS] Improve xDS Code Coverage - Clusters (#18165) test: improve xDS cluster code coverage * NET-4222 take config file consul container (#18218) Net 4222 take config file consul container * Envoy Integration Test Windows (#18007) * [CONSUL-395] Update check_hostport and Usage (#40) * [CONSUL-397] Copy envoy binary from Image (#41) * [CONSUL-382] Support openssl in unique test dockerfile (#43) * [CONSUL-405] Add bats to single container (#44) * [CONSUL-414] Run Prometheus Test Cases and Validate Changes (#46) * [CONSUL-410] Run Jaeger in Single container (#45) * [CONSUL-412] Run test-sds-server in single container (#48) * [CONSUL-408] Clean containers (#47) * [CONSUL-384] Rebase and sync fork (#50) * [CONSUL-415] Create Scenarios Troubleshooting Docs (#49) * [CONSUL-417] Update Docs Single Container (#51) * [CONSUL-428] Add Socat to single container (#54) * [CONSUL-424] Replace pkill in kill_envoy function (#52) * [CONSUL-434] Modify Docker run functions in Helper script (#53) * [CONSUL-435] Replace docker run in set_ttl_check_state & wait_for_agent_service_register functions (#55) * [CONSUL-438] Add netcat (nc) in the Single container Dockerfile (#56) * [CONSUL-429] Replace Docker run with Docker exec (#57) * [CONSUL-436] Curl timeout and run tests (#58) * [CONSUL-443] Create dogstatsd Function (#59) * [CONSUL-431] Update Docs Netcat (#60) * [CONSUL-439] Parse nc Command in function (#61) * [CONSUL-463] Review curl Exec and get_ca_root Func (#63) * [CONSUL-453] Docker hostname in Helper functions (#64) * [CONSUL-461] Test wipe volumes without extra cont (#66) * [CONSUL-454] Check ports in the Server and Agent containers (#65) * [CONSUL-441] Update windows dockerfile with version (#62) * [CONSUL-466] Review case-grpc Failing Test (#67) * [CONSUL-494] Review case-cfg-resolver-svc-failover (#68) * [CONSUL-496] Replace docker_wget & docker_curl (#69) * [CONSUL-499] Cleanup Scripts - Remove nanoserver (#70) * [CONSUL-500] Update Troubleshooting Docs (#72) * [CONSUL-502] Pull & Tag Envoy Windows Image (#73) * [CONSUL-504] Replace docker run in docker_consul (#76) * [CONSUL-505] Change admin_bind * [CONSUL-399] Update envoy to 1.23.1 (#78) * [CONSUL-510] Support case-wanfed-gw on Windows (#79) * [CONSUL-506] Update troubleshooting Documentation (#80) * [CONSUL-512] Review debug_dump_volumes Function (#81) * [CONSUL-514] Add zipkin to Docker Image (#82) * [CONSUL-515] Update Documentation (#83) * [CONSUL-529] Support case-consul-exec (#86) * [CONSUL-530] Update Documentation (#87) * [CONSUL-530] Update default consul version 1.13.3 * [CONSUL-539] Cleanup (#91) * [CONSUL-546] Scripts Clean-up (#92) * [CONSUL-491] Support admin_access_log_path value for Windows (#71) * [CONSUL-519] Implement mkfifo Alternative (#84) * [CONSUL-542] Create OS Specific Files for Envoy Package (#88) * [CONSUL-543] Create exec_supported.go (#89) * [CONSUL-544] Test and Build Changes (#90) * Implement os.DevNull * using mmap instead of disk files * fix import in exec-unix * fix nmap open too many arguemtn * go fmt on file * changelog file * fix go mod * Update .changelog/17694.txt Co-authored-by: Dhia Ayachi <[email protected]> * different mmap library * fix bootstrap json * some fixes * chocolatey version fix and image fix * using different library * fix Map funciton call * fix mmap call * fix tcp dump * fix tcp dump * windows tcp dump * Fix docker run * fix tests * fix go mod * fix version 16.0 * fix version * fix version dev * sleep to debug * fix sleep * fix permission issue * fix permission issue * fix permission issue * fix command * fix command * fix funciton * fix assert config entry status command not found * fix command not found assert_cert_has_cn * fix command not found assert_upstream_missing * fix command not found assert_upstream_missing_once * fix command not found get_upstream_endpoint * fix command not found get_envoy_public_listener_once * fix command not found * fix test cases * windows integration test workflow github * made code similar to unix using npipe * fix go.mod * fix dialing of npipe * dont wait * check size of written json * fix undefined n * running * fix dep * fix syntax error * fix workflow file * windows runner * fix runner * fix from json * fix runs on * merge connect envoy * fix cin path * build * fix file name * fix file name * fix dev build * remove unwanted code * fix upload * fix bin name * fix path * checkout current branch * fix path * fix tests * fix shell bash for windows sh files * fix permission of run-test.sh * removed docker dev * added shell bash for tests * fix tag * fix win=true * fix cd * added dev * fix variable undefined * removed failing tests * fix tcp dump image * fix curl * fix curl * tcp dump path * fix tcpdump path * fix curl * fix curl install * stop removing intermediate containers * fix tcpdump docker image * revert -rm * --rm=false * makeing docker image before * fix tcpdump * removed case consul exec * removed terminating gateway simple * comment case wasm * removed data dog * comment out upload coverage * uncomment case-consul-exec * comment case consul exec * if always * logs * using consul 1.17.0 * fix quotes * revert quotes * redirect to dev null * Revert version * revert consul connect * fix version * removed envoy connect * not using function * change log * docker logs * fix logs * restructure bad authz * rmeoved dev null * output * fix file descriptor * fix cacert * fix cacert * fix ca cert * cacert does not work in windows curl * fix func * removed docker logs * added sleep * fix tls * commented case-consul-exec * removed echo * retry docker consul * fix upload bin * uncomment consul exec * copying consul.exe to docker image * copy fix * fix paths * fix path * github workspace path * latest version * Revert "latest version" This reverts commit 5a7d7b82d9e7553bcb01b02557ec8969f9deba1d. * commented consul exec * added ssl revoke best effort * revert best effort * removed unused files * rename var name and change dir * windows runner * permission * needs setup fix * swtich to github runner * fix file path * fix path * fix path * fix path * fix path * fix path * fix build paths * fix tag * nightly runs * added matrix in github workflow, renamed files * fix job * fix matrix * removed brackes * from json * without using job matrix * fix quotes * revert job matrix * fix workflow * fix comment * added comment * nightly runs * removed datadog ci as it is already measured in linux one * running test * Revert "running test" This reverts commit 7013d15a23732179d18ec5d17336e16b26fab5d4. * pr comment fixes * running test now * running subset of test * running subset of test * job matrix * shell bash * removed bash shell * linux machine for job matrix * fix output * added cat to debug * using ubuntu latest * fix job matrix * fix win true * fix go test * revert job matrix --------- Co-authored-by: Jose Ignacio Lorenzo <[email protected]> Co-authored-by: Franco Bruno Lavayen <[email protected]> Co-authored-by: Ivan K Berlot <[email protected]> Co-authored-by: Ezequiel Fernández Ponce <[email protected]> Co-authored-by: joselo85 <[email protected]> Co-authored-by: Ezequiel Fernández Ponce <[email protected]> Co-authored-by: Dhia Ayachi <[email protected]> * fix typos and update ecs compat table (#18215) * fix typos and update ecs compat table * real info for the ecs compat matrix table * Update website/content/docs/ecs/compatibility.mdx Co-authored-by: Chris Thain <[email protected]> --------- Co-authored-by: Chris Thain <[email protected]> * [OSS] proxystate: add proxystate protos (#18216) * proxystate: add proxystate protos to pbmesh and resolve imports and conflicts between message names * ci: don't verify s390x (#18224) * [CC-5718] Remove HCP token requirement during bootstrap (#18140) * [CC-5718] Remove HCP token requirement during bootstrap * Re-add error for loading HCP management token * Remove old comment * Add changelog entry * Remove extra validation line * Apply suggestions from code review Co-authored-by: lornasong <[email protected]> --------- Co-authored-by: lornasong <[email protected]> * [NET-4122] Doc guidance for federation with externalServers (#18207) Doc guidance for federation with externalServers Add guidance for proper configuration when joining to a secondary cluster using WAN fed with external servers also enabled. Also clarify federation requirements and fix formatting for an unrelated value. Update both the Helm chart reference (synced from `consul-k8s`, see hashicorp/consul-k8s#2583) and the docs on using `externalServers`. * [OSS] Improve xDS Code Coverage - Endpoints and Misc (#18222) test: improve xDS endpoints code coverage * Clarify license reporting timing and GDPR compliance (#18237) Add Alicia's edits to clarify log timing and other details * Fix Github Workflow File (#18241) * [CONSUL-382] Support openssl in unique test dockerfile (#43) * [CONSUL-405] Add bats to single container (#44) * [CONSUL-414] Run Prometheus Test Cases and Validate Changes (#46) * [CONSUL-410] Run Jaeger in Single container (#45) * [CONSUL-412] Run test-sds-server in single container (#48) * [CONSUL-408] Clean containers (#47) * [CONSUL-384] Rebase and sync fork (#50) * [CONSUL-415] Create Scenarios Troubleshooting Docs (#49) * [CONSUL-417] Update Docs Single Container (#51) * [CONSUL-428] Add Socat to single container (#54) * [CONSUL-424] Replace pkill in kill_envoy function (#52) * [CONSUL-434] Modify Docker run functions in Helper script (#53) * [CONSUL-435] Replace docker run in set_ttl_check_state & wait_for_agent_service_register functions (#55) * [CONSUL-438] Add netcat (nc) in the Single container Dockerfile (#56) * [CONSUL-429] Replace Docker run with Docker exec (#57) * [CONSUL-436] Curl timeout and run tests (#58) * [CONSUL-443] Create dogstatsd Function (#59) * [CONSUL-431] Update Docs Netcat (#60) * [CONSUL-439] Parse nc Command in function (#61) * [CONSUL-463] Review curl Exec and get_ca_root Func (#63) * [CONSUL-453] Docker hostname in Helper functions (#64) * [CONSUL-461] Test wipe volumes without extra cont (#66) * [CONSUL-454] Check ports in the Server and Agent containers (#65) * [CONSUL-441] Update windows dockerfile with version (#62) * [CONSUL-466] Review case-grpc Failing Test (#67) * [CONSUL-494] Review case-cfg-resolver-svc-failover (#68) * [CONSUL-496] Replace docker_wget & docker_curl (#69) * [CONSUL-499] Cleanup Scripts - Remove nanoserver (#70) * [CONSUL-500] Update Troubleshooting Docs (#72) * [CONSUL-502] Pull & Tag Envoy Windows Image (#73) * [CONSUL-504] Replace docker run in docker_consul (#76) * [CONSUL-505] Change admin_bind * [CONSUL-399] Update envoy to 1.23.1 (#78) * [CONSUL-510] Support case-wanfed-gw on Windows (#79) * [CONSUL-506] Update troubleshooting Documentation (#80) * [CONSUL-512] Review debug_dump_volumes Function (#81) * [CONSUL-514] Add zipkin to Docker Image (#82) * [CONSUL-515] Update Documentation (#83) * [CONSUL-529] Support case-consul-exec (#86) * [CONSUL-530] Update Documentation (#87) * [CONSUL-530] Update default consul version 1.13.3 * [CONSUL-539] Cleanup (#91) * [CONSUL-546] Scripts Clean-up (#92) * [CONSUL-491] Support admin_access_log_path value for Windows (#71) * [CONSUL-519] Implement mkfifo Alternative (#84) * [CONSUL-542] Create OS Specific Files for Envoy Package (#88) * [CONSUL-543] Create exec_supported.go (#89) * [CONSUL-544] Test and Build Changes (#90) * Implement os.DevNull * using mmap instead of disk files * fix import in exec-unix * fix nmap open too many arguemtn * go fmt on file * changelog file * fix go mod * Update .changelog/17694.txt Co-authored-by: Dhia Ayachi <[email protected]> * different mmap library * fix bootstrap json * some fixes * chocolatey version fix and image fix * using different library * fix Map funciton call * fix mmap call * fix tcp dump * fix tcp dump * windows tcp dump * Fix docker run * fix tests * fix go mod * fix version 16.0 * fix version * fix version dev * sleep to debug * fix sleep * fix permission issue * fix permission issue * fix permission issue * fix command * fix command * fix funciton * fix assert config entry status command not found * fix command not found assert_cert_has_cn * fix command not found assert_upstream_missing * fix command not found assert_upstream_missing_once * fix command not found get_upstream_endpoint * fix command not found get_envoy_public_listener_once * fix command not found * fix test cases * windows integration test workflow github * made code similar to unix using npipe * fix go.mod * fix dialing of npipe * dont wait * check size of written json * fix undefined n * running * fix dep * fix syntax error * fix workflow file * windows runner * fix runner * fix from json * fix runs on * merge connect envoy * fix cin path * build * fix file name * fix file name * fix dev build * remove unwanted code * fix upload * fix bin name * fix path * checkout current branch * fix path * fix tests * fix shell bash for windows sh files * fix permission of run-test.sh * removed docker dev * added shell bash for tests * fix tag * fix win=true * fix cd * added dev * fix variable undefined * removed failing tests * fix tcp dump image * fix curl * fix curl * tcp dump path * fix tcpdump path * fix curl * fix curl install * stop removing intermediate containers * fix tcpdump docker image * revert -rm * --rm=false * makeing docker image before * fix tcpdump * removed case consul exec * removed terminating gateway simple * comment case wasm * removed data dog * comment out upload coverage * uncomment case-consul-exec * comment case consul exec * if always * logs * using consul 1.17.0 * fix quotes * revert quotes * redirect to dev null * Revert version * revert consul connect * fix version * removed envoy connect * not using function * change log * docker logs * fix logs * restructure bad authz * rmeoved dev null * output * fix file descriptor * fix cacert * fix cacert * fix ca cert * cacert does not work in windows curl * fix func * removed docker logs * added sleep * fix tls * commented case-consul-exec * removed echo * retry docker consul * fix upload bin * uncomment consul exec * copying consul.exe to docker image * copy fix * fix paths * fix path * github workspace path * latest version * Revert "latest version" This reverts commit 5a7d7b82d9e7553bcb01b02557ec8969f9deba1d. * commented consul exec * added ssl revoke best effort * revert best effort * removed unused files * rename var name and change dir * windows runner * permission * needs setup fix * swtich to github runner * fix file path * fix path * fix path * fix path * fix path * fix path * fix build paths * fix tag * nightly runs * added matrix in github workflow, renamed files * fix job * fix matrix * removed brackes * from json * without using job matrix * fix quotes * revert job matrix * fix workflow * fix comment * added comment * nightly runs * removed datadog ci as it is already measured in linux one * running test * Revert "running test" This reverts commit 7013d15a23732179d18ec5d17336e16b26fab5d4. * pr comment fixes * running test now * running subset of test * running subset of test * job matrix * shell bash * removed bash shell * linux machine for job matrix * fix output * added cat to debug * using ubuntu latest * fix job matrix * fix win true * fix go test * revert job matrix * Fix tests --------- Co-authored-by: Ivan K Berlot <[email protected]> Co-authored-by: Jose Ignacio Lorenzo <[email protected]> Co-authored-by: Franco Bruno Lavayen <[email protected]> Co-authored-by: Ezequiel Fernández Ponce <[email protected]> Co-authored-by: joselo85 <[email protected]> Co-authored-by: Ezequiel Fernández Ponce <[email protected]> Co-authored-by: Dhia Ayachi <[email protected]> * NET-4996 - filter go-tests and test-integration workflows from running on docs only and ui only changes (#18236) * Align build arch matrix with enterprise (#18235) Ensure that OSS remains in sync w/ Enterprise by aligning the format of arch matrix args for various build jobs. * Revert "NET-4996 - filter go-tests and test-integration workflows from running on docs only and ui only changes" (#18248) Revert "NET-4996 - filter go-tests and test-integration workflows from running on docs only and ui only changes (#18236)" This reverts commit a11dba710e6ce6f172c0fa6c9b61567cc1efffc8. * resource: Add scope to resource type registration [NET-4976] (#18214) Enables querying a resource type's registration to determine if a resource is cluster, partition, or partition and namespace scoped. * Fix some inconsistencies in jwt docs (#18234) * NET-1825: More new ACL token creation docs (#18063) Co-authored-by: trujillo-adam <[email protected]> * grafana: add the panel resource usage of connect injector (#18247) * [NET-3700] Backfill changelog entry for c2bbe67 and 7402d06 (#18259) Backfill changelog entry for c2bbe67 and 7402d06 Add a changelog entry for the follow-up PR since it was specific to the fix and references the original change. * NET-4897 - update comment to include the current issue url from the go team. (#18263) * fix typos, style, and improper links (#18269) * member cli: add -filter expression to flags (#18223) * member cli: add -filter expression to flags * changelog * update doc * Add test cases * use quote * go-tests: disable s390x (#18273) * docs: Update ext-authz documentation for kubernetes (#18281) Update ext-authz documentation for kubernetes * docs: Consul on Kubernetes specific upgrade info (#18230) * Compatibility page - dataplanes mention * Upgrading higher-level dataplane mention * `exec=` string callout * More visible for upgrade page * Apply suggestions from code review Co-authored-by: Tu Nguyen <[email protected]> --------- Co-authored-by: Tu Nguyen <[email protected]> Co-authored-by: John Murret <[email protected]> * Fix typo in Envoy extensions doc (#18284) * docs: K8s secondary DC requirements (#18280) * Requested edit * Apply suggestions from code review Co-authored-by: trujillo-adam <[email protected]> --------- Co-authored-by: trujillo-adam <[email protected]> * api-gateway: subscribe to bound-api-gateway only after receiving api-gateway (#18291) * api-gateway: subscribe to bound-api-gateway only after receiving api-gateway This fixes a race condition due to our dependency on having the listener(s) from the api-gateway config entry in order to fully and properly process the resources on the bound-api-gateway config entry. * Apply suggestions from code review * Add changelog entry * Update K8s changelog to address cloud auto-join change in 1.0.0 (#18293) * Update list of Envoy versions (#18300) Update supported envoy versions to 1.24.10, 1.25.9, 1.26.4, 1.27.0. * [NET-4904] Update list of Envoy versions in docs (#18306) Update list of Envoy versions in docs Update supported Envoy versions across Consul release versions. * Update actions for TSCCR (#18317) Update action versions before deadline * Fix topology view when displaying mixed connect-native/normal services. (#13023) * Fix topoloy intention with mixed connect-native/normal services. If a service is registered twice, once with connect-native and once without, the topology views would prune the existing intentions. This change brings the code more in line with the transparent proxy behavior. * Dedupe nodes in the ServiceTopology ui endpoint (like done with tags). * Consider a service connect-native as soon as one instance is. * Stop JWT provider from being written in non default namespace (#18325) * docs: Fix some comments (#17118) Signed-off-by: cui fliter <[email protected]> * [NET-4703] Prevent partial application of Envoy extensions (#18068) Prevent partial application of Envoy extensions Ensure that non-required extensions do not change xDS resources before exiting on failure by cloning proto messages prior to applying each extension. To support this change, also move `CanApply` checks up a layer and make them prior to attempting extension application, s.t. we avoid unnecessary copies where extensions can't be applied. Last, ensure that we do not allow panics from `CanApply` or `Extend` checks to escape the attempted extension application. * docs: Simplify example jq commands by removing pipes (#18327) Simplify jq command examples by removing pipes to other commands. * fix typo in create a mesh token docs (#18337) * [CC-5719] Add support for builtin global-read-only policy (#18319) * [CC-5719] Add support for builtin global-read-only policy * Add changelog * Add read-only to docs * Fix some minor issues. * Change from ReplaceAll to Sprintf * Change IsValidPolicy name to return an error instead of bool * Fix PolicyList test * Fix other tests * Apply suggestions from code review Co-authored-by: Paul Glass <[email protected]> * Fix state store test for policy list. * Fix naming issues * Update acl/validation.go Co-authored-by: Chris Thain <[email protected]> * Update agent/consul/acl_endpoint.go --------- Co-authored-by: Paul Glass <[email protected]> Co-authored-by: Chris Thain <[email protected]> * [NET-5121] proxystate: move protos to subdirectory to avoid conflicts (#18335) * also makes a few protos updates * resource: adding various helpers for working with resources (#18342) This is a bit of a grab bag of helpers that I found useful for working with them when authoring substantial Controllers. Subsequent PRs will make use of them. * Wasm integration tests for local and remote wasm files (#17756) * wasm integration tests for local and remote wasm files refactoring and cleanup for wasm testing remove wasm debug logging PR feedback, wasm build lock correct path pattern for wasm build files Add new helper function to minimize changes to existing test code Remove extra param mod tidy add custom service setup to test lib add wait until static server sidecar can reach nginx sidecar Doc comments PR feedback Update workflows to compile wasm for integration tests Fix docker build path Fix package name for linter Update makefile, fix redeclared function Update expected wasm filename Debug test ls in workflow remove pwd in favor of relative path more debugging Build wasm in compatability tests as well Build wasm directly in ci rather than in container Debug tinygo and llvm version Change wasm file extension Remove tinygo debugging Remove extra comments * Add compiled wasm and build instructions * [HCP Telemetry] Periodic Refresh for Dynamic Telemetry Configuration (#18168) * OTElExporter now uses an EndpointProvider to discover the endpoint * OTELSink uses a ConfigProvider to obtain filters and labels configuration * improve tests for otel_sink * Regex logic is moved into client for a method on the TelemetryConfig object * Create a telemetry_config_provider and update deps to use it * Fix conversion * fix import newline * Add logger to hcp client and move telemetry_config out of the client.go file * Add a telemetry_config.go to refactor client.go * Update deps * update hcp deps test * Modify telemetry_config_providers * Check for nil filters * PR review updates * Fix comments and move around pieces * Fix comments * Remove context from client struct * Moved ctx out of sink struct and fixed filters, added a test * Remove named imports, use errors.New if not fformatting * Remove HCP dependencies in telemetry package * Add success metric and move lock only to grab the t.cfgHahs * Update hash * fix nits * Create an equals method and add tests * Improve telemetry_config_provider.go tests * Add race test * Add missing godoc * Remove mock for MetricsClient * Avoid goroutine test panics * trying to kick CI lint issues by upgrading mod * imprve test code and add hasher for testing * Use structure logging for filters, fix error constants, and default to allow all regex * removed hashin and modify logic to simplify * Improve race test and fix PR feedback by removing hash equals and avoid testing the timer.Ticker logic, and instead unit test * Ran make go-mod-tidy * Use errtypes in the test * Add changelog * add safety check for exporter endpoint * remove require.Contains by using error types, fix structure logging, and fix success metric typo in exporter * Fixed race test to have changing config values * Send success metric before modifying config * Avoid the defer and move the success metric under * NET-4240 - Snapshots are failing on Windows (#18302) * fix go mod * fix go sum * added change log * ran make go mod tidy * Register ProxyStateTemplate Resource (#18316) Also, change the ProxyState.id to identity. This is because we already have the id of this proxy from the resource, and this id should be name-aligned with the workload it represents. It should also have the owner ref set to the workload ID if we need that. And so the id field seems unnecessary. We do, however, need a reference to workload identity so that we can authorize the proxy when it initially connects to the xDS server. * [NET-5146] security: Update Go version to 1.20.7 and `x/net` to 0.13.0 (#18358) * Update Go version to 1.20.7 This resolves [CVE-2023-29409] (https://nvd.nist.gov/vuln/detail/CVE-2023-29409)(`crypto/tls`). * Bump golang.org/x/net to 0.13.0 Addresses [CVE-2023-3978](https://nvd.nist.gov/vuln/detail/CVE-2023-3978) for security scans (non-impacting). * mesh: adding the protobuf types and resources backing mesh config v2 (#18351) * [OSS] test: xds coverage for routes (#18369) test: xds coverage for routes * Fix policy lookup to allow for slashes (#18347) * Fix policy lookup to allow for slashes * Fix suggestions * Fix other test * Revert some lines * add some initial CODEOWNERS (#18346) * Add redirects for mesh-gateway docs (#18377) * expose grpc as http endpoint (#18221) expose resource grpc endpoints as http endpoints * bimapper: fix a bug and add some more test coverage (#18387) * [docs] Fix ServiceDefaults example in distributed tracing (#17212) Fix ServiceDefaults example in distributed tracing. * Fix #17730 - Dev mode has new line (#18367) * adding new line only in case of pretty in url not in dev mode * change log added * chore: make go-mod-tidy (#18388) * resource: Make resource read tenancy aware (#18397) * Not using chmod - fixed integration test for Enterprise (#18401) * Not using chmod * checking icacls * Removed push * replaced ordered list of rate limit ops with flow diagram (#18398) * replaced ordered list of rate limit ops with flow diagram * made diagram font bigger * updated colors for dark mode img * fixed typo * recentered dark mode image * Apply suggestions from code review Co-authored-by: Jeff Boruszak <[email protected]> --------- Co-authored-by: Jeff Boruszak <[email protected]> * Upgrade test: remove outdated test and disable log due to verbosity (#18403) * remove outdated test * disable log since we have too many parallel tests * bump testcontainers-go from 0.22.0 and remove pinned go version in in… (#18395) * bump testcontainers-go from 0.22.0 and remove pinned go version in integ test * go mod tidy * Replace deprecated target.Authority with target.URL.Host * Add ServiceEndpoints Mutation hook tests (#18404) * Add ServiceEndpoints Mutation hook tests * Move endpoint owner validation into the validation hook Also there were some minor changes to error validation to account for go-cmp not liking to peer through an errors.errorstring type that get created by errors.New * catalog: add FailoverPolicy mutation and validation hooks (#18390) Add most of the validation and mutation hooks for the FailoverPolicy resource. * NET-5115 Add retry + timeout filters for api-gateway (#18324) * squash, implement retry/timeout in consul core * update tests * catalog: adding a controller to reconcile FailoverPolicy resources (#18399) Add most of the semantic cross-resource validation for FailoverPolicy resources using a new controller. * fix the error in ent repo (#18421) fix the error in ent repo * build: updates for 1.16.1 release (#18415) * build: updates for 1.16.1 release * build: fix missing replace directive for envoyextensions * build: go mod tidy testing/deployer * update ECS links (#18419) * docs: fix incorrect proxy-defaults config in Lua Envoy extension (#18418) fix incorrect proxy-defaults config * resource: Make resource write tenancy aware (#18423) * [NET-5217] [OSS] Derive sidecar proxy locality from parent service (#18437) * Add logging to locality policy application In OSS, this is currently a no-op. * Inherit locality when registering sidecars When sidecar locality is not explicitly configured, inherit locality from the proxied service. * NET-4984: Update APIGW Config Entries for JWT Auth (#18366) * Added oss config entries for Policy and JWT on APIGW * Updated structs for config entry * Updated comments, ran deep-copy * Move JWT configuration into OSS file * Add in the config entry OSS file for jwts * Added changelog * fixing proto spacing * Moved to using manually written deep copy method * Use pointers for override/default fields in apigw config entries * Run gen scripts for changed types * k8s compat - Openshift versions (#18307) * Update compatibility.mdx Co-authored-by: trujillo-adam <[email protected]> * APIGW: Update HTTPRouteConfigEntry for JWT Auth (#18422) * Updated httproute config entry for JWT Filters * Added manual deepcopy method for httproute jwt filter * Fix test * Update JWTFilter to be in oss file * Add changelog * Add build tags for deepcopy oss file * [COMPLIANCE] License changes (#18443) * Adding explicit MPL license for sub-package This directory and its subdirectories (packages) contain files licensed with the MPLv2 `LICENSE` file in this directory and are intentionally licensed separately from the BSL `LICENSE` file at the root of this repository. * Adding explicit MPL license for sub-package This directory and its subdirectories (packages) contain files licensed with the MPLv2 `LICENSE` file in this directory and are intentionally licensed separately from the BSL `LICENSE` file at the root of this repository. * Updating the license from MPL to Business Source License Going forward, this project will be licensed under the Business Source License v1.1. Please see our blog post for more details at <Blog URL>, FAQ at www.hashicorp.com/licensing-faq, and details of the license at www.hashicorp.com/bsl. * add missing license headers * Update copyright file headers to BUSL-1.1 * Update copyright file headers to BUSL-1.1 * Update copyright file headers to BUSL-1.1 * Update copyright file headers to BUSL-1.1 * Update copyright file headers to BUSL-1.1 * Update copyright file headers to BUSL-1.1 * Update copyright file headers to BUSL-1.1 * Update copyright file headers to BUSL-1.1 * Update copyright file headers to BUSL-1.1 * Update copyright file headers to BUSL-1.1 * Update copyright file headers to BUSL-1.1 * Update copyright file headers to BUSL-1.1 * Update copyright file headers to BUSL-1.1 * Update copyright file headers to BUSL-1.1 * Update copyright file headers to BUSL-1.1 --------- Co-authored-by: hashicorp-copywrite[bot] <110428419+hashicorp-copywrite[bot]@users.noreply.github.com> * Net-2708/delete resource endpoint (#18420) * feat: add http delete endpoint for resource service * refactor: clean up * Disable deep-copy codegen verification for now. (#18446) Once we figure out what to do with license headers for these files we should re-enable this check. * build: upgrade to latest buf v1.26.0 (#18426) * NET-4952 Add docs for export command (#18425) * Init file for services export command docs * Add link for export command to nav * Add export to list of subcommands for services * Add usage and basic command options * Add API options using includes * Add section for enterprise options * Include argument value in flag format * Add examples * Apply suggestions from code review Co-authored-by: Jeff Boruszak <[email protected]> --------- Co-authored-by: Jeff Boruszak <[email protected]> * Net-2712/resource hcl parsing (#18250) * Initial protohcl implementation Co-authored-by: Matt Keeler <[email protected]> Co-authored-by: Daniel Upton <[email protected]> * resourcehcl: implement resource decoding on top of protohcl Co-authored-by: Daniel Upton <[email protected]> * fix: resolve ci failures * test: add additional unmarshalling tests * refactor: update function test to clean protohcl package imports --------- Co-authored-by: Matt Keeler <[email protected]> Co-authored-by: Daniel Upton <[email protected]> * read endpoint (#18268) implement http read endpoint to expose resource grpc service read method * Net-2707/list resource endpoint (#18444) feat: list resources endpoint * Fix incorrect yaml in examples (#18463) * NET-5187: Upgrade test timeout due to log producer errors (#18461) Stop log producer before restarting container * [NET-4799] [OSS] xdsv2: listeners L4 support for connect proxies (#18436) * refactor to avoid future import cycles * resource: Make resource list tenancy aware (#18475) * [COMPLIANCE] License update (#18479) Update BUSL LICENSE to include licensed product and version. Co-authored-by: hashicorp-copywrite[bot] <110428419+hashicorp-copywrite[bot]@users.noreply.github.com> * docs…
trujillo-adam
added a commit
that referenced
this pull request
Sep 7, 2023
…t monitor (#18699) * tlsutil: Fix check TLS configuration (#17481) * tlsutil: Fix check TLS configuration * Rewording docs. * Update website/content/docs/services/configuration/checks-configuration-reference.mdx Co-authored-by: trujillo-adam <[email protected]> * Fix typos and add changelog entry. --------- Co-authored-by: trujillo-adam <[email protected]> * docs: Deprecations for connect-native SDK and specific connect native APIs (#17937) * Update v1_16_x.mdx * Update connect native golang page --------- Co-authored-by: trujillo-adam <[email protected]> * Revert "Add workflow to verify linux release packages (#17904)" (#17942) This reverts commit 3368f14fab500ebe9f6aeab5631dd1d5f5a453e5. * Fixes Secondary ConnectCA update (#17846) This fixes a bug that was identified which resulted in subsequent ConnectCA configuration update not to persist in the cluster. * fixing typo in link to jwt-validations-with-intentions doc (#17955) * Fix streaming backend link (#17958) * Fix streaming backend link * Update health.mdx * Dynamically create jwks clusters for jwt-providers (#17944) * website: remove deprecated agent rpc docs (#17962) * Fix missing BalanceOutboundConnections in v2 catalog. (#17964) * feature - [NET - 4005] - [Supportability] Reloadable Configuration - enable_debug (#17565) * # This is a combination of 9 commits. # This is the 1st commit message: init without tests # This is the commit message #2: change log # This is the commit message #3: fix tests # This is the commit message #4: fix tests # This is the commit message #5: added tests # This is the commit message #6: change log breaking change # This is the commit message #7: removed breaking change # This is the commit message #8: fix test # This is the commit message #9: keeping the test behaviour same * # This is a combination of 12 commits. # This is the 1st commit message: init without tests # This is the commit message #2: change log # This is the commit message #3: fix tests # This is the commit message #4: fix tests # This is the commit message #5: added tests # This is the commit message #6: change log breaking change # This is the commit message #7: removed breaking change # This is the commit message #8: fix test # This is the commit message #9: keeping the test behaviour same # This is the commit message #10: made enable debug atomic bool # This is the commit message #11: fix lint # This is the commit message #12: fix test true enable debug * parent 10f500e895d92cc3691ade7b74a33db755d22039 author absolutelightning <[email protected]> 1687352587 +0530 committer absolutelightning <[email protected]> 1687352592 +0530 init without tests change log fix tests fix tests added tests change log breaking change removed breaking change fix test keeping the test behaviour same made enable debug atomic bool fix lint fix test true enable debug using enable debug in agent as atomic bool test fixes fix tests fix tests added update on correct locaiton fix tests fix reloadable config enable debug fix tests fix init and acl 403 * revert commit * Fix formatting codeblocks on APIgw docs (#17970) * fix formatting codeblocks * remove unnecessary indents * Remove POC code (#17974) * update doc (#17910) * update doc * update link * Remove duplicate and unused newDecodeConfigEntry func (#17979) * docs: samenessGroup YAML examples (#17984) * configuration entry syntax * Example config * Add changelog entry for 1.16.0 (#17987) * Fix typo (#17198) servcies => services * Expose JWKS cluster config through JWTProviderConfigEntry (#17978) * Expose JWKS cluster config through JWTProviderConfigEntry * fix typos, rename trustedCa to trustedCA * Integration test for ext-authz Envoy extension (#17980) * Fix incorrect protocol for transparent proxy upstreams. (#17894) This PR fixes a bug that was introduced in: https://github.com/hashicorp/consul/pull/16021 A user setting a protocol in proxy-defaults would cause tproxy implicit upstreams to not honor the upstream service's protocol set in its `ServiceDefaults.Protocol` field, and would instead always use the proxy-defaults value. Due to the fact that upstreams configured with "tcp" can successfully contact upstream "http" services, this issue was not recognized until recently (a proxy-defaults with "tcp" and a listening service with "http" would make successful requests, but not the opposite). As a temporary work-around, users experiencing this issue can explicitly set the protocol on the `ServiceDefaults.UpstreamConfig.Overrides`, which should take precedence. The fix in this PR removes the proxy-defaults protocol from the wildcard upstream that tproxy uses to configure implicit upstreams. When the protocol was included, it would always overwrite the value during discovery chain compilation, which was not correct. The discovery chain compiler also consumes proxy defaults to determine the protocol, so simply excluding it from the wildcard upstream config map resolves the issue. * feat: include nodes count in operator usage endpoint and cli command (#17939) * feat: update operator usage api endpoint to include nodes count * feat: update operator usange cli command to includes nodes count * [OSS] Improve Gateway Test Coverage of Catalog Health (#18011) * fix(cli): remove failing check from 'connect envoy' registration for api gateway * test(integration): add tests to check catalog statsus of gateways on startup * remove extra sleep comment * Update test/integration/consul-container/libs/assert/service.go * changelog * Fixes Traffic rate limitting docs (#17997) * Fix removed service-to-service peering links (#17221) * docs: fix removed service-to-service peering links * docs: extend peering-via-mesh-gateways intro (thanks @trujillo-adam) --------- Co-authored-by: trujillo-adam <[email protected]> * docs: Sameness "beta" warning (#18017) * Warning updates * .x * updated typo in tab heading (#18022) * updated typo in tab heading * updated tab group typo, too * Document that DNS lookups can target cluster peers (#17990) Static DNS lookups, in addition to explicitly targeting a datacenter, can target a cluster peer. This was added in 95dc0c7b301b70a6b955a8b7c9737c9b86f03df6 but didn't make the documentation. The driving function for the change is `parseLocality` here: https://github.com/hashicorp/consul/blob/0b1299c28d8127129d61310ee4280055298438e0/agent/dns_oss.go#L25 The biggest change in this is to adjust the standard lookup syntax to tie `.<datacenter>` to `.dc` as required-together, and to append in the similar `.<cluster-peer>.peer` optional argument, both to A record and SRV record lookups. Co-authored-by: David Yu <[email protected]> * Add first integration test for jwt auth with intention (#18005) * fix stand-in text for name field (#18030) * removed sameness conf entry from failover nav (#18033) * docs - add service sync annotations and k8s service weight annotation (#18032) * Docs for https://github.com/hashicorp/consul-k8s/pull/2293 * remove versions for enterprise features since they are old --------- Co-authored-by: Tu Nguyen <[email protected]> * docs - add jobs use case for service mesh k8s (#18037) * docs - add jobs use case for service mesh k8s * add code blocks * address feedback (#18045) * Add verify server hostname to tls default (#17155) * [OSS] Fix initial_fetch_timeout to wait for all xDS resources (#18024) * fix(connect): set initial_fetch_time to wait indefinitely * changelog * PR feedback 1 * ui: fix typos for peer service imports (#17999) * test: fix FIPS inline cert test message (#18076) * Fix a couple typos in Agent Telemetry Metrics docs (#18080) * Fix metrics docs * Add changelog Signed-off-by: josh <[email protected]> --------- Signed-off-by: josh <[email protected]> * docs updates - cluster peering and virtual services (#18069) * Update route-to-virtual-services.mdx * Update establish-peering.mdx * Update service-mesh-compare.mdx (#17279) grammar change * Update helm docs on main (#18085) * ci: use gotestsum v1.10.1 [NET-4042] (#18088) * Docs: Update proxy lifecycle annotations and consul-dataplane flags (#18075) * Update proxy lifecycle annotations and consul-dataplane flags * Pass configured role name to Vault for AWS auth in Connect CA (#17885) * Docs for dataplane upgrade on k8s (#18051) * Docs for dataplane upgrade on k8s --------- Co-authored-by: David Yu <[email protected]> Co-authored-by: trujillo-adam <[email protected]> * docs - update upgrade index page to not recommend consul leave. (#18100) * Displays Consul version of each nodes in UI nodes section (#17754) * update UINodes and UINodeInfo response with consul-version info added as NodeMeta, fetched from serf members * update test cases TestUINodes, TestUINodeInfo * added nil check for map * add consul-version in local agent node metadata * get consul version from serf member and add this as node meta in catalog register request * updated ui mock response to include consul versions as node meta * updated ui trans and added version as query param to node list route * updates in ui templates to display consul version with filter and sorts * updates in ui - model class, serializers,comparators,predicates for consul version feature * added change log for Consul Version Feature * updated to get version from consul service, if for some reason not available from serf * updated changelog text * updated dependent testcases * multiselection version filter * Update agent/consul/state/catalog.go comments updated Co-authored-by: Jared Kirschner <[email protected]> --------- Co-authored-by: Jared Kirschner <[email protected]> * api gw 1.16 updates (#18081) * api gw 1.16 updates * Apply suggestions from code review Co-authored-by: Jeff Boruszak <[email protected]> * update CodeBlockConfig filename * Apply suggestions from code review Co-authored-by: trujillo-adam <[email protected]> Co-authored-by: Jeff Boruszak <[email protected]> * remove non-standard intentions page * Update website/content/docs/api-gateway/configuration/index.mdx Co-authored-by: trujillo-adam <[email protected]> --------- Co-authored-by: Jeff Boruszak <[email protected]> Co-authored-by: trujillo-adam <[email protected]> * [NET-4103] ci: build s390x (#18067) * ci: build s390x * ci: test s390x * ci: dev build s390x * no GOOS * ent only * build: publish s390x * fix syntax error * fix syntax error again * fix syntax error again x2 * test branch * Move s390x conditionals to step level * remove test branch --------- Co-authored-by: emilymianeil <[email protected]> * :ermahgerd "Sevice Mesh" -> "Service Mesh" (#18116) Just a typo in the docs. * Split pbmesh.UpstreamsConfiguration as a resource out of pbmesh.Upstreams (#17991) Configuration that previously was inlined into the Upstreams resource applies to both explicit and implicit upstreams and so it makes sense to split it out into its own resource. It also has other minor changes: - Renames `proxy.proto` proxy_configuration.proto` - Changes the type of `Upstream.destination_ref` from `pbresource.ID` to `pbresource.Reference` - Adds comments to fields that didn't have them * [NET-4895] ci - api tests and consul container tests error because of dependency bugs with go 1.20.6. Pin go to 1.20.5. (#18124) ### Description The following jobs started failing when go 1.20.6 was released: - `go-test-api-1-19` - `go-test-api-1-20` - `compatibility-integration-tests` - `upgrade-integration-tests` `compatibility-integration-tests` and `compatibility-integration-tests` to this testcontainers issue: https://github.com/testcontainers/testcontainers-go/issues/1359. This issue calls for testcontainers to release a new version when one of their dependencies is fixed. When that is done, we will unpin the go versions in `compatibility-integration-tests` and `compatibility-integration-tests`. ### Testing & Reproduction steps See these jobs broken in CI and then see them work with this PR. --------- Co-authored-by: Chris Thain <[email protected]> * Add ingress gateway deprecation notices to docs (#18102) ### Description This adds notices, that ingress gateway is deprecated, to several places in the product docs where ingress gateway is the topic. ### Testing & Reproduction steps Tested with a local copy of the website. ### Links Deprecation of ingress gateway was announced in the Release Notes for Consul 1.16 and Consul-K8s 1.2. See: [https://developer.hashicorp.com/consul/docs/release-notes/consul/v1_16_x#what-s-deprecated](https://developer.hashicorp.com/consul/docs/release-notes/consul/v1_16_x#what-s-deprecated ) [https://developer.hashicorp.com/consul/docs/release-notes/consul-k8s/v1_2_x#what-s-deprecated](https://developer.hashicorp.com/consul/docs/release-notes/consul-k8s/v1_2_x#what-s-deprecated) ### PR Checklist * [N/A] updated test coverage * [X] external facing docs updated * [X] appropriate backport labels added * [X] not a security concern --------- Co-authored-by: trujillo-adam <[email protected]> * Add docs for jwt cluster configuration (#18004) ### Description <!-- Please describe why you're making this change, in plain English. --> - Add jwt-provider docs for jwks cluster configuration. The configuration was added here: https://github.com/hashicorp/consul/pull/17978 * Docs: fix unmatched bracket for health checks page (#18134) * NET-4657/add resource service client (#18053) ### Description <!-- Please describe why you're making this change, in plain English. --> Dan had already started on this [task](https://github.com/hashicorp/consul/pull/17849) which is needed to start building the HTTP APIs. This just needed some cleanup to get it ready for review. Overview: - Rename `internalResourceServiceClient` to `insecureResourceServiceClient` for name consistency - Configure a `secureResourceServiceClient` with auth enabled ### PR Checklist * [ ] ~updated test coverage~ * [ ] ~external facing docs updated~ * [x] appropriate backport labels added * [ ] ~not a security concern~ * Fix bug with Vault CA provider (#18112) Updating RootPKIPath but not IntermediatePKIPath would not update leaf signing certs with the new root. Unsure if this happens in practice but manual testing showed it is a bug that would break mesh and agent connections once the old root is pruned. * [NET-4897] net/http host header is now verified and request.host that contains socked now error (#18129) ### Description This is related to https://github.com/hashicorp/consul/pull/18124 where we pinned the go versions in CI to 1.20.5 and 1.19.10. go 1.20.6 and 1.19.11 now validate request host headers for validity, including the hostname cannot be prefixed with slashes. For local communications (npipe://, unix://), the hostname is not used, but we need valid and meaningful hostname. Prior versions go Go would clean the host header, and strip slashes in the process, but go1.20.6 and go1.19.11 no longer do, and reject the host header. Around the community we are seeing that others are intercepting the req.host and if it starts with a slash or ends with .sock, they changing the host to localhost or another dummy value. [client: define a "dummy" hostname to use for local connections by thaJeztah · Pull Request #45942 · moby/moby](https://github.com/moby/moby/pull/45942) ### Testing & Reproduction steps Check CI tests. ### Links * [ ] updated test coverage * [ ] external facing docs updated * [ ] appropriate backport labels added * [ ] not a security concern * add a conditional around setting LANFilter.AllSegments to make sure it is valid (#18139) ### Description This is to correct a code problem because this assumes all segments, but when you get to Enterprise, you can be in partition that is not the default partition, in which case specifying all segments does not validate and fails. This is to correct the setting of this filter with `AllSegments` to `true` to only occur when in the the `default` partition. ### Testing & Reproduction steps <!-- * In the case of bugs, describe how to replicate * If any manual tests were done, document the steps and the conditions to replicate * Call out any important/ relevant unit tests, e2e tests or integration tests you have added or are adding --> ### Links <!-- Include any links here that might be helpful for people reviewing your PR (Tickets, GH issues, API docs, external benchmarks, tools docs, etc). If there are none, feel free to delete this section. Please be mindful not to leak any customer or confidential information. HashiCorp employees may want to use our internal URL shortener to obfuscate links. --> ### PR Checklist * [ ] updated test coverage * [ ] external facing docs updated * [ ] appropriate backport labels added * [ ] not a security concern * chore: bump upgrade integrations tests to 1.15, 116 [NET-4743] (#18130) * re org resource type registry (#18133) * fix: update delegateMock used in ENT (#18149) ### Description <!-- Please describe why you're making this change, in plain English. --> The mock is used in `http_ent_test` file which caused lint failures. For OSS->ENT parity adding the same change here. ### Links <!-- Include any links here that might be helpful for people reviewing your PR (Tickets, GH issues, API docs, external benchmarks, tools docs, etc). If there are none, feel free to delete this section. Please be mindful not to leak any customer or confidential information. HashiCorp employees may want to use our internal URL shortener to obfuscate links. --> Identified in OSS->ENT [merge PR](https://github.com/hashicorp/consul-enterprise/pull/6328) ### PR Checklist * [ ] ~updated test coverage~ * [ ] ~external facing docs updated~ * [x] appropriate backport labels added * [ ] ~not a security concern~ * Use JWT-auth filter in metadata mode & Delegate validation to RBAC filter (#18062) ### Description <!-- Please describe why you're making this change, in plain English. --> - Currently the jwt-auth filter doesn't take into account the service identity when validating jwt-auth, it only takes into account the path and jwt provider during validation. This causes issues when multiple source intentions restrict access to an endpoint with different JWT providers. - To fix these issues, rather than use the JWT auth filter for validation, we use it in metadata mode and allow it to forward the successful validated JWT token payload to the RBAC filter which will make the decisions. This PR ensures requests with and without JWT tokens successfully go through the jwt-authn filter. The filter however only forwards the data for successful/valid tokens. On the RBAC filter level, we check the payload for claims and token issuer + existing rbac rules. ### Testing & Reproduction steps <!-- * In the case of bugs, describe how to replicate * If any manual tests were done, document the steps and the conditions to replicate * Call out any important/ relevant unit tests, e2e tests or integration tests you have added or are adding --> - This test covers a multi level jwt requirements (requirements at top level and permissions level). It also assumes you have envoy running, you have a redis and a sidecar proxy service registered, and have a way to generate jwks with jwt. I mostly use: https://www.scottbrady91.com/tools/jwt for this. - first write your proxy defaults ``` Kind = "proxy-defaults" name = "global" config { protocol = "http" } ``` - Create two providers ``` Kind = "jwt-provider" Name = "auth0" Issuer = "https://ronald.local" JSONWebKeySet = { Local = { JWKS = "eyJrZXlzIjog....." } } ``` ``` Kind = "jwt-provider" Name = "okta" Issuer = "https://ronald.local" JSONWebKeySet = { Local = { JWKS = "eyJrZXlzIjogW3...." } } ``` - add a service intention ``` Kind = "service-intentions" Name = "redis" JWT = { Providers = [ { Name = "okta" }, ] } Sources = [ { Name = "*" Permissions = [{ Action = "allow" HTTP = { PathPrefix = "/workspace" } JWT = { Providers = [ { Name = "okta" VerifyClaims = [ { Path = ["aud"] Value = "my_client_app" }, { Path = ["sub"] Value = "5be86359073c434bad2da3932222dabe" } ] }, ] } }, { Action = "allow" HTTP = { PathPrefix = "/" } JWT = { Providers = [ { Name = "auth0" }, ] } }] } ] ``` - generate 3 jwt tokens: 1 from auth0 jwks, 1 from okta jwks with different claims than `/workspace` expects and 1 with correct claims - connect to your envoy (change service and address as needed) to view logs and potential errors. You can add: `-- --log-level debug` to see what data is being forwarded ``` consul connect envoy -sidecar-for redis1 -grpc-addr 127.0.0.1:8502 ``` - Make the following requests: ``` curl -s -H "Authorization: Bearer $Auth0_TOKEN" --insecure --cert leaf.cert --key leaf.key --cacert connect-ca.pem https://localhost:20000/workspace -v RBAC filter denied curl -s -H "Authorization: Bearer $Okta_TOKEN_with_wrong_claims" --insecure --cert leaf.cert --key leaf.key --cacert connect-ca.pem https://localhost:20000/workspace -v RBAC filter denied curl -s -H "Authorization: Bearer $Okta_TOKEN_with_correct_claims" --insecure --cert leaf.cert --key leaf.key --cacert connect-ca.pem https://localhost:20000/workspace -v Successful request ``` ### TODO * [x] Update test coverage * [ ] update integration tests (follow-up PR) * [x] appropriate backport labels added * Support Consul Connect Envoy Command on Windows (#17694) ### Description Add support for consul connect envoy command on windows. This PR fixes the comments of PR - https://github.com/hashicorp/consul/pull/15114 ### Testing * Built consul.exe from this branch on windows and hosted here - [AWS S3](https://asheshvidyut-bucket.s3.ap-southeast-2.amazonaws.com/consul.zip) * Updated the [tutorial](https://developer.hashicorp.com/consul/tutorials/developer-mesh/consul-windows-workloads) and changed the `consul_url.default` value to [AWS S3](https://asheshvidyut-bucket.s3.ap-southeast-2.amazonaws.com/consul.zip) * Followed the steps in the tutorial and verified that everything is working as described. ### PR Checklist * [x] updated test coverage * [ ] external facing docs updated * [x] appropriate backport labels added * [x] not a security concern --------- Co-authored-by: Franco Bruno Lavayen <[email protected]> Co-authored-by: Jose Ignacio Lorenzo <[email protected]> Co-authored-by: Jose Ignacio Lorenzo <[email protected]> Co-authored-by: Dhia Ayachi <[email protected]> * Change docs to say 168h instead of 7d for server_rejoin_age_max (#18154) ### Description Addresses https://github.com/hashicorp/consul/pull/17171#issuecomment-1636930705 * [OSS] test: improve xDS listener code coverage (#18138) test: improve xDS listener code coverage * Re-order expected/actual for assertContainerState in consul container tests (#18157) Re-order expected/actual, consul container tests * group and document make file (#17943) * group and document make file * Add `testing/deployer` (neé `consul-topology`) [NET-4610] (#17823) Co-authored-by: R.B. Boyer <[email protected]> Co-authored-by: R.B. Boyer <[email protected]> Co-authored-by: Freddy <[email protected]> * [NET-4792] Add integrations tests for jwt-auth (#18169) * Add FIPS reference to consul enterprise docs (#18028) * Add FIPS reference to consul enterprise docs * Update website/content/docs/enterprise/index.mdx Co-authored-by: David Yu <[email protected]> * remove support for ecs client (fips) --------- Co-authored-by: David Yu <[email protected]> * add peering_commontopo tests [NET-3700] (#17951) Co-authored-by: R.B. Boyer <[email protected]> Co-authored-by: R.B. Boyer <[email protected]> Co-authored-by: Freddy <[email protected]> Co-authored-by: NiniOak <[email protected]> * docs - remove Sentinel from enterprise features list (#18176) * Update index.mdx * Update kv.mdx * Update docs-nav-data.json * delete sentinel.mdx * Update redirects.js --------- Co-authored-by: Tu Nguyen <[email protected]> * [NET-4865] Bump golang.org/x/net to 0.12.0 (#18186) Bump golang.org/x/net to 0.12.0 While not necessary to directly address CVE-2023-29406 (which should be handled by using a patched version of Go when building), an accompanying change to HTTP/2 error handling does impact agent code. See https://go-review.googlesource.com/c/net/+/506995 for the HTTP/2 change. Bump this dependency across our submodules as well for the sake of potential indirect consumers of `x/net/http`. * Call resource mutate hook before validate hook (NET-4907) (#18178) * [NET-4865] security: Update Go version to 1.20.6 (#18190) Update Go version to 1.20.6 This resolves [CVE-2023-29406] (https://nvd.nist.gov/vuln/detail/CVE-2023-29406) for uses of the `net/http` standard library. Note that until the follow-up to #18124 is done, the version of Go used in those impacted tests will need to remain on 1.20.5. * Improve XDS test coverage: JWT auth edition (#18183) * Improve XDS test coverage: JWT auth edition more tests * test: xds coverage for jwt listeners --------- Co-authored-by: DanStough <[email protected]> * update readme.md (#18191) u[date readme.md * Update submodules to latest following 1.16.0 (#18197) Align all our internal use of submodules on the latest versions. * SEC-090: Automated trusted workflow pinning (2023-07-18) (#18174) Result of tsccr-helper -log-level=info -pin-all-workflows . Co-authored-by: hashicorp-tsccr[bot] <hashicorp-tsccr[bot]@users.noreply.github.com> * Fix Backport Assistant PR commenting (#18200) * Fix Backport Assistant failure PR commenting For general comments on a PR, it looks like you have to use the `/issue` endpoint rather than `/pulls`, which requires commit/other review-specific target details. This matches the endpoint used in `backport-reminder.yml`. * Remove Backport Reminder workflow This is noisy (even when adding multiple labels, individual comments per label are generated), and likely no longer needed: we haven't had this work in a long time due to an expired GH token, and we now have better automation for backport PR assignment. * resource: Pass resource to Write ACL hook instead of just resource Id [NET-4908] (#18192) * Explicitly enable WebSocket upgrades (#18150) This PR explicitly enables WebSocket upgrades in Envoy's UpgradeConfig for all proxy types. (API Gateway, Ingress, and Sidecar.) Fixes #8283 * docs: fix the description of client rpc (#18206) * NET-4804: Add dashboard for monitoring consul-k8s (#18208) * [OSS] Improve xDS Code Coverage - Clusters (#18165) test: improve xDS cluster code coverage * NET-4222 take config file consul container (#18218) Net 4222 take config file consul container * Envoy Integration Test Windows (#18007) * [CONSUL-395] Update check_hostport and Usage (#40) * [CONSUL-397] Copy envoy binary from Image (#41) * [CONSUL-382] Support openssl in unique test dockerfile (#43) * [CONSUL-405] Add bats to single container (#44) * [CONSUL-414] Run Prometheus Test Cases and Validate Changes (#46) * [CONSUL-410] Run Jaeger in Single container (#45) * [CONSUL-412] Run test-sds-server in single container (#48) * [CONSUL-408] Clean containers (#47) * [CONSUL-384] Rebase and sync fork (#50) * [CONSUL-415] Create Scenarios Troubleshooting Docs (#49) * [CONSUL-417] Update Docs Single Container (#51) * [CONSUL-428] Add Socat to single container (#54) * [CONSUL-424] Replace pkill in kill_envoy function (#52) * [CONSUL-434] Modify Docker run functions in Helper script (#53) * [CONSUL-435] Replace docker run in set_ttl_check_state & wait_for_agent_service_register functions (#55) * [CONSUL-438] Add netcat (nc) in the Single container Dockerfile (#56) * [CONSUL-429] Replace Docker run with Docker exec (#57) * [CONSUL-436] Curl timeout and run tests (#58) * [CONSUL-443] Create dogstatsd Function (#59) * [CONSUL-431] Update Docs Netcat (#60) * [CONSUL-439] Parse nc Command in function (#61) * [CONSUL-463] Review curl Exec and get_ca_root Func (#63) * [CONSUL-453] Docker hostname in Helper functions (#64) * [CONSUL-461] Test wipe volumes without extra cont (#66) * [CONSUL-454] Check ports in the Server and Agent containers (#65) * [CONSUL-441] Update windows dockerfile with version (#62) * [CONSUL-466] Review case-grpc Failing Test (#67) * [CONSUL-494] Review case-cfg-resolver-svc-failover (#68) * [CONSUL-496] Replace docker_wget & docker_curl (#69) * [CONSUL-499] Cleanup Scripts - Remove nanoserver (#70) * [CONSUL-500] Update Troubleshooting Docs (#72) * [CONSUL-502] Pull & Tag Envoy Windows Image (#73) * [CONSUL-504] Replace docker run in docker_consul (#76) * [CONSUL-505] Change admin_bind * [CONSUL-399] Update envoy to 1.23.1 (#78) * [CONSUL-510] Support case-wanfed-gw on Windows (#79) * [CONSUL-506] Update troubleshooting Documentation (#80) * [CONSUL-512] Review debug_dump_volumes Function (#81) * [CONSUL-514] Add zipkin to Docker Image (#82) * [CONSUL-515] Update Documentation (#83) * [CONSUL-529] Support case-consul-exec (#86) * [CONSUL-530] Update Documentation (#87) * [CONSUL-530] Update default consul version 1.13.3 * [CONSUL-539] Cleanup (#91) * [CONSUL-546] Scripts Clean-up (#92) * [CONSUL-491] Support admin_access_log_path value for Windows (#71) * [CONSUL-519] Implement mkfifo Alternative (#84) * [CONSUL-542] Create OS Specific Files for Envoy Package (#88) * [CONSUL-543] Create exec_supported.go (#89) * [CONSUL-544] Test and Build Changes (#90) * Implement os.DevNull * using mmap instead of disk files * fix import in exec-unix * fix nmap open too many arguemtn * go fmt on file * changelog file * fix go mod * Update .changelog/17694.txt Co-authored-by: Dhia Ayachi <[email protected]> * different mmap library * fix bootstrap json * some fixes * chocolatey version fix and image fix * using different library * fix Map funciton call * fix mmap call * fix tcp dump * fix tcp dump * windows tcp dump * Fix docker run * fix tests * fix go mod * fix version 16.0 * fix version * fix version dev * sleep to debug * fix sleep * fix permission issue * fix permission issue * fix permission issue * fix command * fix command * fix funciton * fix assert config entry status command not found * fix command not found assert_cert_has_cn * fix command not found assert_upstream_missing * fix command not found assert_upstream_missing_once * fix command not found get_upstream_endpoint * fix command not found get_envoy_public_listener_once * fix command not found * fix test cases * windows integration test workflow github * made code similar to unix using npipe * fix go.mod * fix dialing of npipe * dont wait * check size of written json * fix undefined n * running * fix dep * fix syntax error * fix workflow file * windows runner * fix runner * fix from json * fix runs on * merge connect envoy * fix cin path * build * fix file name * fix file name * fix dev build * remove unwanted code * fix upload * fix bin name * fix path * checkout current branch * fix path * fix tests * fix shell bash for windows sh files * fix permission of run-test.sh * removed docker dev * added shell bash for tests * fix tag * fix win=true * fix cd * added dev * fix variable undefined * removed failing tests * fix tcp dump image * fix curl * fix curl * tcp dump path * fix tcpdump path * fix curl * fix curl install * stop removing intermediate containers * fix tcpdump docker image * revert -rm * --rm=false * makeing docker image before * fix tcpdump * removed case consul exec * removed terminating gateway simple * comment case wasm * removed data dog * comment out upload coverage * uncomment case-consul-exec * comment case consul exec * if always * logs * using consul 1.17.0 * fix quotes * revert quotes * redirect to dev null * Revert version * revert consul connect * fix version * removed envoy connect * not using function * change log * docker logs * fix logs * restructure bad authz * rmeoved dev null * output * fix file descriptor * fix cacert * fix cacert * fix ca cert * cacert does not work in windows curl * fix func * removed docker logs * added sleep * fix tls * commented case-consul-exec * removed echo * retry docker consul * fix upload bin * uncomment consul exec * copying consul.exe to docker image * copy fix * fix paths * fix path * github workspace path * latest version * Revert "latest version" This reverts commit 5a7d7b82d9e7553bcb01b02557ec8969f9deba1d. * commented consul exec * added ssl revoke best effort * revert best effort * removed unused files * rename var name and change dir * windows runner * permission * needs setup fix * swtich to github runner * fix file path * fix path * fix path * fix path * fix path * fix path * fix build paths * fix tag * nightly runs * added matrix in github workflow, renamed files * fix job * fix matrix * removed brackes * from json * without using job matrix * fix quotes * revert job matrix * fix workflow * fix comment * added comment * nightly runs * removed datadog ci as it is already measured in linux one * running test * Revert "running test" This reverts commit 7013d15a23732179d18ec5d17336e16b26fab5d4. * pr comment fixes * running test now * running subset of test * running subset of test * job matrix * shell bash * removed bash shell * linux machine for job matrix * fix output * added cat to debug * using ubuntu latest * fix job matrix * fix win true * fix go test * revert job matrix --------- Co-authored-by: Jose Ignacio Lorenzo <[email protected]> Co-authored-by: Franco Bruno Lavayen <[email protected]> Co-authored-by: Ivan K Berlot <[email protected]> Co-authored-by: Ezequiel Fernández Ponce <[email protected]> Co-authored-by: joselo85 <[email protected]> Co-authored-by: Ezequiel Fernández Ponce <[email protected]> Co-authored-by: Dhia Ayachi <[email protected]> * fix typos and update ecs compat table (#18215) * fix typos and update ecs compat table * real info for the ecs compat matrix table * Update website/content/docs/ecs/compatibility.mdx Co-authored-by: Chris Thain <[email protected]> --------- Co-authored-by: Chris Thain <[email protected]> * [OSS] proxystate: add proxystate protos (#18216) * proxystate: add proxystate protos to pbmesh and resolve imports and conflicts between message names * ci: don't verify s390x (#18224) * [CC-5718] Remove HCP token requirement during bootstrap (#18140) * [CC-5718] Remove HCP token requirement during bootstrap * Re-add error for loading HCP management token * Remove old comment * Add changelog entry * Remove extra validation line * Apply suggestions from code review Co-authored-by: lornasong <[email protected]> --------- Co-authored-by: lornasong <[email protected]> * [NET-4122] Doc guidance for federation with externalServers (#18207) Doc guidance for federation with externalServers Add guidance for proper configuration when joining to a secondary cluster using WAN fed with external servers also enabled. Also clarify federation requirements and fix formatting for an unrelated value. Update both the Helm chart reference (synced from `consul-k8s`, see hashicorp/consul-k8s#2583) and the docs on using `externalServers`. * [OSS] Improve xDS Code Coverage - Endpoints and Misc (#18222) test: improve xDS endpoints code coverage * Clarify license reporting timing and GDPR compliance (#18237) Add Alicia's edits to clarify log timing and other details * Fix Github Workflow File (#18241) * [CONSUL-382] Support openssl in unique test dockerfile (#43) * [CONSUL-405] Add bats to single container (#44) * [CONSUL-414] Run Prometheus Test Cases and Validate Changes (#46) * [CONSUL-410] Run Jaeger in Single container (#45) * [CONSUL-412] Run test-sds-server in single container (#48) * [CONSUL-408] Clean containers (#47) * [CONSUL-384] Rebase and sync fork (#50) * [CONSUL-415] Create Scenarios Troubleshooting Docs (#49) * [CONSUL-417] Update Docs Single Container (#51) * [CONSUL-428] Add Socat to single container (#54) * [CONSUL-424] Replace pkill in kill_envoy function (#52) * [CONSUL-434] Modify Docker run functions in Helper script (#53) * [CONSUL-435] Replace docker run in set_ttl_check_state & wait_for_agent_service_register functions (#55) * [CONSUL-438] Add netcat (nc) in the Single container Dockerfile (#56) * [CONSUL-429] Replace Docker run with Docker exec (#57) * [CONSUL-436] Curl timeout and run tests (#58) * [CONSUL-443] Create dogstatsd Function (#59) * [CONSUL-431] Update Docs Netcat (#60) * [CONSUL-439] Parse nc Command in function (#61) * [CONSUL-463] Review curl Exec and get_ca_root Func (#63) * [CONSUL-453] Docker hostname in Helper functions (#64) * [CONSUL-461] Test wipe volumes without extra cont (#66) * [CONSUL-454] Check ports in the Server and Agent containers (#65) * [CONSUL-441] Update windows dockerfile with version (#62) * [CONSUL-466] Review case-grpc Failing Test (#67) * [CONSUL-494] Review case-cfg-resolver-svc-failover (#68) * [CONSUL-496] Replace docker_wget & docker_curl (#69) * [CONSUL-499] Cleanup Scripts - Remove nanoserver (#70) * [CONSUL-500] Update Troubleshooting Docs (#72) * [CONSUL-502] Pull & Tag Envoy Windows Image (#73) * [CONSUL-504] Replace docker run in docker_consul (#76) * [CONSUL-505] Change admin_bind * [CONSUL-399] Update envoy to 1.23.1 (#78) * [CONSUL-510] Support case-wanfed-gw on Windows (#79) * [CONSUL-506] Update troubleshooting Documentation (#80) * [CONSUL-512] Review debug_dump_volumes Function (#81) * [CONSUL-514] Add zipkin to Docker Image (#82) * [CONSUL-515] Update Documentation (#83) * [CONSUL-529] Support case-consul-exec (#86) * [CONSUL-530] Update Documentation (#87) * [CONSUL-530] Update default consul version 1.13.3 * [CONSUL-539] Cleanup (#91) * [CONSUL-546] Scripts Clean-up (#92) * [CONSUL-491] Support admin_access_log_path value for Windows (#71) * [CONSUL-519] Implement mkfifo Alternative (#84) * [CONSUL-542] Create OS Specific Files for Envoy Package (#88) * [CONSUL-543] Create exec_supported.go (#89) * [CONSUL-544] Test and Build Changes (#90) * Implement os.DevNull * using mmap instead of disk files * fix import in exec-unix * fix nmap open too many arguemtn * go fmt on file * changelog file * fix go mod * Update .changelog/17694.txt Co-authored-by: Dhia Ayachi <[email protected]> * different mmap library * fix bootstrap json * some fixes * chocolatey version fix and image fix * using different library * fix Map funciton call * fix mmap call * fix tcp dump * fix tcp dump * windows tcp dump * Fix docker run * fix tests * fix go mod * fix version 16.0 * fix version * fix version dev * sleep to debug * fix sleep * fix permission issue * fix permission issue * fix permission issue * fix command * fix command * fix funciton * fix assert config entry status command not found * fix command not found assert_cert_has_cn * fix command not found assert_upstream_missing * fix command not found assert_upstream_missing_once * fix command not found get_upstream_endpoint * fix command not found get_envoy_public_listener_once * fix command not found * fix test cases * windows integration test workflow github * made code similar to unix using npipe * fix go.mod * fix dialing of npipe * dont wait * check size of written json * fix undefined n * running * fix dep * fix syntax error * fix workflow file * windows runner * fix runner * fix from json * fix runs on * merge connect envoy * fix cin path * build * fix file name * fix file name * fix dev build * remove unwanted code * fix upload * fix bin name * fix path * checkout current branch * fix path * fix tests * fix shell bash for windows sh files * fix permission of run-test.sh * removed docker dev * added shell bash for tests * fix tag * fix win=true * fix cd * added dev * fix variable undefined * removed failing tests * fix tcp dump image * fix curl * fix curl * tcp dump path * fix tcpdump path * fix curl * fix curl install * stop removing intermediate containers * fix tcpdump docker image * revert -rm * --rm=false * makeing docker image before * fix tcpdump * removed case consul exec * removed terminating gateway simple * comment case wasm * removed data dog * comment out upload coverage * uncomment case-consul-exec * comment case consul exec * if always * logs * using consul 1.17.0 * fix quotes * revert quotes * redirect to dev null * Revert version * revert consul connect * fix version * removed envoy connect * not using function * change log * docker logs * fix logs * restructure bad authz * rmeoved dev null * output * fix file descriptor * fix cacert * fix cacert * fix ca cert * cacert does not work in windows curl * fix func * removed docker logs * added sleep * fix tls * commented case-consul-exec * removed echo * retry docker consul * fix upload bin * uncomment consul exec * copying consul.exe to docker image * copy fix * fix paths * fix path * github workspace path * latest version * Revert "latest version" This reverts commit 5a7d7b82d9e7553bcb01b02557ec8969f9deba1d. * commented consul exec * added ssl revoke best effort * revert best effort * removed unused files * rename var name and change dir * windows runner * permission * needs setup fix * swtich to github runner * fix file path * fix path * fix path * fix path * fix path * fix path * fix build paths * fix tag * nightly runs * added matrix in github workflow, renamed files * fix job * fix matrix * removed brackes * from json * without using job matrix * fix quotes * revert job matrix * fix workflow * fix comment * added comment * nightly runs * removed datadog ci as it is already measured in linux one * running test * Revert "running test" This reverts commit 7013d15a23732179d18ec5d17336e16b26fab5d4. * pr comment fixes * running test now * running subset of test * running subset of test * job matrix * shell bash * removed bash shell * linux machine for job matrix * fix output * added cat to debug * using ubuntu latest * fix job matrix * fix win true * fix go test * revert job matrix * Fix tests --------- Co-authored-by: Ivan K Berlot <[email protected]> Co-authored-by: Jose Ignacio Lorenzo <[email protected]> Co-authored-by: Franco Bruno Lavayen <[email protected]> Co-authored-by: Ezequiel Fernández Ponce <[email protected]> Co-authored-by: joselo85 <[email protected]> Co-authored-by: Ezequiel Fernández Ponce <[email protected]> Co-authored-by: Dhia Ayachi <[email protected]> * NET-4996 - filter go-tests and test-integration workflows from running on docs only and ui only changes (#18236) * Align build arch matrix with enterprise (#18235) Ensure that OSS remains in sync w/ Enterprise by aligning the format of arch matrix args for various build jobs. * Revert "NET-4996 - filter go-tests and test-integration workflows from running on docs only and ui only changes" (#18248) Revert "NET-4996 - filter go-tests and test-integration workflows from running on docs only and ui only changes (#18236)" This reverts commit a11dba710e6ce6f172c0fa6c9b61567cc1efffc8. * resource: Add scope to resource type registration [NET-4976] (#18214) Enables querying a resource type's registration to determine if a resource is cluster, partition, or partition and namespace scoped. * Fix some inconsistencies in jwt docs (#18234) * NET-1825: More new ACL token creation docs (#18063) Co-authored-by: trujillo-adam <[email protected]> * grafana: add the panel resource usage of connect injector (#18247) * [NET-3700] Backfill changelog entry for c2bbe67 and 7402d06 (#18259) Backfill changelog entry for c2bbe67 and 7402d06 Add a changelog entry for the follow-up PR since it was specific to the fix and references the original change. * NET-4897 - update comment to include the current issue url from the go team. (#18263) * fix typos, style, and improper links (#18269) * member cli: add -filter expression to flags (#18223) * member cli: add -filter expression to flags * changelog * update doc * Add test cases * use quote * go-tests: disable s390x (#18273) * docs: Update ext-authz documentation for kubernetes (#18281) Update ext-authz documentation for kubernetes * docs: Consul on Kubernetes specific upgrade info (#18230) * Compatibility page - dataplanes mention * Upgrading higher-level dataplane mention * `exec=` string callout * More visible for upgrade page * Apply suggestions from code review Co-authored-by: Tu Nguyen <[email protected]> --------- Co-authored-by: Tu Nguyen <[email protected]> Co-authored-by: John Murret <[email protected]> * Fix typo in Envoy extensions doc (#18284) * docs: K8s secondary DC requirements (#18280) * Requested edit * Apply suggestions from code review Co-authored-by: trujillo-adam <[email protected]> --------- Co-authored-by: trujillo-adam <[email protected]> * api-gateway: subscribe to bound-api-gateway only after receiving api-gateway (#18291) * api-gateway: subscribe to bound-api-gateway only after receiving api-gateway This fixes a race condition due to our dependency on having the listener(s) from the api-gateway config entry in order to fully and properly process the resources on the bound-api-gateway config entry. * Apply suggestions from code review * Add changelog entry * Update K8s changelog to address cloud auto-join change in 1.0.0 (#18293) * Update list of Envoy versions (#18300) Update supported envoy versions to 1.24.10, 1.25.9, 1.26.4, 1.27.0. * [NET-4904] Update list of Envoy versions in docs (#18306) Update list of Envoy versions in docs Update supported Envoy versions across Consul release versions. * Update actions for TSCCR (#18317) Update action versions before deadline * Fix topology view when displaying mixed connect-native/normal services. (#13023) * Fix topoloy intention with mixed connect-native/normal services. If a service is registered twice, once with connect-native and once without, the topology views would prune the existing intentions. This change brings the code more in line with the transparent proxy behavior. * Dedupe nodes in the ServiceTopology ui endpoint (like done with tags). * Consider a service connect-native as soon as one instance is. * Stop JWT provider from being written in non default namespace (#18325) * docs: Fix some comments (#17118) Signed-off-by: cui fliter <[email protected]> * [NET-4703] Prevent partial application of Envoy extensions (#18068) Prevent partial application of Envoy extensions Ensure that non-required extensions do not change xDS resources before exiting on failure by cloning proto messages prior to applying each extension. To support this change, also move `CanApply` checks up a layer and make them prior to attempting extension application, s.t. we avoid unnecessary copies where extensions can't be applied. Last, ensure that we do not allow panics from `CanApply` or `Extend` checks to escape the attempted extension application. * docs: Simplify example jq commands by removing pipes (#18327) Simplify jq command examples by removing pipes to other commands. * fix typo in create a mesh token docs (#18337) * [CC-5719] Add support for builtin global-read-only policy (#18319) * [CC-5719] Add support for builtin global-read-only policy * Add changelog * Add read-only to docs * Fix some minor issues. * Change from ReplaceAll to Sprintf * Change IsValidPolicy name to return an error instead of bool * Fix PolicyList test * Fix other tests * Apply suggestions from code review Co-authored-by: Paul Glass <[email protected]> * Fix state store test for policy list. * Fix naming issues * Update acl/validation.go Co-authored-by: Chris Thain <[email protected]> * Update agent/consul/acl_endpoint.go --------- Co-authored-by: Paul Glass <[email protected]> Co-authored-by: Chris Thain <[email protected]> * [NET-5121] proxystate: move protos to subdirectory to avoid conflicts (#18335) * also makes a few protos updates * resource: adding various helpers for working with resources (#18342) This is a bit of a grab bag of helpers that I found useful for working with them when authoring substantial Controllers. Subsequent PRs will make use of them. * Wasm integration tests for local and remote wasm files (#17756) * wasm integration tests for local and remote wasm files refactoring and cleanup for wasm testing remove wasm debug logging PR feedback, wasm build lock correct path pattern for wasm build files Add new helper function to minimize changes to existing test code Remove extra param mod tidy add custom service setup to test lib add wait until static server sidecar can reach nginx sidecar Doc comments PR feedback Update workflows to compile wasm for integration tests Fix docker build path Fix package name for linter Update makefile, fix redeclared function Update expected wasm filename Debug test ls in workflow remove pwd in favor of relative path more debugging Build wasm in compatability tests as well Build wasm directly in ci rather than in container Debug tinygo and llvm version Change wasm file extension Remove tinygo debugging Remove extra comments * Add compiled wasm and build instructions * [HCP Telemetry] Periodic Refresh for Dynamic Telemetry Configuration (#18168) * OTElExporter now uses an EndpointProvider to discover the endpoint * OTELSink uses a ConfigProvider to obtain filters and labels configuration * improve tests for otel_sink * Regex logic is moved into client for a method on the TelemetryConfig object * Create a telemetry_config_provider and update deps to use it * Fix conversion * fix import newline * Add logger to hcp client and move telemetry_config out of the client.go file * Add a telemetry_config.go to refactor client.go * Update deps * update hcp deps test * Modify telemetry_config_providers * Check for nil filters * PR review updates * Fix comments and move around pieces * Fix comments * Remove context from client struct * Moved ctx out of sink struct and fixed filters, added a test * Remove named imports, use errors.New if not fformatting * Remove HCP dependencies in telemetry package * Add success metric and move lock only to grab the t.cfgHahs * Update hash * fix nits * Create an equals method and add tests * Improve telemetry_config_provider.go tests * Add race test * Add missing godoc * Remove mock for MetricsClient * Avoid goroutine test panics * trying to kick CI lint issues by upgrading mod * imprve test code and add hasher for testing * Use structure logging for filters, fix error constants, and default to allow all regex * removed hashin and modify logic to simplify * Improve race test and fix PR feedback by removing hash equals and avoid testing the timer.Ticker logic, and instead unit test * Ran make go-mod-tidy * Use errtypes in the test * Add changelog * add safety check for exporter endpoint * remove require.Contains by using error types, fix structure logging, and fix success metric typo in exporter * Fixed race test to have changing config values * Send success metric before modifying config * Avoid the defer and move the success metric under * NET-4240 - Snapshots are failing on Windows (#18302) * fix go mod * fix go sum * added change log * ran make go mod tidy * Register ProxyStateTemplate Resource (#18316) Also, change the ProxyState.id to identity. This is because we already have the id of this proxy from the resource, and this id should be name-aligned with the workload it represents. It should also have the owner ref set to the workload ID if we need that. And so the id field seems unnecessary. We do, however, need a reference to workload identity so that we can authorize the proxy when it initially connects to the xDS server. * [NET-5146] security: Update Go version to 1.20.7 and `x/net` to 0.13.0 (#18358) * Update Go version to 1.20.7 This resolves [CVE-2023-29409] (https://nvd.nist.gov/vuln/detail/CVE-2023-29409)(`crypto/tls`). * Bump golang.org/x/net to 0.13.0 Addresses [CVE-2023-3978](https://nvd.nist.gov/vuln/detail/CVE-2023-3978) for security scans (non-impacting). * mesh: adding the protobuf types and resources backing mesh config v2 (#18351) * [OSS] test: xds coverage for routes (#18369) test: xds coverage for routes * Fix policy lookup to allow for slashes (#18347) * Fix policy lookup to allow for slashes * Fix suggestions * Fix other test * Revert some lines * add some initial CODEOWNERS (#18346) * Add redirects for mesh-gateway docs (#18377) * expose grpc as http endpoint (#18221) expose resource grpc endpoints as http endpoints * bimapper: fix a bug and add some more test coverage (#18387) * [docs] Fix ServiceDefaults example in distributed tracing (#17212) Fix ServiceDefaults example in distributed tracing. * Fix #17730 - Dev mode has new line (#18367) * adding new line only in case of pretty in url not in dev mode * change log added * chore: make go-mod-tidy (#18388) * resource: Make resource read tenancy aware (#18397) * Not using chmod - fixed integration test for Enterprise (#18401) * Not using chmod * checking icacls * Removed push * replaced ordered list of rate limit ops with flow diagram (#18398) * replaced ordered list of rate limit ops with flow diagram * made diagram font bigger * updated colors for dark mode img * fixed typo * recentered dark mode image * Apply suggestions from code review Co-authored-by: Jeff Boruszak <[email protected]> --------- Co-authored-by: Jeff Boruszak <[email protected]> * Upgrade test: remove outdated test and disable log due to verbosity (#18403) * remove outdated test * disable log since we have too many parallel tests * bump testcontainers-go from 0.22.0 and remove pinned go version in in… (#18395) * bump testcontainers-go from 0.22.0 and remove pinned go version in integ test * go mod tidy * Replace deprecated target.Authority with target.URL.Host * Add ServiceEndpoints Mutation hook tests (#18404) * Add ServiceEndpoints Mutation hook tests * Move endpoint owner validation into the validation hook Also there were some minor changes to error validation to account for go-cmp not liking to peer through an errors.errorstring type that get created by errors.New * catalog: add FailoverPolicy mutation and validation hooks (#18390) Add most of the validation and mutation hooks for the FailoverPolicy resource. * NET-5115 Add retry + timeout filters for api-gateway (#18324) * squash, implement retry/timeout in consul core * update tests * catalog: adding a controller to reconcile FailoverPolicy resources (#18399) Add most of the semantic cross-resource validation for FailoverPolicy resources using a new controller. * fix the error in ent repo (#18421) fix the error in ent repo * build: updates for 1.16.1 release (#18415) * build: updates for 1.16.1 release * build: fix missing replace directive for envoyextensions * build: go mod tidy testing/deployer * update ECS links (#18419) * docs: fix incorrect proxy-defaults config in Lua Envoy extension (#18418) fix incorrect proxy-defaults config * resource: Make resource write tenancy aware (#18423) * [NET-5217] [OSS] Derive sidecar proxy locality from parent service (#18437) * Add logging to locality policy application In OSS, this is currently a no-op. * Inherit locality when registering sidecars When sidecar locality is not explicitly configured, inherit locality from the proxied service. * NET-4984: Update APIGW Config Entries for JWT Auth (#18366) * Added oss config entries for Policy and JWT on APIGW * Updated structs for config entry * Updated comments, ran deep-copy * Move JWT configuration into OSS file * Add in the config entry OSS file for jwts * Added changelog * fixing proto spacing * Moved to using manually written deep copy method * Use pointers for override/default fields in apigw config entries * Run gen scripts for changed types * k8s compat - Openshift versions (#18307) * Update compatibility.mdx Co-authored-by: trujillo-adam <[email protected]> * APIGW: Update HTTPRouteConfigEntry for JWT Auth (#18422) * Updated httproute config entry for JWT Filters * Added manual deepcopy method for httproute jwt filter * Fix test * Update JWTFilter to be in oss file * Add changelog * Add build tags for deepcopy oss file * [COMPLIANCE] License changes (#18443) * Adding explicit MPL license for sub-package This directory and its subdirectories (packages) contain files licensed with the MPLv2 `LICENSE` file in this directory and are intentionally licensed separately from the BSL `LICENSE` file at the root of this repository. * Adding explicit MPL license for sub-package This directory and its subdirectories (packages) contain files licensed with the MPLv2 `LICENSE` file in this directory and are intentionally licensed separately from the BSL `LICENSE` file at the root of this repository. * Updating the license from MPL to Business Source License Going forward, this project will be licensed under the Business Source License v1.1. Please see our blog post for more details at <Blog URL>, FAQ at www.hashicorp.com/licensing-faq, and details of the license at www.hashicorp.com/bsl. * add missing license headers * Update copyright file headers to BUSL-1.1 * Update copyright file headers to BUSL-1.1 * Update copyright file headers to BUSL-1.1 * Update copyright file headers to BUSL-1.1 * Update copyright file headers to BUSL-1.1 * Update copyright file headers to BUSL-1.1 * Update copyright file headers to BUSL-1.1 * Update copyright file headers to BUSL-1.1 * Update copyright file headers to BUSL-1.1 * Update copyright file headers to BUSL-1.1 * Update copyright file headers to BUSL-1.1 * Update copyright file headers to BUSL-1.1 * Update copyright file headers to BUSL-1.1 * Update copyright file headers to BUSL-1.1 * Update copyright file headers to BUSL-1.1 --------- Co-authored-by: hashicorp-copywrite[bot] <110428419+hashicorp-copywrite[bot]@users.noreply.github.com> * Net-2708/delete resource endpoint (#18420) * feat: add http delete endpoint for resource service * refactor: clean up * Disable deep-copy codegen verification for now. (#18446) Once we figure out what to do with license headers for these files we should re-enable this check. * build: upgrade to latest buf v1.26.0 (#18426) * NET-4952 Add docs for export command (#18425) * Init file for services export command docs * Add link for export command to nav * Add export to list of subcommands for services * Add usage and basic command options * Add API options using includes * Add section for enterprise options * Include argument value in flag format * Add examples * Apply suggestions from code review Co-authored-by: Jeff Boruszak <[email protected]> --------- Co-authored-by: Jeff Boruszak <[email protected]> * Net-2712/resource hcl parsing (#18250) * Initial protohcl implementation Co-authored-by: Matt Keeler <[email protected]> Co-authored-by: Daniel Upton <[email protected]> * resourcehcl: implement resource decoding on top of protohcl Co-authored-by: Daniel Upton <[email protected]> * fix: resolve ci failures * test: add additional unmarshalling tests * refactor: update function test to clean protohcl package imports --------- Co-authored-by: Matt Keeler <[email protected]> Co-authored-by: Daniel Upton <[email protected]> * read endpoint (#18268) implement http read endpoint to expose resource grpc service read method * Net-2707/list resource endpoint (#18444) feat: list resources endpoint * Fix incorrect yaml in examples (#18463) * NET-5187: Upgrade test timeout due to log producer errors (#18461) Stop log producer before restarting container * [NET-4799] [OSS] xdsv2: listeners L4 support for connect proxies (#18436) * refactor to avoid future import cycles * resource: Make resource list tenancy aware (#18475) * [COMPLIANCE] License update (#18479) Update BUSL LICENSE to include licensed product and version. Co-authored-by: hashicorp-copywrite[bot] <110428419+hashicorp-copywrite[bot]@users.noreply.github.com> * docs: specify manual server metadata intervention (#18477) * docs: Update OpenShift compatibility (#18478) Update compatibility.mdx * resource: Make resource delete tenancy aware (#18476) resource: Make resource delete tenancy awarae * Add license-checker action that fails when any backported file contains BUSL header (#18485) * Add license-checker action that fails when any backported file contains BUSL header * Quote echoed variable to retain line breaks * Add ticket to reference for more details * docs: Update K8s TGW tutorial to reliably obtain role ID (#18474) The `grep…
trujillo-adam
pushed a commit
that referenced
this pull request
Sep 18, 2023
* Requested edit * Apply suggestions from code review Co-authored-by: trujillo-adam <[email protected]> --------- Co-authored-by: trujillo-adam <[email protected]> Resolve conflicts and manually cherry-pick
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description
This PR responds to a request to highlight the Kubernetes WAN Federation requirement that when ACLs are enabled, the Kube API of the secondary datacenters must be routable from the primary datacenter.
Links
PR Checklist