Backport of NET-5397 - wire up destination golden tests from sidecar-…

…proxy controller for xds controller and xdsv2 into release/1.17.x (#19350) * server: run the api checks against the path without params (#19205) * Clone proto into deepcopy correctly (#19204) * chore: update version and nightly CI for 1.17 (#19208) Update version file to 1.18-dev, and replace 1.13 nightly test with 1.17. * mesh: add validation hook to proxy configuration (#19186) * mesh: add more validations to Destinations resource (#19202) * catalog, mesh: implement missing ACL hooks (#19143) This change adds ACL hooks to the remaining catalog and mesh resources, excluding any computed ones. Those will for now continue using the default operator:x permissions. It refactors a lot of the common testing functions so that they can be re-used between resources. There are also some types that we don't yet support (e.g. virtual IPs) that this change adds ACL hooks to for future-proofing. * NET-5073 - ProxyConfiguration: implement various connection options (#19187) * NET-5073 - ProxyConfiguration: implement various connection options * PR feedback - LocalConnection and InboundConnection do not affect exposed routes. configure L7 route destinations. fix connection proto sequence numbers. * add timeout to L7 Route Destinations * Relplat 897 copywrite bot workarounds (#19200) Co-authored-by: Ronald Ekambi <[email protected]> * mesh: add xRoute ACL hook tenancy tests (#19177) Enhance the xRoute ACL hook tests to cover tenanted situations. These tests will only execute in enterprise. * resource: enforce lowercase v2 resource names (#19218) * mesh: add DestinationPolicy ACL hook tenancy tests (#19178) Enhance the DestinationPolicy ACL hook tests to cover tenanted situations. These tests will only execute in enterprise. * catalog: add FailoverPolicy ACL hook tenancy test (#19179) * docs: Multi-port corrections (#19224) * typo fixes and instruction corrections * typo * link path correction * Add reason why port 53 is not used by default (#19222) * Update dns-configuration.mdx * Update website/content/docs/services/discovery/dns-configuration.mdx Co-authored-by: Tu Nguyen <[email protected]> * v2tenancy: rename v1alpha1 -> v2beta1 (#19227) * [NET-5944] security: Update Go version to 1.20.10 and `x/net` to 0.17.0 (#19225) * Bump golang.org/x/net to 0.17.0 This resolves [CVE-2023-39325](https://nvd.nist.gov/vuln/detail/CVE-2023-39325) / [CVE-2023-44487](https://nvd.nist.gov/vuln/detail/CVE-2023-44487). * Update Go version to 1.20.10 This resolves [CVE-2023-39325](https://nvd.nist.gov/vuln/detail/CVE-2023-39325) / [CVE-2023-44487](https://nvd.nist.gov/vuln/detail/CVE-2023-44487) (`net/http`). * NET-6097 - sidecar proxy controller - give name to first failover policy target (#19239) * Cc 5545: Upgrade HDS packages and modifiers (#19226) * Upgrade @hashicorp/design-system-tokens to 1.9.0 * Upgrade @hashicorp/design-system-components to 1.8.1 * Upgrade @hashicorp/design-system-components and ember-in-viewport * Explicitly install [email protected] * rename copy-button * Fix how cleanup is done in with-copyable * Update aria-menu modifier for new structure * Update css-prop modifier to new structure * Convert did-upsert to regular class modifier * Update notification modifier for new structure * Update on-oustside modifier for new structure * Move destroy handler registration in with-copyable * Update style modifier for new structure * Update validate modifier for new structure * Guard against setting on destroyed object * Upgrade @hashicorp/design-system-components to 2.14.1 * Remove debugger * Guard against null in aria-menu * Fix undefined hash in validate addon * Upgrade ember-on-resize-modifier * Fix copy button import, missing import and array destructuring --------- Co-authored-by: wenincode <[email protected]> * [NET-5810] CE changes for multiple virtual hosts (#19246) CE changes for multiple virtual hosts * Net 4893- Ensure we're testing all the latest versions of Vault/Nomad (#19119) * NET-5592 - update Nomad integration testing * NET-4893: Ensure we're testing all the latest versions of Vault/Nomad * docs: Fix example control-plane-request-limit HCL and JSON (#19105) The control-plane-request-limit config entry does not support specifying parameter names in snake case format. This commit updates the HCL and JSON examples to use the supported camel case key format. * test: add 1.17 nightly integrations test (#19253) * fix expose paths (#19257) When testing adding http probes to apps, I ran into some issues which I fixed here: - The listener should be listening on the exposed listener port, updated that. - The listener and route names were pointing to the path of the exposed path. In my test, the path was "/" resulting in an empty string path. Also, the path may not be unique across exposed path listeners, so I decided to use the path+exposed port as the unique identifier. * docs: Multiport HCP constraint update (#19261) * Add sentence * link text adjustment * docs: Fix multi-port install (#19262) * Update configure.mdx Co-authored-by: Jeff Boruszak <[email protected]> * Prevent circular dependencies between v2 resources and generate a mermaid diagram with their dependencies (#19230) * build(docker): always publish full and minor version tags for dev images (#19278) * fix nightly integration test: envoy version and n-2 version (#19286) * [NET-6221] Ensure LB policy set for locality-aware routing (CE) (#19283) Ensure LB policy set for locality-aware routing (CE) `overprovisioningFactor` should be overridden with the expected value (100,000) when there are multiple endpoint groups. Update code and tests to enforce this. This is an Enterprise feature. This commit represents the CE portions of the change; tests are added in the corresponding `consul-enterprise` change. * fix: allow snake case keys for ip based rate limit config entry (#19277) * fix: allow snake case keys for ip based rate limit config entry * chore: add changelog * reformatted the JSON schema server conf ref (#19288) * acls,catalog,mesh: properly authorize workload selectors on writes (#19260) To properly enforce writes on resources that have workload selectors with prefixes, we need another service authorization rule that allows us to check whether read is allowed within a given prefix. Specifically we need to only allow writes if the policy prefix allows for a wider set of names than the prefix selector on the resource. We should also not allow policies with exact names for prefix matches. Part of [NET-3993] * NET-6239: Temporarily disable verify envoy check (#19299) * skip verify envoy version * cleanup * Update supported Envoy versions (#19276) * mesh: provide missing domain to route configurations in ProxyStateTemplate (#19298) * add empty domains * update unit tests * enable verify envoy script (#19303) * Vault CA bugfixes (#19285) * Re-add retry logic to Vault token renewal * Fix goroutine leak * Add test for detecting goroutine leak * Add changelog * Rename tests * Add comment * Backout Envoy 1.28.0 (#19306) * added ent to ce downgrade changes (#19311) * added ent to ce downgrade changes * added changelog * added busl headers * skip envoy version check in ci (#19315) * Tenancy Bridge v2 (#19220) * tenancy bridge v2 for v2 resources * add missing copywrite headers * remove branch name causing conflicts (#19319) * mesh: ensure route configs are named uniquely per port (#19323) * [NET-5327] Templated policies api/cli docs (#19270) * More templated policies docs (#19312) [NET-5327]More templated policies docs * Fixing docs to add more templated policies references (#19335) * Upgrade Consul UI to Node 18 (#19252) * Upgrading node to node 18 * Ensure we're on latest version of yarn as well * add comma to make frontend tests run * Use Node 18 Alpine image in UI build dockerfile * delete package-lock.json --------- Co-authored-by: wenincode <[email protected]> Co-authored-by: Ella Cai <[email protected]> Co-authored-by: Ella Cai <[email protected]> * resource: default peername to local in list endpoints (#19340) * NET-5397 - wire up golden tests from sidecar-proxy controller for xds controller and xdsv2 * WIP * WIP * everything matching except leafCerts. need to mock those * single port destinations working except mixed destinations * golden test input to xds controller tests for destinations * proposed fix for failover group naming errors * clean up test to use helper. * clean up test to use helper. * fix test file * add docstring for test function. * add docstring for test function. * fix linting error * backport of commit b5fa4f2 * backport of commit e7ff94b --------- Co-authored-by: R.B. Boyer <[email protected]> Co-authored-by: Ashwin Venkatesh <[email protected]> Co-authored-by: Michael Zalimeni <[email protected]> Co-authored-by: Iryna Shustava <[email protected]> Co-authored-by: John Murret <[email protected]> Co-authored-by: modrake <[email protected]> Co-authored-by: Ronald Ekambi <[email protected]> Co-authored-by: Semir Patel <[email protected]> Co-authored-by: Jeff Boruszak <[email protected]> Co-authored-by: David Yu <[email protected]> Co-authored-by: Tu Nguyen <[email protected]> Co-authored-by: Chris Hut <[email protected]> Co-authored-by: wenincode <[email protected]> Co-authored-by: John Maguire <[email protected]> Co-authored-by: Sophie Gairo <[email protected]> Co-authored-by: Blake Covarrubias <[email protected]> Co-authored-by: Dan Stough <[email protected]> Co-authored-by: Nitya Dhanushkodi <[email protected]> Co-authored-by: Eric Haberkorn <[email protected]> Co-authored-by: cskh <[email protected]> Co-authored-by: Poonam Jadhav <[email protected]> Co-authored-by: trujillo-adam <[email protected]> Co-authored-by: Anita Akaeze <[email protected]> Co-authored-by: Chris Thain <[email protected]> Co-authored-by: Chris S. Kim <[email protected]> Co-authored-by: aahel <[email protected]> Co-authored-by: Dhia Ayachi <[email protected]> Co-authored-by: Ronald <[email protected]> Co-authored-by: Ella Cai <[email protected]> Co-authored-by: Ella Cai <[email protected]>
hashicorp · Oct 24, 2023 · 10f5181 · 10f5181
1 parent 8b35e08
commit 10f5181
Show file tree

Hide file tree

Showing 12 changed files with 2,534 additions and 1 deletion.
diff --git a/internal/mesh/internal/controllers/xds/controller_test.go b/internal/mesh/internal/controllers/xds/controller_test.go
@@ -7,10 +7,14 @@ import (
 	"context"
 	"crypto/x509"
 	"encoding/pem"
+	"fmt"
+	"strings"
 	"testing"
 
+	"github.com/hashicorp/consul/internal/testing/golden"
 	"github.com/stretchr/testify/require"
 	"github.com/stretchr/testify/suite"
+	"google.golang.org/protobuf/encoding/protojson"
 
 	svctest "github.com/hashicorp/consul/agent/grpc-external/services/resource/testing"
 	"github.com/hashicorp/consul/agent/leafcert"
@@ -995,3 +999,158 @@ func (suite *xdsControllerTestSuite) TestReconcile_prevWatchesToCancel() {
 func TestXdsController(t *testing.T) {
 	suite.Run(t, new(xdsControllerTestSuite))
 }
+
+// TestReconcile_SidecarProxyGoldenFileInputs tests the Reconcile() by using
+// the golden test output/expected files from the sidecar proxy tests as inputs
+// to the XDS controller reconciliation.
+// XDS controller reconciles the full ProxyStateTemplate object.  The fields
+// that things that it focuses on are leaf certs, endpoints, and trust bundles,
+// which is just a subset of the ProxyStateTemplate struct.  Prior to XDS controller
+// reconciliation, the sidecar proxy controller will have reconciled the other parts
+// of the ProxyStateTemplate.
+// Since the XDS controller does act on the ProxyStateTemplate, the tests
+// utilize that entire object rather than just the parts that XDS controller
+// internals reconciles.  Namely, by using checking the full ProxyStateTemplate
+// rather than just endpoints, leaf certs, and trust bundles, the test also ensures
+// side effects or change in scope to XDS controller are not introduce mistakenly.
+func (suite *xdsControllerTestSuite) TestReconcile_SidecarProxyGoldenFileInputs() {
+	path := "../sidecarproxy/builder/testdata"
+	cases := []string{
+		// destinations
+		"destination/l4-single-destination-ip-port-bind-address",
+		"destination/l4-single-destination-unix-socket-bind-address",
+		"destination/l4-single-implicit-destination-tproxy",
+		"destination/l4-multi-destination",
+		"destination/l4-multiple-implicit-destinations-tproxy",
+		"destination/l4-implicit-and-explicit-destinations-tproxy",
+		"destination/mixed-multi-destination",
+		"destination/multiport-l4-and-l7-multiple-implicit-destinations-tproxy",
+		"destination/multiport-l4-and-l7-single-implicit-destination-tproxy",
+		"destination/multiport-l4-and-l7-single-implicit-destination-with-multiple-workloads-tproxy",
+
+		//sources
+
+	}
+
+	for _, name := range cases {
+		suite.Run(name, func() {
+			// Create ProxyStateTemplate from the golden file.
+			pst := JSONToProxyTemplate(suite.T(),
+				golden.GetBytesAtFilePath(suite.T(), fmt.Sprintf("%s/%s.golden", path, name)))
+
+			// Destinations will need endpoint refs set up.
+			if strings.Split(name, "/")[0] == "destination" && len(pst.ProxyState.Endpoints) == 0 {
+				suite.addRequiredEndpointsAndRefs(pst)
+			}
+
+			// Store the initial ProxyStateTemplate.
+			proxyStateTemplate := resourcetest.Resource(pbmesh.ProxyStateTemplateType, "test").
+				WithData(suite.T(), pst).
+				Write(suite.T(), suite.client)
+
+			// Check with resource service that it exists.
+			retry.Run(suite.T(), func(r *retry.R) {
+				suite.client.RequireResourceExists(r, proxyStateTemplate.Id)
+			})
+
+			// Track it in the mapper.
+			suite.mapper.TrackItem(proxyStateTemplate.Id, []resource.ReferenceOrID{})
+
+			// Run the reconcile, and since no ProxyStateTemplate is stored, this simulates a deletion.
+			err := suite.ctl.Reconcile(context.Background(), suite.runtime, controller.Request{
+				ID: proxyStateTemplate.Id,
+			})
+			require.NoError(suite.T(), err)
+			require.NotNil(suite.T(), proxyStateTemplate)
+
+			// Get the reconciled proxyStateTemplate to check the reconcile results.
+			reconciledPS := suite.updater.Get(proxyStateTemplate.Id.Name)
+
+			// Verify leaf cert contents then hard code them for comparison
+			// and downstream tests since they change from test run to test run.
+			require.NotEmpty(suite.T(), reconciledPS.LeafCertificates)
+			reconciledPS.LeafCertificates = map[string]*pbproxystate.LeafCertificate{
+				"test-identity": {
+					Cert: "-----BEGIN CERTIFICATE-----\nMIICDjCCAbWgAwIBAgIBAjAKBggqhkjOPQQDAjAUMRIwEAYDVQQDEwlUZXN0IENB\nIDEwHhcNMjMxMDE2MTYxMzI5WhcNMjMxMDE2MTYyMzI5WjAAMFkwEwYHKoZIzj0C\nAQYIKoZIzj0DAQcDQgAErErAIosDPheZQGbxFQ4hYC/e9Fi4MG9z/zjfCnCq/oK9\nta/bGT+5orZqTmdN/ICsKQDhykxZ2u/Xr6845zhcJaOCAQowggEGMA4GA1UdDwEB\n/wQEAwIDuDAdBgNVHSUEFjAUBggrBgEFBQcDAgYIKwYBBQUHAwEwDAYDVR0TAQH/\nBAIwADApBgNVHQ4EIgQg3ogXVz9cqaK2B6xdiJYMa5NtT0KkYv7BA2dR7h9EcwUw\nKwYDVR0jBCQwIoAgq+C1mPlPoGa4lt7sSft1goN5qPGyBIB/3mUHJZKSFY8wbwYD\nVR0RAQH/BGUwY4Zhc3BpZmZlOi8vMTExMTExMTEtMjIyMi0zMzMzLTQ0NDQtNTU1\nNTU1NTU1NTU1LmNvbnN1bC9hcC9kZWZhdWx0L25zL2RlZmF1bHQvaWRlbnRpdHkv\ndGVzdC1pZGVudGl0eTAKBggqhkjOPQQDAgNHADBEAiB6L+t5bzRrBPhiQYNeA7fF\nUCuLWrdjW4Xbv3SLg0IKMgIgfRC5hEx+DqzQxTCP4sexX3hVWMjKoWmHdwiUcg+K\n/IE=\n-----END CERTIFICATE-----\n",
+					Key:  "-----BEGIN EC PRIVATE KEY-----\nMHcCAQEEIFIFkTIL1iUV4O/RpveVHzHs7ZzhSkvYIzbdXDttz9EooAoGCCqGSM49\nAwEHoUQDQgAErErAIosDPheZQGbxFQ4hYC/e9Fi4MG9z/zjfCnCq/oK9ta/bGT+5\norZqTmdN/ICsKQDhykxZ2u/Xr6845zhcJQ==\n-----END EC PRIVATE KEY-----\n",
+				},
+			}
+
+			// Compare actual vs expected.
+			actual := prototest.ProtoToJSON(suite.T(), reconciledPS)
+			expected := golden.Get(suite.T(), actual, name+".golden")
+			require.JSONEq(suite.T(), expected, actual)
+		})
+	}
+}
+
+func (suite *xdsControllerTestSuite) addRequiredEndpointsAndRefs(pst *pbmesh.ProxyStateTemplate) {
+	//get service data
+	serviceData := &pbcatalog.Service{}
+	var vp uint32 = 7000
+	requiredEps := make(map[string]*pbproxystate.EndpointRef)
+
+	// iterate through clusters and set up endpoints for cluster/mesh port.
+	for clusterName := range pst.ProxyState.Clusters {
+		if clusterName == "null_route_cluster" || clusterName == "original-destination" {
+			continue
+		}
+
+		//increment the random port number.
+		vp++
+		clusterNameSplit := strings.Split(clusterName, ".")
+		port := clusterNameSplit[0]
+		svcName := clusterNameSplit[1]
+
+		// set up service data with port info.
+		serviceData.Ports = append(serviceData.Ports, &pbcatalog.ServicePort{
+			TargetPort:  port,
+			VirtualPort: vp,
+			Protocol:    pbcatalog.Protocol_PROTOCOL_TCP,
+		})
+
+		// create service.
+		svc := resourcetest.Resource(pbcatalog.ServiceType, svcName).
+			WithData(suite.T(), &pbcatalog.Service{}).
+			Write(suite.T(), suite.client)
+
+		// create endpoints with svc as owner.
+		eps := resourcetest.Resource(pbcatalog.ServiceEndpointsType, svcName).
+			WithData(suite.T(), &pbcatalog.ServiceEndpoints{Endpoints: []*pbcatalog.Endpoint{
+				{
+					Ports: map[string]*pbcatalog.WorkloadPort{
+						"mesh": {
+							Port:     20000,
+							Protocol: pbcatalog.Protocol_PROTOCOL_MESH,
+						},
+					},
+					Addresses: []*pbcatalog.WorkloadAddress{
+						{
+							Host:  "10.1.1.1",
+							Ports: []string{"mesh"},
+						},
+					},
+				},
+			}}).
+			WithOwner(svc.Id).
+			Write(suite.T(), suite.client)
+
+		// add to working list of required endpoints.
+		requiredEps[clusterName] = &pbproxystate.EndpointRef{
+			Id:   eps.Id,
+			Port: "mesh",
+		}
+	}
+
+	// set working list of required endpoints as proxy state's RequiredEndpoints.
+	pst.RequiredEndpoints = requiredEps
+}
+
+func JSONToProxyTemplate(t *testing.T, json []byte) *pbmesh.ProxyStateTemplate {
+	t.Helper()
+	proxyTemplate := &pbmesh.ProxyStateTemplate{}
+	m := protojson.UnmarshalOptions{}
+	err := m.Unmarshal(json, proxyTemplate)
+	require.NoError(t, err)
+	return proxyTemplate
+}
diff --git a/.../controllers/xds/testdata/destination/l4-implicit-and-explicit-destinations-tproxy.golden b/.../controllers/xds/testdata/destination/l4-implicit-and-explicit-destinations-tproxy.golden
@@ -0,0 +1,182 @@
+{
+  "clusters": {
+    "original-destination": {
+      "endpointGroup": {
+        "passthrough": {
+          "config": {
+            "connectTimeout": "5s"
+          }
+        }
+      },
+      "name": "original-destination"
+    },
+    "tcp.api-1.default.dc1.internal.foo.consul": {
+      "altStatName": "tcp.api-1.default.dc1.internal.foo.consul",
+      "endpointGroup": {
+        "dynamic": {
+          "config": {
+            "connectTimeout": "5s",
+            "disablePanicThreshold": true
+          },
+          "outboundTls": {
+            "alpnProtocols": [
+              "consul~tcp"
+            ],
+            "outboundMesh": {
+              "identityKey": "test-identity",
+              "sni": "api-1.default.dc1.internal.foo.consul",
+              "validationContext": {
+                "spiffeIds": [
+                  "spiffe://foo.consul/ap/default/ns/default/identity/api1-identity"
+                ],
+                "trustBundlePeerNameKey": "local"
+              }
+            }
+          }
+        }
+      },
+      "name": "tcp.api-1.default.dc1.internal.foo.consul"
+    },
+    "tcp.api-2.default.dc1.internal.foo.consul": {
+      "altStatName": "tcp.api-2.default.dc1.internal.foo.consul",
+      "endpointGroup": {
+        "dynamic": {
+          "config": {
+            "connectTimeout": "5s",
+            "disablePanicThreshold": true
+          },
+          "outboundTls": {
+            "alpnProtocols": [
+              "consul~tcp"
+            ],
+            "outboundMesh": {
+              "identityKey": "test-identity",
+              "sni": "api-2.default.dc1.internal.foo.consul",
+              "validationContext": {
+                "spiffeIds": [
+                  "spiffe://foo.consul/ap/default/ns/default/identity/api2-identity"
+                ],
+                "trustBundlePeerNameKey": "local"
+              }
+            }
+          }
+        }
+      },
+      "name": "tcp.api-2.default.dc1.internal.foo.consul"
+    }
+  },
+  "identity": {
+    "name": "test-identity",
+    "tenancy": {
+      "namespace": "default",
+      "partition": "default",
+      "peerName": "local"
+    },
+    "type": {
+      "group": "auth",
+      "groupVersion": "v2beta1",
+      "kind": "WorkloadIdentity"
+    }
+  },
+  "listeners": [
+    {
+      "direction": "DIRECTION_OUTBOUND",
+      "hostPort": {
+        "host": "1.1.1.1",
+        "port": 1234
+      },
+      "name": "default/local/default/api-1:tcp:1.1.1.1:1234",
+      "routers": [
+        {
+          "l4": {
+            "cluster": {
+              "name": "tcp.api-1.default.dc1.internal.foo.consul"
+            },
+            "statPrefix": "upstream.tcp.api-1.default.default.dc1"
+          }
+        }
+      ]
+    },
+    {
+      "capabilities": [
+        "CAPABILITY_TRANSPARENT"
+      ],
+      "defaultRouter": {
+        "l4": {
+          "cluster": {
+            "name": "original-destination"
+          },
+          "statPrefix": "upstream.original-destination"
+        }
+      },
+      "direction": "DIRECTION_OUTBOUND",
+      "hostPort": {
+        "host": "127.0.0.1",
+        "port": 15001
+      },
+      "name": "outbound_listener",
+      "routers": [
+        {
+          "l4": {
+            "cluster": {
+              "name": "tcp.api-2.default.dc1.internal.foo.consul"
+            },
+            "statPrefix": "upstream.tcp.api-2.default.default.dc1"
+          },
+          "match": {
+            "destinationPort": 7070,
+            "prefixRanges": [
+              {
+                "addressPrefix": "2.2.2.2",
+                "prefixLen": 32
+              },
+              {
+                "addressPrefix": "3.3.3.3",
+                "prefixLen": 32
+              }
+            ]
+          }
+        }
+      ]
+    }
+  ],
+  "endpoints": {
+    "tcp.api-1.default.dc1.internal.foo.consul": {
+      "endpoints": [
+        {
+          "healthStatus": "HEALTH_STATUS_HEALTHY",
+          "hostPort": {
+            "host": "10.1.1.1",
+            "port": 20000
+          }
+        }
+      ]
+    },
+    "tcp.api-2.default.dc1.internal.foo.consul": {
+      "endpoints": [
+        {
+          "healthStatus": "HEALTH_STATUS_HEALTHY",
+          "hostPort": {
+            "host": "10.1.1.1",
+            "port": 20000
+          }
+        }
+      ]
+    }
+  },
+  "trustBundles": {
+    "local": {
+      "roots": [
+        "some-root",
+        "some-other-root"
+      ],
+      "trustDomain": "some-trust-domain"
+    }
+  },
+  "leafCertificates": {
+    "test-identity": {
+      "cert": "-----BEGIN CERTIFICATE-----\nMIICDjCCAbWgAwIBAgIBAjAKBggqhkjOPQQDAjAUMRIwEAYDVQQDEwlUZXN0IENB\nIDEwHhcNMjMxMDE2MTYxMzI5WhcNMjMxMDE2MTYyMzI5WjAAMFkwEwYHKoZIzj0C\nAQYIKoZIzj0DAQcDQgAErErAIosDPheZQGbxFQ4hYC/e9Fi4MG9z/zjfCnCq/oK9\nta/bGT+5orZqTmdN/ICsKQDhykxZ2u/Xr6845zhcJaOCAQowggEGMA4GA1UdDwEB\n/wQEAwIDuDAdBgNVHSUEFjAUBggrBgEFBQcDAgYIKwYBBQUHAwEwDAYDVR0TAQH/\nBAIwADApBgNVHQ4EIgQg3ogXVz9cqaK2B6xdiJYMa5NtT0KkYv7BA2dR7h9EcwUw\nKwYDVR0jBCQwIoAgq+C1mPlPoGa4lt7sSft1goN5qPGyBIB/3mUHJZKSFY8wbwYD\nVR0RAQH/BGUwY4Zhc3BpZmZlOi8vMTExMTExMTEtMjIyMi0zMzMzLTQ0NDQtNTU1\nNTU1NTU1NTU1LmNvbnN1bC9hcC9kZWZhdWx0L25zL2RlZmF1bHQvaWRlbnRpdHkv\ndGVzdC1pZGVudGl0eTAKBggqhkjOPQQDAgNHADBEAiB6L+t5bzRrBPhiQYNeA7fF\nUCuLWrdjW4Xbv3SLg0IKMgIgfRC5hEx+DqzQxTCP4sexX3hVWMjKoWmHdwiUcg+K\n/IE=\n-----END CERTIFICATE-----\n",
+      "key": "-----BEGIN EC PRIVATE KEY-----\nMHcCAQEEIFIFkTIL1iUV4O/RpveVHzHs7ZzhSkvYIzbdXDttz9EooAoGCCqGSM49\nAwEHoUQDQgAErErAIosDPheZQGbxFQ4hYC/e9Fi4MG9z/zjfCnCq/oK9ta/bGT+5\norZqTmdN/ICsKQDhykxZ2u/Xr6845zhcJQ==\n-----END EC PRIVATE KEY-----\n"
+    }
+  }
+}