Add Custom Resource for mirror target (#2023)

* add mirror definition and update proto dependencies Signed-off-by: hlts2 <[email protected]> * Add mirror gateway component (#1949) * add mirror component Signed-off-by: hlts2 <[email protected]> * fix mirror dockerfile path Signed-off-by: hlts2 <[email protected]> * fix deepsource warning Signed-off-by: hlts2 <[email protected]> * fix insert rpc handler Signed-off-by: hlts2 <[email protected]> * deleted unused file Signed-off-by: hlts2 <[email protected]> * fix api resource name Signed-off-by: hlts2 <[email protected]> * add mirror service and fix handler logic Signed-off-by: hlts2 <[email protected]> * fix values and helm template for new configuration Signed-off-by: hlts2 <[email protected]> * Format code with gofumpt and prettier * fix template rendering bug of mirror configmap Signed-off-by: hlts2 <[email protected]> * fix nil pointer error Signed-off-by: hlts2 <[email protected]> * bugfix to advertize different address Signed-off-by: hlts2 <[email protected]> * returns error when there is no other mirror gateways Signed-off-by: hlts2 <[email protected]> * add info log Signed-off-by: hlts2 <[email protected]> * extract only mirror addresses Signed-off-by: hlts2 <[email protected]> * fix log variable Signed-off-by: hlts2 <[email protected]> * add logging Signed-off-by: hlts2 <[email protected]> * bugfix mirror connection logic Signed-off-by: hlts2 <[email protected]> * add remove api proxy implementation Signed-off-by: hlts2 <[email protected]> * add error handling for register API Signed-off-by: hlts2 <[email protected]> * fix debug message Signed-off-by: hlts2 <[email protected]> * refactor discover logic Signed-off-by: hlts2 <[email protected]> * refactor Signed-off-by: hlts2 <[email protected]> * add remove api Signed-off-by: hlts2 <[email protected]> * Format code with prettier and gofumpt * add remove api and remove api rollback logic Signed-off-by: hlts2 <[email protected]> * refactor Signed-off-by: hlts2 <[email protected]> * add update rpc handler Signed-off-by: hlts2 <[email protected]> * refactor context variable for grpc method Signed-off-by: hlts2 <[email protected]> * fix trace span name Signed-off-by: hlts2 <[email protected]> * add getObjects method Signed-off-by: hlts2 <[email protected]> * refactor update and remove handler Signed-off-by: hlts2 <[email protected]> * Apply suggestions from code review Co-authored-by: Kiichiro YUKAWA <[email protected]> * add multi handler implementation Signed-off-by: hlts2 <[email protected]> * fix mirror client trace span Signed-off-by: hlts2 <[email protected]> * rename discover service to mirror service Signed-off-by: hlts2 <[email protected]> * fix build error of bidirectional stream Signed-off-by: hlts2 <[email protected]> * add more error handling to search handler Signed-off-by: hlts2 <[email protected]> * add debug comment Signed-off-by: hlts2 <[email protected]> * add status code log Signed-off-by: hlts2 <[email protected]> * add status code log Signed-off-by: hlts2 <[email protected]> * add status code log Signed-off-by: hlts2 <[email protected]> * fix named return bug Signed-off-by: hlts2 <[email protected]> * add debug log Signed-off-by: hlts2 <[email protected]> * add debug command Signed-off-by: hlts2 <[email protected]> * add debug command Signed-off-by: hlts2 <[email protected]> * add debug command Signed-off-by: hlts2 <[email protected]> * add force status wrap Signed-off-by: hlts2 <[email protected]> * execute upsert method to local cluster's lb Signed-off-by: hlts2 <[email protected]> * add target debug log Signed-off-by: hlts2 <[email protected]> * fix delete unwrap of circuitbreaker Signed-off-by: hlts2 <[email protected]> * refactor error handling for new status parse function Signed-off-by: hlts2 <[email protected]> * Format code with prettier and gofumpt * refactor error handling Signed-off-by: hlts2 <[email protected]> * style: Format code with prettier and gofumpt * fix error handling for broadcast operation of getobject Signed-off-by: hlts2 <[email protected]> * fix error join bug of getobject Signed-off-by: hlts2 <[email protected]> * refactor Signed-off-by: hlts2 <[email protected]> * deleted unused code Signed-off-by: hlts2 <[email protected]> * fix status handling and add address to resource name Signed-off-by: hlts2 <[email protected]> * use vald client for update operation Signed-off-by: hlts2 <[email protected]> * add rollback test for mirror crud Signed-off-by: hlts2 <[email protected]> * style: Format code with prettier and gofumpt * fix variable name Signed-off-by: hlts2 <[email protected]> * refactor mirror servie Signed-off-by: hlts2 <[email protected]> * deleted unused code Signed-off-by: hlts2 <[email protected]> * fix trace attribute and refactor variable name Signed-off-by: hlts2 <[email protected]> * add error handling when crud rpc fails Signed-off-by: hlts2 <[email protected]> * deleted unused value Signed-off-by: hlts2 <[email protected]> * make format Signed-off-by: hlts2 <[email protected]> * Revert "make format" This reverts commit f605563. * Apply suggestions from code review Co-authored-by: Kiichiro YUKAWA <[email protected]> * apply suggestion Signed-off-by: hlts2 <[email protected]> * bugfix and refactor mirror client Signed-off-by: hlts2 <[email protected]> --------- Signed-off-by: hlts2 <[email protected]> Co-authored-by: deepsource-autofix[bot] <62050782+deepsource-autofix[bot]@users.noreply.github.com> Co-authored-by: Kiichiro YUKAWA <[email protected]> * add discovery logic for other mirror target Signed-off-by: hlts2 <[email protected]> * add k8s template file Signed-off-by: hlts2 <[email protected]> * add crd for mirror target resource Signed-off-by: hlts2 <[email protected]> * add new values for mirror discovery logic Signed-off-by: hlts2 <[email protected]> * style: Format code with gofumpt and prettier * fix schema register error and fix manifest to multi vald cluster Signed-off-by: hlts2 <[email protected]> * style: Format code with gofumpt and prettier * bugfix: adapt the service account and add early return Signed-off-by: hlts2 <[email protected]> * bugfix: usage of errors join Signed-off-by: hlts2 <[email protected]> * bugfix: type conversion bug Signed-off-by: hlts2 <[email protected]> * fix metadata hash logic and add mirror group function Signed-off-by: hlts2 <[email protected]> * add status handling for network connection Signed-off-by: hlts2 <[email protected]> * refactoring Signed-off-by: hlts2 <[email protected]> * bugfix: status update handling Signed-off-by: hlts2 <[email protected]> * bugfix: update status using status writer Signed-off-by: hlts2 <[email protected]> * add status timestamp Signed-off-by: hlts2 <[email protected]> * fix yaml field format Signed-off-by: hlts2 <[email protected]> * improve status changes handling Signed-off-by: hlts2 <[email protected]> * reduced number of status updates Signed-off-by: hlts2 <[email protected]> * added status check handling before changing status Signed-off-by: hlts2 <[email protected]> * update vald mirror target definition and add new cluster role Signed-off-by: hlts2 <[email protected]> * style: Format code with gofumpt and prettier * deploy multiple vald clusters and mirror target resource to connect with each other Signed-off-by: hlts2 <[email protected]> * fix e2d test to mirror-gateway and crd view field Signed-off-by: hlts2 <[email protected]> * define mirror app name Signed-off-by: hlts2 <[email protected]> * deleted unused code Signed-off-by: hlts2 <[email protected]> * fix golangci warning Signed-off-by: hlts2 <[email protected]> * crd generate command for mirror target using mirror-target-values Signed-off-by: hlts2 <[email protected]> * add string field validation Signed-off-by: hlts2 <[email protected]> * fix build error Signed-off-by: hlts2 <[email protected]> * add configuration comment Signed-off-by: hlts2 <[email protected]> * style: Format code with gofumpt and prettier * add image tag for debug Signed-off-by: hlts2 <[email protected]> * fix rebase error Signed-off-by: hlts2 <[email protected]> * fix wait timeout option Signed-off-by: hlts2 <[email protected]> * add new option to set mirror address Signed-off-by: hlts2 <[email protected]> * add new comment Signed-off-by: hlts2 <[email protected]> * bugfix for ingress configuration Signed-off-by: hlts2 <[email protected]> * not create lb ingress when mirror is enabled Signed-off-by: hlts2 <[email protected]> * not create filter ingress when mirror is enabled Signed-off-by: hlts2 <[email protected]> * add error handling for unimplemented codes error Signed-off-by: hlts2 <[email protected]> * bugfix register addr process Signed-off-by: hlts2 <[email protected]> * fix build error Signed-off-by: hlts2 <[email protected]> * Revert "update docker file for build error" This reverts commit f40986d. * add new package for docker build error Signed-off-by: hlts2 <[email protected]> * Add mirror gateway example (#2082) * add example code Signed-off-by: hlts2 <[email protected]> * execute search and getobject request to all vald clusters Signed-off-by: hlts2 <[email protected]> * fixed to send data to each cluster based on the dataset Signed-off-by: hlts2 <[email protected]> --------- Signed-off-by: hlts2 <[email protected]> * add mirror target metrics Signed-off-by: hlts2 <[email protected]> * fix example ngt dimension Signed-off-by: hlts2 <[email protected]> * make proto/all & make format Signed-off-by: hlts2 <[email protected]> --------- Signed-off-by: hlts2 <[email protected]> Co-authored-by: deepsource-autofix[bot] <62050782+deepsource-autofix[bot]@users.noreply.github.com> Co-authored-by: Kiichiro YUKAWA <[email protected]>
vdaas · Jul 4, 2023 · 96339a8 · 96339a8
1 parent fa08bb5
commit 96339a8
Show file tree

Hide file tree

Showing 38 changed files with 2,039 additions and 135 deletions.
diff --git a/Makefile b/Makefile
@@ -294,7 +294,7 @@ E2E_UPSERT_COUNT                   ?= 10
 E2E_REMOVE_COUNT                   ?= 3
 E2E_WAIT_FOR_CREATE_INDEX_DURATION ?= 8m
 E2E_TARGET_NAME                    ?= vald-lb-gateway
-E2E_TARGET_POD_NAME                ?= $(eval E2E_TARGET_POD_NAME := $(shell kubectl get pods --selector=app=$(E2E_TARGET_NAME) | tail -1 | cut -f1 -d " "))$(E2E_TARGET_POD_NAME)
+E2E_TARGET_POD_NAME                ?= $(eval E2E_TARGET_POD_NAME := $(shell kubectl get pods --selector=app=$(E2E_TARGET_NAME) -n $(E2E_TARGET_NAMESPACE) | tail -1 | cut -f1 -d " "))$(E2E_TARGET_POD_NAME)
 E2E_TARGET_NAMESPACE               ?= default
 E2E_TARGET_PORT                    ?= 8081
 E2E_PORTFORWARD_ENABLED            ?= true

diff --git a/Makefile.d/helm.mk b/Makefile.d/helm.mk
@@ -121,3 +121,14 @@ helm/schema/crd/vald-helm-operator: \
 	charts/vald-helm-operator/values.yaml > $(TEMP_DIR)/valdhelmoperatorrelease-spec.yaml
 	$(BINDIR)/yq eval-all 'select(fileIndex==0).spec.versions[0].schema.openAPIV3Schema.properties.spec = select(fileIndex==1).spec | select(fileIndex==0)' \
 	$(TEMP_DIR)/valdhelmoperatorrelease.yaml $(TEMP_DIR)/valdhelmoperatorrelease-spec.yaml > charts/vald-helm-operator/crds/valdhelmoperatorrelease.yaml
+
+.PHONY: helm/schema/crd/vald/mirror-target
+## generate OpenAPI v3 schema for ValdMirrorTarget
+helm/schema/crd/vald/mirror-target: \
+	yq/install
+	mv charts/vald/crds/valdmirrortarget.yaml  $(TEMP_DIR)/valdmirrortarget.yaml
+	GOPRIVATE=$(GOPRIVATE) \
+	go run -mod=readonly hack/helm/schema/crd/main.go \
+	charts/vald/schemas/mirror-target-values.yaml > $(TEMP_DIR)/valdmirrortarget-spec.yaml
+	$(BINDIR)/yq eval-all 'select(fileIndex==0).spec.versions[0].schema.openAPIV3Schema.properties.spec = select(fileIndex==1).spec | select(fileIndex==0)' \
+	$(TEMP_DIR)/valdmirrortarget.yaml $(TEMP_DIR)/valdmirrortarget-spec.yaml > charts/vald/crds/valdmirrortarget.yaml
diff --git a/Makefile.d/k8s.mk b/Makefile.d/k8s.mk
@@ -17,6 +17,7 @@
 MIRROR01_NAMESPACE = vald-01
 MIRROR02_NAMESPACE = vald-02
 MIRROR03_NAMESPACE = vald-03
+MIRROR_APP_NAME    = vald-mirror-gateway
 
 .PHONY: k8s/manifest/clean
 ## clean k8s manifests
@@ -123,6 +124,11 @@ k8s/multi/vald/deploy:
 		-f ./charts/vald/values/multi-vald/dev-vald-with-mirror.yaml \
 		-f ./charts/vald/values/multi-vald/dev-vald-03.yaml \
 		-n $(MIRROR03_NAMESPACE)
+	kubectl wait --for=condition=ready pod -l app=$(MIRROR_APP_NAME) --timeout=120s -n $(MIRROR01_NAMESPACE)
+	kubectl wait --for=condition=ready pod -l app=$(MIRROR_APP_NAME) --timeout=120s -n $(MIRROR02_NAMESPACE)
+	kubectl wait --for=condition=ready pod -l app=$(MIRROR_APP_NAME) --timeout=120s -n $(MIRROR03_NAMESPACE)
+	kubectl apply -f ./charts/vald/values/multi-vald/mirror-target.yaml \
+		-n $(MIRROR03_NAMESPACE)
 
 .PHONY: k8s/multi/vald/delete
 ## delete multiple vald sample clusters to k8s

diff --git a/apis/docs/v1/docs.md b/apis/docs/v1/docs.md
@@ -101,12 +101,12 @@
   - [Object](#vald-v1-Object)
 - [apis/proto/v1/vald/remove.proto](#apis_proto_v1_vald_remove-proto)
   - [Remove](#vald-v1-Remove)
+- [apis/proto/v1/vald/search.proto](#apis_proto_v1_vald_search-proto)
+  - [Search](#vald-v1-Search)
 - [apis/proto/v1/vald/update.proto](#apis_proto_v1_vald_update-proto)
   - [Update](#vald-v1-Update)
 - [apis/proto/v1/vald/upsert.proto](#apis_proto_v1_vald_upsert-proto)
   - [Upsert](#vald-v1-Upsert)
-- [apis/proto/v1/vald/search.proto](#apis_proto_v1_vald_search-proto)
-  - [Search](#vald-v1-Search)
 - [apis/proto/v1/mirror/mirror.proto](#apis_proto_v1_mirror_mirror-proto)
   - [Mirror](#mirror-v1-Mirror)
 - [Scalar Value Types](#scalar-value-types)
@@ -1043,6 +1043,33 @@ Remove service provides ways to remove indexed vectors.
 | StreamRemove | [.payload.v1.Remove.Request](#payload-v1-Remove-Request) stream    | [.payload.v1.Object.StreamLocation](#payload-v1-Object-StreamLocation) stream | A method to remove multiple indexed vectors by bidirectional streaming. |
 | MultiRemove  | [.payload.v1.Remove.MultiRequest](#payload-v1-Remove-MultiRequest) | [.payload.v1.Object.Locations](#payload-v1-Object-Locations)                  | A method to remove multiple indexed vectors in a single request.        |
 
+<a name="apis_proto_v1_vald_search-proto"></a>
+
+<p align="right"><a href="#top">Top</a></p>
+
+## apis/proto/v1/vald/search.proto
+
+<a name="vald-v1-Search"></a>
+
+### Search
+
+Search service provides ways to search indexed vectors.
+
+| Method Name            | Request Type                                                           | Response Type                                                                 | Description                                                                        |
+| ---------------------- | ---------------------------------------------------------------------- | ----------------------------------------------------------------------------- | ---------------------------------------------------------------------------------- |
+| Search                 | [.payload.v1.Search.Request](#payload-v1-Search-Request)               | [.payload.v1.Search.Response](#payload-v1-Search-Response)                    | A method to search indexed vectors by a raw vector.                                |
+| SearchByID             | [.payload.v1.Search.IDRequest](#payload-v1-Search-IDRequest)           | [.payload.v1.Search.Response](#payload-v1-Search-Response)                    | A method to search indexed vectors by ID.                                          |
+| StreamSearch           | [.payload.v1.Search.Request](#payload-v1-Search-Request) stream        | [.payload.v1.Search.StreamResponse](#payload-v1-Search-StreamResponse) stream | A method to search indexed vectors by multiple vectors.                            |
+| StreamSearchByID       | [.payload.v1.Search.IDRequest](#payload-v1-Search-IDRequest) stream    | [.payload.v1.Search.StreamResponse](#payload-v1-Search-StreamResponse) stream | A method to search indexed vectors by multiple IDs.                                |
+| MultiSearch            | [.payload.v1.Search.MultiRequest](#payload-v1-Search-MultiRequest)     | [.payload.v1.Search.Responses](#payload-v1-Search-Responses)                  | A method to search indexed vectors by multiple vectors in a single request.        |
+| MultiSearchByID        | [.payload.v1.Search.MultiIDRequest](#payload-v1-Search-MultiIDRequest) | [.payload.v1.Search.Responses](#payload-v1-Search-Responses)                  | A method to search indexed vectors by multiple IDs in a single request.            |
+| LinearSearch           | [.payload.v1.Search.Request](#payload-v1-Search-Request)               | [.payload.v1.Search.Response](#payload-v1-Search-Response)                    | A method to linear search indexed vectors by a raw vector.                         |
+| LinearSearchByID       | [.payload.v1.Search.IDRequest](#payload-v1-Search-IDRequest)           | [.payload.v1.Search.Response](#payload-v1-Search-Response)                    | A method to linear search indexed vectors by ID.                                   |
+| StreamLinearSearch     | [.payload.v1.Search.Request](#payload-v1-Search-Request) stream        | [.payload.v1.Search.StreamResponse](#payload-v1-Search-StreamResponse) stream | A method to linear search indexed vectors by multiple vectors.                     |
+| StreamLinearSearchByID | [.payload.v1.Search.IDRequest](#payload-v1-Search-IDRequest) stream    | [.payload.v1.Search.StreamResponse](#payload-v1-Search-StreamResponse) stream | A method to linear search indexed vectors by multiple IDs.                         |
+| MultiLinearSearch      | [.payload.v1.Search.MultiRequest](#payload-v1-Search-MultiRequest)     | [.payload.v1.Search.Responses](#payload-v1-Search-Responses)                  | A method to linear search indexed vectors by multiple vectors in a single request. |
+| MultiLinearSearchByID  | [.payload.v1.Search.MultiIDRequest](#payload-v1-Search-MultiIDRequest) | [.payload.v1.Search.Responses](#payload-v1-Search-Responses)                  | A method to linear search indexed vectors by multiple IDs in a single request.     |
+
 <a name="apis_proto_v1_vald_update-proto"></a>
 
 <p align="right"><a href="#top">Top</a></p>
@@ -1079,33 +1106,6 @@ Upsert service provides ways to insert/update vectors.
 | StreamUpsert | [.payload.v1.Upsert.Request](#payload-v1-Upsert-Request) stream    | [.payload.v1.Object.StreamLocation](#payload-v1-Object-StreamLocation) stream | A method to insert/update multiple vectors by bidirectional streaming. |
 | MultiUpsert  | [.payload.v1.Upsert.MultiRequest](#payload-v1-Upsert-MultiRequest) | [.payload.v1.Object.Locations](#payload-v1-Object-Locations)                  | A method to insert/update multiple vectors in a single request.        |
 
-<a name="apis_proto_v1_vald_search-proto"></a>
-
-<p align="right"><a href="#top">Top</a></p>
-
-## apis/proto/v1/vald/search.proto
-
-<a name="vald-v1-Search"></a>
-
-### Search
-
-Search service provides ways to search indexed vectors.
-
-| Method Name            | Request Type                                                           | Response Type                                                                 | Description                                                                        |
-| ---------------------- | ---------------------------------------------------------------------- | ----------------------------------------------------------------------------- | ---------------------------------------------------------------------------------- |
-| Search                 | [.payload.v1.Search.Request](#payload-v1-Search-Request)               | [.payload.v1.Search.Response](#payload-v1-Search-Response)                    | A method to search indexed vectors by a raw vector.                                |
-| SearchByID             | [.payload.v1.Search.IDRequest](#payload-v1-Search-IDRequest)           | [.payload.v1.Search.Response](#payload-v1-Search-Response)                    | A method to search indexed vectors by ID.                                          |
-| StreamSearch           | [.payload.v1.Search.Request](#payload-v1-Search-Request) stream        | [.payload.v1.Search.StreamResponse](#payload-v1-Search-StreamResponse) stream | A method to search indexed vectors by multiple vectors.                            |
-| StreamSearchByID       | [.payload.v1.Search.IDRequest](#payload-v1-Search-IDRequest) stream    | [.payload.v1.Search.StreamResponse](#payload-v1-Search-StreamResponse) stream | A method to search indexed vectors by multiple IDs.                                |
-| MultiSearch            | [.payload.v1.Search.MultiRequest](#payload-v1-Search-MultiRequest)     | [.payload.v1.Search.Responses](#payload-v1-Search-Responses)                  | A method to search indexed vectors by multiple vectors in a single request.        |
-| MultiSearchByID        | [.payload.v1.Search.MultiIDRequest](#payload-v1-Search-MultiIDRequest) | [.payload.v1.Search.Responses](#payload-v1-Search-Responses)                  | A method to search indexed vectors by multiple IDs in a single request.            |
-| LinearSearch           | [.payload.v1.Search.Request](#payload-v1-Search-Request)               | [.payload.v1.Search.Response](#payload-v1-Search-Response)                    | A method to linear search indexed vectors by a raw vector.                         |
-| LinearSearchByID       | [.payload.v1.Search.IDRequest](#payload-v1-Search-IDRequest)           | [.payload.v1.Search.Response](#payload-v1-Search-Response)                    | A method to linear search indexed vectors by ID.                                   |
-| StreamLinearSearch     | [.payload.v1.Search.Request](#payload-v1-Search-Request) stream        | [.payload.v1.Search.StreamResponse](#payload-v1-Search-StreamResponse) stream | A method to linear search indexed vectors by multiple vectors.                     |
-| StreamLinearSearchByID | [.payload.v1.Search.IDRequest](#payload-v1-Search-IDRequest) stream    | [.payload.v1.Search.StreamResponse](#payload-v1-Search-StreamResponse) stream | A method to linear search indexed vectors by multiple IDs.                         |
-| MultiLinearSearch      | [.payload.v1.Search.MultiRequest](#payload-v1-Search-MultiRequest)     | [.payload.v1.Search.Responses](#payload-v1-Search-Responses)                  | A method to linear search indexed vectors by multiple vectors in a single request. |
-| MultiLinearSearchByID  | [.payload.v1.Search.MultiIDRequest](#payload-v1-Search-MultiIDRequest) | [.payload.v1.Search.Responses](#payload-v1-Search-Responses)                  | A method to linear search indexed vectors by multiple IDs in a single request.     |
-
 <a name="apis_proto_v1_mirror_mirror-proto"></a>
 
 <p align="right"><a href="#top">Top</a></p>

diff --git a/apis/proto/v1/mirror/mirror.proto b/apis/proto/v1/mirror/mirror.proto
@@ -22,26 +22,24 @@ import "github.com/googleapis/googleapis/google/api/annotations.proto";
 
 option go_package = "github.com/vdaas/vald/apis/grpc/v1/mirror";
 option java_multiple_files = true;
-option java_package = "org.vdaas.vald.api.v1.mirror";
 option java_outer_classname = "ValdMirror";
-
+option java_package = "org.vdaas.vald.api.v1.mirror";
 
 // Represent the mirror service.
 service Mirror {
-
   // Register is the RPC to register other mirror servers.
   rpc Register(payload.v1.Mirror.Targets) returns (payload.v1.Mirror.Targets) {
     option (google.api.http) = {
-      post : "/mirror/register"
-      body : "*"
+      post: "/mirror/register"
+      body: "*"
     };
   }
 
   // Advertise is the RPC to advertise other mirror servers.
   rpc Advertise(payload.v1.Mirror.Targets) returns (payload.v1.Mirror.Targets) {
     option (google.api.http) = {
-      post : "/mirror/advertise"
-      body : "*"
+      post: "/mirror/advertise"
+      body: "*"
     };
   }
 }
diff --git a/apis/proto/v1/payload/payload.proto b/apis/proto/v1/payload/payload.proto
@@ -543,7 +543,6 @@ message Info {
 
 // Mirror related messages.
 message Mirror {
-
   // Represent server information.
   message Target {
     // The target hostname.

diff --git a/charts/vald/crds/valdmirrortarget.yaml b/charts/vald/crds/valdmirrortarget.yaml
@@ -0,0 +1,101 @@
+#
+# Copyright (C) 2019-2023 vdaas.org vald team <[email protected]>
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#    https://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+#
+apiVersion: apiextensions.k8s.io/v1
+kind: CustomResourceDefinition
+metadata:
+  name: valdmirrortargets.vald.vdaas.org
+spec:
+  group: vald.vdaas.org
+  names:
+    kind: ValdMirrorTarget
+    shortNames:
+      - vmt
+      - vmts
+    listKind: ValdMirrorTargetList
+    plural: valdmirrortargets
+    singular: valdmirrortarget
+  scope: Namespaced
+  versions:
+    - name: v1
+      served: true
+      storage: true
+      subresources:
+        status: {}
+      additionalPrinterColumns:
+        - name: Host
+          type: string
+          jsonPath: .spec.target.host
+          priority: 1
+        - name: Port
+          type: integer
+          jsonPath: .spec.target.port
+          priority: 1
+        - name: Status
+          type: string
+          jsonPath: .status.phase
+        - name: "Last Transition Time"
+          type: string
+          jsonPath: .status.lastTransitionTime
+        - name: Age
+          type: date
+          jsonPath: .metadata.creationTimestamp
+      schema:
+        openAPIV3Schema:
+          description: ValdMirrorTarget is the Schema for the valdmirrortargets API
+          type: object
+          properties:
+            apiVersion:
+              description: "APIVersion defines the versioned schema of this representation of an object. Servers should convert recognized schemas to the latest internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources"
+              type: string
+            kind:
+              description: "Kind is a string value representing the REST resource this object represents. Servers may infer this from the endpoint the client submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds"
+              type: string
+            metadata:
+              type: object
+            status:
+              description: ValdMirrorTargetStatus defines the observed state of ValdMirrorTarget
+              type: object
+              default: {}
+              properties:
+                phase:
+                  type: string
+                  enum:
+                    - Pending
+                    - Connected
+                    - Disconnected
+                    - Unknown
+                  default: Pending
+                lastTransitionTime:
+                  type: string
+            spec:
+              type: object
+              properties:
+                colocation:
+                  type: string
+                  minLength: 1
+                target:
+                  type: object
+                  properties:
+                    host:
+                      type: string
+                      minLength: 1
+                    port:
+                      type: integer
+                      maximum: 65535
+                      minimum: 0
+                  required:
+                    - host
+                    - port
diff --git a/charts/vald/schemas/mirror-target-values.yaml b/charts/vald/schemas/mirror-target-values.yaml
@@ -0,0 +1,25 @@
+#
+# Copyright (C) 2019-2023 vdaas.org vald team <[email protected]>
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#    https://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+#
+colocation: dc1
+# @schema {"name": "target", "type": "object", "required": ["host", "port"]}
+# target -- Mirror target information
+target:
+  # @schema {"name": "target.host", "type": "string", "minLength": 1}
+  # host -- Mirror target host name
+  host: "vald-mirror-gateway.vald.svc.cluster.local"
+  # @schema {"name": "target.port", "type": "integer", "minimum": 0, "maximum": 65535}
+  # target.port -- Mirror target port
+  port: 8081
diff --git a/charts/vald/templates/gateway/filter/ing.yaml b/charts/vald/templates/gateway/filter/ing.yaml
@@ -14,7 +14,8 @@
 # limitations under the License.
 #
 {{- $gateway := .Values.gateway.filter -}}
-{{- if and $gateway.enabled $gateway.ingress.enabled }}
+{{- $mgateway := .Values.gateway.mirror -}}
+{{- if and $gateway.enabled $gateway.ingress.enabled (not $mgateway.enabled) }}
 apiVersion: networking.k8s.io/v1
 kind: Ingress
 metadata:

diff --git a/charts/vald/templates/gateway/lb/ing.yaml b/charts/vald/templates/gateway/lb/ing.yaml
@@ -14,7 +14,8 @@
 # limitations under the License.
 #
 {{- $gateway := .Values.gateway.lb -}}
-{{- if and $gateway.enabled $gateway.ingress.enabled }}
+{{- $mgateway := .Values.gateway.mirror -}}
+{{- if and $gateway.enabled $gateway.ingress.enabled (not $mgateway.enabled) }}
 apiVersion: networking.k8s.io/v1
 kind: Ingress
 metadata: