Skip to content

Commit

Permalink
Merge branch 'main' into feature/mirror-gateway-definition
Browse files Browse the repository at this point in the history
  • Loading branch information
hlts2 committed Dec 6, 2023
2 parents 21b781f + 4fddfb0 commit 1aef5da
Show file tree
Hide file tree
Showing 77 changed files with 5,993 additions and 1,104 deletions.
2 changes: 1 addition & 1 deletion .github/ISSUE_TEMPLATE/bug_report.md
Original file line number Diff line number Diff line change
Expand Up @@ -22,7 +22,7 @@ assignees: ""

<!--- Please change the versions below along with your environment -->

- Go Version: 1.21.4
- Go Version: 1.21.5
- Docker Version: 20.10.8
- Kubernetes Version: v1.28.4
- NGT Version: 2.1.5
2 changes: 1 addition & 1 deletion .github/ISSUE_TEMPLATE/security_issue_report.md
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,7 @@ assignees: ""

<!--- Please change the versions below along with your environment -->

- Go Version: 1.21.4
- Go Version: 1.21.5
- Docker Version: 20.10.8
- Kubernetes Version: v1.28.4
- NGT Version: 2.1.5
2 changes: 1 addition & 1 deletion .github/PULL_REQUEST_TEMPLATE.md
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,7 @@

<!--- Please change the versions below along with your environment -->

- Go Version: 1.21.4
- Go Version: 1.21.5
- Docker Version: 20.10.8
- Kubernetes Version: v1.28.4
- NGT Version: 2.1.5
Expand Down
78 changes: 78 additions & 0 deletions .github/workflows/dockers-readreplica-rotate.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,78 @@
#
# Copyright (C) 2019-2023 vdaas.org vald team <[email protected]>
#
# Licensed under the Apache License, Version 2.0 (the "License");
# You may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# https://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
name: "Build docker image: readreplica-rotate"
on:
push:
branches:
- main
tags:
- "*.*.*"
- "v*.*.*"
- "*.*.*-*"
- "v*.*.*-*"
paths:
- ".github/actions/docker-build/actions.yaml"
- ".github/workflows/dockers-readreplica-rotate.yml"
- "go.mod"
- "go.sum"
- "internal/**"
- "!internal/**/*_test.go"
- "!internal/db/**"
- "!internal/k8s/**"
- "apis/grpc/**"
- "pkg/index/job/readreplica/rotate/**"
- "cmd/index/job/readreplica/rotate/**"
- "dockers/index/job/readreplica/rotate/Dockerfile"
- "versions/GO_VERSION"
pull_request:
paths:
- ".github/actions/docker-build/actions.yaml"
- ".github/workflows/_docker-image.yaml"
- ".github/workflows/dockers-readreplica-rotate.yml"
- "go.mod"
- "go.sum"
- "internal/**"
- "!internal/**/*_test.go"
- "!internal/db/**"
- "!internal/k8s/**"
- "apis/grpc/**"
- "pkg/index/job/readreplica/rotate/**"
- "cmd/index/job/readreplica/rotate/**"
- "dockers/index/job/readreplica/rotate/Dockerfile"
- "versions/GO_VERSION"
pull_request_target:
paths:
- ".github/actions/docker-build/actions.yaml"
- ".github/workflows/_docker-image.yaml"
- ".github/workflows/dockers-readreplica-rotate.yml"
- "go.mod"
- "go.sum"
- "internal/**"
- "!internal/**/*_test.go"
- "!internal/db/**"
- "!internal/k8s/**"
- "apis/grpc/**"
- "pkg/index/job/readreplica/rotate/**"
- "cmd/index/job/readreplica/rotate/**"
- "dockers/index/job/readreplica/rotate/Dockerfile"
- "versions/GO_VERSION"

jobs:
build:
uses: ./.github/workflows/_docker-image.yaml
with:
target: readreplica-rotate
secrets: inherit
114 changes: 114 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,119 @@
# CHANGELOG

## v1.7.9

### Docker images

<table>
<tr>
<th>component</th>
<th>Docker pull</th>
</tr>
<tr>
<td>Agent NGT</td>
<td>
<code>docker pull vdaas/vald-agent-ngt:v1.7.9</code><br/>
<code>docker pull ghcr.io/vdaas/vald/vald-agent-ngt:v1.7.9</code>
</td>
</tr>
<tr>
<td>Agent sidecar</td>
<td>
<code>docker pull vdaas/vald-agent-sidecar:v1.7.9</code><br/>
<code>docker pull ghcr.io/vdaas/vald/vald-agent-sidecar:v1.7.9</code>
</td>
</tr>
<tr>
<td>Discoverers</td>
<td>
<code>docker pull vdaas/vald-discoverer-k8s:v1.7.9</code><br/>
<code>docker pull ghcr.io/vdaas/vald/vald-discoverer-k8s:v1.7.9</code>
</td>
</tr>
<tr>
<td>Gateways</td>
<td>
<code>docker pull vdaas/vald-lb-gateway:v1.7.9</code><br/>
<code>docker pull ghcr.io/vdaas/vald/vald-lb-gateway:v1.7.9</code><br/>
<code>docker pull vdaas/vald-filter-gateway:v1.7.9</code><br/>
<code>docker pull ghcr.io/vdaas/vald/vald-filter-gateway:v1.7.9</code>
</td>
</tr>
<tr>
<td>Index Manager</td>
<td>
<code>docker pull vdaas/vald-manager-index:v1.7.9</code><br/>
<code>docker pull ghcr.io/vdaas/vald/vald-manager-index:v1.7.9</code>
</td>
</tr>
<tr>
<td>Helm Operator</td>
<td>
<code>docker pull vdaas/vald-helm-operator:v1.7.9</code><br/>
<code>docker pull ghcr.io/vdaas/vald/vald-helm-operator:v1.7.9</code>
</td>
</tr>
</table>

### Documents

- [GoDoc](https://pkg.go.dev/github.com/vdaas/[email protected])
- [Helm Chart Reference](https://github.com/vdaas/vald/blob/v1.7.9/charts/vald/README.md)
- [Helm Operator Chart Reference](https://github.com/vdaas/vald/blob/v1.7.9/charts/vald-helm-operator/README.md)

### Changes

:sparkles: New feature

- Add read replica rotator [#2241](https://github.com/vdaas/vald/pull/2241)
- Implement manifests for Index Management Job [#2235](https://github.com/vdaas/vald/pull/2235)
- Add job error to report index correction error status [#2231](https://github.com/vdaas/vald/pull/2231)
- Add implementation for save index job [#2227](https://github.com/vdaas/vald/pull/2227)
- Add implementation for create index job [#2223](https://github.com/vdaas/vald/pull/2223)
- Add index correction metrics [#2215](https://github.com/vdaas/vald/pull/2215)
- Add index correction document [#2217](https://github.com/vdaas/vald/pull/2217)
- Add make command to update template [#2212](https://github.com/vdaas/vald/pull/2212)
- Add job to check format difference [#2214](https://github.com/vdaas/vald/pull/2214)
- Add verification for index correction e2e and add clusterrole cronjobs for operator to deploy index correction [#2205](https://github.com/vdaas/vald/pull/2205)
- Add StreamListObject to LB [#2203](https://github.com/vdaas/vald/pull/2203)
- Add index correction helm templates and E2E [#2200](https://github.com/vdaas/vald/pull/2200)
- Add index correction internal logic [#2194](https://github.com/vdaas/vald/pull/2194)
- Add bbolt as internal/db/kvs [#2177](https://github.com/vdaas/vald/pull/2177)

:zap: Improve performance

- Improve index correction performance [#2234](https://github.com/vdaas/vald/pull/2234)

:recycle: Refactor

- Refactor Index Management Job [#2232](https://github.com/vdaas/vald/pull/2232)
- Fix invalid network policy schema [#2230](https://github.com/vdaas/vald/pull/2230)
- Add minikube to create volume snapshot development environment locally [#2228](https://github.com/vdaas/vald/pull/2228)
- Enable ingress resource in the get started document [#2211](https://github.com/vdaas/vald/pull/2211)
- Add step to get k3s latest version [#2206](https://github.com/vdaas/vald/pull/2206)
- Update telepresence and helm-docs installer and update deps [#2195](https://github.com/vdaas/vald/pull/2195)
- Replace x/slices with standard slices pkg [#2193](https://github.com/vdaas/vald/pull/2193)
- add benchmark and check program for core ngt [#2179](https://github.com/vdaas/vald/pull/2179)

:bug: Bugfix

- Revert vtpool for ResourceExhausted problem [#2255](https://github.com/vdaas/vald/pull/2255)
- Fix deleted contour ingress controller apply [#2229](https://github.com/vdaas/vald/pull/2229)

:pencil2: Document

- Add document for RemoveByTimestamp RPC [#2238](https://github.com/vdaas/vald/pull/2238)

:green_heart: CI

- Disable exhaustruct [#2240](https://github.com/vdaas/vald/pull/2240)
- Fix fails when there are format differences [#2226](https://github.com/vdaas/vald/pull/2226)

:arrow_up: Update dependencies

- update deps [#2208](https://github.com/vdaas/vald/pull/2208)
- update dependencies [#2260](https://github.com/vdaas/vald/pull/2260)

## v1.7.8

### Docker images
Expand Down
1 change: 1 addition & 0 deletions Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -34,6 +34,7 @@ LOADTEST_IMAGE = $(NAME)-loadtest
INDEX_CORRECTION_IMAGE = $(NAME)-index-correction
INDEX_CREATION_IMAGE = $(NAME)-index-creation
INDEX_SAVE_IMAGE = $(NAME)-index-save
READREPLICA_ROTATE_IMAGE = $(NAME)-readreplica-rotate
MANAGER_INDEX_IMAGE = $(NAME)-manager-index
MAINTAINER = "$(ORG).org $(NAME) team <$(NAME)@$(ORG).org>"

Expand Down
29 changes: 29 additions & 0 deletions Makefile.d/build.mk
Original file line number Diff line number Diff line change
Expand Up @@ -321,6 +321,35 @@ cmd/index/job/save/index-save: \
$(dir $@)main.go
$@ -version

cmd/index/job/readreplica/rotate/readreplica-rotate: \
$(GO_SOURCES_INTERNAL) \
$(PBGOS) \
$(shell find $(ROOTDIR)/cmd/index/job/readreplica/rotate -type f -name '*.go' -not -name '*_test.go' -not -name 'doc.go') \
$(shell find $(ROOTDIR)/pkg/index/job/readreplica/rotate -type f -name '*.go' -not -name '*_test.go' -not -name 'doc.go')
$(eval CGO_ENABLED = 0)
CGO_ENABLED=$(CGO_ENABLED) \
GO111MODULE=on \
GOPRIVATE=$(GOPRIVATE) \
go build \
--ldflags "-w -extldflags=-static \
-X '$(GOPKG)/internal/info.Version=$(VERSION)' \
-X '$(GOPKG)/internal/info.GitCommit=$(GIT_COMMIT)' \
-X '$(GOPKG)/internal/info.BuildTime=$(DATETIME)' \
-X '$(GOPKG)/internal/info.GoVersion=$(GO_VERSION)' \
-X '$(GOPKG)/internal/info.GoOS=$(GOOS)' \
-X '$(GOPKG)/internal/info.GoArch=$(GOARCH)' \
-X '$(GOPKG)/internal/info.CGOEnabled=$(CGO_ENABLED)' \
-X '$(GOPKG)/internal/info.BuildCPUInfoFlags=$(CPU_INFO_FLAGS)' \
-buildid=" \
-mod=readonly \
-modcacherw \
-a \
-tags "osusergo netgo static_build" \
-trimpath \
-o $@ \
$(dir $@)main.go
$@ -version

.PHONY: binary/build/zip
## build all binaries and zip them
binary/build/zip: \
Expand Down
14 changes: 14 additions & 0 deletions Makefile.d/docker.mk
Original file line number Diff line number Diff line change
Expand Up @@ -245,3 +245,17 @@ docker/build/index-save:
-t $(ORG)/$(INDEX_SAVE_IMAGE):$(TAG) . \
--build-arg MAINTAINER=$(MAINTAINER) \
--build-arg GO_VERSION=$(GO_VERSION)

.PHONY: docker/name/readreplica-rotate
docker/name/readreplica-rotate:
@echo "$(ORG)/$(READREPLICA_ROTATE_IMAGE)"

.PHONY: docker/build/readreplica-rotate
## build readreplica-rotate image
docker/build/readreplica-rotate:
$(DOCKER) build \
$(DOCKER_OPTS) \
-f dockers/index/job/readreplica/rotate/Dockerfile \
-t $(ORG)/$(READREPLICA_ROTATE_IMAGE):$(TAG) . \
--build-arg MAINTAINER=$(MAINTAINER) \
--build-arg GO_VERSION=$(GO_VERSION)
8 changes: 8 additions & 0 deletions Makefile.d/k8s.mk
Original file line number Diff line number Diff line change
Expand Up @@ -42,13 +42,15 @@ k8s/manifest/update: \
mkdir -p k8s/gateway
mkdir -p k8s/manager
mkdir -p k8s/index/job
mkdir -p k8s/index/job/readreplica
mv $(TEMP_DIR)/vald/templates/agent k8s/agent
mv $(TEMP_DIR)/vald/templates/discoverer k8s/discoverer
mv $(TEMP_DIR)/vald/templates/gateway/lb k8s/gateway/lb
mv $(TEMP_DIR)/vald/templates/manager/index k8s/manager/index
mv $(TEMP_DIR)/vald/templates/index/job/correction k8s/index/job/correction
mv $(TEMP_DIR)/vald/templates/index/job/creation k8s/index/job/creation
mv $(TEMP_DIR)/vald/templates/index/job/save k8s/index/job/save
mv $(TEMP_DIR)/vald/templates/index/job/readreplica/rotate k8s/index/job/readreplica/rotate
rm -rf $(TEMP_DIR)

.PHONY: k8s/manifest/helm-operator/clean
Expand Down Expand Up @@ -95,6 +97,9 @@ k8s/vald/deploy:
kubectl apply -f $(TEMP_DIR)/vald/crds || true
kubectl apply -f $(TEMP_DIR)/vald/templates/gateway/mirror || true
kubectl apply -f $(TEMP_DIR)/vald/templates/index/job/correction || true
kubectl apply -f $(TEMP_DIR)/vald/templates/index/job/creation || true
kubectl apply -f $(TEMP_DIR)/vald/templates/index/job/save || true
kubectl apply -f $(TEMP_DIR)/vald/templates/index/job/readreplica/rotate || true
rm -rf $(TEMP_DIR)
kubectl get pods -o jsonpath="{.items[*].spec.containers[*].image}" | tr " " "\n"

Expand All @@ -115,6 +120,9 @@ k8s/vald/delete:
--output-dir $(TEMP_DIR) \
charts/vald
kubectl delete -f $(TEMP_DIR)/vald/templates/gateway/mirror
kubectl delete -f $(TEMP_DIR)/vald/templates/index/job/readreplica/rotate
kubectl delete -f $(TEMP_DIR)/vald/templates/index/job/save
kubectl delete -f $(TEMP_DIR)/vald/templates/index/job/creation
kubectl delete -f $(TEMP_DIR)/vald/templates/index/job/correction
kubectl delete -f $(TEMP_DIR)/vald/templates/gateway/lb
kubectl delete -f $(TEMP_DIR)/vald/templates/manager/index
Expand Down
3 changes: 3 additions & 0 deletions Makefile.d/test.mk
Original file line number Diff line number Diff line change
Expand Up @@ -200,6 +200,9 @@ test/create-empty:
echo "Creating empty test file $$f"; \
package="$$(dirname $$f)" ; \
package="$$(basename $$package)" ; \
if [ "$$(basename $$f)" = "main.go" ]; then \
package="main"; \
fi; \
echo "package $$package" >> "$$f"; \
fi; \
done
Expand Down
2 changes: 1 addition & 1 deletion charts/vald-helm-operator/Chart.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,7 @@

apiVersion: v2
name: vald-helm-operator
version: v1.7.8
version: v1.7.9
description: A Helm chart for vald-helm-operator
type: application
keywords:
Expand Down
10 changes: 5 additions & 5 deletions charts/vald-helm-operator/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@

This is a Helm chart to install vald-helm-operator.

Current chart version is `v1.7.8`
Current chart version is `v1.7.9`

## Table of Contents

Expand All @@ -26,13 +26,13 @@ Run the following command to install the chart,

Please upgrade the CRDs first because Helm doesn't have a support to upgrade CRDs.

$ kubectl replace -f https://raw.githubusercontent.com/vdaas/vald/v1.7.8/charts/vald-helm-operator/crds/valdrelease.yaml
$ kubectl replace -f https://raw.githubusercontent.com/vdaas/vald/v1.7.8/charts/vald-helm-operator/crds/valdhelmoperatorrelease.yaml
$ kubectl replace -f https://raw.githubusercontent.com/vdaas/vald/v1.7.9/charts/vald-helm-operator/crds/valdrelease.yaml
$ kubectl replace -f https://raw.githubusercontent.com/vdaas/vald/v1.7.9/charts/vald-helm-operator/crds/valdhelmoperatorrelease.yaml

After upgrading CRDs, you can upgrade the operator.
If you're using `valdhelmoperatorrelease` (or `vhor`) resource, please update the `spec.image.tag` field of it.

$ kubectl patch vhor vhor-release -p '{"spec":{"image":{"tag":"v1.7.8"}}}'
$ kubectl patch vhor vhor-release -p '{"spec":{"image":{"tag":"v1.7.9"}}}'

On the other hand, please update the operator's deployment manually.

Expand Down Expand Up @@ -79,7 +79,7 @@ spec: {}
| healthPort | int | `8081` | port of health endpoint |
| image.pullPolicy | string | `"Always"` | image pull policy |
| image.repository | string | `"vdaas/vald-helm-operator"` | image repository |
| image.tag | string | `"v1.7.8"` | image tag |
| image.tag | string | `"v1.7.9"` | image tag |
| leaderElectionID | string | `"vald-helm-operator"` | name of the configmap that is used for holding the leader lock. |
| livenessProbe.enabled | bool | `true` | enable liveness probe. |
| livenessProbe.failureThreshold | int | `2` | liveness probe failure threshold |
Expand Down
Loading

0 comments on commit 1aef5da

Please sign in to comment.