-
Notifications
You must be signed in to change notification settings - Fork 77
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Implement base of continuous benchmark tool (#1776)
* Create Continuous Bench Search Job tool (#1733) * ✨ create bench job search tools Signed-off-by: vankichi <[email protected]> * ✨ add load hdf5 functions Signed-off-by: vankichi <[email protected]> * ♻️ fix format Signed-off-by: vankichi <[email protected]> * ♻️ fix docker and use hdf5 data Signed-off-by: vankichi <[email protected]> * ♻️ fix Signed-off-by: vankichi <[email protected]> * ♻️ refactor benchmark job Signed-off-by: vankichi <[email protected]> * ♻️ fix Signed-off-by: vankichi <[email protected]> * ♻️ fix proto Signed-off-by: vankichi <[email protected]> * 💚 add benchmark job image build ci Signed-off-by: vankichi <[email protected]> * 💚 invest Signed-off-by: vankichi <[email protected]> * Revert "💚 invest" This reverts commit f0f585c. * Apply suggestions from code review Co-authored-by: Yusuke Kato <[email protected]> * ♻️ apply code review Signed-off-by: vankichi <[email protected]> * Apply suggestions from code review Co-authored-by: Hiroto Funakoshi <[email protected]> * ✨ apply from feedback Signed-off-by: vankichi <[email protected]> * Update internal/config/benchmark.go Co-authored-by: Yusuke Kato <[email protected]> * ♻️ change directory path Signed-off-by: vankichi <[email protected]> Signed-off-by: vankichi <[email protected]> Co-authored-by: Yusuke Kato <[email protected]> Co-authored-by: Hiroto Funakoshi <[email protected]> * Add crds for continuous benchmark tools (#1789) * ✨ add crds for continuous benchmark operator Signed-off-by: vankichi <[email protected]> * ✨ add benchmark operator/job scheme Signed-off-by: vankichi <[email protected]> * ✨ rename package names and add doc.go Signed-off-by: vankichi <[email protected]> * ✨ create runtime object Signed-off-by: vankichi <[email protected]> * Apply suggestions from code review Co-authored-by: Yusuke Kato <[email protected]> * ♻️ apply feedback Signed-off-by: vankichi <[email protected]> Signed-off-by: vankichi <[email protected]> Co-authored-by: Yusuke Kato <[email protected]> * Add Job reconciler & Change directory constitution of internal/k8s for benchmark (#1825) * ✨ ♻️ add Job reconciler & use scenario instead of operator Signed-off-by: vankichi <[email protected]> * ♻️ fix format & rename file Signed-off-by: vankichi <[email protected]> Signed-off-by: vankichi <[email protected]> * Add benchmark operator framework (#1916) * ✨ impl benchmark reconciler Signed-off-by: vankichi <[email protected]> * ✨ create benchmark operator framework Signed-off-by: vankichi <[email protected]> * ♻️ remove unness changes Signed-off-by: vankichi <[email protected]> Signed-off-by: vankichi <[email protected]> * Format code with prettier and gofumpt * impl reconcile logic for create benchmark job (#1923) * ✨ impl reconcile logic for create benchmark job Signed-off-by: vankichi <[email protected]> * Format code with prettier and gofumpt * ♻️ fix Signed-off-by: vankichi <[email protected]> * ♻️ refactor continuous benchmark's crds Signed-off-by: vankichi <[email protected]> * ♻️ resolve error due to update conn bench crds for pkg/tools/benchmark/job Signed-off-by: vankichi <[email protected]> * ♻️ refactor continuous benchmark job logic Signed-off-by: vankichi <[email protected]> * Format code with prettier and gofumpt * ♻️ update charts Signed-off-by: vankichi <[email protected]> * Format code with prettier and gofumpt * ♻️ rafactor con bench config and bug fix reconcile logic Signed-off-by: vankichi <[email protected]> * 🐛 Bugfix: fix typo and recall function logic Signed-off-by: vankichi <[email protected]> * ♻️ refactor pkg benchmark job Signed-off-by: vankichi <[email protected]> --------- Signed-off-by: vankichi <[email protected]> Co-authored-by: deepsource-autofix[bot] <62050782+deepsource-autofix[bot]@users.noreply.github.com> --------- Signed-off-by: vankichi <[email protected]> Co-authored-by: Yusuke Kato <[email protected]> Co-authored-by: Hiroto Funakoshi <[email protected]> Co-authored-by: deepsource-autofix[bot] <62050782+deepsource-autofix[bot]@users.noreply.github.com>
- Loading branch information
4 people
committed
Dec 7, 2023
1 parent
9e9d684
commit 2696959
Showing
80 changed files
with
7,103 additions
and
5 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,129 @@ | ||
| # | ||
| # Copyright (C) 2019-2022 vdaas.org vald team <[email protected]> | ||
| # | ||
| # Licensed under the Apache License, Version 2.0 (the "License"); | ||
| # you may not use this file except in compliance with the License. | ||
| # You may obtain a copy of the License at | ||
| # | ||
| # https://www.apache.org/licenses/LICENSE-2.0 | ||
| # | ||
| # Unless required by applicable law or agreed to in writing, software | ||
| # distributed under the License is distributed on an "AS IS" BASIS, | ||
| # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. | ||
| # See the License for the specific language governing permissions and | ||
| # limitations under the License. | ||
| # | ||
| name: "Build docker image: benchmark-job" | ||
| on: | ||
| push: | ||
| branches: | ||
| - master | ||
| tags: | ||
| - "*.*.*" | ||
| - "v*.*.*" | ||
| - "*.*.*-*" | ||
| - "v*.*.*-*" | ||
| paths: | ||
| - ".github/actions/docker-build/actions.yaml" | ||
| - ".github/workflows/dockers-benchmak-job-image.yml" | ||
| - "go.mod" | ||
| - "go.sum" | ||
| - "internal/**" | ||
| - "!internal/**/*_test.go" | ||
| - "!internal/db/**" | ||
| - "!internal/k8s/**" | ||
| - "apis/grpc/**" | ||
| - "pkg/benchmark/job/**" | ||
| - "cmd/benchmark/job/**" | ||
| - "dockers/benchmark/job/Dockerfile" | ||
| - "versions/GO_VERSION" | ||
| - "versions/NGT_VERSION" | ||
| pull_request: | ||
| paths: | ||
| - ".github/actions/docker-build/actions.yaml" | ||
| - ".github/workflows/dockers-benchmak-job-image.yml" | ||
| - "go.mod" | ||
| - "go.sum" | ||
| - "internal/**" | ||
| - "!internal/**/*_test.go" | ||
| - "!internal/db/**" | ||
| - "!internal/k8s/**" | ||
| - "apis/grpc/**" | ||
| - "pkg/benchmark/job/**" | ||
| - "cmd/benchmark/job/**" | ||
| - "dockers/benchmark/job/Dockerfile" | ||
| - "versions/GO_VERSION" | ||
| - "versions/NGT_VERSION" | ||
|
|
||
| jobs: | ||
| build: | ||
| strategy: | ||
| max-parallel: 4 | ||
| runs-on: ubuntu-latest | ||
| steps: | ||
| - uses: actions/checkout@v3 | ||
| - name: Setup QEMU | ||
| uses: docker/setup-qemu-action@v1 | ||
| with: | ||
| platforms: all | ||
| - name: Setup Docker Buildx | ||
| id: buildx | ||
| uses: docker/setup-buildx-action@v1 | ||
| with: | ||
| buildkitd-flags: "--debug" | ||
| - name: Login to DockerHub | ||
| uses: docker/login-action@v1 | ||
| with: | ||
| username: ${{ secrets.DOCKERHUB_USER }} | ||
| password: ${{ secrets.DOCKERHUB_PASS }} | ||
| - name: Login to GitHub Container Registry | ||
| uses: docker/login-action@v1 | ||
| with: | ||
| registry: ghcr.io | ||
| username: ${{ secrets.PACKAGE_USER }} | ||
| password: ${{ secrets.PACKAGE_TOKEN }} | ||
| - name: Build and Publish | ||
| id: build_and_publish | ||
| uses: ./.github/actions/docker-build | ||
| with: | ||
| target: benchmark-job | ||
| builder: ${{ steps.buildx.outputs.name }} | ||
| - name: Initialize CodeQL | ||
| if: startsWith( github.ref, 'refs/tags/') | ||
| uses: github/codeql-action/init@v2 | ||
| - name: Run vulnerability scanner (table) | ||
| if: startsWith( github.ref, 'refs/tags/') | ||
| uses: aquasecurity/trivy-action@master | ||
| with: | ||
| image-ref: "${{ steps.build_and_publish.outputs.IMAGE_NAME }}:${{ steps.build_and_publish.outputs.PRIMARY_TAG }}" | ||
| format: "table" | ||
| - name: Run vulnerability scanner (sarif) | ||
| if: startsWith( github.ref, 'refs/tags/') | ||
| uses: aquasecurity/trivy-action@master | ||
| with: | ||
| image-ref: "${{ steps.build_and_publish.outputs.IMAGE_NAME }}:${{ steps.build_and_publish.outputs.PRIMARY_TAG }}" | ||
| format: "template" | ||
| template: "@/contrib/sarif.tpl" | ||
| output: "trivy-results.sarif" | ||
| - name: Upload Trivy scan results to Security tab | ||
| if: startsWith( github.ref, 'refs/tags/') | ||
| uses: github/codeql-action/upload-sarif@v2 | ||
| with: | ||
| sarif_file: "trivy-results.sarif" | ||
| slack: | ||
| name: Slack notification | ||
| needs: build | ||
| runs-on: ubuntu-latest | ||
| if: github.ref == 'refs/heads/master' || startsWith( github.ref, 'refs/tags/') | ||
| steps: | ||
| - uses: technote-space/workflow-conclusion-action@v2 | ||
| with: | ||
| GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }} | ||
| - uses: 8398a7/action-slack@v3 | ||
| with: | ||
| author_name: benchmark-job image build | ||
| status: ${{ env.WORKFLOW_CONCLUSION }} | ||
| only_mention_fail: channel | ||
| env: | ||
| GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }} | ||
| SLACK_WEBHOOK_URL: ${{ secrets.SLACK_NOTIFY_WEBHOOK_URL }} |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.