Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Admission Controller (Webhook) for Akri Configuration(s) #206

Merged
merged 83 commits into from
Feb 24, 2021
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
83 commits
Select commit Hold shift + click to select a range
64b306d
Helm Chart updates
DazWilkin Jan 14, 2021
0c8ce09
Add 'webhooks' to the project
DazWilkin Jan 14, 2021
9e42e89
Commented Helm Chart `webhook-configuration`
DazWilkin Jan 14, 2021
080530d
Template `webhook-configuration`
DazWilkin Jan 14, 2021
3f43c1e
Webhook service
DazWilkin Jan 14, 2021
7c5e816
Dockerfile (draft)
DazWilkin Jan 14, 2021
b0e4af2
GitHub Action (draft)
DazWilkin Jan 14, 2021
562143d
Force build
DazWilkin Jan 14, 2021
492b555
Trying again
DazWilkin Jan 14, 2021
a288591
And again
DazWilkin Jan 14, 2021
a483149
Added GHCR secret to fork
DazWilkin Jan 14, 2021
3c33620
Grrr
DazWilkin Jan 14, 2021
21ffa76
Grrr
DazWilkin Jan 14, 2021
1117b5c
Issue w/ GitHub Action build
DazWilkin Jan 14, 2021
b6eca93
Undo
DazWilkin Jan 14, 2021
4174259
Consistency
DazWilkin Jan 14, 2021
50c4c28
Revised Rust Webhook to not require these
DazWilkin Jan 14, 2021
01d4ded
Preliminary README
DazWilkin Jan 14, 2021
6478473
Update `version.sh` and run `./version.sh -u -s`
DazWilkin Jan 15, 2021
f717254
Added `webhook-configuration`
DazWilkin Jan 15, 2021
530b436
Does this resolve Helm Linting concern?
DazWilkin Jan 15, 2021
801d3f0
r/AMD32V7/AMR32V7
DazWilkin Jan 15, 2021
2761bde
Are these required?
DazWilkin Jan 15, 2021
fe3d133
Follow pattern
DazWilkin Jan 15, 2021
5dd09a2
Follow Akri pattern
DazWilkin Jan 15, 2021
596c39b
r/webhook-configuration/webhookConfiguration
DazWilkin Jan 15, 2021
0d40538
correct path references
DazWilkin Jan 15, 2021
e11f5cd
Merge branch 'main' into webhook-configuration
DazWilkin Jan 15, 2021
106c27d
Cross-build Rust
DazWilkin Jan 15, 2021
1ccac5c
Exceptions no longer required
DazWilkin Jan 15, 2021
acc8cd8
Assume `./target/release/webhook-configuration`
DazWilkin Jan 15, 2021
2b37785
Expose x-plat targets to docker
DazWilkin Jan 15, 2021
16b6513
Omission
DazWilkin Jan 15, 2021
b0538f3
Improve handling of flags
DazWilkin Jan 15, 2021
60f2581
Tidy
DazWilkin Jan 15, 2021
cc4640b
Correct indendation
DazWilkin Jan 16, 2021
5e1c340
Version bump
DazWilkin Jan 16, 2021
ec75674
Update `CHANGELOG`
DazWilkin Jan 16, 2021
8340bfc
`cargo.lock` versions now bumped too
DazWilkin Jan 19, 2021
8ab5bbb
Temporarily (!) bumping build timeouts
DazWilkin Jan 19, 2021
93158ce
Corrected component reference
DazWilkin Jan 19, 2021
7ee4e38
Missed `build-arg=CROSS_BUILD_TARGET=...`
DazWilkin Jan 19, 2021
f98b976
Avoid Helm issue with un-`named` `Lists`
DazWilkin Jan 20, 2021
365dd09
Revert `CHANGELOG`
DazWilkin Jan 27, 2021
b32f50f
Added unit tests
DazWilkin Feb 3, 2021
9387a04
Filters Akri Configuration manifest so that it validates
DazWilkin Feb 4, 2021
cc52a7b
Added test for filtered `generation`
DazWilkin Feb 4, 2021
42d6dd8
Added test for extended (complete) Config
DazWilkin Feb 5, 2021
28f0f3d
To avoid GitHub Actions build failure
DazWilkin Feb 8, 2021
999e827
End-to-End Webhook tests
DazWilkin Feb 8, 2021
7fe7a67
Integration tests require `actix-rt`
DazWilkin Feb 8, 2021
345700d
Merge branch 'main' into webhook-configuration
DazWilkin Feb 8, 2021
1ac3dc5
Include `webhook-configuration` container image
DazWilkin Feb 8, 2021
9ac2365
Learns all the things!
DazWilkin Feb 8, 2021
d7e0bbc
Typo
DazWilkin Feb 9, 2021
bf8e62c
Added `webhook` to `test-case`s
DazWilkin Feb 9, 2021
019709a
Removed `get_agent_and_controller` reference
DazWilkin Feb 9, 2021
8dbe9c4
Simplified Actions `matrix`; Updated Webhook certs
DazWilkin Feb 9, 2021
3588d13
Override `webhookConfiguration.image.tag` to `pr-amd64`
DazWilkin Feb 9, 2021
5e82996
Add debugging
DazWilkin Feb 9, 2021
03adc3e
Typo
DazWilkin Feb 9, 2021
ff846af
sudo'ing `kubectl` commands
DazWilkin Feb 9, 2021
d5c9eab
Try subprocess; reduce test matrix
DazWilkin Feb 10, 2021
2a79b16
Add `shell=True`
DazWilkin Feb 10, 2021
6a811fb
More `describe`s
DazWilkin Feb 10, 2021
79b0a49
Correct `kubectl --selector` commands
DazWilkin Feb 10, 2021
effd520
Reflect r/ENTRYPOINT/CMD in Dockerfile
DazWilkin Feb 10, 2021
5dc2623
Try matrix
DazWilkin Feb 10, 2021
c27b5f7
Debugging Kubenernetes 1.16
DazWilkin Feb 10, 2021
56ee546
Additional MicroK8s 1.16+1.17 and debugging
DazWilkin Feb 10, 2021
206336f
Generate: `*-webhook-log` from GitHub Actions
DazWilkin Feb 11, 2021
2e5a53a
Add some debugging statements
DazWilkin Feb 11, 2021
b1e3e90
K8s 1.16 expects AdmissionReview to be v1beta1
DazWilkin Feb 11, 2021
56e9584
Checking K3s 1.16+1.17
DazWilkin Feb 11, 2021
a720388
Ordering may be significant
DazWilkin Feb 11, 2021
67e2ccc
Reverted
DazWilkin Feb 12, 2021
26de1fe
Don't delete CRDs; Remove redundant debugging
DazWilkin Feb 12, 2021
53bb1bc
Limit to distro-versions known to work for E2Es
DazWilkin Feb 12, 2021
5f4a640
Merge branch 'main' into webhook-configuration
DazWilkin Feb 22, 2021
600b93a
Bumped OpenAPI-generated sources due to Akri security audit issues
DazWilkin Feb 23, 2021
438061a
Documents `run-webook.py`
DazWilkin Feb 24, 2021
58c876f
Typo
DazWilkin Feb 24, 2021
6fc791e
`expect` only when providing useful message
DazWilkin Feb 24, 2021
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 8 additions & 0 deletions .dockerignore
Original file line number Diff line number Diff line change
Expand Up @@ -39,18 +39,26 @@
!target/x86_64-unknown-linux-gnu/release/agent
!target/x86_64-unknown-linux-gnu/debug/udev-video-broker
!target/x86_64-unknown-linux-gnu/release/udev-video-broker
!target/x86_64-unknown-linux-gnu/debug/webhook-configuration
!target/x86_64-unknown-linux-gnu/release/webhook-configuration

!target/aarch64-unknown-linux-gnu/debug/controller
!target/aarch64-unknown-linux-gnu/release/controller
!target/aarch64-unknown-linux-gnu/debug/agent
!target/aarch64-unknown-linux-gnu/release/agent
!target/aarch64-unknown-linux-gnu/debug/udev-video-broker
!target/aarch64-unknown-linux-gnu/release/udev-video-broker
!target/aarch64-unknown-linux-gnu/debug/webhook-configuration
!target/aarch64-unknown-linux-gnu/release/webhook-configuration

!target/armv7-unknown-linux-gnueabihf/debug/controller
!target/armv7-unknown-linux-gnueabihf/release/controller
!target/armv7-unknown-linux-gnueabihf/debug/agent
!target/armv7-unknown-linux-gnueabihf/release/agent
!target/armv7-unknown-linux-gnueabihf/debug/udev-video-broker
!target/armv7-unknown-linux-gnueabihf/release/udev-video-broker
!target/armv7-unknown-linux-gnueabihf/debug/webhook-configuration
!target/armv7-unknown-linux-gnueabihf/release/webhook-configuration

# Cross toml file needs to be available for making the cross build containers
!Cross.toml
Expand Down
106 changes: 106 additions & 0 deletions .github/workflows/build-webhook-configuration-container.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,106 @@
name: Build Webhook Configuration

on:
workflow_dispatch:
inputs:
push:
branches: [main]
paths:
- .github/actions/build-component-per-arch/**
- .github/actions/build-component-multi-arch/**
- .github/workflows/build-webhook-configuration-container.yml
- build/containers/Dockerfile.webhook-configuration
- webhooks/validating/configuration
- version.txt
- build/akri-containers.mk
- Makefile
pull_request:
branches: [main]
paths:
- .github/actions/build-component-per-arch/**
- .github/actions/build-component-multi-arch/**
- .github/workflows/build-workflow-configuration-container.yml
- build/containers/Dockerfile.workflow-configuration
- webhooks/validating/configuration
- version.txt
- build/akri-containers.mk
- Makefile
release:
types:
- published

env:
AKRI_COMPONENT: webhook-configuration
MAKEFILE_COMPONENT: webhook-configuration

jobs:
per-arch:
runs-on: ubuntu-latest
timeout-minutes: 60
strategy:
matrix:
arch:
- arm64v8
- arm32v7
- amd64

steps:
- name: Checkout the head commit of the branch
uses: actions/checkout@v2
with:
persist-credentials: false

- name: Prepare To Install
uses: actions/setup-node@v1
with:
node-version: 12
- name: Install Deps
run: |
yarn install
yarn add @actions/core @actions/github @actions/exec fs

- name: Run Per-Arch component build for ${{ env.AKRI_COMPONENT }}
uses: ./.github/actions/build-component-per-arch
with:
github_event_name: ${{ github.event_name }}
github_ref: ${{ github.ref }}
github_event_action: ${{ github.event.action }}
github_merged: ${{ github.event.pull_request.merged }}
container_name: ${{ env.AKRI_COMPONENT }}
container_prefix: ghcr.io/deislabs/akri
container_registry_base_url: ghcr.io
container_registry_username: ${{ secrets.crUsername }}
container_registry_password: ${{ secrets.crPassword }}
makefile_component_name: ${{ env.MAKEFILE_COMPONENT }}
platform: ${{ matrix.arch }}
build_rust: "1"

multi-arch:
if: (github.event_name == 'release') || (github.event_name == 'push' && github.ref == 'refs/heads/main') || (startsWith(github.event_name, 'pull_request') && github.event.action == 'closed' && github.event.pull_request.merged == true && github.ref != 'refs/heads/main')
needs: per-arch
runs-on: ubuntu-latest
timeout-minutes: 60

steps:
- name: Checkout repo
uses: actions/checkout@v2

- name: Prepare To Install
uses: actions/setup-node@v1
with:
node-version: 12
- name: Install Deps
run: |
yarn install
yarn add @actions/core @actions/github @actions/exec fs

- name: Run Multi-Arch component build for ${{ env.AKRI_COMPONENT }}
uses: ./.github/actions/build-component-multi-arch
with:
github_event_name: ${{ github.event_name }}
container_name: ${{ env.AKRI_COMPONENT }}
container_prefix: ghcr.io/deislabs/akri
container_registry_base_url: ghcr.io
container_registry_username: ${{ secrets.crUsername }}
container_registry_password: ${{ secrets.crPassword }}
makefile_component_name: ${{ env.MAKEFILE_COMPONENT }}
29 changes: 27 additions & 2 deletions .github/workflows/run-test-cases.yml
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,7 @@ on:
- test/run-end-to-end.py
- test/run-conservation-of-broker-pod.py
- test/run-helm-install-delete.py
- test/run-webhook.py
- test/shared_test_code.py
- .github/workflows/run-test-cases.yml
- build/containers/Dockerfile.agent
Expand All @@ -26,6 +27,7 @@ on:
- test/run-end-to-end.py
- test/run-conservation-of-broker-pod.py
- test/run-helm-install-delete.py
- test/run-webhook.py
- test/shared_test_code.py
- .github/workflows/run-test-cases.yml
- build/containers/Dockerfile.agent
Expand Down Expand Up @@ -64,8 +66,10 @@ jobs:
make akri-build
make controller-build-amd64
make agent-build-amd64
make webhook-configuration-build-amd64
docker save ${PREFIX}/agent:${LABEL_PREFIX}-amd64 > agent.tar
docker save ${PREFIX}/controller:${LABEL_PREFIX}-amd64 > controller.tar
docker save ${PREFIX}/webhook-configuration:${LABEL_PREFIX}-amd64 > webhook-configuration.tar

- name: Upload Agent container as artifact
if: startsWith(github.event_name, 'pull_request')
Expand All @@ -79,6 +83,12 @@ jobs:
with:
name: controller.tar
path: controller.tar
- name: Upload Webhook-Configuration container as artifact
if: startsWith(github.event_name, 'pull_request')
uses: actions/upload-artifact@v2
with:
name: webhook-configuration.tar
path: webhook-configuration.tar

test-cases:
needs: build-containers
Expand Down Expand Up @@ -137,6 +147,8 @@ jobs:
test:
- case: end-to-end
file: test/run-end-to-end.py
- case: webhook
file: test/run-webhook.py

steps:
- name: Checkout the head commit of the branch
Expand All @@ -163,6 +175,11 @@ jobs:
uses: actions/download-artifact@v2
with:
name: controller.tar
- name: Download Webhook-Configuration container artifact
if: startsWith(github.event_name, 'pull_request')
uses: actions/download-artifact@v2
with:
name: webhook-configuration.tar

- if: startsWith(matrix.kube.runtime, 'K3s')
name: Install K3s
Expand Down Expand Up @@ -197,6 +214,7 @@ jobs:
run: |
sudo k3s ctr image import agent.tar
sudo k3s ctr image import controller.tar
sudo k3s ctr image import webhook-configuration.tar

- if: startsWith(matrix.kube.runtime, 'Kubernetes')
name: Install Kubernetes
Expand All @@ -222,6 +240,7 @@ jobs:
run: |
sudo docker load --input agent.tar
sudo docker load --input controller.tar
sudo docker load --input webhook-configuration.tar

- if: startsWith(matrix.kube.runtime, 'MicroK8s')
name: Install MicroK8s
Expand All @@ -234,7 +253,6 @@ jobs:
sudo chown -f -R $USER $HOME/.kube --verbose
sudo sh -c "microk8s.kubectl config view --raw >~/.kube/config"
sudo cat ~/.kube/config
# sudo microk8s.enable helm3
sudo microk8s.enable rbac
sudo microk8s.enable dns
until sudo microk8s.status --wait-ready; do sleep 5s; echo "Try again"; done
Expand All @@ -253,6 +271,7 @@ jobs:
sudo microk8s ctr images ls
sudo microk8s ctr --debug --timeout 10s images import agent.tar
sudo microk8s ctr --debug --timeout 10s images import controller.tar
sudo microk8s ctr --debug --timeout 10s images import webhook-configuration.tar
sudo microk8s ctr images ls

- name: Add Akri Helm Chart
Expand All @@ -270,7 +289,7 @@ jobs:
run: |
git fetch origin main
git show origin/main:version.txt > /tmp/version_to_test.txt
echo '--set agent.image.pullPolicy=Never,agent.image.tag=pr-amd64,controller.image.pullPolicy=Never,controller.image.tag=pr-amd64' > /tmp/extra_helm_args.txt
echo '--set agent.image.pullPolicy=Never,agent.image.tag=pr-amd64,controller.image.pullPolicy=Never,controller.image.tag=pr-amd64,webhookConfiguration.image.pullPolicy=Never,webhookConfiguration.image.tag=pr-amd64' > /tmp/extra_helm_args.txt
# For non-PR (i.e. push, release, manual), version.txt is corresponds
# to an existing Helm chart.
- if: (!(startsWith(github.event_name, 'pull_request')))
Expand Down Expand Up @@ -311,3 +330,9 @@ jobs:
with:
name: ${{ matrix.kube.runtime }}-${{ matrix.test.case }}-controller-log
path: /tmp/controller_log.txt
- name: Upload webhook log as artifact
if: always()
uses: actions/upload-artifact@v2
with:
name: ${{ matrix.kube.runtime }}-${{ matrix.test.case }}-webhook-log
path: /tmp/webhook_log.txt
8 changes: 4 additions & 4 deletions Cargo.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

7 changes: 1 addition & 6 deletions Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -4,9 +4,4 @@
h2 = { git = "https://github.com/kate-goldenring/h2", branch = "master" }

[workspace]
members = [
"shared",
"controller",
"agent",
"samples/brokers/udev-video-broker"
]
members = ["shared", "controller", "agent", "samples/brokers/udev-video-broker", "webhooks/validating/configuration"]
2 changes: 1 addition & 1 deletion agent/Cargo.toml
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
[package]
name = "agent"
version = "0.1.18"
version = "0.2.0"
authors = ["Kate Goldenring <[email protected]>", "<[email protected]>"]
edition = "2018"

Expand Down
Loading