Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: Bootstrap webhook/component/integration/performance tests #225

Merged
merged 48 commits into from
Nov 7, 2022

Conversation

RealAnna
Copy link
Contributor

@RealAnna RealAnna commented Oct 24, 2022

#173

Research results

While DT uses fakeclients mostly, all other operator repo uses kubebuilder and ginkgo combo for E2E tests.
An example of suite test with configs file to setup testenv is here
Example of webhook test is here
An example of integration/Component test is here
Example integration test with kuttl

Kubebatch scheduler uses ginkgo also for performance/load tests https://github.com/kubernetes-sigs/kube-batch/tree/master/test/kubemark

Poc Description

In this Poc there are examples made for the operator and for the scheduler. Unit test uses a fakeclient while E2E/Component and load tests use kubebuilder+ testenv+ ginkgo. An integration test for the full cluster uses kuttl.

A simple report is generated for component and e2e tests:

suite_test.go  | passed
KeptnAppController should update the status of the CR  | passed
KeptnAppController should update the spans | passed
[Feature:Performance] Load KeptnAppController should create the app version CR | passed

Also a simple load test report :
Overall AppVersions created 100/100 Creation times: [2022-10-28 12:45:04.3396049 ...

Instructions on contributions are available :

component test instructions
e2e instructions

make file has been updated to have performance test, e2e and unit test too

Further improvements

  1. Define requirements for load tests
  2. List improvements for pipelines

@RealAnna RealAnna changed the title poc: create first example tests feat: create first example tests (poc) Oct 25, 2022
@RealAnna RealAnna force-pushed the poc/create_first_tests branch 2 times, most recently from 2c650fe to c4e7885 Compare October 28, 2022 09:35
@RealAnna RealAnna marked this pull request as ready for review October 28, 2022 11:06
@RealAnna RealAnna changed the title feat: create first example tests (poc) feat: Created first example tests(poc) Oct 28, 2022
@RealAnna RealAnna added the on hold Do not merge yet label Oct 28, 2022
thschue
thschue previously approved these changes Oct 31, 2022
Copy link
Contributor

@thschue thschue left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm

@codecov-commenter
Copy link

codecov-commenter commented Nov 4, 2022

Codecov Report

Merging #225 (58ea611) into main (847a460) will increase coverage by 6.29%.
The diff coverage is n/a.

Additional details and impacted files
@@            Coverage Diff            @@
##            main     #225      +/-   ##
=========================================
+ Coverage   5.03%   11.33%   +6.29%     
=========================================
  Files          9       12       +3     
  Lines        794     1032     +238     
=========================================
+ Hits          40      117      +77     
- Misses       749      902     +153     
- Partials       5       13       +8     
Impacted Files Coverage Δ
operator/controllers/keptnapp/controller.go 69.79% <ø> (ø)
scheduler/pkg/klcpermit/workflow_manager.go 11.23% <0.00%> (ø)
scheduler/pkg/klcpermit/permit.go 0.00% <0.00%> (ø)
Flag Coverage Δ
keptn-lifecycle-operator 13.19% <ø> (+7.59%) ⬆️
scheduler 4.52% <ø> (+4.52%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

@thschue thschue modified the milestones: 0.5.x, 0.4.x Nov 4, 2022
Signed-off-by: RealAnna <[email protected]>

feat: added example of unit and component test

Signed-off-by: RealAnna <[email protected]>

feat: fixed imports

Signed-off-by: RealAnna <[email protected]>

feat: example ginkgo test for app controller

Signed-off-by: RealAnna <[email protected]>

feat: example ginkgo test for app controller

Signed-off-by: RealAnna <[email protected]>

feat: example ginkgo test for app controller

Signed-off-by: RealAnna <[email protected]>

feat: example ginkgo test for app controller

Signed-off-by: RealAnna <[email protected]>

feat: added example unit test with mock

Signed-off-by: RealAnna <[email protected]>

feat: fixed bug

Signed-off-by: RealAnna <[email protected]>

feat: added test

Signed-off-by: RealAnna <[email protected]>

feat: update test suite with clean E2E test best practice

Signed-off-by: RealAnna <[email protected]>

feat: update test to avoid vet failure

Signed-off-by: RealAnna <[email protected]>

feat: init fake

Signed-off-by: RealAnna <[email protected]>

feat: added fake client for unit test of reconcilers

Signed-off-by: RealAnna <[email protected]>

feat: fixed appcontroller test

Signed-off-by: RealAnna <[email protected]>

feat: added a Developer.md file

Signed-off-by: RealAnna <[email protected]>

feat: fixed for autogen

Signed-off-by: RealAnna <[email protected]>

feat: clean up test add traces

Signed-off-by: RealAnna <[email protected]>

feat: added more tips

Signed-off-by: RealAnna <[email protected]>

feat: added stuff from k8s developer guide

Signed-off-by: RealAnna <[email protected]>

feat: added fix for main changes and silly load test example

Signed-off-by: RealAnna <[email protected]>

feat: added fix for maikefile

Signed-off-by: RealAnna <[email protected]>

feat: added fix for maikefile

Signed-off-by: RealAnna <[email protected]>

feat: added more robust init of tests

Signed-off-by: RealAnna <[email protected]>

feat: go mod

Signed-off-by: RealAnna <[email protected]>

feat: new developer instructions

Signed-off-by: RealAnna <[email protected]>

feat: added simple test reporter and load test fake metrics

Signed-off-by: RealAnna <[email protected]>

feat: added description on reporters

Signed-off-by: RealAnna <[email protected]>

feat: cleanup

Signed-off-by: RealAnna <[email protected]>

feat: cleanup

Signed-off-by: RealAnna <[email protected]>

feat: cleanup

Signed-off-by: RealAnna <[email protected]>

feat: cleanup

Signed-off-by: RealAnna <[email protected]>
Signed-off-by: RealAnna <[email protected]>
Signed-off-by: RealAnna <[email protected]>
Signed-off-by: RealAnna <[email protected]>
Signed-off-by: RealAnna <[email protected]>
Signed-off-by: RealAnna <[email protected]>
.github/workflows/e2e-test.yaml Show resolved Hide resolved
.github/workflows/e2e-test.yaml Show resolved Hide resolved
.github/workflows/e2e-test.yaml Outdated Show resolved Hide resolved
.github/workflows/integration-test.yaml Show resolved Hide resolved
.github/workflows/integration-test.yaml Show resolved Hide resolved
operator/Makefile Show resolved Hide resolved
scheduler/Makefile Show resolved Hide resolved
scheduler/Makefile Show resolved Hide resolved
scheduler/test/e2e/DEVELOPER.md Outdated Show resolved Hide resolved
test/integration/simple-deployment/01-assert.yaml Outdated Show resolved Hide resolved
Signed-off-by: RealAnna <[email protected]>
Signed-off-by: RealAnna <[email protected]>
@RealAnna RealAnna force-pushed the poc/create_first_tests branch from f8c3d73 to c4a44b7 Compare November 7, 2022 09:43
Signed-off-by: RealAnna <[email protected]>
Signed-off-by: RealAnna <[email protected]>
Signed-off-by: RealAnna <[email protected]>
thschue
thschue previously requested changes Nov 7, 2022
.github/workflows/release.yml Outdated Show resolved Hide resolved
.github/workflows/e2e-test.yaml Show resolved Hide resolved
operator/test/component/suite_test.go Outdated Show resolved Hide resolved
operator/test/e2e/suite_test.go Outdated Show resolved Hide resolved
Signed-off-by: RealAnna <[email protected]>
@RealAnna RealAnna requested a review from thschue November 7, 2022 12:15
kuttl-test.yaml Outdated Show resolved Hide resolved
@thisthat thisthat changed the title feat: Created first example tests feat: bootstrap webhook/component/integration/performance tests Nov 7, 2022
Signed-off-by: RealAnna <[email protected]>
@mowies mowies changed the title feat: bootstrap webhook/component/integration/performance tests feat: Bootstrap webhook/component/integration/performance tests Nov 7, 2022
Signed-off-by: RealAnna <[email protected]>
Signed-off-by: RealAnna <[email protected]>
@RealAnna RealAnna merged commit dbe08c0 into main Nov 7, 2022
@RealAnna RealAnna deleted the poc/create_first_tests branch November 7, 2022 15:02
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants