Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Proposed requirements for integration testing #310

Merged
merged 1 commit into from
Mar 2, 2023
Merged

Proposed requirements for integration testing #310

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
38 changes: 38 additions & 0 deletions docs/design/proposal/e2e-testing-v2.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,38 @@
# ACK e2e testing v2

## Goal

Increasing confidence in the release quality of core ACK libraries ([code-generator](https://github.com/aws-controllers-k8s/code-generator), [runtime](https://github.com/aws-controllers-k8s/runtime)) and controllers.

## Proposed requirements

### High-priority maintenance work

* [dockershim goes away in k8s v1.24](https://kubernetes.io/blog/2022/01/07/kubernetes-is-moving-on-from-dockershim) — our tests currently rely heavily on Docker and *we need our testing framework to be supported to k8s v1.24 and beyond*

### Individual controller testing (expanding on the existing e2e testing framework)

1. e2e tests that can be written as declarative YAML test files
* A list of steps in a YAML file - each step gets some configuration
* The only Python code that implementers need to write is glue for how to connect to their service
* Standardizes the way tests are written — potentially improves the quality of e2e tests and reduces amount of time required for implementing and reviewing test code

### Multi-controller testing framework (**new**)

1. The ability to test the following common run-time features with n >= 1 controllers:
* [AdoptedResource](https://aws-controllers-k8s.github.io/community/reference/common/v1alpha1/adoptedresource/)
* [FieldExport](https://aws-controllers-k8s.github.io/community/reference/common/v1alpha1/fieldexport/)
* [DeletionPolicy](https://aws-controllers-k8s.github.io/community/docs/user-docs/deletion-policy/)
* [Cross-account resource management](https://aws-controllers-k8s.github.io/community/docs/user-docs/cross-account-resource-management/)
* [Drift remediation](https://github.com/aws-controllers-k8s/community/issues/1367)
* [Multi-version support](https://github.com/aws-controllers-k8s/community/issues/835) ([related documentation task](https://github.com/aws-controllers-k8s/community/issues/1432))
jljaco marked this conversation as resolved.
Show resolved Hide resolved
* All CLI flag options
* Possible stretch goals:
* Cross-Region resource management
* [metrics](https://github.com/aws-controllers-k8s/runtime/tree/main/pkg/metrics)
jljaco marked this conversation as resolved.
Show resolved Hide resolved
* [tags](https://github.com/aws-controllers-k8s/runtime/tree/main/pkg/tags)
* [conditions management](https://github.com/aws-controllers-k8s/runtime/blob/main/pkg/runtime/reconciler.go#L310)

2. The interaction of multiple controllers in a cluster
* Cross-controller references - these are only applicable in certain concrete situations (e.g., EC2 referencing other services, IAM referencing EC2, MemoryDB referencing IAM, etc.)
jljaco marked this conversation as resolved.
Show resolved Hide resolved
* Soak tests with multiple controllers in the same cluster