chore(ci): add end to end testing mechanism

[mploski]

Issue number: #1226

Summary

This PR implements proposal shared in RFC

IT add mechanism to run an end to end tests on Lambda Powertools library using real AWS Services (Lambda, DynamoDB) in real AWS Account.
Initially, tests can be run manually by maintainers on specific branch to ensure expected feature works.
Tests should be triggered in GitHub but also maintainers/contributors should be able to run them in their local environment using their own AWS Account.

In order to run them manually:

1. Export your AWS_PROFILE to point tests to your AWS Account2.
2. Run make e2e-test from project root directory

Changes

1. Modify Makefile to add new e2e-test command automating test execution. pytest uses x-dist plugin to parallelize execution over test groups.
2. Add new test groups (logger/metrics/tracer) under tests/e2e directory. They contains pytest tests that calls lambda endpoint and fetch information about execution from CloudWatch Metrics/CloudWatch Logs/X-Ray API. We store Lambda handler code under handler/ sub directory. We use this handlers to spawn new lambdas during infrastructure deployment
3. Under tests/e2e/utils directory we store:

• infrastructure.py - contains class responsible for spawning infrastructure in test account
• helpers.py - stores helpers methods here that simplify fetching data from CloudWatch Metrics/CloudWatch Logs/X-Ray API
• Dockerfile - used by CDK code to create custom lambda powertools layer

4. Add session fixture in conftest.py file responsible for creating infrastructure for every test group and removing it at the end of test group execution. Fixture uses infrastructure class methods for creating/removing infrastructure
5. Modify GitHub workflow mechanism by adding run-e2e-tests.yaml. User triggers new workflow manually from github. Workflow authorize to our test AWS account using OIDC and then run end to end tests against it

User experience

Users are not affected by this change. It enables new testing capabilities for maintainers and contributors to ensure highest possible code quality

Maintainer experience

Maintainer can run this job either from GitHub Actions Page

Or from local shell

Checklist

If your change doesn't seem to apply, please leave them unchecked.

• Meet tenets criteria
• I have performed a self-review of my this change
• Changes are tested
• Changes are documented
• PR title follows conventional commit semantics

Is this a breaking change?

No **RFC issue number**: https://github.com//issues/1226

Checklist:

• Migration process documented
• Implement warnings (if it can live side by side)

Acknowledgment

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

Disclaimer: We value your time and bandwidth. As such, any pull requests created on non-triaged issues might not be successful.