Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[WIP] Run E2E tests on suites #5025

Closed
wants to merge 2 commits into from

Conversation

mcandeia
Copy link
Contributor

Description

This is an attempt to make our e2e test stability better. The general idea is to divide our tests in N suites that could be configurable by just setting up an environment variable "E2E_TEST_SUITE_COUNT". The test chunks are arbitrarily chosen, following the lexicographical order of the folders' name. If we intentionally want some tests to execute in same suite one option is just name the folder as 1-testA and 1-testB.

We expect the following improvements:

  • The total test duration will decrease
  • The SNAT ports exaustion should not happen very often

Issue reference

Please reference the issue this PR will close: #4962

Checklist

Please make sure you've completed the relevant tasks for this PR, out of the following list:

@mcandeia mcandeia changed the title Feat/test parallel suites [WIP] Run e2e tests on suites Aug 17, 2022
@mcandeia mcandeia changed the title [WIP] Run e2e tests on suites [WIP] Run E2E tests on suites Aug 17, 2022
@mcandeia mcandeia force-pushed the feat/test-parallel-suites branch from f61027b to bb886ed Compare August 17, 2022 17:48
remaining=$(expr $i + $double)

# if it can't be divided exactly so the last test will include the remaining tests.
if [[ $remaining -lt $total_tests ]]; then
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Note that some tests should always be in the same suite (i.e. executed in parallel). For example, all pubsub tests should be executed in parallel to avoid overloading Service Bus

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In this case all pubsub tests should be executed sequentially rather than parallel, is that you meant?

If yes, indeed, as you can see the test suite division is not explicit, I was wondering if it really needs to be. Thoughts?

An option might be prefixing the folder name with the desired test suite, something like:

1-pubsub_test_b
1-pubsub_test_a

Folders that not contains the prefix wouldn't have any restriction, how about that?

Another option is having subfolders

/pubsub/test_a
/pubsub/test_b

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, sorry, I meant sequentially.

Any implementation would be fine.

Signed-off-by: Marcos Candeia <marrcooos@gmail.com>
Signed-off-by: Marcos Candeia <marrcooos@gmail.com>
@dapr-bot
Copy link
Collaborator

This pull request has been automatically marked as stale because it has not had activity in the last 60 days. It will be closed in 7 days if no further activity occurs. Please feel free to give a status update now, ping for review, or re-open when it's ready. Thank you for your contributions!

@dapr-bot dapr-bot added the stale Issues and PRs without response label Oct 17, 2022
@dapr-bot
Copy link
Collaborator

This pull request has been automatically closed because it has not had activity in the last 67 days. Please feel free to give a status update now, ping for review, or re-open when it's ready. Thank you for your contributions!

@dapr-bot dapr-bot closed this Oct 24, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
stale Issues and PRs without response
Projects
None yet
Development

Successfully merging this pull request may close these issues.

E2E tests: connect to AKS using a Wireguard tunnel
3 participants