-
Notifications
You must be signed in to change notification settings - Fork 820
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
ci: windows smoke test list #11797
Merged
Merged
ci: windows smoke test list #11797
Changes from all commits
Commits
Show all changes
75 commits
Select commit
Hold shift + click to select a range
52007b9
ci: migrated to new test strategy
awsluja 52979de
chore: add other tests and migration too
awsluja dc39171
chore: cleanup
awsluja d560079
chore: cleanup
awsluja 0a19d69
chore: cleanup unused e2e tests
awsluja 00ea815
chore: remove generated file
awsluja c38f04c
chore: remove parallelization blocker
awsluja 72bc406
chore: force parallel runs
awsluja 8886c1e
chore: cleanup test list
awsluja f85c2ff
chore: disable retries and cleanup exclusions
awsluja e1c82af
chore: bump test count to fill gaps
awsluja 5c7a51c
chore: move random id earlier in project name
awsluja 1c408b7
chore: update yarn version
awsluja e93880e
chore: fix typo
awsluja a5d077e
chore: try to fix delete issues
awsluja dcb75f7
chore: fix teardown on delete error
awsluja f0f7787
Merge branch 'dev' into e2e-multi-core-optimizations
awsluja a9e08cb
ci: create custom file system mutex
awsluja 82fed6e
chore: cleanup logs
awsluja e1b32af
ci: increase randomness of initial lock creation
awsluja 7afc0ad
chore: move migration tests to small isolated vms
awsluja 55111a8
chore: adjust concurrency
awsluja eed73be
chore: update naming
awsluja ebd0be2
ci: update reporter to better handle concurrent tests
awsluja b38e298
ci: add use parent account exceptions
awsluja 4842214
chore: cleanup naming, fix windows artifacts
awsluja 52094aa
chore: use data to optimize test scheduling
awsluja 4341105
chore: cleanup
awsluja 11d2e94
chore: cleanup
awsluja 3b7efb3
chore: fix call flow
awsluja 62b9f9e
chore: workaround duplicate spinner bugs
awsluja c1678e1
ci: split api_2 test
awsluja 304c345
chore: increase push timeout
awsluja c1c2ef9
ci: add method to run solo tests
awsluja bcde99e
ci: configure retries compatible with worker pools
awsluja 9482eb4
ci: workaround concurrent polling limits
awsluja ae166d6
chore: fix bug with solo tests
awsluja 31c0f81
chore: move import & geo tests to run solo
awsluja 1d110fe
chore: add artifact scripts
awsluja ce925d0
chore: trigger workflow results
awsluja e64f8b1
ci: fix artifact path
awsluja c0b086e
Merge branch 'dev' into e2e-multi-core-optimizations
awsluja 3673c22
Merge branch 'dev' into e2e-multi-core-optimizations
awsluja bdd4eea
chore: cleanup
awsluja 817c60e
chore: add timer spy
awsluja c0e2df5
Merge branch 'dev' into e2e-multi-core-optimizations
awsluja 1c3ad43
ci: update test lists
awsluja 565fe8f
chore: cleanup
awsluja fe810ab
ci: fix retries
awsluja a6645fe
ci: move delays to createProjectDir
awsluja 6f93e9a
ci: use same script for e2e as linux on windows
awsluja d6fafa1
chore: split storage-1 test
awsluja 6e1df49
ci: fix config
awsluja 5f0919b
ci: add bash env
awsluja b02b80e
ci: reduce timeout on push
awsluja 0a136ca
ci: update test lists
awsluja 3ec4e50
ci: update test lists
awsluja 70ff919
ci: lower timeout and enable concurrent migration testing
awsluja be2e50f
ci: update test list
awsluja 214fd0f
ci: restructure migration test setups
awsluja 538a33f
ci: cleanup to use file names instead of old job names
awsluja d3054fc
chore: refactor
awsluja 6fb874e
ci: fix dependencies
awsluja fb590c5
ci: attempt to fix retry on windows
awsluja fa1b2d0
chore: revert retry changes
awsluja c31f7e7
chore: undo changes to addCircleCiTags
awsluja 01f46b0
chore: address PR feedback
awsluja 8370b71
chore: undo changes to spinners
awsluja ae885d2
chore: cleanup unused files
awsluja a88c2b4
chore: lint & formatting
awsluja 06d60f7
chore: lint, remove unused import
awsluja b9d7daa
chore: undo changes to initialize-env.ts
awsluja ef7ffb8
Merge branch 'dev' into e2e-multi-core-optimizations
awsluja bce4eff
ci: windows smoke test list for non-dev branches
awsluja 07357b6
Merge branch 'dev' into windows-smoke-test-list
awsluja File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change | ||||||||
---|---|---|---|---|---|---|---|---|---|---|
|
@@ -33,6 +33,66 @@ const RUN_SOLO = [ | |||||||||
'src/__tests__/schema-auth-15.test.ts', | ||||||||||
'src/__tests__/schema-connection-1.test.ts', | ||||||||||
]; | ||||||||||
/** | ||||||||||
* Most Windows tests only run on 'dev', except for this list of smoke tests. | ||||||||||
* This list should contain ~30 high quality tests that provide good overall coverage. | ||||||||||
* | ||||||||||
* All other tests will eventually run on 'dev'. | ||||||||||
*/ | ||||||||||
const WINDOWS_SMOKE_TESTS = [ | ||||||||||
// api, lambda, multi-env | ||||||||||
'src/__tests__/api_lambda_auth_1.test.ts', | ||||||||||
// auth with api, trigger, and function that depends on api | ||||||||||
'src/__tests__/auth_2e.test.ts', | ||||||||||
// js with with all auth options | ||||||||||
'src/__tests__/auth_3c.test.ts', | ||||||||||
// headless auth with totp & sms | ||||||||||
'src/__tests__/auth_5b.test.ts', | ||||||||||
// headless auth update resource | ||||||||||
'src/__tests__/auth_5e.test.ts', | ||||||||||
// js with all auth options and front end config | ||||||||||
'src/__tests__/auth_6.test.ts', | ||||||||||
// auth import | ||||||||||
'src/__tests__/auth_7b.test.ts', | ||||||||||
// flutter with auth | ||||||||||
'src/__tests__/auth_8b.test.ts', | ||||||||||
// android | ||||||||||
'src/__tests__/auth_11.test.ts', | ||||||||||
// auth with multiple triggers | ||||||||||
'src/__tests__/auth-trigger.test.ts', | ||||||||||
// configure | ||||||||||
'src/__tests__/congifure-project.test.ts', | ||||||||||
// api with containers and secrets | ||||||||||
'src/__tests__/containers-api-secrets.test.ts', | ||||||||||
// api with dynamodb & lambda, custom policies | ||||||||||
'src/__tests__/custom_policies_function.test.ts', | ||||||||||
// env | ||||||||||
'src/__tests__/env-1.test.ts', | ||||||||||
// export and pull | ||||||||||
'src/__tests__/export-pull-a.test.ts', | ||||||||||
// functions | ||||||||||
'src/__tests__/function_10.test.ts', | ||||||||||
// notifications with function permissions | ||||||||||
'src/__tests__/function-permissions.test.ts', | ||||||||||
// geo | ||||||||||
'src/__tests__/geo-add-a.test.ts', | ||||||||||
// global sandbox | ||||||||||
'src/__tests__/global_sandbox-b.test.ts', | ||||||||||
// hooks | ||||||||||
'src/__tests__/hooks-b.test.ts', | ||||||||||
// hosting | ||||||||||
'src/__tests__/hostingPROD.test.ts', | ||||||||||
// import, s3 | ||||||||||
'src/__tests__/import_s3_2b.test.ts', | ||||||||||
// interactions | ||||||||||
'src/__tests__/interactions.test.ts', | ||||||||||
// schema auth test | ||||||||||
'src/__tests__/schema-auth-1a.test.ts', | ||||||||||
// schema model test | ||||||||||
'src/__tests__/schema-model-e.test.ts', | ||||||||||
// interative deployments | ||||||||||
'src/__tests__/schema-iterative-update-1.test.ts', | ||||||||||
]; | ||||||||||
const TEST_EXCLUSIONS: { l: string[]; w: string[] } = { | ||||||||||
l: [], | ||||||||||
w: [ | ||||||||||
|
@@ -170,6 +230,11 @@ export const splitTestsV2 = function splitTests( | |||||||||
if (TEST_EXCLUSIONS[os].find(excluded => test === excluded)) { | ||||||||||
continue; | ||||||||||
} | ||||||||||
// when we are not running E2E on 'dev', we only run a subset of tests on Windows | ||||||||||
const isNotDevBranch = process.env.CIRCLE_BRANCH !== 'dev'; | ||||||||||
if (isNotDevBranch && os === 'w' && !WINDOWS_SMOKE_TESTS.includes(test)) { | ||||||||||
Comment on lines
+234
to
+235
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
Suggested change
|
||||||||||
continue; // skip this test | ||||||||||
} | ||||||||||
const US_WEST_2 = FORCE_US_WEST_2.find(t => test.startsWith(t)); | ||||||||||
const USE_PARENT = USE_PARENT_ACCOUNT.some(usesParent => test.startsWith(usesParent)); | ||||||||||
|
||||||||||
|
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
https://circleci.com/docs/env-vars/