Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Avoid unsafe conditionals in integration tests #1891

Merged

Conversation

jillr
Copy link
Collaborator

@jillr jillr commented Nov 30, 2023

SUMMARY

Update a batch of assertions in integration tests to avoid templating

See: ansible/ansible-documentation#866 and ansible/ansible-documentation#866 for more information

This PR covers about half of the impacted test tasks for this repo but is already going to be a lot to get through CI so additional PRs will follow.

Affected tasks identified with:
https://gist.github.com/sivel/7b012ae2ebf4ab4cb6225379a72cc502

ISSUE TYPE
  • Bugfix Pull Request
COMPONENT NAME

tests

ADDITIONAL INFORMATION

There are unsupported tests in this change. The cloudtrail tests seem to be broken independent of this change, iam_group tests pass for me locally

This comment was marked as resolved.

- "'engine_mode' in _result_tag_db_cluster"
- _result_tag_db_cluster.engine_mode == "provisioned"
- "'engine_version' in _result_tag_db_cluster"
- "'master_username' in _result_tag_db_cluster"
- _result_tag_db_cluster.master_username == "{{ username }}"
- _result_tag_db_cluster.master_username == username
- "'port' in _result_tag_db_cluster"
- _result_tag_db_cluster.port == {{ port }}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

what about this kind of assertion?

Copy link
Contributor

@tremble tremble left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

still digging into failures...

This comment was marked as resolved.

This comment was marked as resolved.

This comment was marked as outdated.

Copy link
Contributor

Build failed.
https://ansible.softwarefactory-project.io/zuul/buildset/1d6bacd19f9d4787bf80b78287c25003

✔️ ansible-galaxy-importer SUCCESS in 6m 09s
✔️ build-ansible-collection SUCCESS in 14m 09s
✔️ ansible-test-splitter SUCCESS in 5m 20s
✔️ integration-amazon.aws-1 SUCCESS in 14m 58s
✔️ integration-amazon.aws-2 SUCCESS in 41m 08s
✔️ integration-amazon.aws-3 SUCCESS in 24m 48s
✔️ integration-amazon.aws-4 SUCCESS in 44m 13s
✔️ integration-amazon.aws-5 SUCCESS in 39m 00s
✔️ integration-amazon.aws-6 SUCCESS in 32m 44s
✔️ integration-amazon.aws-7 SUCCESS in 11m 51s
✔️ integration-amazon.aws-8 SUCCESS in 17m 34s
✔️ integration-amazon.aws-9 SUCCESS in 12m 44s
✔️ integration-amazon.aws-10 SUCCESS in 30m 34s
✔️ integration-amazon.aws-11 SUCCESS in 11m 30s
✔️ integration-amazon.aws-12 SUCCESS in 19m 45s
✔️ integration-amazon.aws-13 SUCCESS in 28m 22s
✔️ integration-amazon.aws-14 SUCCESS in 26m 30s
✔️ integration-amazon.aws-15 SUCCESS in 26m 06s
✔️ integration-amazon.aws-16 SUCCESS in 17m 41s
✔️ integration-amazon.aws-17 SUCCESS in 18m 12s
integration-amazon.aws-18 FAILURE in 23m 12s
✔️ integration-amazon.aws-19 SUCCESS in 21m 15s
✔️ integration-amazon.aws-20 SUCCESS in 16m 21s
✔️ integration-amazon.aws-21 SUCCESS in 15m 02s
Skipped 23 jobs

Copy link
Contributor

Build failed.
https://ansible.softwarefactory-project.io/zuul/buildset/c0bea16585c346919ef32b10c92968b8

✔️ ansible-galaxy-importer SUCCESS in 3m 43s
✔️ build-ansible-collection SUCCESS in 14m 00s
✔️ ansible-test-splitter SUCCESS in 5m 44s
✔️ integration-amazon.aws-1 SUCCESS in 13m 08s
✔️ integration-amazon.aws-2 SUCCESS in 30m 41s
✔️ integration-amazon.aws-3 SUCCESS in 23m 23s
✔️ integration-amazon.aws-4 SUCCESS in 39m 56s
✔️ integration-amazon.aws-5 SUCCESS in 36m 52s
integration-amazon.aws-6 RETRY_LIMIT in 6m 57s
✔️ integration-amazon.aws-7 SUCCESS in 10m 10s
✔️ integration-amazon.aws-8 SUCCESS in 16m 03s
✔️ integration-amazon.aws-9 SUCCESS in 9m 20s
✔️ integration-amazon.aws-10 SUCCESS in 26m 47s
✔️ integration-amazon.aws-11 SUCCESS in 12m 19s
✔️ integration-amazon.aws-12 SUCCESS in 19m 36s
✔️ integration-amazon.aws-13 SUCCESS in 23m 46s
✔️ integration-amazon.aws-14 SUCCESS in 27m 12s
✔️ integration-amazon.aws-15 SUCCESS in 23m 15s
✔️ integration-amazon.aws-16 SUCCESS in 10m 28s
✔️ integration-amazon.aws-17 SUCCESS in 17m 22s
integration-amazon.aws-18 FAILURE in 23m 44s
✔️ integration-amazon.aws-19 SUCCESS in 20m 19s
✔️ integration-amazon.aws-20 SUCCESS in 20m 15s
✔️ integration-amazon.aws-21 SUCCESS in 14m 59s
Skipped 23 jobs

Copy link
Contributor

Build failed.
https://ansible.softwarefactory-project.io/zuul/buildset/520a830ec2f64922b9965b55dbe3f6f7

✔️ ansible-galaxy-importer SUCCESS in 5m 22s
✔️ build-ansible-collection SUCCESS in 15m 32s
✔️ ansible-test-splitter SUCCESS in 5m 38s
✔️ integration-amazon.aws-1 SUCCESS in 14m 01s
✔️ integration-amazon.aws-2 SUCCESS in 56m 43s
✔️ integration-amazon.aws-3 SUCCESS in 25m 19s
✔️ integration-amazon.aws-4 SUCCESS in 41m 18s
integration-amazon.aws-5 RETRY_LIMIT in 1m 44s
✔️ integration-amazon.aws-6 SUCCESS in 26m 07s
integration-amazon.aws-7 RETRY_LIMIT in 1m 44s
✔️ integration-amazon.aws-8 SUCCESS in 16m 09s
✔️ integration-amazon.aws-9 SUCCESS in 9m 57s
integration-amazon.aws-10 RETRY_LIMIT in 1m 43s
✔️ integration-amazon.aws-11 SUCCESS in 10m 04s
✔️ integration-amazon.aws-12 SUCCESS in 18m 02s
✔️ integration-amazon.aws-13 SUCCESS in 23m 37s
✔️ integration-amazon.aws-14 SUCCESS in 27m 31s
integration-amazon.aws-15 RETRY_LIMIT in 1m 48s
✔️ integration-amazon.aws-16 SUCCESS in 10m 34s
✔️ integration-amazon.aws-17 SUCCESS in 19m 09s
integration-amazon.aws-18 FAILURE in 22m 51s
✔️ integration-amazon.aws-19 SUCCESS in 21m 08s
✔️ integration-amazon.aws-20 SUCCESS in 18m 54s
✔️ integration-amazon.aws-21 SUCCESS in 14m 36s
Skipped 23 jobs

@tremble tremble force-pushed the port_asserts branch 3 times, most recently from 1b67b18 to 46d08d3 Compare December 4, 2023 16:24
Copy link
Contributor

Build succeeded.
https://ansible.softwarefactory-project.io/zuul/buildset/2002b9832f92434791125b832adec4d6

✔️ ansible-galaxy-importer SUCCESS in 5m 29s
✔️ build-ansible-collection SUCCESS in 14m 58s
✔️ ansible-test-splitter SUCCESS in 6m 11s
✔️ integration-amazon.aws-1 SUCCESS in 13m 18s
✔️ integration-amazon.aws-2 SUCCESS in 30m 03s
✔️ integration-amazon.aws-3 SUCCESS in 27m 11s
✔️ integration-amazon.aws-4 SUCCESS in 38m 52s
✔️ integration-amazon.aws-5 SUCCESS in 31m 30s
✔️ integration-amazon.aws-6 SUCCESS in 29m 54s
✔️ integration-amazon.aws-7 SUCCESS in 11m 04s
✔️ integration-amazon.aws-8 SUCCESS in 16m 59s
✔️ integration-amazon.aws-9 SUCCESS in 10m 33s
✔️ integration-amazon.aws-10 SUCCESS in 29m 10s
✔️ integration-amazon.aws-11 SUCCESS in 11m 49s
✔️ integration-amazon.aws-12 SUCCESS in 22m 52s
✔️ integration-amazon.aws-13 SUCCESS in 21m 05s
✔️ integration-amazon.aws-14 SUCCESS in 20m 31s
✔️ integration-amazon.aws-15 SUCCESS in 20m 29s
✔️ integration-amazon.aws-16 SUCCESS in 13m 35s
✔️ integration-amazon.aws-17 SUCCESS in 17m 50s
✔️ integration-amazon.aws-18 SUCCESS in 22m 00s
✔️ integration-amazon.aws-19 SUCCESS in 18m 27s
✔️ integration-amazon.aws-20 SUCCESS in 25m 09s
✔️ integration-amazon.aws-21 SUCCESS in 15m 40s
Skipped 23 jobs

@tremble tremble added the mergeit Merge the PR (SoftwareFactory) label Dec 4, 2023
Copy link
Contributor

Build failed.
https://ansible.softwarefactory-project.io/zuul/buildset/5ccfd87d1da34ff7b665839efff04bfb

✔️ ansible-galaxy-importer SUCCESS in 3m 50s
✔️ build-ansible-collection SUCCESS in 16m 53s
✔️ ansible-test-splitter SUCCESS in 8m 19s
✔️ integration-amazon.aws-1 SUCCESS in 14m 19s
✔️ integration-amazon.aws-2 SUCCESS in 35m 47s
✔️ integration-amazon.aws-3 SUCCESS in 24m 13s
✔️ integration-amazon.aws-4 SUCCESS in 47m 04s
✔️ integration-amazon.aws-5 SUCCESS in 35m 33s
✔️ integration-amazon.aws-6 SUCCESS in 31m 16s
✔️ integration-amazon.aws-7 SUCCESS in 11m 08s
✔️ integration-amazon.aws-8 SUCCESS in 19m 36s
✔️ integration-amazon.aws-9 SUCCESS in 9m 42s
integration-amazon.aws-10 RETRY_LIMIT in 1m 43s
✔️ integration-amazon.aws-11 SUCCESS in 10m 16s
integration-amazon.aws-12 FAILURE in 29m 37s
✔️ integration-amazon.aws-13 SUCCESS in 26m 07s
✔️ integration-amazon.aws-14 SUCCESS in 22m 18s
✔️ integration-amazon.aws-15 SUCCESS in 27m 17s
✔️ integration-amazon.aws-16 SUCCESS in 15m 36s
✔️ integration-amazon.aws-17 SUCCESS in 21m 24s
integration-amazon.aws-18 RETRY_LIMIT in 1m 43s
✔️ integration-amazon.aws-19 SUCCESS in 21m 51s
integration-amazon.aws-20 FAILURE in 21m 58s
✔️ integration-amazon.aws-21 SUCCESS in 14m 51s
Skipped 23 jobs

@tremble
Copy link
Contributor

tremble commented Dec 5, 2023

recheck

Copy link
Contributor

Build failed.
https://ansible.softwarefactory-project.io/zuul/buildset/1105fbdc84654860a0c63b6a87bc29ea

✔️ ansible-galaxy-importer SUCCESS in 5m 28s
✔️ build-ansible-collection SUCCESS in 13m 53s
✔️ ansible-test-splitter SUCCESS in 5m 10s
✔️ integration-amazon.aws-1 SUCCESS in 13m 53s
✔️ integration-amazon.aws-2 SUCCESS in 32m 27s
✔️ integration-amazon.aws-3 SUCCESS in 24m 45s
✔️ integration-amazon.aws-4 SUCCESS in 41m 43s
✔️ integration-amazon.aws-5 SUCCESS in 36m 30s
integration-amazon.aws-6 FAILURE in 54m 23s
✔️ integration-amazon.aws-7 SUCCESS in 12m 44s
✔️ integration-amazon.aws-8 SUCCESS in 18m 16s
✔️ integration-amazon.aws-9 SUCCESS in 12m 08s
✔️ integration-amazon.aws-10 SUCCESS in 29m 51s
✔️ integration-amazon.aws-11 SUCCESS in 11m 02s
✔️ integration-amazon.aws-12 SUCCESS in 23m 41s
✔️ integration-amazon.aws-13 SUCCESS in 24m 37s
✔️ integration-amazon.aws-14 SUCCESS in 27m 49s
✔️ integration-amazon.aws-15 SUCCESS in 19m 49s
✔️ integration-amazon.aws-16 SUCCESS in 14m 57s
✔️ integration-amazon.aws-17 SUCCESS in 23m 55s
✔️ integration-amazon.aws-18 SUCCESS in 22m 02s
✔️ integration-amazon.aws-19 SUCCESS in 22m 11s
✔️ integration-amazon.aws-20 SUCCESS in 16m 17s
✔️ integration-amazon.aws-21 SUCCESS in 16m 26s
Skipped 23 jobs

@tremble
Copy link
Contributor

tremble commented Dec 5, 2023

recheck

Copy link
Contributor

Build succeeded.
https://ansible.softwarefactory-project.io/zuul/buildset/41e13d39f01b4dd89d34f76a274e27f1

✔️ ansible-galaxy-importer SUCCESS in 5m 17s
✔️ build-ansible-collection SUCCESS in 13m 49s
✔️ ansible-test-splitter SUCCESS in 5m 56s
✔️ integration-amazon.aws-1 SUCCESS in 13m 41s
✔️ integration-amazon.aws-2 SUCCESS in 31m 14s
✔️ integration-amazon.aws-3 SUCCESS in 24m 20s
✔️ integration-amazon.aws-4 SUCCESS in 41m 15s
✔️ integration-amazon.aws-5 SUCCESS in 36m 17s
✔️ integration-amazon.aws-6 SUCCESS in 30m 11s
✔️ integration-amazon.aws-7 SUCCESS in 11m 33s
✔️ integration-amazon.aws-8 SUCCESS in 18m 42s
✔️ integration-amazon.aws-9 SUCCESS in 11m 07s
✔️ integration-amazon.aws-10 SUCCESS in 41m 57s
✔️ integration-amazon.aws-11 SUCCESS in 10m 23s
✔️ integration-amazon.aws-12 SUCCESS in 23m 34s
✔️ integration-amazon.aws-13 SUCCESS in 24m 19s
✔️ integration-amazon.aws-14 SUCCESS in 25m 25s
✔️ integration-amazon.aws-15 SUCCESS in 26m 51s
✔️ integration-amazon.aws-16 SUCCESS in 14m 45s
✔️ integration-amazon.aws-17 SUCCESS in 21m 42s
✔️ integration-amazon.aws-18 SUCCESS in 23m 32s
✔️ integration-amazon.aws-19 SUCCESS in 24m 30s
✔️ integration-amazon.aws-20 SUCCESS in 16m 22s
✔️ integration-amazon.aws-21 SUCCESS in 16m 22s
Skipped 23 jobs

Copy link
Contributor

Build succeeded (gate pipeline).
https://ansible.softwarefactory-project.io/zuul/buildset/45cfc70901a04bfa8a2e4da896c0b397

✔️ ansible-galaxy-importer SUCCESS in 5m 00s
✔️ build-ansible-collection SUCCESS in 13m 54s
✔️ ansible-test-splitter SUCCESS in 5m 57s
✔️ integration-amazon.aws-1 SUCCESS in 13m 18s
✔️ integration-amazon.aws-2 SUCCESS in 30m 44s
✔️ integration-amazon.aws-3 SUCCESS in 27m 21s
✔️ integration-amazon.aws-4 SUCCESS in 40m 01s
✔️ integration-amazon.aws-5 SUCCESS in 38m 52s
✔️ integration-amazon.aws-6 SUCCESS in 27m 12s
✔️ integration-amazon.aws-7 SUCCESS in 10m 21s
✔️ integration-amazon.aws-8 SUCCESS in 17m 01s
✔️ integration-amazon.aws-9 SUCCESS in 9m 35s
✔️ integration-amazon.aws-10 SUCCESS in 29m 24s
✔️ integration-amazon.aws-11 SUCCESS in 11m 03s
✔️ integration-amazon.aws-12 SUCCESS in 21m 10s
✔️ integration-amazon.aws-13 SUCCESS in 24m 38s
✔️ integration-amazon.aws-14 SUCCESS in 27m 03s
✔️ integration-amazon.aws-15 SUCCESS in 25m 36s
✔️ integration-amazon.aws-16 SUCCESS in 18m 53s
✔️ integration-amazon.aws-17 SUCCESS in 18m 23s
✔️ integration-amazon.aws-18 SUCCESS in 22m 06s
✔️ integration-amazon.aws-19 SUCCESS in 22m 53s
✔️ integration-amazon.aws-20 SUCCESS in 19m 42s
✔️ integration-amazon.aws-21 SUCCESS in 16m 48s
Skipped 23 jobs

@softwarefactory-project-zuul softwarefactory-project-zuul bot merged commit dc08059 into ansible-collections:main Dec 5, 2023
39 checks passed
@tremble tremble added the backport-7 PR should be backported to the stable-7 branch label Dec 5, 2023
Copy link

patchback bot commented Dec 5, 2023

Backport to stable-7: 💚 backport PR created

✅ Backport PR branch: patchback/backports/stable-7/dc0805937cc748bb26b4618ea7189dca7984a546/pr-1891

Backported as #1903

🤖 @patchback
I'm built with octomachinery and
my source is open — https://github.com/sanitizers/patchback-github-app.

patchback bot pushed a commit that referenced this pull request Dec 5, 2023
Avoid unsafe conditionals in integration tests

SUMMARY
Update a batch of assertions in integration tests to avoid templating
See: ansible/ansible-documentation#866 and ansible/ansible-documentation#866 for more information
This PR covers about half of the impacted test tasks for this repo but is already going to be a lot to get through CI so additional PRs will follow.
Affected tasks identified with:
https://gist.github.com/sivel/7b012ae2ebf4ab4cb6225379a72cc502
ISSUE TYPE

Bugfix Pull Request

COMPONENT NAME
tests
ADDITIONAL INFORMATION
There are unsupported tests in this change.  The cloudtrail tests seem to be broken independent of this change, iam_group tests pass for me locally

Reviewed-by: Bikouo Aubin
Reviewed-by: Mark Chappell
Reviewed-by: Alina Buzachis
Reviewed-by: Mike Graves <[email protected]>
(cherry picked from commit dc08059)
softwarefactory-project-zuul bot pushed a commit that referenced this pull request Dec 5, 2023
[PR #1891/dc080593 backport][stable-7] Avoid unsafe conditionals in integration tests

This is a backport of PR #1891 as merged into main (dc08059).
SUMMARY
Update a batch of assertions in integration tests to avoid templating
See: ansible/ansible-documentation#866 and ansible/ansible-documentation#866 for more information
This PR covers about half of the impacted test tasks for this repo but is already going to be a lot to get through CI so additional PRs will follow.
Affected tasks identified with:
https://gist.github.com/sivel/7b012ae2ebf4ab4cb6225379a72cc502
ISSUE TYPE

Bugfix Pull Request

COMPONENT NAME
tests
ADDITIONAL INFORMATION
There are unsupported tests in this change.  The cloudtrail tests seem to be broken independent of this change, iam_group tests pass for me locally

Reviewed-by: Mark Chappell
@gravesm gravesm mentioned this pull request Feb 21, 2024
softwarefactory-project-zuul bot pushed a commit that referenced this pull request Feb 22, 2024
Fix unsafe conditional

SUMMARY

This fixes an unsafe conditional in integration tests that was missed by our previous mass updates for unsafe conditionals (#1891 and #1911).

ISSUE TYPE


Bugfix Pull Request

COMPONENT NAME

ADDITIONAL INFORMATION

Reviewed-by: Mark Chappell
Reviewed-by: Alina Buzachis
patchback bot pushed a commit that referenced this pull request Feb 22, 2024
Fix unsafe conditional

SUMMARY

This fixes an unsafe conditional in integration tests that was missed by our previous mass updates for unsafe conditionals (#1891 and #1911).

ISSUE TYPE

Bugfix Pull Request

COMPONENT NAME

ADDITIONAL INFORMATION

Reviewed-by: Mark Chappell
Reviewed-by: Alina Buzachis
(cherry picked from commit dc7a3fb)
patchback bot pushed a commit that referenced this pull request Feb 22, 2024
Fix unsafe conditional

SUMMARY

This fixes an unsafe conditional in integration tests that was missed by our previous mass updates for unsafe conditionals (#1891 and #1911).

ISSUE TYPE

Bugfix Pull Request

COMPONENT NAME

ADDITIONAL INFORMATION

Reviewed-by: Mark Chappell
Reviewed-by: Alina Buzachis
(cherry picked from commit dc7a3fb)
softwarefactory-project-zuul bot pushed a commit that referenced this pull request Feb 23, 2024
[PR #1984/dc7a3fb2 backport][stable-7] Fix unsafe conditional

This is a backport of PR #1984 as merged into main (dc7a3fb).
SUMMARY

This fixes an unsafe conditional in integration tests that was missed by our previous mass updates for unsafe conditionals (#1891 and #1911).

ISSUE TYPE


Bugfix Pull Request

COMPONENT NAME

ADDITIONAL INFORMATION
softwarefactory-project-zuul bot pushed a commit that referenced this pull request Feb 26, 2024
[PR #1984/dc7a3fb2 backport][stable-6] Fix unsafe conditional

This is a backport of PR #1984 as merged into main (dc7a3fb).
SUMMARY

This fixes an unsafe conditional in integration tests that was missed by our previous mass updates for unsafe conditionals (#1891 and #1911).

ISSUE TYPE


Bugfix Pull Request

COMPONENT NAME

ADDITIONAL INFORMATION

Reviewed-by: Mark Chappell
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
backport-7 PR should be backported to the stable-7 branch mergeit Merge the PR (SoftwareFactory)
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants