Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Fix 8142: Timeout failure of fuzzer in RandomNodeOperationsTest.test_node_operations #8176

Merged
merged 4 commits into from
Jan 13, 2023
Merged

Fix 8142: Timeout failure of fuzzer in RandomNodeOperationsTest.test_node_operations #8176

merged 4 commits into from
Jan 13, 2023

Conversation

rystsov
Copy link
Contributor

@rystsov rystsov commented Jan 12, 2023 •
edited
Loading

Parsing rpk.acl_create_allow_cluster & rpk.describe_topic output to avoid slipping failures as success.

Fixes #8142, Fixes #8178

Backports Required

  • none - not a bug fix
  • none - issue does not exist in previous branches
  • none - papercut/not impactful enough to backport
  • v22.3.x
  • v22.2.x
  • v22.1.x

UX Changes

  • none

Release Notes

  • none

@rystsov rystsov mentioned this pull request Jan 12, 2023
Closed
@rystsov
Copy link
Contributor Author

rystsov commented Jan 12, 2023 •
edited
Loading

New issues

Rare issues without an (obvious) connection to the changes in the PR, most likely flaky tests reveled by running tests ~200x times on repeat

  1. CI Failure (timeout on wait_until recommissioned)
  2. BadLogLines (500 on decommission)
  3. Stuck consumer in PartitionMovementUpgradeTest.test_basic_upgrade
  4. Search victim assert in ControllerEraseTest.test_erase_controller_log
  5. Internal redpanda assert! in NodesDecommissioningTest.test_flipping_decommission_recommission
  6. Storage space alert in PartitionBalancerTest.test_full_nodes

An per-existing problem which was revealed by hardening of the rpk.acl_create_allow_cluster method in this PR (the test is put behind on_to_fail)

  1. CI Failure (acl_create_allow_cluster CLUSTER_AUTHORIZATION_FAILED)

Frequent issues without an (obvious) connection to the changes in the PR

  1. Assert expected_license_contents failure in FeaturesMultiNodeTest.test_license_upload_and_query
  2. Assert expected_license_contents failure in RpkClusterTest.test_upload_and_query_cluster_license_rpk

@rystsov rystsov force-pushed the issue-8142 branch 2 times, most recently from 03facc6 to 65bd692 Compare January 13, 2023 00:39
@rystsov
ducky: parse rpk.acl_create_allow_cluster output
3b69796 
parsing the output to fail on anything but success to avoid
slipping failures as success

fixes redpanda-data#8142
@rystsov
ducky: parse rpk.describe_topic output
658ec19 
parsing the output to fail on anything but success to avoid
slipping failures as success

fixes redpanda-data#8178
@rystsov
Revert "tests: mark unstable random node operations tests ok_to_fail"
c21c936 
This reverts commit 89ec2db.
@rystsov
tests: mark unstable acls tests ok_to_fail
bb45823 
redpanda-data#8202
@rystsov
Copy link
Contributor Author

rystsov commented Jan 13, 2023

/ci-repeat 5

@piyushredpanda piyushredpanda merged commit da248ae into redpanda-data:dev Jan 13, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@mmaslankaprv mmaslankaprv mmaslankaprv approved these changes

@bharathv bharathv Awaiting requested review from bharathv

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
3 participants
@rystsov @mmaslankaprv @piyushredpanda