Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

CI Failure (Segmentation fault on shard 0) in RandomNodeOperationsTest.test_node_operations #16510

Closed
vbotbuildovich opened this issue Feb 7, 2024 · 1 comment
Assignees
Labels
auto-triaged used to know which issues have been opened from a CI job ci-failure performance

Comments

@vbotbuildovich
Copy link
Collaborator

vbotbuildovich commented Feb 7, 2024

https://buildkite.com/redpanda/vtools/builds/11733

Module: rptest.tests.random_node_operations_test
Class: RandomNodeOperationsTest
Method: test_node_operations
Arguments: {
    "num_to_upgrade": 3,
    "enable_failures": true,
    "with_tiered_storage": false
}
test_id:    RandomNodeOperationsTest.test_node_operations
status:     FAIL
run time:   1306.897 seconds

<NodeCrash ip-172-31-47-188: Segmentation fault on shard 0.
>
Traceback (most recent call last):
  File "/home/ubuntu/redpanda/tests/rptest/services/cluster.py", line 99, in wrapped
    r = f(self, *args, **kwargs)
  File "/home/ubuntu/redpanda/tests/rptest/tests/random_node_operations_test.py", line 428, in test_node_operations
    executor.execute_operation(op)
  File "/home/ubuntu/redpanda/tests/rptest/utils/node_operations.py", line 442, in execute_operation
    node_id = self.node_id(operation.node)
  File "/home/ubuntu/redpanda/tests/rptest/utils/node_operations.py", line 238, in node_id
    return self.redpanda.node_id(self.redpanda.get_node(idx),
  File "/home/ubuntu/redpanda/tests/rptest/services/redpanda.py", line 1289, in node_id
    node_id = wait_until_result(
  File "/home/ubuntu/redpanda/tests/rptest/util.py", line 90, in wait_until_result
    wait_until(wrapped_condition, *args, **kwargs)
  File "/usr/local/lib/python3.10/dist-packages/ducktape/utils/util.py", line 57, in wait_until
    raise TimeoutError(err_msg() if callable(err_msg) else err_msg) from last_exception
ducktape.errors.TimeoutError: couldn't reach admin endpoint for ip-172-31-47-188

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.10/dist-packages/ducktape/tests/runner_client.py", line 184, in _do_run
    data = self.run_test()
  File "/usr/local/lib/python3.10/dist-packages/ducktape/tests/runner_client.py", line 269, in run_test
    return self.test_context.function(self.test)
  File "/usr/local/lib/python3.10/dist-packages/ducktape/mark/_mark.py", line 481, in wrapper
    return functools.partial(f, *args, **kwargs)(*w_args, **w_kwargs)
  File "/home/ubuntu/redpanda/tests/rptest/services/cluster.py", line 120, in wrapped
    redpanda.raise_on_crash(log_allow_list=log_allow_list)
  File "/home/ubuntu/redpanda/tests/rptest/services/redpanda.py", line 2805, in raise_on_crash
    raise NodeCrash(crashes)
rptest.services.utils.NodeCrash: <NodeCrash ip-172-31-47-188: Segmentation fault on shard 0.
>

JIRA Link: CORE-1760

@vbotbuildovich vbotbuildovich added auto-triaged used to know which issues have been opened from a CI job ci-failure labels Feb 7, 2024
@travisdowns travisdowns self-assigned this Feb 8, 2024
@travisdowns travisdowns changed the title CI Failure (key symptom) in RandomNodeOperationsTest.test_node_operations CI Failure (Segmentation fault on shard 0) in RandomNodeOperationsTest.test_node_operations Feb 9, 2024
@ztlpn
Copy link
Contributor

ztlpn commented May 14, 2024

Likely fixed by #18305

@ztlpn ztlpn closed this as completed May 14, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
auto-triaged used to know which issues have been opened from a CI job ci-failure performance
Projects
None yet
Development

No branches or pull requests

4 participants