Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

*IT tests in x-pack/plugin/core silently ignored? #53407

Closed
ywelsch opened this issue Mar 11, 2020 · 3 comments · Fixed by #53444
Closed

*IT tests in x-pack/plugin/core silently ignored? #53407

ywelsch opened this issue Mar 11, 2020 · 3 comments · Fixed by #53444
Labels
:Delivery/Build Build or test infrastructure Team:Delivery Meta label for Delivery team

Comments

@ywelsch
Copy link
Contributor

ywelsch commented Mar 11, 2020

*IT tests in x-pack/plugin/coreseems to be silently ignored. I was working on a patch for source only snapshots and noticed that SourceOnlySnapshotIT is failing on master (without my changes), but this is not being picked up by CI. Same probably applies to ReloadSynonymAnalyzerIT and other *IT tests?

This looks potentially dangerous as test coverage is not what we would expect it to be.
@ywelsch ywelsch added the :Delivery/Build Build or test infrastructure label Mar 11, 2020
@elasticmachine
Copy link
Collaborator

Pinging @elastic/es-core-infra (:Core/Infra/Build)

@rjernst
Copy link
Member

rjernst commented Mar 11, 2020

I've found the problem, and it is very unfortunate. It appears all IT tests in xpack have been disabled for over a year. The IT tests in xpack are run under a separate test runner task called internalClusterTest. In moving where this task is created, it was accidentally not added to the check task, so CI has not known to run it.

I'll open a PR to re-enable, and mute any failing tests with test-failure issues as we would any other failing test.

rjernst added a commit to rjernst/elasticsearch that referenced this issue Mar 12, 2020
@rjernst
Add internalClusterTest to check task
0ac74db 
This commit adds internalClusterTest in xpack core to run as part of
check. This was accidentally removed in a refactoring. Other xpack
modules already do this, but core was left out. This commit also mutes 2
tests that currently fail.

closes elastic#53407
@rjernst rjernst mentioned this issue Mar 12, 2020
Merged
@rjernst
Copy link
Member

rjernst commented Mar 12, 2020

After going through all the xpack modules defining internalClusterTask, only the core tests were left out of check. This should be fixed by #53444.

rjernst added a commit that referenced this issue Mar 17, 2020
@rjernst
Add internalClusterTest to check task (#53444)
3608be6 
This commit adds internalClusterTest in xpack core to run as part of
check. This was accidentally removed in a refactoring. Other xpack
modules already do this, but core was left out. This commit also mutes 2
tests that currently fail.

closes #53407
rjernst added a commit that referenced this issue Mar 17, 2020
@rjernst
Add internalClusterTest to check task (#53444)
e7f3867 
This commit adds internalClusterTest in xpack core to run as part of
check. This was accidentally removed in a refactoring. Other xpack
modules already do this, but core was left out. This commit also mutes 2
tests that currently fail.

closes #53407
rjernst added a commit that referenced this issue Mar 17, 2020
@rjernst
Add internalClusterTest to check task (#53444)
d1e411c 
This commit adds internalClusterTest in xpack core to run as part of
check. This was accidentally removed in a refactoring. Other xpack
modules already do this, but core was left out. This commit also mutes 2
tests that currently fail.

closes #53407
@mark-vieira mark-vieira added the Team:Delivery Meta label for Delivery team label Nov 11, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
:Delivery/Build Build or test infrastructure Team:Delivery Meta label for Delivery team
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Add internalClusterTest to check task rjernst/elasticsearch
4 participants
@rjernst @ywelsch @mark-vieira @elasticmachine