Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Remove usage of max_local_storage_nodes in test infrastructure #41652

Merged
merged 10 commits into from
May 22, 2019

Conversation

ywelsch
Copy link
Contributor

@ywelsch ywelsch commented Apr 29, 2019

Moves the test infrastructure away from using node.max_local_storage_nodes, allowing us in a follow-up PR to deprecate this setting in 7.x and to remove it in 8.0.

This also changes the behavior of InternalTestCluster so that starting up nodes will not automatically reuse data folders of previously stopped nodes. If this behavior is desired, it needs to be explicitly done by passing the data path from the stopped node to the new node that is started.

@ywelsch ywelsch requested a review from DaveCTurner April 29, 2019 18:00
@elasticmachine
Copy link
Collaborator

Pinging @elastic/es-core-infra

Copy link
Contributor

@DaveCTurner DaveCTurner left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I left a couple of minor comments, but this looks good. CI failed without giving any useful feedback:

org.elasticsearch.gradle.testclusters.TestClustersPluginIT > testMultiProject FAILED
org.gradle.testkit.runner.UnexpectedBuildFailure at TestClustersPluginIT.java:88


ensureGreen();

// index is gone.
assertFalse(indexExists(indexName));
}

private void executeRepurposeCommandForOrdinal(Settings settings, String indexUUID, int ordinal,
private void executeRepurposeCommandForOrdinal(Settings settings, String indexUUID,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This method name mensions ordinal but ordinal is no longer an argument.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

fixed in c10b9a4

updatedSettings.put(Environment.PATH_DATA_SETTING.getKey(), baseDir.resolve(name));
}

updatedSettings.put(Environment.PATH_SHARED_DATA_SETTING.getKey(), baseDir.resolve(name + "-shared"));
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we still need this? It seems strange that it's set in every integration test when it's so rarely used in production.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yes, it's used by ESIntegTestCase which 30% of the time uses custom data path (see ESIntegTestCase#indexSettings())

@ywelsch
Copy link
Contributor Author

ywelsch commented May 6, 2019

The above test failures are not reproducible after merging in latest master.

@ywelsch ywelsch requested a review from DaveCTurner May 6, 2019 10:57
@ywelsch
Copy link
Contributor Author

ywelsch commented May 8, 2019

@elasticmachine run elasticsearch-ci/1
@elasticmachine run elasticsearch-ci/packaging-sample

@ywelsch
Copy link
Contributor Author

ywelsch commented May 8, 2019

@elasticmachine run elasticsearch-ci/1
@elasticmachine run elasticsearch-ci/default-distro

Copy link
Contributor

@DaveCTurner DaveCTurner left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

(I'm not excluding the possibility that some other tests need adjustment too, but CI didn't find them here)

@ywelsch
Copy link
Contributor Author

ywelsch commented May 21, 2019

@elasticmachine run elasticsearch-ci/1

2 similar comments
@ywelsch
Copy link
Contributor Author

ywelsch commented May 21, 2019

@elasticmachine run elasticsearch-ci/1

@ywelsch
Copy link
Contributor Author

ywelsch commented May 22, 2019

@elasticmachine run elasticsearch-ci/1

@ywelsch ywelsch merged commit b31482e into elastic:master May 22, 2019
ywelsch added a commit that referenced this pull request May 22, 2019
Moves the test infrastructure away from using node.max_local_storage_nodes, allowing us in a
follow-up PR to deprecate this setting in 7.x and to remove it in 8.0.

This also changes the behavior of InternalTestCluster so that starting up nodes will not automatically
reuse data folders of previously stopped nodes. If this behavior is desired, it needs to be explicitly
done by passing the data path from the stopped node to the new node that is started.
gurkankaymak pushed a commit to gurkankaymak/elasticsearch that referenced this pull request May 27, 2019
…ic#41652)

Moves the test infrastructure away from using node.max_local_storage_nodes, allowing us in a
follow-up PR to deprecate this setting in 7.x and to remove it in 8.0.

This also changes the behavior of InternalTestCluster so that starting up nodes will not automatically
reuse data folders of previously stopped nodes. If this behavior is desired, it needs to be explicitly
done by passing the data path from the stopped node to the new node that is started.
ywelsch added a commit that referenced this pull request Dec 17, 2019
With node ordinals gone, there's no longer a need for such a complicated full cluster restart
procedure (as we can now uniquely associate nodes to data folders).

Follow-up to #41652
ywelsch added a commit that referenced this pull request Dec 17, 2019
With node ordinals gone, there's no longer a need for such a complicated full cluster restart
procedure (as we can now uniquely associate nodes to data folders).

Follow-up to #41652
SivagurunathanV pushed a commit to SivagurunathanV/elasticsearch that referenced this pull request Jan 23, 2020
With node ordinals gone, there's no longer a need for such a complicated full cluster restart
procedure (as we can now uniquely associate nodes to data folders).

Follow-up to elastic#41652
@mark-vieira mark-vieira added the Team:Delivery Meta label for Delivery team label Nov 11, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
:Delivery/Build Build or test infrastructure >non-issue Team:Delivery Meta label for Delivery team v7.2.0 v8.0.0-alpha1
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants