Update the QA project to use test cluster api #1430
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Also update the project to run fixtures on the java 8 runtime. Java home can now be set on cluster configs and is used by fixtures. A cluster can be configured with a dependent ElasticsearchCluster. This cluster is depended on for any configuration tasks in case its cluster address is used. Settings are now stored as objects in cluster configurations instead of Strings since they may be GStrings with closures inside of them that must be resolved ONLY at configuration writing time. This is needed to resolve test cluster http addresses which are only discoverable once the cluster is started. In order to make this resolution process simpler, removed most of the Map<String, String> types in the code and replaced them with the FileSettings type which can resolve closures as needed when writing the configuration to a file. Upgraded Spark QA runtime to 2.3.4 as the older version has been removed from apache's mirrors. AbstractClusterTask now extends DefaultTestClustersTask, as it is the only way to ensure that Elasticsearch is still running when starting the task. In order to configure JAVA_HOME consistently, each AbstractClusterTask now has its environment variables resolved in its super class instead of with duplicated code in each task.
jbaiera
changed the title
jakelandis
reviewed
Feb 25, 2020
...ain/groovy/org/elasticsearch/hadoop/gradle/fixture/hadoop/HadoopClusterFormationTasks.groovy
Outdated
Show resolved
Hide resolved
...ain/groovy/org/elasticsearch/hadoop/gradle/fixture/hadoop/HadoopClusterFormationTasks.groovy
Outdated
Show resolved
Hide resolved
mark-vieira
approved these changes
Feb 28, 2020
| @@ -69,7 +72,7 @@ class HadoopClusterFormationTasks { | |||
| * <p> | |||
| * Returns a list of NodeInfo objects for each node in the cluster. | |||
| * | |||
| * Based on {@link org.elasticsearch.gradle.test.ClusterFormationTasks} | |||
| * Based on (now removed) org.elasticsearch.gradle.test.ClusterFormationTasks | |||
There was a problem hiding this comment.
Is this really necessary then?
| static String unapplyString(Object value) { | ||
| if (value == null) { | ||
| return null | ||
| } else if (value instanceof Closure) { |
There was a problem hiding this comment.
You can make this even more generic and just use Callable.
jbaiera
added a commit
that referenced
this pull request
May 13, 2020
Also update the project to run fixtures on the java 8 runtime. Java home can now be set on cluster configs and is used by fixtures. A cluster can be configured with a dependent ElasticsearchCluster. This cluster is depended on for any configuration tasks in case its cluster address is used. Settings are now stored as objects in cluster configurations instead of Strings since they may be GStrings with closures inside of them that must be resolved ONLY at configuration writing time. This is needed to resolve test cluster http addresses which are only discoverable once the cluster is started. In order to make this resolution process simpler, removed most of the Map<String, String> types in the code and replaced them with the FileSettings type which can resolve closures as needed when writing the configuration to a file. Upgraded Spark QA runtime to 2.3.4 as the older version has been removed from apache's mirrors. AbstractClusterTask now extends DefaultTestClustersTask, as it is the only way to ensure that Elasticsearch is still running when starting the task. In order to configure JAVA_HOME consistently, each AbstractClusterTask now has its environment variables resolved in its super class instead of with duplicated code in each task.
|
Relates #1423 |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This PR includes changes necessary to run the Kerberos QA tests using the new test clusters functionality in build-tools.
This also updates the project to run fixtures on the Java 8 runtime. JAVA_HOME can now be set on cluster configs and is used by fixtures.
A cluster can be configured with a dependent ElasticsearchCluster. This cluster is depended on for any configuration tasks in case its cluster address is used.
Settings are now stored as objects in cluster configurations instead of Strings since they may be GStrings with closures inside of them that must be resolved ONLY at configuration writing time. This is needed to resolve test cluster http addresses which are only discoverable once the cluster is started.
In order to make this resolution process simpler, removed most of the Map<String, String> types in the code and replaced them with the FileSettings type which can resolve closures as needed when writing the configuration to a file.
Upgraded Spark QA runtime to 2.3.4 as the older version has been removed from apache's mirrors.
AbstractClusterTask now extends DefaultTestClustersTask, as it is the only way to ensure that Elasticsearch is still running when starting the task.
In order to configure JAVA_HOME consistently, each AbstractClusterTask now has its environment variables resolved in its super class instead of with duplicated code in each task.