Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[ML][Data Frame] add node attr to GET _stats #43842

Merged

Conversation

benwtrent
Copy link
Member

This adds node information to the _stats call. I made the node field part of the state sub-object. This made the most sense to me as state also contains the task state information, and the reason for its current state.

closes #43743

@elasticmachine
Copy link
Collaborator

Pinging @elastic/ml-core

@davidkyle
Copy link
Member

retest this please

Copy link
Member

@davidkyle davidkyle left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. left some minor comments

@@ -37,7 +38,9 @@ public void testFromXContent() throws IOException {
DataFrameTransformStateTests::toXContent,
DataFrameTransformState::fromXContent)
.supportsUnknownFields(true)
.randomFieldsExcludeFilter(field -> field.equals("current_position"))
.randomFieldsExcludeFilter(field -> field.equals("current_position") ||
field.equals("node") ||
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could be simplified to field.startsWith("node") but shouldn't it just be the map node.attributes that is ignored.

@@ -64,7 +64,7 @@ protected boolean supportsUnknownFields() {

@Override
protected Predicate<String> getRandomFieldsExcludeFilter() {
return field -> field.equals("state.current_position");
return field -> field.equals("state.current_position") || field.equals("state.node") || field.equals("state.node.attributes");
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

as above

node.getEphemeralId(),
node.getAddress().toString(),
// TODO add data_frame attributes when/if they are added
Collections.emptyMap());
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe make attributes nullable so it is not written as an empty object attributes: {} in toXContent


@Override
protected Predicate<String> getRandomFieldsExcludeFilter() {
return field -> !field.isEmpty();
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If another nested object besides the attribute map was added to NodeAttributes it would be excluded here.

private static void setNodeAttributes(DataFrameTransformStateAndStats dataFrameTransformStateAndStats,
PersistentTasksCustomMetaData persistentTasksCustomMetaData,
ClusterState state) {
var pTask = persistentTasksCustomMetaData.getTask(dataFrameTransformStateAndStats.getTransformId());
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

var - how very modern

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🎩
😄

@benwtrent
Copy link
Member Author

@elasticmachine update branch

@benwtrent
Copy link
Member Author

jenkins retest this please

@benwtrent
Copy link
Member Author

run elasticsearch-ci/2

@benwtrent
Copy link
Member Author

run elasticsearch-ci/packaging-sample

1 similar comment
@benwtrent
Copy link
Member Author

run elasticsearch-ci/packaging-sample

@benwtrent benwtrent merged commit 8f4ae63 into elastic:master Jul 2, 2019
@benwtrent benwtrent deleted the feature/ml-df-add-node-attr-to-_stats branch July 2, 2019 19:32
benwtrent added a commit to benwtrent/elasticsearch that referenced this pull request Jul 2, 2019
* [ML][Data Frame] add node attr to GET _stats

* addressing testing issues with node.attributes
benwtrent added a commit that referenced this pull request Jul 3, 2019
* [ML][Data Frame] add node attr to GET _stats (#43842)

* [ML][Data Frame] add node attr to GET _stats

* addressing testing issues with node.attributes

* adjusting for backport
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[ML] Add node info to data frame _stats
4 participants