Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[ML] Add a "verbose" option to the data frame analytics stats endpoint #59125

Closed
droberts195 opened this issue Jul 7, 2020 · 2 comments
Closed
Labels
>enhancement :ml Machine learning

Comments

@droberts195
Copy link
Contributor

The data frame analytics stats endpoint can return extremely verbose information that is only useful if it is plotted on a chart. For example:

Screenshot 2020-04-07 at 12 52 24

For REST API clients that do not have the ability to plot the fold_values on a chart the information is just clutter that makes it hard to see other elements of the stats.

To avoid this problem we should have a ?verbose option on the REST endpoint for data frame analytics stats. (This does not need to be implemented in the Java clients as they can always retrieve the whole object and the user just doesn't call the methods that return verbose information if they don't want it.)

An example of a parameter that only affects the X-Content representation of the results is ?explain on the find_file_structure endpoint. This doesn't go into the request but instead gets passed in the params argument to the toXContent methods that render the response in JSON so they can decide not to print certain things. So ?verbose for data frame analytics stats can be done similarly.

One complication is that these same toXContent methods also create the JSON when the stats are indexed. In this case we need to keep all the fields. One way to make this work would be to make them write all the fields by default and only skip the detail when the verbose parameter was explicitly false, but default verbose to false if not specified in the REST handler. Another way would be to have a FOR_INTERNAL_STORAGE parameter that we pass from the places where we're persisting to an index.

The final question is exactly which fields to omit when ?verbose is not specified. Certainly fold_values, but maybe there are others. Please have a look and propose something that seems reasonable.

@droberts195 droberts195 added >enhancement :ml Machine learning labels Jul 7, 2020
@elasticmachine
Copy link
Collaborator

Pinging @elastic/ml-core (:ml)

@droberts195
Copy link
Contributor Author

Fixed by #59589

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
>enhancement :ml Machine learning
Projects
None yet
Development

No branches or pull requests

3 participants