Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Adding ML HLRC wrapper and put_job API call #32726

Merged
merged 2 commits into from
Aug 10, 2018

Conversation

benwtrent
Copy link
Member

This adds the Machine Learning HLRC wrapper class and creates the first request/response objects necessary for the PUT /job/<job_id> request.

@elasticmachine
Copy link
Collaborator

Pinging @elastic/ml-core

return generator.ofCodePointsLength(random(), 10, 10);
}

private static Job buildRandomJob(String jobId) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Most of this PR looks really good, but I wonder if in the future we'd regret creating a totally random job in the integration test (which runs against a real cluster, so no mocking). As more endpoints need to be tested the tests will have to build on each other. For example, you can't test opening a job until you've created one, and you can't test closing a job until you've created one and opened it. So maybe replace this with a buildTestJob(String jobId) method that always builds the same structure of job. This one would be a reasonable choice: https://www.elastic.co/guide/en/elasticsearch/reference/current/ml-put-job.html

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@droberts195 yeah, I had this same debate with myself. I initially had just a consistent job being created. However to better test locally, I ran numerous tests with this random creation and just never reverted back to the consistent job.

I have no qualms making it a semi-statically created job. Will update shortly.

@benwtrent
Copy link
Member Author

Jenkins retest this please

Copy link
Contributor

@droberts195 droberts195 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM


DataDescription.Builder dataDescription = new DataDescription.Builder();
dataDescription.setTimeFormat(randomFrom(DataDescription.EPOCH_MS, DataDescription.EPOCH));
dataDescription.setTimeField(randomAlphaOfLength(10));
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

To make jobs built with this method reusable when we come to send data to them in other tests, I think it would be better to fix the time format to EPOCH_MS and time field name to a specific value, e.g. "time".

However, since it takes hours to get the PR CI build to complete and we'll be changing this same file in future PRs that implement the other endpoints I'm happy to leave this as-is for now.

@benwtrent benwtrent merged commit 94a9b25 into elastic:master Aug 10, 2018
benwtrent added a commit that referenced this pull request Aug 10, 2018
* Adding ML HLRC wrapper and put_job API call

* Changing integration test job to have consistent stucture
@javanna
Copy link
Member

javanna commented Aug 10, 2018

heya stumbled upon this merging conflicts in another PR, I noticed that this new API is missing in the docs. Not sure whether this was done on purpose or not but I thought I would raise it.

@droberts195
Copy link
Contributor

@javanna yes, I'm going to do the docs for this endpoint in a separate PR because they'll be large and complicated and @benwtrent has only recently joined us. put_job is by far our most complicated endpoint in terms of options that need to be documented. It would have been nice to start with a simpler one but unfortunately the other ML endpoints require that a job has been created, so we were forced to do the most complicated endpoint first.

@elasticmachine
Copy link
Collaborator

Pinging @elastic/es-core-infra

@benwtrent benwtrent deleted the feature/ml-hlrc-put-job branch November 26, 2018 16:40
@jimczi jimczi added v7.0.0-beta1 and removed v7.0.0 labels Feb 7, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

6 participants