Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[7.x][ML] Use unique named pipes for data frame analytics (#70918) #70926

Conversation

dimitris-athanasiou
Copy link
Contributor

When a data frame analytics job is stopped because of
a call to the _stop API, if the process is running it is killed.
Depending on the OS, it may take some time to delete all the
used named pipes. This means that in a scenario where the job
is restarted immediately after it is possible that the old
named pipes are used which results to the new process not
properly communicating with java.

This has been the underlying issue of #70698 and #67581.

This commit fixes it by using unique identifiers for the named
pipes.

Closes #70698

Backport of #70918

)

When a data frame analytics job is stopped because of
a call to the _stop API, if the process is running it is killed.
Depending on the OS, it may take some time to delete all the
used named pipes. This means that in a scenario where the job
is restarted immediately after it is possible that the old
named pipes are used which results to the new process not
properly communicating with java.

This has been the underlying issue of elastic#70698 and elastic#67581.

This commit fixes it by using unique identifiers for the named
pipes.

Closes elastic#70698

Backport of elastic#70918
@dimitris-athanasiou dimitris-athanasiou added >test Issues or PRs that are addressing/adding tests :ml Machine learning backport labels Mar 26, 2021
@elasticmachine elasticmachine added the Team:ML Meta label for the ML team label Mar 26, 2021
@elasticmachine
Copy link
Collaborator

Pinging @elastic/ml-core (Team:ML)

@dimitris-athanasiou dimitris-athanasiou merged commit 64ff5f6 into elastic:7.x Mar 26, 2021
@dimitris-athanasiou dimitris-athanasiou deleted the use-unique-named-pipes-for-dfa-7x branch March 26, 2021 17:28
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
backport :ml Machine learning Team:ML Meta label for the ML team >test Issues or PRs that are addressing/adding tests
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants