[ML] Migrate unallocated jobs and datafeeds #37536
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
Pinging @elastic/ml-core |
43 tasks
davidkyle
commented
Jan 16, 2019
|
|
||
| // TODO is eligible for migration check (all nodes are at the same version) | ||
| if (params.getJob() == null) { | ||
| return AWAITING_MIGRATION; |
There was a problem hiding this comment.
The major difference between this and 6.6 is that in a mixed cluster an open job will not get an assignment i.e. a job left open during rolling upgrade will not be assigned until it has been migrated and the persistent task parameters changed which will only happen once all nodes in the cluster are at least version 6.6.0. 6.7 is capable of running un-migrated jobs so we could allow the un-migrated job to run in a mixed cluster depending on the result of MlConfigMigrationEligibilityCheck.jobIsEligibleForMigration.
Either way I need to remove the TODO
Migrate ml job and datafeed config of open jobs and update the parameters of the persistent tasks as they become unallocated during a rolling upgrade. Block allocation of ml persistent tasks until the configs are migrated.
davidkyle
force-pushed
the
migrate-unallocated
branch
from
3f71c36 to
ccd29f3
Compare
benwtrent
approved these changes
Jan 23, 2019
|
run elasticsearch-ci/2 |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Backport of #37430 but some tests have changed. Reproducing the original description here:
The final step of the process of migrating ml configs in #32905. This change migrates jobs & datafeed configs of open jobs once the persistent task becomes unallocated, the persistent task parameters must be updated for open jobs.
OpenJobPersistentTasksExecutor.getAssignment will not assign a task that has the missing parameters
WIP as this change cannot be merged until the issue described in #36810 (comment) has been resolved otherwise the rolling upgrade tests will fail intermittently.The cause was fixed in #37656