Skip to content

Commit

Permalink
[DOCS] Simplify ML upgrade step (elastic#40006)
Browse files Browse the repository at this point in the history
  • Loading branch information
lcawl committed Mar 25, 2019
1 parent 495dc11 commit 6f03a6c
Show file tree
Hide file tree
Showing 4 changed files with 43 additions and 29 deletions.
48 changes: 25 additions & 23 deletions docs/reference/upgrade/close-ml.asciidoc
Original file line number Diff line number Diff line change
Expand Up @@ -11,33 +11,35 @@ POST _ml/set_upgrade_mode?enabled=false
// TEARDOWN
////////////

If your {ml} indices were created earlier than the previous major version, they
must be reindexed. In those circumstances, there must be no machine learning
jobs running during the upgrade.

In all other circumstances, there is no requirement to close your {ml} jobs.
There are, however, advantages to doing so. If you choose to leave your jobs
running during the upgrade, they are affected when you stop the {ml} nodes. The
jobs move to another {ml} node and restore the model states. This scenario has
the least disruption to the active {ml} jobs but incurs the highest load on the
cluster.

To close all {ml} jobs before you upgrade, see
{stack-ov}/stopping-ml.html[Stopping {ml}]. This method persists the model
state at the moment of closure, which means that when you open your jobs after
the upgrade, they use the exact same model. This scenario takes the most time,
however, especially if you have many jobs or jobs with large model states.

To temporarily halt the tasks associated with your {ml} jobs and {dfeeds} and
prevent new jobs from opening, use the <<ml-set-upgrade-mode,set upgrade mode API>>:
If your {ml} indices were created before {prev-major-version}, you must
<<reindex-upgrade,reindex the indices>>.

If your {ml} indices were created in {prev-major-version}, you can:

* Leave your {ml} jobs running during the upgrade. When you shut down a
{ml} node, its jobs automatically move to another node and restore the model
states. This option enables your jobs to continue running during the upgrade but
it puts increased load on the cluster.
* Temporarily halt the tasks associated with your {ml} jobs and {dfeeds} and
prevent new jobs from opening by using the
<<ml-set-upgrade-mode,set upgrade mode API>>:
+
--
[source,js]
--------------------------------------------------
POST _ml/set_upgrade_mode?enabled=true
--------------------------------------------------
// CONSOLE
This method does not persist the absolute latest model state, rather it uses the
last model state that was automatically saved. By halting the tasks, you avoid
incurring the cost of managing active jobs during the upgrade and it's quicker
than stopping {dfeeds} and closing jobs.
When you disable upgrade mode, the jobs resume using the last model
state that was automatically saved. This option avoids the overhead of managing
active jobs during the upgrade and is faster than explicitly stopping {dfeeds}
and closing jobs.
--
* {stack-ov}/stopping-ml.html[Stop all {dfeeds} and close all jobs]. This option
saves the model state at the time of closure. When you reopen the jobs after the
upgrade, they use the exact same model. However, saving the latest model state
takes longer than using upgrade mode, especially if you have a lot of jobs or
jobs with large model states.
2 changes: 1 addition & 1 deletion docs/reference/upgrade/cluster_restart.asciidoc
Original file line number Diff line number Diff line change
Expand Up @@ -26,7 +26,7 @@ recovery.
include::synced-flush.asciidoc[]
--

. *Stop any machine learning jobs that are running.*
. *Temporarily stop the tasks associated with active {ml} jobs and {dfeeds}.* (Optional)
+
--
include::close-ml.asciidoc[]
Expand Down
20 changes: 16 additions & 4 deletions docs/reference/upgrade/reindex_upgrade.asciidoc
Original file line number Diff line number Diff line change
Expand Up @@ -59,9 +59,10 @@ ifdef::include-xpack[]
[TIP]
====
If you use {ml-features} and your {ml} indices were created before
{prev-major-version}, you must
{stack-ov}/stopping-ml.html[stop all {dfeeds} and close all {ml} jobs] before
you reindex the indices.
{prev-major-version}, you must temporarily halt the tasks associated with your
{ml} jobs and {dfeeds} and prevent new jobs from opening during the reindex. Use
the <<ml-set-upgrade-mode,set upgrade mode API>> or
{stack-ov}/stopping-ml.html[stop all {dfeeds} and close all {ml} jobs].
If you use {es} {security-features}, before you reindex `.security*` internal
indices it is a good idea to create a temporary superuser account in the `file`
Expand Down Expand Up @@ -112,6 +113,17 @@ indices from the previous major version to be upgraded to the
current major version. Skipping a major version means that you must
resolve any backward compatibility issues yourself.
ifdef::include-xpack[]
If you use {ml-features} and you're migrating indices from a 6.5 or earlier
cluster, the job and {dfeed} configuration information are not stored in an
index. You must recreate your {ml} jobs in the new cluster. If you are migrating
from a 6.6 or later cluster, it is a good idea to temporarily halt the tasks
associated with your {ml} jobs and {dfeeds} to prevent inconsistencies between
different {ml} indices that are reindexed at slightly different times. Use the
<<ml-set-upgrade-mode,set upgrade mode API>> or
{stack-ov}/stopping-ml.html[stop all {dfeeds} and close all {ml} jobs].
endif::include-xpack[]
=============================================

To migrate your indices:
Expand Down Expand Up @@ -184,4 +196,4 @@ monitor progress of the reindex job with the <<tasks,task API>>:
`30s` and `1`).

.. Once reindexing is complete and the status of the new index is `green`,
you can delete the old index.
you can delete the old index.
2 changes: 1 addition & 1 deletion docs/reference/upgrade/rolling_upgrade.asciidoc
Original file line number Diff line number Diff line change
Expand Up @@ -35,7 +35,7 @@ include::synced-flush.asciidoc[]

--

. *Stop any machine learning jobs that are running.*
. *Temporarily stop the tasks associated with active {ml} jobs and {dfeeds}.* (Optional)
+
--
include::close-ml.asciidoc[]
Expand Down

0 comments on commit 6f03a6c

Please sign in to comment.