[DOCS] Clarify migration from warm to cold tier for searchable snapshots

[arteam]

Specify that we can't just reuse the local data as a cold cache and
the migration incurs data transfer costs.

Closes #74385

[elasticmachine]

Pinging @elastic/es-docs (Team:Docs)

[elasticmachine]

Pinging @elastic/es-distributed (Team:Distributed)

[jrodewig]

Thanks for this update @arteam!

Is this isolated to the warm->cold transition?

It seems like you'd incur this cost whenever you create a searchable snapshot, regardless of the tier or phase transition. For example, if someone goes directly from hot->frozen, I think you'd incur the same data transfer costs.

[henningandersen]

I think we should elaborate a bit here, mentioning that this is also true when hot/warm and cold are co-located. I think this is the part that can confuse some users. Also, we should note that if you already take snapshots of your data, the actual data uploaded to the repo will be minimal. Finally, I am a bit worried about the terminology "data transfer costs", I think we should use clarify that it requires a download of data which may have costs depending on operating environment. Perhaps link to what David wrote in #77607 rather than elaborate too much here on the costs.

I think frozen is less confusing since:

1. We recommend using dedicated frozen nodes.
2. Partially mounted indices will only download a minimum set of data initially.
3. The bulk of the download happens on search.

Still, adding a note may be worthwhile

[arteam]

I think #77607 kind of supersedes this change and makes it superfluous. #77607 explains in detail all the points about the migration between the tiers including the structure of "data transfer costs".

[DaveCTurner]

I just came across this, sorry I didn't know you were working on these docs too @arteam (hence why I opened #77607). Echoing what Henning says, yes, I think we need to write more words here as this is the sort of thing that comes up frequently when speaking to customers. If we just said this we would be giving the impression that searchable snapshots normally cost more in terms of data transfer than regular indices, which isn't the case.

[arteam]

Totally agree, that's why I left my comment #77583 (comment). The doc change #77607 is definitely much more clear, detailed, and articulate than this change which is too blunt. I totally agree that we shouldn't give a "data transfer costs" warning by default because for the majority of customers these costs would be negligible.

I think we can close this issue is favour of #77607 when it's merged.

[henningandersen]

I think the specific point in #74385 is not really covered by #77607 and could still deserve an explicit call-out, either in #77607 or as a follow-up. When hot/warm and cold are co-located, it is not obvious from docs that it will still need to be downloaded (even when no "relocation" happens).

[DaveCTurner]

I think it is pretty much covered in #77607. We say these things

Takes a snapshot of the managed index in the configured repository and mounts it
[...]
Mounting and relocating the shards of {search-snap} indices involves copying data from the snapshot repository.

I've adjusted this last bit to say "... involves copying the shard contents ..." in 93e0141. Do you think that's enough?

[arteam]

Superseded by #77607