Downsampling: copyindex.hidden setting from source

[csoulios]

Rollup indices are initially created as hidden (index.hidden: true).

At the end of the rollup process, we must set this setting to the value the source index has

[elasticsearchmachine]

Pinging @elastic/es-analytics-geo (Team:Analytics)

[csoulios]

There's a small catch here. If the setting index.hidden was not set at all in the source index, the default value is false. Here, we explicitly set the value to false. Hopefully, this does not interfere with anything else.

[dakrone]

Could we do something like

if (sourceIndexMetadata.isHidden() == false && sourceIndexMetadata.getSettings().keySet().contains(IndexMetadata.SETTING_INDEX_HIDDEN)) {
    settings.put(IndexMetadata.SETTING_INDEX_HIDDEN, false);
}

To only set the setting if it does indeed exist on the source as explicitly set?

[csoulios]

When the rollup index is created setting index.hidden is set to true. So, after rollup ends we want to revert this value.

I changed the code so that it removes setting (calling setNull()) if it is not set at all in the source index.

[dakrone]

Sounds good to me 👍

[salvatore-campagna]

Just a minor comment. For the rest LGTM.

[dakrone]

LGTM, I left one suggestion though, about how we might be able to do it without the explicit setting.

[dakrone]

Could we do something like

if (sourceIndexMetadata.isHidden() == false && sourceIndexMetadata.getSettings().keySet().contains(IndexMetadata.SETTING_INDEX_HIDDEN)) {
    settings.put(IndexMetadata.SETTING_INDEX_HIDDEN, false);
}

To only set the setting if it does indeed exist on the source as explicitly set?

[salvatore-campagna]

LGTM