Use value of kibana.index parameter as prefix for all kibana system indices

[parosio]

I'd suggest the use of kibana.index param value as prefix for all kibana system indices.

Use case:

Multiple kibana installations insisting on same Elastic cluster, each one with its kibana.index: It'd be useful if ancillary kibana system indices were named (by default) after the main kibana index name, avoiding the need to explicitly specify them.
See also this thread on discuss.elastic about .kibana_task_manager index.

I'd also suggest to enhance the current documentation of kibana settings specifying the list of indices created by kibana, expecially for those needed (#61199): the only references I've found about the .kibana_task_manager index are in release notes and in the developer guide.

Related to:

#61199

[elasticmachine]

Pinging @elastic/kibana-platform (Team:Platform)

[pgayvallet]

@rudolf Any idea of the implications, and globally, WDYT?

[rudolf]

Historically kibana.index was the way to configure two Kibana "tenants" on one ES cluster. I agree with @parosio that we've never had good documentation for this and with us adding more indices that don't follow this pattern it's gotten even harder for users to set up Kibana correctly in this way.

However, with the introduction of Spaces and Cross Cluster Search we're not aware of strong use cases for continuing to support kibana multi-tenancy. The main discussion has been happening in #60053

@parosio Can you elaborate why you'd like to use multiple tenants on one ES cluster? Do you have a problem which Spaces or Cross Cluster Search can't solve? What would be the impact on you if multi-tenant support was removed in 8.x?

[parosio]

@rudolf: to answer your question:
we're still using 6.7 (updated from a preexisting 5.4), with a 3rd party Elastic plugin to implement user authorization to specific indices, which served us fine (meaning: never really watched into elastic basic authZ features).
Everything fine until we began "hosting" what is essentially a new application, managed by a different group: the less intrusive solution was to give a separate kibana instance (with its kibana.index) to the new group.

Having no experience at all with Spaces, I don't know what effort is needed to set up the equivalent of two totally different applications. I suspect, in any case, it would require a superuser who could manage both (what we'd like to avoid).

About the impact of removal of multi-tenant support: I suppose we'd either never update to 8, or completely split the environments, with a full stack for each application. But the latter option requires a lot more resources, so it'd be easier to never update.

[rudolf]

I'd encourage you to investigate Spaces, it's available with the free Basic licence and was designed for exactly the use case you seem to have. You could create a space for each group so that their dashboards and other saved objects are isolated. Note: although you can use spaces to enforce security this is not necessary, you can decide to give all users access to all spaces. This way there's no user management overhead, but you still get isolation.

[rudolf]

We will be removing the ability to use Kibana "multi-tenancy" and removing the kibana.index configuration setting in 8.0 #82020

[rudolf]

Given that this will is deprecated and will be removed in 8.0 we would not want to encourage more users to adopt it in 7.x so going to close this with the recommendation to adopt spaces instead.