System indices ignore all user templates #87260
Conversation
When creating an index with a system index descriptor, ignore all templates. Update tests that check template behavior.
williamrandolph
added
:Core/Infra/Core
|
Hi @williamrandolph, I've created a changelog YAML for you. |
| @@ -632,6 +639,52 @@ private ClusterState applyCreateIndexRequestWithV2Template( | |||
| ); | |||
| } | |||
|
|
|||
| private ClusterState applyCreateIndexRequestForSystemIndex( | |||
There was a problem hiding this comment.
At first, I used applyCreateIndexRequestWithV1Templates and a list of empty templates, but that seemed very unsatisfying.
|
Kibana does use templates for at least one system index Alternatively we can ignore user templates for all MANAGED system indices. |
|
Here's the related Kibana issue to migrate the security_session indices to supply the index options at creation time: elastic/kibana#134897 |
|
Pinging @elastic/es-core-infra (Team:Core/Infra) |
|
To mitigate the fact that Kibana is using templates we introduced the allowsTemplates flag on SystemIndexDescriptor (#87933). We now use this flag in MetadataCreateIndexService to only allow templates to impact index creation on index descriptors that have this flag. Tests have been adjusted accordingly to reflect this behaviour. We ensure that templates have impact only on index descriptors that have the flag explicitly on. |
docs/changelog/87260.yaml
Outdated
| summary: System indices ignore all user templates | ||
| area: Infra/Core | ||
| type: bug | ||
| issues: [] |
There was a problem hiding this comment.
Remember to add the relevant issue links here.
| 2 | ||
| ); | ||
|
|
||
| assertAcked(client().admin().indices().prepareDeleteTemplate("*").get()); |
There was a problem hiding this comment.
This line isn't necessary, it's done in the @After (and this should be handled by the test infrastructure anyway, I thought).
| public void testAutoCreateSystemAliasViaV1Template() throws Exception { | ||
| var nonPrimaryIndex = autoCreateSystemAliasViaV1Template(INDEX_NAME); | ||
|
|
||
| assertAliasesHidden(nonPrimaryIndex, Set.of(INDEX_NAME), 1); | ||
|
|
||
| assertAcked(client().admin().indices().prepareDeleteTemplate("*").get()); |
There was a problem hiding this comment.
Ah, it looks like this line was copied - can you remove it from here as well? It's equally unnecessary.
| 2 | ||
| ); | ||
|
|
||
| assertAcked(client().admin().indices().prepareDeleteTemplate("*").get()); |
There was a problem hiding this comment.
Same here, this is done in the @After.
|
|
||
| assertHasAliases(Set.of(".test-index", ".test-index-legacy-alias")); | ||
| assertHasAliases(Set.of(INDEX_NAME), INDEX_NAME, PRIMARY_INDEX_NAME, 1); | ||
|
|
||
| assertAcked(client().admin().indices().prepareDeleteTemplate("*").get()); |
...rnalClusterTest/java/org/elasticsearch/indices/TestSystemIndexDescriptorAllowsTemplates.java
Show resolved
Hide resolved
|
|
||
| assertAliasesHidden(nonPrimaryIndex, Set.of(INDEX_NAME), 1); | ||
|
|
||
| assertAcked( |
There was a problem hiding this comment.
I'm pretty sure this is unnecessary as well despite not being explicitly in the @After, I ran it locally without this x10 and every thing, passed, and :server:check passed as well.
There was a problem hiding this comment.
I'm inclined to leave it, it does clean up the template explicitly and at the same time it tests we can clean up composable templates.
...nalClusterTest/java/org/elasticsearch/action/admin/indices/create/CreateSystemIndicesIT.java
Show resolved
Hide resolved
|
@elasticmachine update branch |
|
Hey @gwbrown, I think I addressed all of your feedback. Do you mind taking another look? |
The fix for this has been merged to Kibana's 'main' branch. |
System indices should ignore all user templates, except when explicitly stated (allowTemplates flag on SystemIndexDescriptor). This was the intended behavior for 8.0.
If merged, this will fix the longstanding issue #42508, as well as the stepping-stone issue #74271.