[Index Management] Add support for index mode #197874

ElenaStoeva · 2024-10-25T16:32:17Z

Closes #195772

Summary

This PR displays the index mode setting in Index templates (at Review step in creation flow and at template details flyout) and Data streams (in ds table and ds details flyout).

How to test:

Creating a Logsdb index template and data stream:

Go to Index Managament -> Index templates and start creating a new template
Add a name and an index pattern test-logsdb and then go to the Settings step
Add the setting "index.mode": "logsdb.
Go to last step (review) and verify that the index mode is correctly displayed in Summary. Save the template.
In the template flyout, verify that the index mode displays the correct label.
Go to Console and create a data stream that mathes the index pattern of the created index template: PUT _data_stream/test-logsdb
Go to Index Management -> Data streams
In the data streams table, verify that the index mode column is correct for the new data stream.
Open the new data stream and verify that the details flyout displays the correct index mode.

Creating a Time series index template and data stream:

Go to Index Managament -> Index templates and start creating a new template
Add a name and an index pattern test-tsds and then go to the Settings step
Add the setting "index.mode": "time_series.
For time series index template, we also need to add a mapping with a time_series_dimension property. Go to the Mappings step, click on "Load JSON" and add the following mappings object:

{
  "properties": {
    "id": {
      "type": "keyword",
      "time_series_dimension": true
    }
  }
}

Go to last step (review) and verify that the index mode is correctly displayed in Summary. Save the template.
In the template flyout, verify that the index mode displays the correct label.
Go to Console and create a data stream that mathes the index pattern of the created index template: PUT _data_stream/test-tsds
Go to Index Management -> Data streams
In the data streams table, verify that the index mode column is correct for the new data stream.
Open the new data stream and verify that the details flyout displays the correct index mode.

Checklist

Any text added follows EUI's writing guidelines, uses sentence case text and includes i18n support
Unit or functional tests were updated or added to match the most common scenarios
Flaky Test Runner was used on any tests changed: https://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/7288
Any UI touched in this PR is usable by keyboard only (learn more about keyboard accessibility)
Any UI touched in this PR does not create any new axe failures (run axe in browser: FF, Chrome)
This renders correctly on smaller devices using a responsive layout. (You can test this in your browser)
This was checked for cross-browser compatibility

bitzandeb · 2024-10-28T12:21:13Z

Hey @ElenaStoeva - could we update the value formatting in the UI so they are like this?

Time series
LogsDB
Standard

Thank you!

florent-leborgne

Copy LGTM overall. I'm just wondering if we can make the tooltip a bit clearer? I don't know if my suggestion makes sense, but happy to brainstorm if needed

...ication/sections/home/data_stream_list/data_stream_detail_panel/data_stream_detail_panel.tsx

elasticmachine · 2024-10-28T19:08:18Z

Pinging @elastic/kibana-management (Team:Kibana Management)

…-fix'

kibanamachine · 2024-10-28T21:16:35Z

Flaky Test Runner Stats

🎉 All tests passed! - kibana-flaky-test-suite-runner#7288

[✅] x-pack/test/functional/apps/index_management/config.ts: 200/200 tests passed.

see run history

…a/kibana into im/index-mode-support

ElenaStoeva · 2024-10-29T11:24:18Z

x-pack/plugins/index_management/common/types/indices.ts


-export interface IndexModule {


Removing this manual type in favor of IndicesIndexSettingsKeys type from Es client since the manual one has missing properties (e.g. mode).

ElenaStoeva · 2024-10-29T11:26:20Z

...anagement/public/application/sections/home/index_list/index_table/index_table_pagination.tsx

@@ -27,7 +27,7 @@ export const IndexTablePagination = ({
  readURLParams,
  setURLParam,
 }: IndexTablePaginationProps) => {
-  const { pageSize, onTableChange } = useEuiTablePersist<IndexModule>({
+  const { pageSize, onTableChange } = useEuiTablePersist<Index>({


It seems here we used an incorrect type - it should be Index.

x-pack/plugins/index_management/public/application/lib/index_mode_labels.ts

mattkime · 2024-10-30T02:53:57Z

x-pack/test/functional/apps/index_management/data_streams_tab/data_streams_tab.ts

@@ -88,6 +88,16 @@ export default ({ getPageObjects, getService }: FtrProviderContext) => {
      await testSubjects.click('closeDetailsButton');
    });

+    it('shows the correct index mode in the details flyout', async () => {
+      // Open details flyout


It would be good to have tests that also check for logsdb

Added with 2590ba4.

mattkime

Left some minor change requests and it could use a couple more tests.

elasticmachine · 2024-10-30T11:44:13Z

Pinging @elastic/obs-ux-management-team (Team:obs-ux-management)

github-actions · 2024-10-30T11:44:21Z

🤖 GitHub comments

Expand to view the GitHub comments

Just comment with:

/oblt-deploy : Deploy a Kibana instance using the Observability test environments.
run docs-build : Re-trigger the docs validation. (use unformatted text in the comment!)

ElenaStoeva · 2024-10-30T12:03:48Z

Thanks for the review @mattkime! I added a test as suggested and responded to your questions. Let me know if you have any other suggestions.

elasticmachine · 2024-10-30T16:04:46Z

💚 Build Succeeded

Buildkite Build
Commit: ab31a8f
Kibana Serverless Image: docker.elastic.co/kibana-ci/kibana-serverless:pr-197874-ab31a8f91f55

Metrics [docs]

Module Count

Fewer modules leads to a faster build time

id	before	after	diff
`indexManagement`	694	695	+1

Public APIs missing comments

Total count of every public API that lacks a comment. Target amount is 0. Run node scripts/build_api_docs --plugin [yourplugin] --stats comments for more detailed information.

id	before	after	diff
`indexManagement`	245	236	-9

Async chunks

Total size of all lazy-loaded chunks that will be downloaded as the user navigates the app

id	before	after	diff
`indexManagement`	688.3KB	690.0KB	+1.7KB
`synthetics`	1.1MB	1.1MB	+21.0B
total			+1.7KB

Unknown metric groups

API count

id	before	after	diff
`indexManagement`	250	241	-9

History

cc @ElenaStoeva

mattkime

Changes look good and work well!

kibanamachine · 2024-10-31T09:25:32Z

Starting backport for target branches: 8.x

https://github.com/elastic/kibana/actions/runs/11609320372

Closes elastic#195772 ## Summary This PR displays the index mode setting in Index templates (at Review step in creation flow and at template details flyout) and Data streams (in ds table and ds details flyout). <img width="1523" alt="Screenshot 2024-10-28 at 14 39 18" src="https://github.com/user-attachments/assets/9d40c1cd-d08a-4950-a679-29d9910ee845"> <img width="1523" alt="Screenshot 2024-10-28 at 19 05 29" src="https://github.com/user-attachments/assets/463e549d-68cd-4ddf-ae0a-cfe43e47af10"> <img width="1523" alt="Screenshot 2024-10-28 at 14 41 01" src="https://github.com/user-attachments/assets/4ccf8727-5913-4857-9dc2-48c40da3a356"> <img width="1523" alt="Screenshot 2024-10-28 at 14 41 09" src="https://github.com/user-attachments/assets/a6b6a0a9-7ae2-4201-8ff9-3bead915fb2a"> ### How to test: **Creating a Logsdb index template and data stream:** 1. Go to Index Managament -> Index templates and start creating a new template 2. Add a name and an index pattern `test-logsdb` and then go to the Settings step 3. Add the setting `"index.mode": "logsdb`. 4. Go to last step (review) and verify that the index mode is correctly displayed in Summary. Save the template. 5. In the template flyout, verify that the index mode displays the correct label. 6. Go to Console and create a data stream that mathes the index pattern of the created index template: `PUT _data_stream/test-logsdb` 7. Go to Index Management -> Data streams 8. In the data streams table, verify that the index mode column is correct for the new data stream. 9. Open the new data stream and verify that the details flyout displays the correct index mode. **Creating a Time series index template and data stream:** 1. Go to Index Managament -> Index templates and start creating a new template 2. Add a name and an index pattern `test-tsds` and then go to the Settings step 3. Add the setting `"index.mode": "time_series`. 4. For time series index template, we also need to add a mapping with a `time_series_dimension` property. Go to the Mappings step, click on "Load JSON" and add the following mappings object: ``` { "properties": { "id": { "type": "keyword", "time_series_dimension": true } } } ``` 6. Go to last step (review) and verify that the index mode is correctly displayed in Summary. Save the template. 7. In the template flyout, verify that the index mode displays the correct label. 8. Go to Console and create a data stream that mathes the index pattern of the created index template: `PUT _data_stream/test-tsds` 9. Go to Index Management -> Data streams 10. In the data streams table, verify that the index mode column is correct for the new data stream. 11. Open the new data stream and verify that the details flyout displays the correct index mode. ### Checklist - [x] Any text added follows [EUI's writing guidelines](https://elastic.github.io/eui/#/guidelines/writing), uses sentence case text and includes [i18n support](https://github.com/elastic/kibana/blob/main/packages/kbn-i18n/README.md) - [x] [Unit or functional tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html) were updated or added to match the most common scenarios - [x] [Flaky Test Runner](https://ci-stats.kibana.dev/trigger_flaky_test_runner/1) was used on any tests changed: https://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/7288 - [x] Any UI touched in this PR is usable by keyboard only (learn more about [keyboard accessibility](https://webaim.org/techniques/keyboard/)) - [x] Any UI touched in this PR does not create any new axe failures (run axe in browser: [FF](https://addons.mozilla.org/en-US/firefox/addon/axe-devtools/), [Chrome](https://chrome.google.com/webstore/detail/axe-web-accessibility-tes/lhdoppojpmngadmnindnejefpokejbdd?hl=en-US)) - [x] This renders correctly on smaller devices using a responsive layout. (You can test this [in your browser](https://www.browserstack.com/guide/responsive-testing-on-local-server)) - [x] This was checked for [cross-browser compatibility](https://www.elastic.co/support/matrix#matrix_browsers) --------- Co-authored-by: kibanamachine <[email protected]> (cherry picked from commit 40ddfbc)

kibanamachine · 2024-10-31T09:30:09Z

💚 All backports created successfully

Status	Branch	Result
✅	8.x

Note: Successful backport PRs will be merged automatically after passing CI.

Questions ?

Please refer to the Backport tool documentation

# Backport This will backport the following commits from `main` to `8.x`: - [[Index Management] Add support for index mode (#197874)](#197874)  ### Questions ? Please refer to the [Backport tool documentation](https://github.com/sqren/backport)  Co-authored-by: Elena Stoeva <[email protected]>

Fixes #203553 ## Summary This PR fixes the template schema so that the server allows creating a template with undefined index mode. The issue is that when we exposed the index mode info in #197874, we assumed that, by default, the index mode is `standard` so `indexMode` was added as a required field. However, in #199521, we added the index mode field only if data streams toggle is enabled, so the `indexMode` is `undefined`. The PR also fixes a smaller issuer where Index mode is displayed even when it is undefined (when data streams are disabled): **Before:** <img width="905" alt="Screenshot 2024-12-10 at 19 01 09" src="https://github.com/user-attachments/assets/e22cab14-eade-4591-8da8-f9aec5ad6c43"> **Now:** <img width="905" alt="Screenshot 2024-12-10 at 18 19 29" src="https://github.com/user-attachments/assets/e7768cdc-9610-4a25-b403-0320b7137db5"> **How to test:** 1. Go to Index Management -> Index templates and start creating an index template. 2. Specify the name and the index pattern and disable the "Data streams" toggle. 3. Go to Review step and verify that index mode is not displayed in the summary and that creating the index template is successful.

Fixes elastic#203553 ## Summary This PR fixes the template schema so that the server allows creating a template with undefined index mode. The issue is that when we exposed the index mode info in elastic#197874, we assumed that, by default, the index mode is `standard` so `indexMode` was added as a required field. However, in elastic#199521, we added the index mode field only if data streams toggle is enabled, so the `indexMode` is `undefined`. The PR also fixes a smaller issuer where Index mode is displayed even when it is undefined (when data streams are disabled): **Before:** <img width="905" alt="Screenshot 2024-12-10 at 19 01 09" src="https://github.com/user-attachments/assets/e22cab14-eade-4591-8da8-f9aec5ad6c43"> **Now:** <img width="905" alt="Screenshot 2024-12-10 at 18 19 29" src="https://github.com/user-attachments/assets/e7768cdc-9610-4a25-b403-0320b7137db5"> **How to test:** 1. Go to Index Management -> Index templates and start creating an index template. 2. Specify the name and the index pattern and disable the "Data streams" toggle. 3. Go to Review step and verify that index mode is not displayed in the summary and that creating the index template is successful. (cherry picked from commit bdc8032)

…203724) # Backport This will backport the following commits from `main` to `8.17`: - [[Index Management] Fix schema for index templates (#203552)](#203552)  ### Questions ? Please refer to the [Backport tool documentation](https://github.com/sqren/backport)  Co-authored-by: Elena Stoeva <[email protected]>

…03725) # Backport This will backport the following commits from `main` to `8.x`: - [[Index Management] Fix schema for index templates (#203552)](#203552)  ### Questions ? Please refer to the [Backport tool documentation](https://github.com/sqren/backport)  Co-authored-by: Elena Stoeva <[email protected]>

Fixes elastic#203553 ## Summary This PR fixes the template schema so that the server allows creating a template with undefined index mode. The issue is that when we exposed the index mode info in elastic#197874, we assumed that, by default, the index mode is `standard` so `indexMode` was added as a required field. However, in elastic#199521, we added the index mode field only if data streams toggle is enabled, so the `indexMode` is `undefined`. The PR also fixes a smaller issuer where Index mode is displayed even when it is undefined (when data streams are disabled): **Before:** <img width="905" alt="Screenshot 2024-12-10 at 19 01 09" src="https://github.com/user-attachments/assets/e22cab14-eade-4591-8da8-f9aec5ad6c43"> **Now:** <img width="905" alt="Screenshot 2024-12-10 at 18 19 29" src="https://github.com/user-attachments/assets/e7768cdc-9610-4a25-b403-0320b7137db5"> **How to test:** 1. Go to Index Management -> Index templates and start creating an index template. 2. Specify the name and the index pattern and disable the "Data streams" toggle. 3. Go to Review step and verify that index mode is not displayed in the summary and that creating the index template is successful.

[Index Management] Add support for index mode

09a5e42

ElenaStoeva self-assigned this Oct 25, 2024

ElenaStoeva added 2 commits October 25, 2024 18:24

Add api integration tests and index mode type

e781775

Add index mode to index template details summary

ce38555

ElenaStoeva added v9.0.0 v8.17.0 labels Oct 28, 2024

ElenaStoeva requested a review from florent-leborgne October 28, 2024 12:09

ElenaStoeva added 2 commits October 28, 2024 14:41

Add translated labels

6cbad6a

Add functional tests

adab4e3

florent-leborgne reviewed Oct 28, 2024

View reviewed changes

...ication/sections/home/data_stream_list/data_stream_detail_panel/data_stream_detail_panel.tsx Outdated Show resolved Hide resolved

ElenaStoeva added 2 commits October 28, 2024 15:59

Small fix

c47c473

Update copy of tooltip

e55abf4

ElenaStoeva marked this pull request as ready for review October 28, 2024 19:08

ElenaStoeva requested a review from a team as a code owner October 28, 2024 19:08

ElenaStoeva and others added 4 commits October 28, 2024 19:10

Merge branch 'main' into im/index-mode-support

f0507d4

[CI] Auto-commit changed files from 'node scripts/yarn_deduplicate'

249c9d3

[CI] Auto-commit changed files from 'node scripts/eslint --no-cache -…

0bbec28

…-fix'

[CI] Auto-commit changed files from 'node scripts/yarn_deduplicate'

19e0e1d

ElenaStoeva added 3 commits October 29, 2024 10:18

Fix failing jest tests

c059169

Merge branch 'im/index-mode-support' of https://github.com/ElenaStoev…

71566ec

…a/kibana into im/index-mode-support

Resolve type check errors

8eef79b

ElenaStoeva commented Oct 29, 2024

View reviewed changes

mattkime reviewed Oct 30, 2024

View reviewed changes

x-pack/plugins/index_management/public/application/lib/index_mode_labels.ts Show resolved Hide resolved

mattkime reviewed Oct 30, 2024

View reviewed changes

mattkime requested changes Oct 30, 2024

View reviewed changes

Fix failing test and type error, add suggested test

2590ba4

ElenaStoeva requested a review from a team as a code owner October 30, 2024 11:44

botelastic bot added ci:project-deploy-observability Create an Observability project Team:obs-ux-management Observability Management User Experience Team labels Oct 30, 2024

ElenaStoeva requested a review from mattkime October 30, 2024 12:02

Merge branch 'main' into im/index-mode-support

ab31a8f

mattkime approved these changes Oct 30, 2024

View reviewed changes

shahzad31 approved these changes Oct 31, 2024

View reviewed changes

sebelga approved these changes Oct 31, 2024

View reviewed changes

ElenaStoeva merged commit 40ddfbc into elastic:main Oct 31, 2024
28 checks passed

kibanamachine mentioned this pull request Oct 31, 2024

[8.x] [Index Management] Add support for index mode (#197874) #198481

Merged

This was referenced Dec 10, 2024

[Index Management] Fix schema for index templates #203552

Merged

[Index Management] Index mode info should only be displayed for index templates with enabled data streams #203589

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[Index Management] Add support for index mode #197874

[Index Management] Add support for index mode #197874

ElenaStoeva commented Oct 25, 2024 •

edited by kibanamachine

Loading

bitzandeb commented Oct 28, 2024

florent-leborgne left a comment

elasticmachine commented Oct 28, 2024

kibanamachine commented Oct 28, 2024

ElenaStoeva Oct 29, 2024 •

edited

Loading

ElenaStoeva Oct 29, 2024

mattkime Oct 30, 2024

ElenaStoeva Oct 30, 2024

mattkime left a comment

elasticmachine commented Oct 30, 2024

github-actions bot commented Oct 30, 2024

ElenaStoeva commented Oct 30, 2024

elasticmachine commented Oct 30, 2024 •

edited

Loading

API count

mattkime left a comment

kibanamachine commented Oct 31, 2024

kibanamachine commented Oct 31, 2024

[Index Management] Add support for index mode #197874

[Index Management] Add support for index mode #197874

Conversation

ElenaStoeva commented Oct 25, 2024 • edited by kibanamachine Loading

Summary

How to test:

Checklist

bitzandeb commented Oct 28, 2024

florent-leborgne left a comment

Choose a reason for hiding this comment

elasticmachine commented Oct 28, 2024

kibanamachine commented Oct 28, 2024

Flaky Test Runner Stats

🎉 All tests passed! - kibana-flaky-test-suite-runner#7288

ElenaStoeva Oct 29, 2024 • edited Loading

Choose a reason for hiding this comment

ElenaStoeva Oct 29, 2024

Choose a reason for hiding this comment

mattkime Oct 30, 2024

Choose a reason for hiding this comment

ElenaStoeva Oct 30, 2024

Choose a reason for hiding this comment

mattkime left a comment

Choose a reason for hiding this comment

elasticmachine commented Oct 30, 2024

github-actions bot commented Oct 30, 2024

🤖 GitHub comments

ElenaStoeva commented Oct 30, 2024

elasticmachine commented Oct 30, 2024 • edited Loading

💚 Build Succeeded

Metrics [docs]

Module Count

Public APIs missing comments

Async chunks

API count

History

mattkime left a comment

Choose a reason for hiding this comment

kibanamachine commented Oct 31, 2024

kibanamachine commented Oct 31, 2024

💚 All backports created successfully

Questions ?

ElenaStoeva commented Oct 25, 2024 •

edited by kibanamachine

Loading

ElenaStoeva Oct 29, 2024 •

edited

Loading

elasticmachine commented Oct 30, 2024 •

edited

Loading