[discover-next][bug] add max height to dataset navigator and use memoization

[kavilla]

Description

For a lot of indices I give it a max height on the popover. Also using memoization helps a lot with the performance and rerendering.

Issues Resolved

n/a

Screenshot

Testing the changes

locally

Changelog

• fix: dataset explorer performance and handling overflow

Check List

• All tests pass
  • yarn test:jest
  • yarn test:jest_integration
• New functionality includes testing.
• New functionality has been documented.
• Update CHANGELOG.md
• Commits are signed per the DCO using --signoff

[codecov]

Codecov Report

Attention: Patch coverage is 4.68750% with 122 lines in your changes missing coverage. Please review.

Project coverage is 63.64%. Comparing base (2d8c743) to head (335978d).
Report is 245 commits behind head on main.

Files with missing lines	Patch %	Lines
.../public/ui/dataset_navigator/dataset_navigator.tsx	0.97%	102 Missing ⚠️
.../public/query/state_sync/connect_to_query_state.ts	16.66%	8 Missing and 2 partials ⚠️
...ta/public/query/dataset_manager/dataset_manager.ts	12.50%	6 Missing and 1 partial ⚠️
...ata/public/antlr/opensearch_sql/code_completion.ts	0.00%	1 Missing ⚠️
...query/state_sync/create_global_query_observable.ts	0.00%	1 Missing ⚠️
...ublic/query_assist/components/query_assist_bar.tsx	0.00%	1 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #7540      +/-   ##
==========================================
- Coverage   63.65%   63.64%   -0.01%     
==========================================
  Files        3629     3629              
  Lines       79519    79520       +1     
  Branches    12602    12601       -1     
==========================================
- Hits        50618    50613       -5     
- Misses      25834    25840       +6     
  Partials     3067     3067              

Flag	Coverage Δ
Linux_1	30.76% <0.80%> (-0.01%)	⬇️
Linux_2	55.58% <ø> (ø)
Linux_3	40.23% <3.20%> (-0.01%)	⬇️
Linux_4	31.57% <2.34%> (-0.01%)	⬇️
Windows_1	30.77% <0.80%> (-0.01%)	⬇️
Windows_2	55.53% <ø> (ø)
Windows_3	40.23% <3.20%> (-0.02%)	⬇️
Windows_4	31.57% <2.34%> (-0.01%)	⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[sejli]

Changes look good and much needed 🙌 . Test failures look unrelated though.

     │1)    doc views
     │       custom doc views with datagrid table
     │         should render react doc view:
     │
     │      Error: expected 'logstash-2015.09.20' to equal 'logstash-2015.09.22'
     │       at Assertion.assert (packages\osd-expect\expect.js:111:11)
     │       at Assertion.be.Assertion.equal (packages\osd-expect\expect.js:238:8)
     │       at Assertion.be (packages\osd-expect\expect.js:80:22)
     │       at Context.<anonymous> (test\plugin_functional\test_suites\doc_views\/doc_views.ts:58:54)
     │       at processTicksAndRejections (node:internal/process/task_queues:95:5)
     │       at Object.apply (packages\osd-test\src\functional_test_runner\lib\mocha\wrap_function.js:95:16)

[ashwin-pc]

Has a few fast follows that are needed, but LGTM

[ashwin-pc]

This can just be setDataset

[ashwin-pc]

We should just use the EUI Scrollbar mixin here

[ashwin-pc]

Do we want a fixed width?

[ashwin-pc]

We should get rid of this limitation

[opensearch-trigger-bot]

The backport to 2.x failed:

The process '/usr/bin/git' failed with exit code 128

To backport manually, run these commands in your terminal:

# Navigate to the root of your repository
cd $(git rev-parse --show-toplevel)
# Fetch latest updates from GitHub
git fetch
# Create a new working tree
git worktree add ../.worktrees/OpenSearch-Dashboards/backport-2.x 2.x
# Navigate to the new working tree
pushd ../.worktrees/OpenSearch-Dashboards/backport-2.x
# Create a new branch
git switch --create backport/backport-7540-to-2.x
# Cherry-pick the merged commit of this pull request and resolve the conflicts
git cherry-pick -x --mainline 1 611f56bc63b56546437b028ae43f76719118aed8
# Push it to GitHub
git push --set-upstream origin backport/backport-7540-to-2.x
# Go back to the original working tree
popd
# Delete the working tree
git worktree remove ../.worktrees/OpenSearch-Dashboards/backport-2.x

Then, create a pull request where the base branch is 2.x and the compare/head branch is backport/backport-7540-to-2.x.

[opensearch-trigger-bot]

The backport to 2.16 failed:

The process '/usr/bin/git' failed with exit code 1

To backport manually, run these commands in your terminal:

# Navigate to the root of your repository
cd $(git rev-parse --show-toplevel)
# Fetch latest updates from GitHub
git fetch
# Create a new working tree
git worktree add ../.worktrees/OpenSearch-Dashboards/backport-2.16 2.16
# Navigate to the new working tree
pushd ../.worktrees/OpenSearch-Dashboards/backport-2.16
# Create a new branch
git switch --create backport/backport-7540-to-2.16
# Cherry-pick the merged commit of this pull request and resolve the conflicts
git cherry-pick -x --mainline 1 611f56bc63b56546437b028ae43f76719118aed8
# Push it to GitHub
git push --set-upstream origin backport/backport-7540-to-2.16
# Go back to the original working tree
popd
# Delete the working tree
git worktree remove ../.worktrees/OpenSearch-Dashboards/backport-2.16

Then, create a pull request where the base branch is 2.16 and the compare/head branch is backport/backport-7540-to-2.16.

[AMoo-Miki]

Backported with #7574