Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Discover] One Discover context awareness #183797

Merged
merged 54 commits into from
Jun 10, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
54 commits
Select commit Hold shift + click to select a range
a0785b9
Initial pass at context provider services
davismcphee May 13, 2024
386a18c
Initial context awareness implementation
davismcphee May 14, 2024
89e2a62
Add profile service
davismcphee May 15, 2024
3cd00c5
Add use_profile_accessor
davismcphee May 15, 2024
b9519d4
Add custom cell renderers and grid flyout
davismcphee May 15, 2024
ac6f2e5
Attach document profiles to data table records
davismcphee May 16, 2024
de7c276
Make root profile service async and add profiles provider
davismcphee May 16, 2024
00669f7
Make data source profile service async and set it via profiles provider
davismcphee May 16, 2024
b19bd7e
Move example profiles to separate file
davismcphee May 16, 2024
5e4318f
Improve example profiles
davismcphee May 17, 2024
70d43dd
Add default columns extension
davismcphee May 17, 2024
d3eee08
Don't display message field in cell renderer
davismcphee May 18, 2024
d2eee9c
Add ProfilesManager
davismcphee May 19, 2024
a96bc15
Use ProfilesManager for accessing and setting profiles
davismcphee May 19, 2024
b6984ce
Change tsx files to ts
davismcphee May 21, 2024
0e5f2e1
Handle ProfilesManager abort controllers internally
davismcphee May 21, 2024
ff90e2b
Rename ProfilesManager methods
davismcphee May 21, 2024
1de8849
Remove static profile services and instead instantiate them in plugin…
davismcphee May 22, 2024
807ee46
Don't export useProfiles
davismcphee May 22, 2024
3829c28
Retrieve ProfilesManager from services
davismcphee May 22, 2024
f753702
Remove show_confirm_panel since it's no longer used
davismcphee May 22, 2024
8d7edd6
Rename contextCollector to proflesCollector
davismcphee May 22, 2024
2939c99
Revert discover_data_state_container change
davismcphee May 22, 2024
d66c1d8
Fix broken types, Jest tests, and translations
davismcphee May 22, 2024
f887d88
Simplify useProfileAccessor
davismcphee May 27, 2024
f0d8bc1
Attach context with profile ID to data table records
davismcphee May 28, 2024
3597c90
Don't retrigger useProfiles on init
davismcphee May 28, 2024
a8060f4
Move data source resolution before data fetching
davismcphee May 28, 2024
ebfcfae
Log and fallback to default on context resolution failure
davismcphee May 30, 2024
fb050ca
Remove profile provider ordering
davismcphee May 30, 2024
ff97785
Add profiles support to saved search embeddable
davismcphee May 30, 2024
659c94a
Add Jest tests
davismcphee May 31, 2024
0217bae
Clean up ProfilesManager
davismcphee May 31, 2024
b80e57f
More Jest tests
davismcphee Jun 1, 2024
f16a506
await resolveRootProfile in saved search embeddable
davismcphee Jun 3, 2024
c992dfe
Fix broken Jest tests
davismcphee Jun 3, 2024
b4477ef
Remove unused import
davismcphee Jun 3, 2024
5752651
Remove extension points except for getCellRenderers
davismcphee Jun 3, 2024
3814f23
Comment out example cell renderers
davismcphee Jun 3, 2024
411d736
Add remaining Jest tests
davismcphee Jun 3, 2024
9ff0c83
Comment out example profile registrations
davismcphee Jun 3, 2024
ce85a6a
[CI] Auto-commit changed files from 'node scripts/eslint --no-cache -…
kibanamachine Jun 3, 2024
113f66f
Update resolveDocumentProfile to return a Proxy for attaching context
davismcphee Jun 5, 2024
6607524
Ensure _id and _index are defined in EsHitRecord
davismcphee Jun 6, 2024
0bfbe8a
Fix mappedDoc in createLogsAIAssistantRenderer and revert change to L…
davismcphee Jun 6, 2024
f5fd842
Use createContextAwarenessMocks in mock Discover services
davismcphee Jun 6, 2024
3f1ba48
Updated fetchEsql params
davismcphee Jun 6, 2024
2b8f6bf
Use mockReturnValue instead of overwriting getActiveSolutionNavId$
davismcphee Jun 6, 2024
aa78635
Merge branch 'main' into one-discover-context-awareness
davismcphee Jun 6, 2024
0c1d023
Revert "Ensure _id and _index are defined in EsHitRecord"
davismcphee Jun 6, 2024
8bab582
Omit getCellRenderers in DocumentProfile
davismcphee Jun 6, 2024
f2e6788
export hooks from index.ts
davismcphee Jun 7, 2024
643e98d
Pass empty ProfilesManager to DiscoverContainer
davismcphee Jun 7, 2024
965b97f
Merge branch 'main' into one-discover-context-awareness
davismcphee Jun 10, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
12 changes: 12 additions & 0 deletions packages/kbn-discover-utils/src/utils/build_data_record.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -30,5 +30,17 @@ describe('Data table record utils', () => {
expect(doc).toHaveProperty('isAnchor');
});
});

test('should support processing each record', () => {
const result = buildDataTableRecordList(esHitsMock, dataViewMock, {
processRecord: (record) => ({ ...record, id: 'custom-id' }),
});
result.forEach((doc) => {
expect(doc).toHaveProperty('id', 'custom-id');
expect(doc).toHaveProperty('raw');
expect(doc).toHaveProperty('flattened');
expect(doc).toHaveProperty('isAnchor');
});
});
});
});
10 changes: 7 additions & 3 deletions packages/kbn-discover-utils/src/utils/build_data_record.ts
Original file line number Diff line number Diff line change
Expand Up @@ -35,9 +35,13 @@ export function buildDataTableRecord(
* @param docs Array of documents returned from Elasticsearch
* @param dataView this current data view
*/
export function buildDataTableRecordList(
export function buildDataTableRecordList<T extends DataTableRecord = DataTableRecord>(
docs: EsHitRecord[],
dataView?: DataView
dataView?: DataView,
{ processRecord }: { processRecord?: (record: DataTableRecord) => T } = {}
Comment on lines +38 to +41
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: as dataView previously was the last optional argument, this function signature made sense. But if we add a parameter and invoke it without having a data view, we then need to force passing a second argument to pass then a third one, and it'll start looking something like

buildDataTableRecordList(docs, undefined, {processRecord})

Shall we refactor the function to get named parameters as we are adding the processRecord now?

buildDataTableRecordList({docs, dataView, processRecord})

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I fully agree with this, but buildDataTableRecordList is used outside of Discover and I'd like to avoid pinging other teams for review on this PR. I can make this change in a followup though to avoid blocking this one.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I opened a PR based on this one for the signature change here: #184976.

): DataTableRecord[] {
return docs.map((doc) => buildDataTableRecord(doc, dataView));
return docs.map((doc) => {
const record = buildDataTableRecord(doc, dataView);
return processRecord ? processRecord(record) : record;
});
}
3 changes: 3 additions & 0 deletions src/plugins/discover/public/__mocks__/services.ts
Original file line number Diff line number Diff line change
Expand Up @@ -41,6 +41,7 @@ import { SearchSourceDependencies } from '@kbn/data-plugin/common';
import { SearchResponse } from '@elastic/elasticsearch/lib/api/types';
import { urlTrackerMock } from './url_tracker.mock';
import { createElement } from 'react';
import { createContextAwarenessMocks } from '../context_awareness/__mocks__';

export function createDiscoverServicesMock(): DiscoverServices {
const dataPlugin = dataPluginMock.createStartContract();
Expand Down Expand Up @@ -137,6 +138,7 @@ export function createDiscoverServicesMock(): DiscoverServices {
...uiSettingsMock,
};

const { profilesManagerMock } = createContextAwarenessMocks();
const theme = themeServiceMock.createSetupContract({ darkMode: false });

corePluginMock.theme = theme;
Expand Down Expand Up @@ -236,6 +238,7 @@ export function createDiscoverServicesMock(): DiscoverServices {
contextLocator: { getRedirectUrl: jest.fn(() => '') },
singleDocLocator: { getRedirectUrl: jest.fn(() => '') },
urlTracker: urlTrackerMock,
profilesManager: profilesManagerMock,
setHeaderActionMenu: jest.fn(),
} as unknown as DiscoverServices;
}
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -31,6 +31,7 @@ const customisationService = createCustomizationService();

async function mountComponent(fetchStatus: FetchStatus, hits: EsHitRecord[]) {
const services = discoverServiceMock;

services.data.query.timefilter.timefilter.getTime = () => {
return { from: '2020-05-14T11:05:13.590', to: '2020-05-14T11:20:13.590' };
};
Expand Down Expand Up @@ -69,6 +70,10 @@ async function mountComponent(fetchStatus: FetchStatus, hits: EsHitRecord[]) {
}

describe('Discover documents layout', () => {
beforeEach(() => {
jest.clearAllMocks();
});

test('render loading when loading and no documents', async () => {
const component = await mountComponent(FetchStatus.LOADING, []);
expect(component.find('.dscDocuments__loading').exists()).toBeTruthy();
Expand Down Expand Up @@ -131,4 +136,22 @@ describe('Discover documents layout', () => {
expect(discoverGridComponent.prop('externalCustomRenderers')).toBeDefined();
expect(discoverGridComponent.prop('customGridColumnsConfiguration')).toBeDefined();
});

describe('context awareness', () => {
it('should pass cell renderers from profile', async () => {
customisationService.set({
id: 'data_table',
logsEnabled: true,
});
await discoverServiceMock.profilesManager.resolveRootProfile({ solutionNavId: 'test' });
const component = await mountComponent(FetchStatus.COMPLETE, esHitsMock);
const discoverGridComponent = component.find(DiscoverGrid);
expect(discoverGridComponent.exists()).toBeTruthy();
expect(Object.keys(discoverGridComponent.prop('externalCustomRenderers')!)).toEqual([
'content',
'resource',
'rootProfile',
]);
});
});
});
Original file line number Diff line number Diff line change
Expand Up @@ -68,6 +68,7 @@ import { onResizeGridColumn } from '../../../../utils/on_resize_grid_column';
import { useContextualGridCustomisations } from '../../hooks/grid_customisations';
import { useIsEsqlMode } from '../../hooks/use_is_esql_mode';
import { useAdditionalFieldGroups } from '../../hooks/sidebar/use_additional_field_groups';
import { useProfileAccessor } from '../../../../context_awareness';

const containerStyles = css`
position: relative;
Expand Down Expand Up @@ -263,6 +264,12 @@ function DiscoverDocumentsComponent({
useContextualGridCustomisations() || {};
const additionalFieldGroups = useAdditionalFieldGroups();

const getCellRenderersAccessor = useProfileAccessor('getCellRenderers');
const cellRenderers = useMemo(() => {
const getCellRenderers = getCellRenderersAccessor(() => customCellRenderer ?? {});
return getCellRenderers();
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I might be misunderstanding, but isn't it necessary to get the cell renderer for each row with the corresponding record instead of just once?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Unified Data Table cell renders are column based currently, so they don't work on individual records and can't be supported by DocumentProfile right now. I'm not sure if we need cell renderers on a per-record basis yet, but maybe there are some good use cases for it.

If we decide we need it, I'd recommend we add support for them and do the necessary Unified Data Table refactoring in a followup.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah, sorry, I didn't know this was an intentional omission. Could this indicate that narrowing the composable profile interface by context type might be good? Because I would also have made that mistake when trying to implement a custom cell renderer.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, that would make sense and it's actually already supported, I just forgot to omit getCellRenderers from the DocumentProfile type when I reverted the example extension points. Updated it here for now so it will be enforced in providers, and we can revisit this later if we find a need for document specific cell renderers: 8bab582.

}, [customCellRenderer, getCellRenderersAccessor]);

const documents = useObservable(stateContainer.dataState.data$.documents$);

const callouts = useMemo(
Expand Down Expand Up @@ -373,66 +380,64 @@ function DiscoverDocumentsComponent({
</>
)}
{!isLegacy && (
<>
<div className="unifiedDataTable">
<CellActionsProvider
getTriggerCompatibleActions={uiActions.getTriggerCompatibleActions}
>
<DiscoverGridMemoized
ariaLabelledBy="documentsAriaLabel"
columns={currentColumns}
columnsMeta={columnsMeta}
expandedDoc={expandedDoc}
dataView={dataView}
loadingState={
isDataLoading
? DataLoadingState.loading
: isMoreDataLoading
? DataLoadingState.loadingMore
: DataLoadingState.loaded
}
rows={rows}
sort={(sort as SortOrder[]) || []}
searchDescription={savedSearch.description}
searchTitle={savedSearch.title}
setExpandedDoc={setExpandedDoc}
showTimeCol={showTimeCol}
settings={grid}
onFilter={onAddFilter as DocViewFilterFn}
onSetColumns={onSetColumns}
onSort={onSort}
onResize={onResizeDataGrid}
useNewFieldsApi={useNewFieldsApi}
configHeaderRowHeight={3}
headerRowHeightState={headerRowHeight}
onUpdateHeaderRowHeight={onUpdateHeaderRowHeight}
rowHeightState={rowHeight}
onUpdateRowHeight={onUpdateRowHeight}
isSortEnabled={true}
isPlainRecord={isEsqlMode}
rowsPerPageState={rowsPerPage ?? getDefaultRowsPerPage(services.uiSettings)}
onUpdateRowsPerPage={onUpdateRowsPerPage}
maxAllowedSampleSize={getMaxAllowedSampleSize(services.uiSettings)}
sampleSizeState={getAllowedSampleSize(sampleSizeState, services.uiSettings)}
onUpdateSampleSize={!isEsqlMode ? onUpdateSampleSize : undefined}
onFieldEdited={onFieldEdited}
configRowHeight={uiSettings.get(ROW_HEIGHT_OPTION)}
showMultiFields={uiSettings.get(SHOW_MULTIFIELDS)}
maxDocFieldsDisplayed={uiSettings.get(MAX_DOC_FIELDS_DISPLAYED)}
renderDocumentView={renderDocumentView}
renderCustomToolbar={renderCustomToolbarWithElements}
services={services}
totalHits={totalHits}
onFetchMoreRecords={onFetchMoreRecords}
componentsTourSteps={TOUR_STEPS}
externalCustomRenderers={customCellRenderer}
customGridColumnsConfiguration={customGridColumnsConfiguration}
customControlColumnsConfiguration={customControlColumnsConfiguration}
additionalFieldGroups={additionalFieldGroups}
/>
</CellActionsProvider>
</div>
</>
<div className="unifiedDataTable">
<CellActionsProvider
getTriggerCompatibleActions={uiActions.getTriggerCompatibleActions}
>
<DiscoverGridMemoized
ariaLabelledBy="documentsAriaLabel"
columns={currentColumns}
columnsMeta={columnsMeta}
expandedDoc={expandedDoc}
dataView={dataView}
loadingState={
isDataLoading
? DataLoadingState.loading
: isMoreDataLoading
? DataLoadingState.loadingMore
: DataLoadingState.loaded
}
rows={rows}
sort={(sort as SortOrder[]) || []}
searchDescription={savedSearch.description}
searchTitle={savedSearch.title}
setExpandedDoc={setExpandedDoc}
showTimeCol={showTimeCol}
settings={grid}
onFilter={onAddFilter as DocViewFilterFn}
onSetColumns={onSetColumns}
onSort={onSort}
onResize={onResizeDataGrid}
useNewFieldsApi={useNewFieldsApi}
configHeaderRowHeight={3}
headerRowHeightState={headerRowHeight}
onUpdateHeaderRowHeight={onUpdateHeaderRowHeight}
rowHeightState={rowHeight}
onUpdateRowHeight={onUpdateRowHeight}
isSortEnabled={true}
isPlainRecord={isEsqlMode}
rowsPerPageState={rowsPerPage ?? getDefaultRowsPerPage(services.uiSettings)}
onUpdateRowsPerPage={onUpdateRowsPerPage}
maxAllowedSampleSize={getMaxAllowedSampleSize(services.uiSettings)}
sampleSizeState={getAllowedSampleSize(sampleSizeState, services.uiSettings)}
onUpdateSampleSize={!isEsqlMode ? onUpdateSampleSize : undefined}
onFieldEdited={onFieldEdited}
configRowHeight={uiSettings.get(ROW_HEIGHT_OPTION)}
showMultiFields={uiSettings.get(SHOW_MULTIFIELDS)}
maxDocFieldsDisplayed={uiSettings.get(MAX_DOC_FIELDS_DISPLAYED)}
renderDocumentView={renderDocumentView}
renderCustomToolbar={renderCustomToolbarWithElements}
services={services}
totalHits={totalHits}
onFetchMoreRecords={onFetchMoreRecords}
componentsTourSteps={TOUR_STEPS}
externalCustomRenderers={cellRenderers}
customGridColumnsConfiguration={customGridColumnsConfiguration}
customControlColumnsConfiguration={customControlColumnsConfiguration}
additionalFieldGroups={additionalFieldGroups}
/>
</CellActionsProvider>
</div>
)}
</EuiFlexItem>
</>
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -64,7 +64,7 @@ export function fetchAll(
savedSearch,
abortController,
} = fetchDeps;
const { data } = services;
const { data, expressions, profilesManager } = services;
const searchSource = savedSearch.searchSource.createChild();

try {
Expand Down Expand Up @@ -100,14 +100,15 @@ export function fetchAll(

// Start fetching all required requests
const response = isEsqlQuery
? fetchEsql(
? fetchEsql({
query,
dataView,
data,
services.expressions,
abortSignal: abortController.signal,
inspectorAdapters,
abortController.signal
)
data,
expressions,
profilesManager,
})
: fetchDocuments(searchSource, fetchDeps);
const fetchType = isEsqlQuery ? 'fetchTextBased' : 'fetchDocuments';
const startTime = window.performance.now();
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -30,6 +30,10 @@ const getDeps = () =>
} as unknown as FetchDeps);

describe('test fetchDocuments', () => {
beforeEach(() => {
jest.clearAllMocks();
});

test('resolves with returned documents', async () => {
const hits = [
{ _id: '1', foo: 'bar' },
Expand All @@ -38,10 +42,17 @@ describe('test fetchDocuments', () => {
const documents = hits.map((hit) => buildDataTableRecord(hit, dataViewMock));
savedSearchMock.searchSource.fetch$ = <T>() =>
of({ rawResponse: { hits: { hits } } } as IKibanaSearchResponse<SearchResponse<T>>);
const resolveDocumentProfileSpy = jest.spyOn(
discoverServiceMock.profilesManager,
'resolveDocumentProfile'
);
expect(await fetchDocuments(savedSearchMock.searchSource, getDeps())).toEqual({
interceptedWarnings: [],
records: documents,
});
expect(resolveDocumentProfileSpy).toHaveBeenCalledTimes(2);
expect(resolveDocumentProfileSpy).toHaveBeenCalledWith({ record: documents[0] });
expect(resolveDocumentProfileSpy).toHaveBeenCalledWith({ record: documents[1] });
});

test('rejects on query failure', async () => {
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -67,7 +67,9 @@ export const fetchDocuments = (
.pipe(
filter((res) => !isRunningResponse(res)),
map((res) => {
return buildDataTableRecordList(res.rawResponse.hits.hits as EsHitRecord[], dataView);
return buildDataTableRecordList(res.rawResponse.hits.hits as EsHitRecord[], dataView, {
processRecord: (record) => services.profilesManager.resolveDocumentProfile({ record }),
});
})
);

Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,66 @@
/*
* Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one
* or more contributor license agreements. Licensed under the Elastic License
* 2.0 and the Server Side Public License, v 1; you may not use this file except
* in compliance with, at your election, the Elastic License 2.0 or the Server
* Side Public License, v 1.
*/

import type { EsHitRecord } from '@kbn/discover-utils';
import type { ExecutionContract } from '@kbn/expressions-plugin/common';
import { RequestAdapter } from '@kbn/inspector-plugin/common';
import { of } from 'rxjs';
import { dataViewWithTimefieldMock } from '../../../__mocks__/data_view_with_timefield';
import { discoverServiceMock } from '../../../__mocks__/services';
import { fetchEsql } from './fetch_esql';

describe('fetchEsql', () => {
beforeEach(() => {
jest.clearAllMocks();
});

it('resolves with returned records', async () => {
const hits = [
{ _id: '1', foo: 'bar' },
{ _id: '2', foo: 'baz' },
] as unknown as EsHitRecord[];
const records = hits.map((hit, i) => ({
id: String(i),
raw: hit,
flattened: hit,
}));
const expressionsExecuteSpy = jest.spyOn(discoverServiceMock.expressions, 'execute');
expressionsExecuteSpy.mockReturnValueOnce({
cancel: jest.fn(),
getData: jest.fn(() =>
of({
result: {
columns: ['_id', 'foo'],
rows: hits,
},
})
),
} as unknown as ExecutionContract);
const resolveDocumentProfileSpy = jest.spyOn(
discoverServiceMock.profilesManager,
'resolveDocumentProfile'
);
expect(
await fetchEsql({
query: { esql: 'from *' },
dataView: dataViewWithTimefieldMock,
inspectorAdapters: { requests: new RequestAdapter() },
data: discoverServiceMock.data,
expressions: discoverServiceMock.expressions,
profilesManager: discoverServiceMock.profilesManager,
})
).toEqual({
records,
esqlQueryColumns: ['_id', 'foo'],
esqlHeaderWarning: undefined,
});
expect(resolveDocumentProfileSpy).toHaveBeenCalledTimes(2);
expect(resolveDocumentProfileSpy).toHaveBeenCalledWith({ record: records[0] });
expect(resolveDocumentProfileSpy).toHaveBeenCalledWith({ record: records[1] });
});
});
Loading
Loading