Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Save Object Aggregation View] Fix for export all after scroll count response changed in PR#2656 #2696

Merged
merged 19 commits into from
Nov 2, 2022
Merged
Show file tree
Hide file tree
Changes from 15 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 2 additions & 1 deletion CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -33,8 +33,9 @@ Inspired from [Keep a Changelog](https://keepachangelog.com/en/1.0.0/)
- [Vis Builder] Rename wizard to visBuilder in class name, type name and function name ([#2639](https://github.com/opensearch-project/OpenSearch-Dashboards/pull/2639))
- [Vis Builder] Rename wizard on save modal and visualization table ([#2645](https://github.com/opensearch-project/OpenSearch-Dashboards/pull/2645))
- Change save object type, wizard id and name to visBuilder #2673 ([#2673](https://github.com/opensearch-project/OpenSearch-Dashboards/pull/2673))
- Add extension point in saved object management to register namespaces and show filter ([#2656](https://github.com/opensearch-project/OpenSearch-Dashboards/pull/2656))
- [Multi DataSource] Update MD data source documentation link ([#2693](https://github.com/opensearch-project/OpenSearch-Dashboards/pull/2693))
- [Save Object Aggregation View] Add extension point in saved object management to register namespaces and show filter ([#2656](https://github.com/opensearch-project/OpenSearch-Dashboards/pull/2656))
cwperks marked this conversation as resolved.
Show resolved Hide resolved
- [Save Object Aggregation View] Fix for export all after scroll count response changed in PR#2656 ([#2696](https://github.com/opensearch-project/OpenSearch-Dashboards/pull/2696))

### 🐛 Bug Fixes

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -30,14 +30,18 @@

import { HttpStart } from 'src/core/public';

interface SavedObjectCountOptions {
cwperks marked this conversation as resolved.
Show resolved Hide resolved
typesToInclude: string[];
namespacesToInclude?: string[];
searchString?: string;
}

export async function getSavedObjectCounts(
http: HttpStart,
typesToInclude: string[],
namespacesToInclude: string[],
searchString?: string
options: SavedObjectCountOptions
): Promise<Record<string, number>> {
return await http.post<Record<string, number>>(
`/api/opensearch-dashboards/management/saved_objects/scroll/counts`,
{ body: JSON.stringify({ typesToInclude, namespacesToInclude, searchString }) }
{ body: JSON.stringify(options) }
);
}
Original file line number Diff line number Diff line change
Expand Up @@ -137,10 +137,12 @@ describe('SavedObjectsTable', () => {
http.post.mockResolvedValue([]);

getSavedObjectCountsMock.mockReturnValue({
'index-pattern': 0,
visualization: 0,
dashboard: 0,
search: 0,
type: {
'index-pattern': 0,
visualization: 0,
dashboard: 0,
search: 0,
},
});

defaultProps = {
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -178,43 +178,62 @@ export class SavedObjectsTable extends Component<SavedObjectsTableProps, SavedOb
}

fetchCounts = async () => {
const { allowedTypes } = this.props;
const { allowedTypes, namespaceRegistry } = this.props;
const { queryText, visibleTypes, visibleNamespaces } = parseQuery(this.state.activeQuery);

const filteredTypes = allowedTypes.filter(
(type) => !visibleTypes || visibleTypes.includes(type)
);

let availableNamespaces = namespaceRegistry.getAll() || [];
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I briefly looked into the idea of having a convention for _all namespaces to reduce the size of the body of the request, but it looks like this would require changes in src/core/server/saved_objects/service/lib/repository.ts (https://github.com/opensearch-project/OpenSearch-Dashboards/blob/main/src/core/server/saved_objects/service/lib/repository.ts#L722) and I wasn't able to determine what would be required to implement this.

@cliu123 Can we include all registered namespaces in the request or should we look into providing an _all request?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No, we cannot. availableNamespaces provides by security backend plugin that evaluates the user's tenant permissions. Not all users have permissions to all tenants.

availableNamespaces = availableNamespaces.map((ns) => ns.id);

const filteredCountOptions = {
typesToInclude: filteredTypes,
searchString: queryText,
};

if (availableNamespaces?.length) {
cwperks marked this conversation as resolved.
Show resolved Hide resolved
cwperks marked this conversation as resolved.
Show resolved Hide resolved
const filteredNamespaces = availableNamespaces.filter(
(ns) => !visibleNamespaces || visibleNamespaces.includes(ns)
cwperks marked this conversation as resolved.
Show resolved Hide resolved
);
cwperks marked this conversation as resolved.
Show resolved Hide resolved
filteredCountOptions.namespacesToInclude = filteredNamespaces;
}

// These are the saved objects visible in the table.
const filteredSavedObjectCounts = await getSavedObjectCounts(
this.props.http,
filteredTypes,
visibleNamespaces,
queryText
filteredCountOptions
);

const exportAllOptions: ExportAllOption[] = [];
const exportAllSelectedOptions: Record<string, boolean> = {};

Object.keys(filteredSavedObjectCounts).forEach((id) => {
const filteredTypeCounts = filteredSavedObjectCounts.type || {};

Object.keys(filteredTypeCounts).forEach((id) => {
// Add this type as a bulk-export option.
exportAllOptions.push({
id,
label: `${id} (${filteredSavedObjectCounts[id] || 0})`,
label: `${id} (${filteredTypeCounts[id] || 0})`,
});

// Select it by default.
exportAllSelectedOptions[id] = true;
});

const countOptions = {
typesToInclude: allowedTypes,
searchString: queryText,
};

if (availableNamespaces?.length) {
cwperks marked this conversation as resolved.
Show resolved Hide resolved
countOptions.namespacesToInclude = availableNamespaces;
}

// Fetch all the saved objects that exist so we can accurately populate the counts within
// the table filter dropdown.
const savedObjectCounts = await getSavedObjectCounts(
this.props.http,
allowedTypes,
[],
queryText
);
const savedObjectCounts = await getSavedObjectCounts(this.props.http, countOptions);

this.setState((state) => ({
...state,
Expand All @@ -234,7 +253,7 @@ export class SavedObjectsTable extends Component<SavedObjectsTableProps, SavedOb

debouncedFetchObjects = debounce(async () => {
const { activeQuery: query, page, perPage } = this.state;
const { notifications, http, allowedTypes } = this.props;
const { notifications, http, allowedTypes, namespaceRegistry } = this.props;
const { queryText, visibleTypes, visibleNamespaces } = parseQuery(query);
const filteredTypes = allowedTypes.filter(
(type) => !visibleTypes || visibleTypes.includes(type)
Expand All @@ -247,8 +266,17 @@ export class SavedObjectsTable extends Component<SavedObjectsTableProps, SavedOb
page: page + 1,
fields: ['id'],
type: filteredTypes,
namespaces: visibleNamespaces,
};

let availableNamespaces = namespaceRegistry.getAll();
availableNamespaces = availableNamespaces.map((ns) => ns.id);
cwperks marked this conversation as resolved.
Show resolved Hide resolved
if (availableNamespaces?.length) {
cwperks marked this conversation as resolved.
Show resolved Hide resolved
const filteredNamespaces = availableNamespaces.filter(
(ns) => !visibleNamespaces || visibleNamespaces.includes(ns)
);
findOptions.namespaces = filteredNamespaces;
}
cwperks marked this conversation as resolved.
Show resolved Hide resolved

cwperks marked this conversation as resolved.
Show resolved Hide resolved
if (findOptions.type.length > 1) {
findOptions.sortField = 'type';
}
Expand Down Expand Up @@ -803,7 +831,7 @@ export class SavedObjectsTable extends Component<SavedObjectsTableProps, SavedOb
];

const availableNamespaces = namespaceRegistry.getAll() || [];
if (availableNamespaces && availableNamespaces.length > 0) {
if (availableNamespaces?.length) {
cwperks marked this conversation as resolved.
Show resolved Hide resolved
const nsCounts = savedObjectCounts.namespaces || {};
const nsFilterOptions = availableNamespaces.map((ns) => {
return {
Expand Down
14 changes: 0 additions & 14 deletions src/plugins/saved_objects_management/server/routes/find.ts
Original file line number Diff line number Diff line change
Expand Up @@ -69,20 +69,6 @@ export const registerFindRoute = (
const managementService = await managementServicePromise;
const { client } = context.core.savedObjects;
const searchTypes = Array.isArray(req.query.type) ? req.query.type : [req.query.type];
if ('namespaces' in req.query) {
const namespacesToInclude = Array.isArray(req.query.namespaces)
? req.query.namespaces
: [req.query.namespaces];
const searchNamespaces = [];
namespacesToInclude.forEach((ns) => {
if (ns == null) {
searchNamespaces.push('default');
} else {
searchNamespaces.push(ns);
}
});
req.query.namespaces = searchNamespaces;
}
const includedFields = Array.isArray(req.query.fields)
? req.query.fields
: [req.query.fields];
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -46,24 +46,21 @@ export const registerScrollForCountRoute = (router: IRouter) => {
},
router.handleLegacyErrors(async (context, req, res) => {
const { client } = context.core.savedObjects;
const namespaces = [];
if (req.body.namespacesToInclude && req.body.namespacesToInclude.length > 0) {
req.body.namespacesToInclude.forEach((ns) => {
if (ns === null) {
namespaces.push('default');
} else {
namespaces.push(ns);
}
});
}
const counts = {
type: {},
};

const findOptions: SavedObjectsFindOptions = {
type: req.body.typesToInclude,
perPage: 1000,
};

if (namespaces.length > 0) {
findOptions.namespaces = namespaces;
const requestHasNamespaces =
Array.isArray(req.body.namespacesToInclude) && req.body.namespacesToInclude.length;

if (requestHasNamespaces) {
counts.namespaces = {};
findOptions.namespaces = req.body.namespacesToInclude;
}

if (req.body.searchString) {
Expand All @@ -73,14 +70,9 @@ export const registerScrollForCountRoute = (router: IRouter) => {

const objects = await findAll(client, findOptions);

const counts = {
type: {},
namespaces: {},
};

objects.forEach((result) => {
const type = result.type;
if (req.body.namespacesToInclude) {
if (requestHasNamespaces) {
const resultNamespaces = (result.namespaces || []).flat();
resultNamespaces.forEach((ns) => {
if (ns === null) {
Expand Down