Skip to content

Commit

Permalink
Merge branch 'main' into workspace_filter
Browse files Browse the repository at this point in the history
Signed-off-by: Hailong Cui <[email protected]>
  • Loading branch information
Hailong-am authored Apr 17, 2024
2 parents 18edfb8 + d911fa7 commit 905c5b3
Show file tree
Hide file tree
Showing 105 changed files with 3,162 additions and 323 deletions.
12 changes: 12 additions & 0 deletions .github/pull_request_template.md
Original file line number Diff line number Diff line change
Expand Up @@ -19,6 +19,18 @@
the functionality of your change
-->

## Changelog
<!--
Add a short but concise sentence about the impact of this pull request. Prefix an entry with the type of change they correspond to: breaking, chore, deprecate, doc, feat, fix, infra, refactor, test.
- fix: Update the graph
- feat: Add a new feature
If this change does not need to added to the changelog, just add a single `skip` line e.g.
- skip
Descriptions following the prefixes must be 100 characters long or less
-->

### Check List

- [ ] All tests pass
Expand Down
19 changes: 0 additions & 19 deletions .github/workflows/changelog_verifier.yml

This file was deleted.

23 changes: 23 additions & 0 deletions .github/workflows/opensearch_changelog_workflow.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,23 @@
name: OpenSearch Changelog Workflow

on:
pull_request_target:
types: [opened, reopened, edited]

permissions:
contents: read
issues: write
pull-requests: write

jobs:
update-changelog:
runs-on: ubuntu-latest
steps:
- name: Check out repository
uses: actions/checkout@v4
- name: Parse changelog entries and submit request for changset creation
uses: BigSamu/[email protected]
with:
token: ${{secrets.GITHUB_TOKEN}}
CHANGELOG_PR_BRIDGE_URL_DOMAIN: ${{secrets.CHANGELOG_PR_BRIDGE_URL_DOMAIN}}
CHANGELOG_PR_BRIDGE_API_KEY: ${{secrets.CHANGELOG_PR_BRIDGE_API_KEY}}
9 changes: 9 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -67,6 +67,7 @@ Inspired from [Keep a Changelog](https://keepachangelog.com/en/1.0.0/)
- [Multiple Datasource] Add icon in datasource table page to show the default datasource ([#6231](https://github.com/opensearch-project/OpenSearch-Dashboards/pull/6231))
- [Multiple Datasource] Add TLS configuration for multiple data sources ([#6171](https://github.com/opensearch-project/OpenSearch-Dashboards/pull/6171))
- [Multiple Datasource] Add multi selectable data source component ([#6211](https://github.com/opensearch-project/OpenSearch-Dashboards/pull/6211))
- [Multiple Datasource] Do not support import data source object to Local cluster when not enable data source ([#6395](https://github.com/opensearch-project/OpenSearch-Dashboards/pull/6395))
- [Multiple Datasource] Refactor data source menu and interface to allow cleaner selection of component and related configurations ([#6256](https://github.com/opensearch-project/OpenSearch-Dashboards/pull/6256))
- [Multiple Datasource] Allow top nav menu to mount data source menu for use case when both menus are mounted ([#6268](https://github.com/opensearch-project/OpenSearch-Dashboards/pull/6268))
- [Workspace] Add create workspace page ([#6179](https://github.com/opensearch-project/OpenSearch-Dashboards/pull/6179))
Expand All @@ -82,18 +83,24 @@ Inspired from [Keep a Changelog](https://keepachangelog.com/en/1.0.0/)
- [Workspace] Add permission control logic ([#6052](https://github.com/opensearch-project/OpenSearch-Dashboards/pull/6052))
- [Multiple Datasource] Add default icon for selectable component and make sure the default datasource shows automatically ([#6327](https://github.com/opensearch-project/OpenSearch-Dashboards/pull/6327))
- [Multiple Datasource] Pass selected data sources to plugin consumers when the multi-select component initially loads ([#6333](https://github.com/opensearch-project/OpenSearch-Dashboards/pull/6333))
- [Mulitple Datasource] Add multi data source support to TSVB ([#6298](https://github.com/opensearch-project/OpenSearch-Dashboards/pull/6298))
- [Multiple Datasource] Add installedPlugins list to data source saved object ([#6348](https://github.com/opensearch-project/OpenSearch-Dashboards/pull/6348))
- [Multiple Datasource] Add default icon in multi-selectable picker ([#6357](https://github.com/opensearch-project/OpenSearch-Dashboards/pull/6357))
- [Workspace] Add APIs to support plugin state in request ([#6303](https://github.com/opensearch-project/OpenSearch-Dashboards/pull/6303))
- [Workspace] Filter left nav menu items according to the current workspace ([#6234](https://github.com/opensearch-project/OpenSearch-Dashboards/pull/6234))
- [Multiple Datasource] Add multi data source support to Timeline ([#6385](https://github.com/opensearch-project/OpenSearch-Dashboards/pull/6385))
- [Multiple Datasource] Refactor data source selector component to include placeholder and add tests ([#6372](https://github.com/opensearch-project/OpenSearch-Dashboards/pull/6372))
- Replace control characters before logging ([#6402](https://github.com/opensearch-project/OpenSearch-Dashboards/pull/6402))
- [Dynamic Configurations] Improve dynamic configurations by adding cache and simplifying client fetch ([#6364](https://github.com/opensearch-project/OpenSearch-Dashboards/pull/6364))
- [CSP Handler] Update CSP handler to only query and modify frame ancestors instead of all CSP directives ([#6398](https://github.com/opensearch-project/OpenSearch-Dashboards/pull/6398))
- [MD] Add OpenSearch cluster group label to top of single selectable dropdown ([#6400](https://github.com/opensearch-project/OpenSearch-Dashboards/pull/6400))
- [Multiple Datasource] Add error state to all data source menu components to show error component and consolidate all fetch errors ([#6440](https://github.com/opensearch-project/OpenSearch-Dashboards/pull/6440))
- [Workspace] Support workspace in saved objects client in server side. ([#6365](https://github.com/opensearch-project/OpenSearch-Dashboards/pull/6365))
- [MD] Add dropdown header to data source single selector ([#6431](https://github.com/opensearch-project/OpenSearch-Dashboards/pull/6431))
- [Workspace] Add permission tab to workspace create update page ([#6378](https://github.com/opensearch-project/OpenSearch-Dashboards/pull/6378))
- [Workspace] Add workspaces filter to saved objects page. ([#6458](https://github.com/opensearch-project/OpenSearch-Dashboards/pull/6458))


### 🐛 Bug Fixes

- [Chore] Update deprecated url methods (url.parse(), url.format()) ([#2910](https://github.com/opensearch-project/OpenSearch-Dashboards/pull/2910))
Expand All @@ -119,6 +126,8 @@ Inspired from [Keep a Changelog](https://keepachangelog.com/en/1.0.0/)
- [BUG][Multiple Datasource]Read hideLocalCluster setting from yml and set in data source selector and data source menu ([#6361](https://github.com/opensearch-project/OpenSearch-Dashboards/pull/6361))
- [BUG][Multiple Datasource] Refactor read-only component to cover more edge cases ([#6416](https://github.com/opensearch-project/OpenSearch-Dashboards/pull/6416))
- [BUG] Fix for checkForFunctionProperty so that order does not matter ([#6248](https://github.com/opensearch-project/OpenSearch-Dashboards/pull/6248))
- [Dynamic Configurations] Fix dynamic config API calls to pass correct input ([#6474](https://github.com/opensearch-project/OpenSearch-Dashboards/pull/6474))
- [BUG][Multiple Datasource] Modify the button of selectable component to fix the title overflow issue ([#6465](https://github.com/opensearch-project/OpenSearch-Dashboards/pull/6465))
- [BUG][Multiple Datasource] Validation succeed as long as status code in response is 200 ([#6399](https://github.com/opensearch-project/OpenSearch-Dashboards/pull/6399))
- [BUG][Multiple Datasource] Add validation for title length to be no longer than 32 characters [#6452](https://github.com/opensearch-project/OpenSearch-Dashboards/pull/6452))

Expand Down
5 changes: 5 additions & 0 deletions changelogs/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
# Changelog and Release Notes

For information regarding the changelog and release notes process, please consult the README in the GitHub Actions repository that this process utilizes. To view this README, follow the link below:

[GitHub Actions Workflow README](https://github.com/BigSamu/OpenSearch_Change_Set_Create_Action/blob/main/README.md)
1 change: 1 addition & 0 deletions package.json
Original file line number Diff line number Diff line change
Expand Up @@ -79,6 +79,7 @@
"osd:bootstrap": "scripts/use_node scripts/build_ts_refs && scripts/use_node scripts/register_git_hook",
"spec_to_console": "scripts/use_node scripts/spec_to_console",
"pkg-version": "scripts/use_node -e \"console.log(require('./package.json').version)\"",
"release_note:generate": "scripts/use_node scripts/generate_release_note",
"cypress:run-without-security": "env TZ=America/Los_Angeles NO_COLOR=1 cypress run --headless --env SECURITY_ENABLED=false",
"cypress:run-with-security": "env TZ=America/Los_Angeles NO_COLOR=1 cypress run --headless --env SECURITY_ENABLED=true,openSearchUrl=https://localhost:9200,WAIT_FOR_LOADER_BUFFER_MS=500",
"osd:ciGroup10": "echo \"dashboard_sanity_test_spec.js\"",
Expand Down
8 changes: 8 additions & 0 deletions scripts/generate_release_note.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
/*
* Copyright OpenSearch Contributors
* SPDX-License-Identifier: Apache-2.0
*/

require('../src/setup_node_env');
require('../src/dev/generate_release_note');
require('../src/dev/generate_release_note_helper');
105 changes: 103 additions & 2 deletions src/core/server/saved_objects/import/import_saved_objects.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -96,11 +96,12 @@ describe('#importSavedObjectsFromStream', () => {
let savedObjectsClient: jest.Mocked<SavedObjectsClientContract>;
let typeRegistry: jest.Mocked<ISavedObjectTypeRegistry>;
const namespace = 'some-namespace';
const testDataSourceId = 'some-datasource';
const testDataSourceId = uuidv4();

const setupOptions = (
createNewCopies: boolean = false,
dataSourceId: string | undefined = undefined
dataSourceId: string | undefined = undefined,
dataSourceEnabled: boolean | undefined = false
): SavedObjectsImportOptions => {
readStream = new Readable();
savedObjectsClient = savedObjectsClientMock.create();
Expand Down Expand Up @@ -135,6 +136,17 @@ describe('#importSavedObjectsFromStream', () => {
attributes: { title: 'some-title' },
};
};

const createDataSourceObject = (): SavedObject<{
title: string;
}> => {
return {
type: 'data-source',
id: uuidv4(),
references: [],
attributes: { title: 'some-title' },
};
};
const createError = (): SavedObjectsImportError => {
const title = 'some-title';
return {
Expand Down Expand Up @@ -589,5 +601,94 @@ describe('#importSavedObjectsFromStream', () => {
const expectedErrors = errors.map(({ type, id }) => expect.objectContaining({ type, id }));
expect(result).toEqual({ success: false, successCount: 0, errors: expectedErrors });
});

test('early return if import data source objects to non-MDS cluster', async () => {
const options = setupOptions(false, testDataSourceId, false);
const dsObj = createDataSourceObject();
const dsExportedObj = createObject(testDataSourceId);
const collectedObjects = [dsObj, dsExportedObj];

const errors = [
{
type: dsObj.type,
id: dsObj.id,
title: dsObj.attributes.title,
meta: { title: dsObj.attributes.title },
error: { type: 'unsupported_type' },
},
{
type: dsExportedObj.type,
id: dsExportedObj.id,
title: dsExportedObj.attributes.title,
meta: { title: dsExportedObj.attributes.title },
error: { type: 'unsupported_type' },
},
];
getMockFn(collectSavedObjects).mockResolvedValue({
errors: [],
collectedObjects,
importIdMap: new Map(),
});
const result = await importSavedObjectsFromStream(options);
const expectedErrors = errors.map(({ type, id }) => expect.objectContaining({ type, id }));
expect(result).toEqual({ success: false, successCount: 0, errors: expectedErrors });
});

test('early return if import mixed non/data source objects to non-MDS cluster', async () => {
const options = setupOptions(false, testDataSourceId, false);
const dsObj = createDataSourceObject();
const dsExportedObj = createObject(testDataSourceId);
const nonDsExportedObj = createObject();
const collectedObjects = [dsObj, dsExportedObj, nonDsExportedObj];

const errors = [
{
type: dsObj.type,
id: dsObj.id,
title: dsObj.attributes.title,
meta: { title: dsObj.attributes.title },
error: { type: 'unsupported_type' },
},
{
type: dsExportedObj.type,
id: dsExportedObj.id,
title: dsExportedObj.attributes.title,
meta: { title: dsExportedObj.attributes.title },
error: { type: 'unsupported_type' },
},
];
getMockFn(collectSavedObjects).mockResolvedValue({
errors: [],
collectedObjects,
importIdMap: new Map(),
});
const result = await importSavedObjectsFromStream(options);
const expectedErrors = errors.map(({ type, id }) => expect.objectContaining({ type, id }));
expect(result).toEqual({ success: false, successCount: 0, errors: expectedErrors });
});

test('early return if import single data source objects to non-MDS cluster', async () => {
const options = setupOptions(false, testDataSourceId, false);
const dsObj = createDataSourceObject();
const collectedObjects = [dsObj];

const errors = [
{
type: dsObj.type,
id: dsObj.id,
title: dsObj.attributes.title,
meta: { title: dsObj.attributes.title },
error: { type: 'unsupported_type' },
},
];
getMockFn(collectSavedObjects).mockResolvedValue({
errors: [],
collectedObjects,
importIdMap: new Map(),
});
const result = await importSavedObjectsFromStream(options);
const expectedErrors = errors.map(({ type, id }) => expect.objectContaining({ type, id }));
expect(result).toEqual({ success: false, successCount: 0, errors: expectedErrors });
});
});
});
25 changes: 25 additions & 0 deletions src/core/server/saved_objects/import/import_saved_objects.ts
Original file line number Diff line number Diff line change
Expand Up @@ -33,13 +33,15 @@ import {
SavedObjectsImportError,
SavedObjectsImportResponse,
SavedObjectsImportOptions,
SavedObjectsImportUnsupportedTypeError,
} from './types';
import { validateReferences } from './validate_references';
import { checkOriginConflicts } from './check_origin_conflicts';
import { createSavedObjects } from './create_saved_objects';
import { checkConflicts } from './check_conflicts';
import { regenerateIds } from './regenerate_ids';
import { checkConflictsForDataSource } from './check_conflict_for_data_source';
import { isSavedObjectWithDataSource } from './validate_object_id';

/**
* Import saved objects from given stream. See the {@link SavedObjectsImportOptions | options} for more
Expand All @@ -58,6 +60,7 @@ export async function importSavedObjectsFromStream({
dataSourceId,
dataSourceTitle,
workspaces,
dataSourceEnabled,
}: SavedObjectsImportOptions): Promise<SavedObjectsImportResponse> {
let errorAccumulator: SavedObjectsImportError[] = [];
const supportedTypes = typeRegistry.getImportableAndExportableTypes().map((type) => type.name);
Expand All @@ -69,6 +72,28 @@ export async function importSavedObjectsFromStream({
supportedTypes,
dataSourceId,
});
// if not enable data_source, throw error early
if (!dataSourceEnabled) {
const notSupportedErrors: SavedObjectsImportError[] = collectSavedObjectsResult.collectedObjects.reduce(
(errors: SavedObjectsImportError[], obj) => {
if (obj.type === 'data-source' || isSavedObjectWithDataSource(obj.id)) {
const error: SavedObjectsImportUnsupportedTypeError = { type: 'unsupported_type' };
const { title } = obj.attributes;
errors.push({ error, type: obj.type, id: obj.id, title, meta: { title } });
}
return errors; // Return the accumulator in each iteration
},
[]
);
if (notSupportedErrors?.length > 0) {
return {
successCount: 0,
success: false,
errors: notSupportedErrors,
};
}
}

errorAccumulator = [...errorAccumulator, ...collectSavedObjectsResult.errors];
/** Map of all IDs for objects that we are attempting to import; each value is empty by default */
let importIdMap = collectSavedObjectsResult.importIdMap;
Expand Down
2 changes: 1 addition & 1 deletion src/core/server/saved_objects/import/types.ts
Original file line number Diff line number Diff line change
Expand Up @@ -189,7 +189,7 @@ export interface SavedObjectsImportOptions {
createNewCopies: boolean;
dataSourceId?: string;
dataSourceTitle?: string;
/** if specified, will import in given workspaces */
dataSourceEnabled?: boolean;
workspaces?: SavedObjectsBaseOptions['workspaces'];
}

Expand Down
59 changes: 59 additions & 0 deletions src/core/server/saved_objects/import/validate_object_id.test.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,59 @@
/*
* Copyright OpenSearch Contributors
* SPDX-License-Identifier: Apache-2.0
*/

import { isSavedObjectWithDataSource } from './validate_object_id';

describe('isObjectWithDataSource', () => {
test('should return false for valid object with data source ID but in wrong format', () => {
// Valid ID with two parts separated by underscore, and both parts being UUIDs
const inValidId = 'invalid_uuid_1234-invalid_uuid_5678';
expect(isSavedObjectWithDataSource(inValidId)).toBe(false);
});

test('should return false for invalid IDs', () => {
// Missing underscore
const invalidId1 = 'missingunderscore';
expect(isSavedObjectWithDataSource(invalidId1)).toBe(false);

// Invalid UUID in the second part
const invalidId2 = 'valid_uuid_1234-invalid_uuid';
expect(isSavedObjectWithDataSource(invalidId2)).toBe(false);

// Missing second part
const invalidId3 = 'valid_uuid_1234';
expect(isSavedObjectWithDataSource(invalidId3)).toBe(false);

// More than two parts
const invalidId4 = 'valid_uuid_1234-valid_uuid_5678-extra_part';
expect(isSavedObjectWithDataSource(invalidId4)).toBe(false);
});

test('should return false for non-UUID parts', () => {
// First part is not a UUID
const invalidId1 = 'not_a_uuid_valid_uuid_1234';
expect(isSavedObjectWithDataSource(invalidId1)).toBe(false);

// Second part is not a UUID
const invalidId2 = 'valid_uuid_1234_not_a_uuid';
expect(isSavedObjectWithDataSource(invalidId2)).toBe(false);

// Both parts are not UUIDs
const invalidId3 = 'not_a_uuid_not_a_uuid';
expect(isSavedObjectWithDataSource(invalidId3)).toBe(false);
});

test('should return false for string with underscore but not with UUID', () => {
// First part is not a UUID
const invalidId = 'saved_object_with_index_pattern_conflict';
expect(isSavedObjectWithDataSource(invalidId)).toBe(false);
});

test('should return false for string with underscore but with three UUIDs', () => {
// First part is not a UUID
const invalidId =
'7cbd2350-2223-11e8-b802-5bcf64c2cfb4_7cbd2350-2223-11e8-b802-5bcf64c2cfb4_7cbd2350-2223-11e8-b802-5bcf64c2cfb4';
expect(isSavedObjectWithDataSource(invalidId)).toBe(false);
});
});
Loading

0 comments on commit 905c5b3

Please sign in to comment.