Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[MD]Use placeholder for data source credentials fields when export saved object #6928

Merged
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions changelogs/fragments/6928.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
feat:
- [MD]Use placeholder for data source credentials fields when export saved object ([#6928](https://github.com/opensearch-project/OpenSearch-Dashboards/pull/6928))
Original file line number Diff line number Diff line change
Expand Up @@ -28,7 +28,10 @@
* under the License.
*/

import { exportSavedObjectsToStream } from './get_sorted_objects_for_export';
import {
DATA_SOURCE_CREDENTIALS_PLACEHOLDER,
exportSavedObjectsToStream,
} from './get_sorted_objects_for_export';
import { savedObjectsClientMock } from '../service/saved_objects_client.mock';
import { Readable } from 'stream';
import { createPromiseFromStreams, createConcatStream } from '../../utils/streams';
Expand Down Expand Up @@ -706,6 +709,50 @@ describe('getSortedObjectsForExport()', () => {
]);
});

test('modifies return results to update `credentials` of data-source to use placeholder', async () => {
const createDataSourceSavedObject = (id: string, auth: any) => ({
id,
type: 'data-source',
attributes: { auth },
references: [],
});

const dataSourceNoAuthInfo = { type: 'no_auth' };
const dataSourceBasicAuthInfo = {
type: 'username_password',
credentials: { username: 'foo', password: 'bar' },
};

const redactedDataSourceBasicAuthInfo = {
type: 'username_password',
credentials: {
username: DATA_SOURCE_CREDENTIALS_PLACEHOLDER,
password: DATA_SOURCE_CREDENTIALS_PLACEHOLDER,
},
};

savedObjectsClient.bulkGet.mockResolvedValueOnce({
saved_objects: [
createDataSourceSavedObject('1', dataSourceNoAuthInfo),
createDataSourceSavedObject('2', dataSourceBasicAuthInfo),
],
});
const exportStream = await exportSavedObjectsToStream({
exportSizeLimit: 10000,
savedObjectsClient,
objects: [
{ type: 'data-source', id: '1' },
{ type: 'data-source', id: '2' },
],
});
const response = await readStreamToCompletion(exportStream);
expect(response).toEqual([
createDataSourceSavedObject('1', dataSourceNoAuthInfo),
createDataSourceSavedObject('2', redactedDataSourceBasicAuthInfo),
expect.objectContaining({ exportedCount: 2 }),
]);
});

test('includes nested dependencies when passed in', async () => {
savedObjectsClient.bulkGet.mockResolvedValueOnce({
saved_objects: [
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -34,6 +34,8 @@ import { SavedObjectsClientContract, SavedObject, SavedObjectsBaseOptions } from
import { fetchNestedDependencies } from './inject_nested_depdendencies';
import { sortObjects } from './sort_objects';

export const DATA_SOURCE_CREDENTIALS_PLACEHOLDER = 'pleaseUpdateCredentials';

/**
* Options controlling the export operation.
* @public
Expand Down Expand Up @@ -185,10 +187,40 @@ export async function exportSavedObjectsToStream({
({ namespaces, ...object }) => object
);

// update the credential fields from "data-source" saved object to use placeholder to avoid exporting sensitive information
const redactedObjectsWithoutCredentials = redactedObjects.map<SavedObject<unknown>>((object) => {
if (object.type === 'data-source') {
const { auth, ...rest } = object.attributes as {
auth: { type: string; credentials?: any };
};
const hasCredentials = auth && auth.credentials;
const updatedCredentials = hasCredentials
? Object.keys(auth.credentials).reduce((acc, key) => {
acc[key] = DATA_SOURCE_CREDENTIALS_PLACEHOLDER;
Copy link
Member

@xinruiba xinruiba Jun 5, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for this change~
Just one question, SigV4 and TokenExchange also have regions under credential attribute, will this logic have any side effect?

Maybe we can also test SigV4 scenario while waiting CI to complete? Thanks!

Copy link
Member Author

@zhongnansu zhongnansu Jun 5, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

every field under "credentials" attributes will be replaced to use placeholder value, including region. I have tested sigV4 case, you can find it from the video. The exported sigV4 data source will look like this

{
   "attributes": {
      "auth": {
         "credentials": {
            "accessKey": "pleaseUpdateCredentials",
            "region": "pleaseUpdateCredentials",
            "secretKey": "pleaseUpdateCredentials",
            "service": "pleaseUpdateCredentials"
         },
         "type": "sigv4"
      },
      "dataSourceVersion": "",
      "description": "",
      "endpoint": "https://mjdty4727oifwax8ycdk.us-west-2.aoss.amazonaws.com",
      "installedPlugins": [],
      "title": "testSigV4"
   },
   "id": "7f8e13c0-2378-11ef-a345-bd55de69299f",
   "migrationVersion": {
      "data-source": "2.4.0"
   },
   "references": [],
   "type": "data-source",
   "updated_at": "2024-06-05T20:16:31.484Z",
   "version": "WzQsMV0="
}

return acc;
}, {} as { [key: string]: any })
: undefined;
return {
...object,
attributes: {
...rest,
auth: {
type: auth.type,
...(hasCredentials && { credentials: updatedCredentials }),
},
},
};
}
return object;
});

const exportDetails: SavedObjectsExportResultDetails = {
exportedCount: exportedObjects.length,
missingRefCount: missingReferences.length,
missingReferences,
};
return createListStream([...redactedObjects, ...(excludeExportDetails ? [] : [exportDetails])]);
return createListStream([
...redactedObjectsWithoutCredentials,
...(excludeExportDetails ? [] : [exportDetails]),
]);
}
Loading