[MD] Register data source plugin interface to route handler context #2121

zhongnansu · 2022-08-10T00:10:37Z

Signed-off-by: Zhongnan Su [email protected]

Description

Create data source plugin context class that contains the interface to retrieve a client by datasource id.
Register data source plugin context to route handler context.
The caller plugin, for example, search plugin will use the interface as below to retrieve a data source client. It will be further implementer in other tasks. Refactor data plugin -> search related logic #1988

const dataSourceClient = await context.dataSource.opensearch.getClient(id)

Init DataSourceClient class, with a dummy implementation of functions required by interfaces , it will be further implemented in other PRs
Init DataSourceService class, with interfaces defined

Issues Resolved

#1979
#1980
#1984
#1982

Check List

codecov-commenter · 2022-08-10T01:04:44Z

Codecov Report

Merging #2121 (025dcca) into feature/multi-datasource (456b5c7) will increase coverage by 0.01%.
The diff coverage is n/a.

@@                     Coverage Diff                      @@
##           feature/multi-datasource    #2121      +/-   ##
============================================================
+ Coverage                     67.47%   67.48%   +0.01%     
============================================================
  Files                          3080     3080              
  Lines                         59246    59246              
  Branches                       9010     9010              
============================================================
+ Hits                          39977    39983       +6     
+ Misses                        17080    17076       -4     
+ Partials                       2189     2187       -2

Impacted Files	Coverage Δ
...ic/application/models/sense_editor/sense_editor.ts	`64.88% <0.00%> (+0.88%)`	⬆️
packages/osd-optimizer/src/node/cache.ts	`52.77% <0.00%> (+2.77%)`	⬆️
...s/osd-optimizer/src/node/node_auto_tranpilation.ts	`87.75% <0.00%> (+4.08%)`	⬆️

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

src/plugins/data_source/server/client/data_source_client.ts

src/plugins/data_source/server/data_source_route_handler_context.ts

src/plugins/data_source/server/client/data_source_client.ts

src/plugins/data_source/server/data_source_route_handler_context.ts

src/plugins/data_source/server/plugin.ts

noCharger

LGTM, would like to see comment from @zengyan-amazon resolved before merging

src/plugins/credential_management/public/components/breadcrumbs.ts

src/plugins/data/common/index_patterns/index_patterns/index_pattern.ts

src/plugins/data_source/server/client/data_source_client.ts

src/plugins/data_source/server/data_source_route_handler_context.ts

src/plugins/data_source/server/data_source_service.ts

src/plugins/data_source/server/plugin.ts

src/plugins/data_source/server/data_source_service.ts

src/plugins/data_source/server/data_source_route_handler_context.ts

src/plugins/data_source/server/client/data_source_client.ts

zengyan-amazon

looks good

zengyan-amazon · 2022-08-17T19:03:54Z

src/plugins/data_source/server/plugin.ts

+              );
+            } catch (error: any) {
+              logger.error(
+                `Fail to get data source client for dataSourceId: [${dataSourceId}]. Detail: ${error.messages}`


this may cause 500 error, let's make a note here to handle that and returns 4xx as a follow up

Sure, I'll track that

Signed-off-by: Zhongnan Su <[email protected]>

seraphjiang

look forward for more coverage

zhongnansu · 2022-08-18T00:47:57Z

look forward for more coverage

definitely, the existing tests can pass now. I am merging in to unblock others. Tracking UT improvement here for this module.
#2163

src/core/server/opensearch/client/configure_client.test.mocks.ts

Signed-off-by: Zhongnan Su <[email protected]>

… handler context (opensearch-project#2121) * Add client management module and register `getClient()` interface to route handler context Signed-off-by: Zhongnan Su <[email protected]>

… handler context (opensearch-project#2121) * Add client management module and register `getClient()` interface to route handler context Signed-off-by: Zhongnan Su <[email protected]> Signed-off-by: Kristen Tian <[email protected]>

* Instantiate credential management plugin code structure (#1996) Signed-off-by: Kristen Tian <[email protected]> * Data source inside stack management setup (#2017) (#2030) Signed-off-by: Kristen Tian <[email protected]> * enable CI for feature branch (#2010) Signed-off-by: Zhongnan Su <[email protected]> Signed-off-by: Kristen Tian <[email protected]> * Add empty data source plugin (#2052) Adds empty data source plugin. Signed-off-by: Kristen Tian <[email protected]> * [MD] Add initial credential management CRUD pages (#2040) * Add credential management CRUD pages 1. List all credentials 2. Create your saved credential 3. Edit your credential 4. Delete credentials Signed-off-by: Louis Chu <[email protected]> Signed-off-by: Kristen Tian <[email protected]> * Register Data source to savedObject & Update license header (#2037) Signed-off-by: Kristen Tian <[email protected]> * Move credential saved object to data source plugin (#2062) Signed-off-by: Louis Chu <[email protected]> Move credential saved object to data source plugin Resolve follow up comments on UI Signed-off-by: Kristen Tian <[email protected]> * breadcrumbfix for datasource management (#2066) * breadcrumbfix for datasource management Signed-off-by: mpabba3003 <[email protected]> * breadcrumbfix for datasource management - refactoring code Signed-off-by: mpabba3003 <[email protected]> * using services to update breadcrumb on data sources management page Signed-off-by: mpabba3003 <[email protected]> * Changing the license header on breadcrumbs.ts datasource management Signed-off-by: Kristen Tian <[email protected]> * Fix breadcrumb on listing page and update saved object mapping (#2069) Signed-off-by: Louis Chu <[email protected]> Signed-off-by: Kristen Tian <[email protected]> * Add data source step into IndexPattern with Mock switch (#2064) (#2086) Signed-off-by: Kristen Tian <[email protected]> * Add delete button for credential detailed page (#2067) Signed-off-by: Yibo Wang <[email protected]> Signed-off-by: Kristen Tian <[email protected]> * Link datasource to indexpattern (#2118) Signed-off-by: Kristen Tian <[email protected]> Signed-off-by: Kristen Tian <[email protected]> * Add encrypt/decrypt module on data source plugin (#2120) Signed-off-by: Louis Chu <[email protected]> 1. Add encrypt/decrypt module with UT 2. Add client factory wrapper for encrypt credential 3. Add encryption config support 4. Bugfix on Jest interpret Buffer Signed-off-by: Kristen Tian <[email protected]> * Integrate index pattern with new data client (#2146) Signed-off-by: Kristen Tian <[email protected]> * Add noAuth to dataSource attributes (#2154) Signed-off-by: Kristen Tian <[email protected]> Signed-off-by: Kristen Tian <[email protected]> * [MD] Datasource Management - creation & listing - UI only (#2128) * data source management - creation & Listing UI only * data source management - creation & Listing UI only * Create/edit data source feature * toggling default value * refactoring code as per review comments * toggling server flag to false Signed-off-by: mpabba3003 <[email protected]> Signed-off-by: Kristen Tian <[email protected]> * Apply dataSource plugin as dependent for cm and dsm plugins (#2150) Signed-off-by: Louis Chu <[email protected]> Apply dataSource plugin as dependent for cm and dsm plugins (#2150) Signed-off-by: Kristen Tian <[email protected]> * Leverage datasource enablement in index pattern management Signed-off-by: Kristen Tian <[email protected]> * [MD] Add client management module and register `getClient()` to route handler context (#2121) * Add client management module and register `getClient()` interface to route handler context Signed-off-by: Zhongnan Su <[email protected]> Signed-off-by: Kristen Tian <[email protected]> * Make step info in index pattern creation dynamic (#2164) Signed-off-by: Kristen Tian <[email protected]> Signed-off-by: Kristen Tian <[email protected]> * Refactor for credential listing page & add loading effect (#2142) Signed-off-by: Yibo Wang <[email protected]> Signed-off-by: Yibo Wang <[email protected]> Signed-off-by: Kristen Tian <[email protected]> * Integration sequal - replace data client placeholders (#2167) Signed-off-by: Kristen Tian <[email protected]> Signed-off-by: Kristen Tian <[email protected]> * Integrate with crypto module to decrpt password (#2170) Signed-off-by: Zhongnan Su <[email protected]> Signed-off-by: Kristen Tian <[email protected]> * [MD] noAuth integration, credential & endpoint validation (#2165) * noAuth integration, credential & endpoint validation Signed-off-by: mpabba3003 <[email protected]> * Refactoring validation message Signed-off-by: mpabba3003 <[email protected]> * Adding back accidentally deleted file home/tutorials/haproxy_metrics/index.ts Signed-off-by: mpabba3003 <[email protected]> Signed-off-by: mpabba3003 <[email protected]> Signed-off-by: Kristen Tian <[email protected]> * [MD] Refactor search strategy to conditionally user datasource client (#2171) Signed-off-by: Kristen Tian <[email protected]> * adding relation between credential selection and no auth checkbox (#2175) Signed-off-by: mpabba3003 <[email protected]> Signed-off-by: mpabba3003 <[email protected]> Signed-off-by: Kristen Tian <[email protected]> * Update getIndicesViaSearch with datasource (#2176) Signed-off-by: Kristen Tian <[email protected]> Signed-off-by: Kristen Tian <[email protected]> * Add null header to child client spawn (#2188) Signed-off-by: Su <[email protected]> Signed-off-by: Kristen Tian <[email protected]> * [MD][IP]Update button position & Add UT & Add skip option (#2195) * Add UT - index pattern related Signed-off-by: Kristen Tian <[email protected]> * Update button position Signed-off-by: Kristen Tian <[email protected]> * Add skip option to allow use default os data source Signed-off-by: Kristen Tian <[email protected]> Signed-off-by: Kristen Tian <[email protected]> * [MD]Refactor layout and validate input fields for listing and create pages (#2202) Signed-off-by: Louis Chu <[email protected]> Signed-off-by: Louis Chu <[email protected]> Signed-off-by: Kristen Tian <[email protected]> * Enable datasource link in saveObjectManagement (#2209) Signed-off-by: Kristen Tian <[email protected]> Signed-off-by: Kristen Tian <[email protected]> * Update configure data source per UX input (#2235) Signed-off-by: Kristen Tian <[email protected]> Signed-off-by: Kristen Tian <[email protected]> * Enable data source audit log to file (#2215) Signed-off-by: Kristen Tian <[email protected]> Signed-off-by: Kristen Tian <[email protected]> * [MD] Refactor of credential editing page layout & refactor backend field validation method (#2222) * Refactor of credential editing page layout & refactor backend field validation method * Resolved the comments & fix the multiple call for one operation Signed-off-by: Yibo Wang <[email protected]> Signed-off-by: Yibo Wang <[email protected]> Signed-off-by: Kristen Tian <[email protected]> * Tweak fetch data back to original (#2238) Signed-off-by: Kristen Tian <[email protected]> Signed-off-by: Kristen Tian <[email protected]> * [MD] Revamped UX for data source management (#2239) * revamped UX for data source management Signed-off-by: mpabba3003 <[email protected]> * refactored datasource screens as per PR comments Signed-off-by: mpabba3003 <[email protected]> Signed-off-by: mpabba3003 <[email protected]> Signed-off-by: Kristen Tian <[email protected]> * [MD] Credentials security redesign (#2253) 1. Data model changes for data source saved object 2. Server side changes for data source saved object a. Implement data_source_saved_objects_client_wrapper to integrate with CryptographyClient for password encryption / decryption. b. Change data_source_service to fetch credentials directly from data source (still decrypt via CryptographyClient) c. Fix unit tests accordingly Signed-off-by: Louis Chu <[email protected]> Signed-off-by: Kristen Tian <[email protected]> * Add step data source UI test (#2264) Signed-off-by: Kristen Tian <[email protected]> Signed-off-by: Kristen Tian <[email protected]> * [MD]Improve datasource server side error handling (#2236) Signed-off-by: Su <[email protected]> Signed-off-by: Kristen Tian <[email protected]> * [MD] Datasource management new API change integrations (#2282) * changing datasource management design to integrate with new API changes Signed-off-by: mpabba3003 <[email protected]> * Update edit_data_source_form.tsx moving masked password to constants Signed-off-by: mpabba3003 <[email protected]> Signed-off-by: Kristen Tian <[email protected]> * Update stream test to bypass CI env generate domain attribute Signed-off-by: Kristen Tian <[email protected]> * Delete credential management Signed-off-by: Kristen Tian <[email protected]> * Address comments Signed-off-by: Kristen Tian <[email protected]> * [MD] Datasource Management - Create data source - Unit tests (#2341) * Unit test cases for data source management - create Signed-off-by: mpabba3003 <[email protected]> * adding tests to utils.ts & changing it to test Signed-off-by: mpabba3003 <[email protected]> Signed-off-by: mpabba3003 <[email protected]> Signed-off-by: Kristen Tian <[email protected]> * Fix update data source & block update endpint (#2364) Signed-off-by: Kristen Tian <[email protected]> Signed-off-by: Kristen Tian <[email protected]> * MD datasource management-datasource table-UTs (#2350) Signed-off-by: Yibo Wang <[email protected]> Signed-off-by: Yibo Wang <[email protected]> Signed-off-by: Kristen Tian <[email protected]> * reafctor based on PR 2334 comments to merge to main (#2375) Signed-off-by: mpabba3003 <[email protected]> Signed-off-by: mpabba3003 <[email protected]> Signed-off-by: Kristen Tian <[email protected]> * removing invalid urls as CI fails on link checker for inavlid urls in git (#2376) Signed-off-by: mpabba3003 <[email protected]> Signed-off-by: mpabba3003 <[email protected]> Signed-off-by: Kristen Tian <[email protected]> * Address comments Signed-off-by: Kristen Tian <[email protected]> * Fix DS snapshot test Signed-off-by: Kristen Tian <[email protected]> * Add https://test.com/ to lychee exclude Signed-off-by: Kristen Tian <[email protected]> * Address comments Signed-off-by: Kristen Tian <[email protected]> * Remove unnessacry check Signed-off-by: Kristen Tian <[email protected]> * Remove not needed check Signed-off-by: Kristen Tian <[email protected]> Signed-off-by: Kristen Tian <[email protected]> Signed-off-by: Zhongnan Su <[email protected]> Signed-off-by: Louis Chu <[email protected]> Signed-off-by: Yibo Wang <[email protected]> Signed-off-by: mpabba3003 <[email protected]> Signed-off-by: Su <[email protected]> Co-authored-by: Louis Chu <[email protected]> Co-authored-by: Zhongnan Su <[email protected]> Co-authored-by: Yan Zeng <[email protected]> Co-authored-by: Manideep Pabba <[email protected]> Co-authored-by: Yibo Wang <[email protected]>

* Instantiate credential management plugin code structure (opensearch-project#1996) Signed-off-by: Kristen Tian <[email protected]> * Data source inside stack management setup (opensearch-project#2017) (opensearch-project#2030) Signed-off-by: Kristen Tian <[email protected]> * enable CI for feature branch (opensearch-project#2010) Signed-off-by: Zhongnan Su <[email protected]> Signed-off-by: Kristen Tian <[email protected]> * Add empty data source plugin (opensearch-project#2052) Adds empty data source plugin. Signed-off-by: Kristen Tian <[email protected]> * [MD] Add initial credential management CRUD pages (opensearch-project#2040) * Add credential management CRUD pages 1. List all credentials 2. Create your saved credential 3. Edit your credential 4. Delete credentials Signed-off-by: Louis Chu <[email protected]> Signed-off-by: Kristen Tian <[email protected]> * Register Data source to savedObject & Update license header (opensearch-project#2037) Signed-off-by: Kristen Tian <[email protected]> * Move credential saved object to data source plugin (opensearch-project#2062) Signed-off-by: Louis Chu <[email protected]> Move credential saved object to data source plugin Resolve follow up comments on UI Signed-off-by: Kristen Tian <[email protected]> * breadcrumbfix for datasource management (opensearch-project#2066) * breadcrumbfix for datasource management Signed-off-by: mpabba3003 <[email protected]> * breadcrumbfix for datasource management - refactoring code Signed-off-by: mpabba3003 <[email protected]> * using services to update breadcrumb on data sources management page Signed-off-by: mpabba3003 <[email protected]> * Changing the license header on breadcrumbs.ts datasource management Signed-off-by: Kristen Tian <[email protected]> * Fix breadcrumb on listing page and update saved object mapping (opensearch-project#2069) Signed-off-by: Louis Chu <[email protected]> Signed-off-by: Kristen Tian <[email protected]> * Add data source step into IndexPattern with Mock switch (opensearch-project#2064) (opensearch-project#2086) Signed-off-by: Kristen Tian <[email protected]> * Add delete button for credential detailed page (opensearch-project#2067) Signed-off-by: Yibo Wang <[email protected]> Signed-off-by: Kristen Tian <[email protected]> * Link datasource to indexpattern (opensearch-project#2118) Signed-off-by: Kristen Tian <[email protected]> Signed-off-by: Kristen Tian <[email protected]> * Add encrypt/decrypt module on data source plugin (opensearch-project#2120) Signed-off-by: Louis Chu <[email protected]> 1. Add encrypt/decrypt module with UT 2. Add client factory wrapper for encrypt credential 3. Add encryption config support 4. Bugfix on Jest interpret Buffer Signed-off-by: Kristen Tian <[email protected]> * Integrate index pattern with new data client (opensearch-project#2146) Signed-off-by: Kristen Tian <[email protected]> * Add noAuth to dataSource attributes (opensearch-project#2154) Signed-off-by: Kristen Tian <[email protected]> Signed-off-by: Kristen Tian <[email protected]> * [MD] Datasource Management - creation & listing - UI only (opensearch-project#2128) * data source management - creation & Listing UI only * data source management - creation & Listing UI only * Create/edit data source feature * toggling default value * refactoring code as per review comments * toggling server flag to false Signed-off-by: mpabba3003 <[email protected]> Signed-off-by: Kristen Tian <[email protected]> * Apply dataSource plugin as dependent for cm and dsm plugins (opensearch-project#2150) Signed-off-by: Louis Chu <[email protected]> Apply dataSource plugin as dependent for cm and dsm plugins (opensearch-project#2150) Signed-off-by: Kristen Tian <[email protected]> * Leverage datasource enablement in index pattern management Signed-off-by: Kristen Tian <[email protected]> * [MD] Add client management module and register `getClient()` to route handler context (opensearch-project#2121) * Add client management module and register `getClient()` interface to route handler context Signed-off-by: Zhongnan Su <[email protected]> Signed-off-by: Kristen Tian <[email protected]> * Make step info in index pattern creation dynamic (opensearch-project#2164) Signed-off-by: Kristen Tian <[email protected]> Signed-off-by: Kristen Tian <[email protected]> * Refactor for credential listing page & add loading effect (opensearch-project#2142) Signed-off-by: Yibo Wang <[email protected]> Signed-off-by: Yibo Wang <[email protected]> Signed-off-by: Kristen Tian <[email protected]> * Integration sequal - replace data client placeholders (opensearch-project#2167) Signed-off-by: Kristen Tian <[email protected]> Signed-off-by: Kristen Tian <[email protected]> * Integrate with crypto module to decrpt password (opensearch-project#2170) Signed-off-by: Zhongnan Su <[email protected]> Signed-off-by: Kristen Tian <[email protected]> * [MD] noAuth integration, credential & endpoint validation (opensearch-project#2165) * noAuth integration, credential & endpoint validation Signed-off-by: mpabba3003 <[email protected]> * Refactoring validation message Signed-off-by: mpabba3003 <[email protected]> * Adding back accidentally deleted file home/tutorials/haproxy_metrics/index.ts Signed-off-by: mpabba3003 <[email protected]> Signed-off-by: mpabba3003 <[email protected]> Signed-off-by: Kristen Tian <[email protected]> * [MD] Refactor search strategy to conditionally user datasource client (opensearch-project#2171) Signed-off-by: Kristen Tian <[email protected]> * adding relation between credential selection and no auth checkbox (opensearch-project#2175) Signed-off-by: mpabba3003 <[email protected]> Signed-off-by: mpabba3003 <[email protected]> Signed-off-by: Kristen Tian <[email protected]> * Update getIndicesViaSearch with datasource (opensearch-project#2176) Signed-off-by: Kristen Tian <[email protected]> Signed-off-by: Kristen Tian <[email protected]> * Add null header to child client spawn (opensearch-project#2188) Signed-off-by: Su <[email protected]> Signed-off-by: Kristen Tian <[email protected]> * [MD][IP]Update button position & Add UT & Add skip option (opensearch-project#2195) * Add UT - index pattern related Signed-off-by: Kristen Tian <[email protected]> * Update button position Signed-off-by: Kristen Tian <[email protected]> * Add skip option to allow use default os data source Signed-off-by: Kristen Tian <[email protected]> Signed-off-by: Kristen Tian <[email protected]> * [MD]Refactor layout and validate input fields for listing and create pages (opensearch-project#2202) Signed-off-by: Louis Chu <[email protected]> Signed-off-by: Louis Chu <[email protected]> Signed-off-by: Kristen Tian <[email protected]> * Enable datasource link in saveObjectManagement (opensearch-project#2209) Signed-off-by: Kristen Tian <[email protected]> Signed-off-by: Kristen Tian <[email protected]> * Update configure data source per UX input (opensearch-project#2235) Signed-off-by: Kristen Tian <[email protected]> Signed-off-by: Kristen Tian <[email protected]> * Enable data source audit log to file (opensearch-project#2215) Signed-off-by: Kristen Tian <[email protected]> Signed-off-by: Kristen Tian <[email protected]> * [MD] Refactor of credential editing page layout & refactor backend field validation method (opensearch-project#2222) * Refactor of credential editing page layout & refactor backend field validation method * Resolved the comments & fix the multiple call for one operation Signed-off-by: Yibo Wang <[email protected]> Signed-off-by: Yibo Wang <[email protected]> Signed-off-by: Kristen Tian <[email protected]> * Tweak fetch data back to original (opensearch-project#2238) Signed-off-by: Kristen Tian <[email protected]> Signed-off-by: Kristen Tian <[email protected]> * [MD] Revamped UX for data source management (opensearch-project#2239) * revamped UX for data source management Signed-off-by: mpabba3003 <[email protected]> * refactored datasource screens as per PR comments Signed-off-by: mpabba3003 <[email protected]> Signed-off-by: mpabba3003 <[email protected]> Signed-off-by: Kristen Tian <[email protected]> * [MD] Credentials security redesign (opensearch-project#2253) 1. Data model changes for data source saved object 2. Server side changes for data source saved object a. Implement data_source_saved_objects_client_wrapper to integrate with CryptographyClient for password encryption / decryption. b. Change data_source_service to fetch credentials directly from data source (still decrypt via CryptographyClient) c. Fix unit tests accordingly Signed-off-by: Louis Chu <[email protected]> Signed-off-by: Kristen Tian <[email protected]> * Add step data source UI test (opensearch-project#2264) Signed-off-by: Kristen Tian <[email protected]> Signed-off-by: Kristen Tian <[email protected]> * [MD]Improve datasource server side error handling (opensearch-project#2236) Signed-off-by: Su <[email protected]> Signed-off-by: Kristen Tian <[email protected]> * [MD] Datasource management new API change integrations (opensearch-project#2282) * changing datasource management design to integrate with new API changes Signed-off-by: mpabba3003 <[email protected]> * Update edit_data_source_form.tsx moving masked password to constants Signed-off-by: mpabba3003 <[email protected]> Signed-off-by: Kristen Tian <[email protected]> * Update stream test to bypass CI env generate domain attribute Signed-off-by: Kristen Tian <[email protected]> * Delete credential management Signed-off-by: Kristen Tian <[email protected]> * Address comments Signed-off-by: Kristen Tian <[email protected]> * [MD] Datasource Management - Create data source - Unit tests (opensearch-project#2341) * Unit test cases for data source management - create Signed-off-by: mpabba3003 <[email protected]> * adding tests to utils.ts & changing it to test Signed-off-by: mpabba3003 <[email protected]> Signed-off-by: mpabba3003 <[email protected]> Signed-off-by: Kristen Tian <[email protected]> * Fix update data source & block update endpint (opensearch-project#2364) Signed-off-by: Kristen Tian <[email protected]> Signed-off-by: Kristen Tian <[email protected]> * MD datasource management-datasource table-UTs (opensearch-project#2350) Signed-off-by: Yibo Wang <[email protected]> Signed-off-by: Yibo Wang <[email protected]> Signed-off-by: Kristen Tian <[email protected]> * reafctor based on PR 2334 comments to merge to main (opensearch-project#2375) Signed-off-by: mpabba3003 <[email protected]> Signed-off-by: mpabba3003 <[email protected]> Signed-off-by: Kristen Tian <[email protected]> * removing invalid urls as CI fails on link checker for inavlid urls in git (opensearch-project#2376) Signed-off-by: mpabba3003 <[email protected]> Signed-off-by: mpabba3003 <[email protected]> Signed-off-by: Kristen Tian <[email protected]> * Address comments Signed-off-by: Kristen Tian <[email protected]> * Fix DS snapshot test Signed-off-by: Kristen Tian <[email protected]> * Add https://test.com/ to lychee exclude Signed-off-by: Kristen Tian <[email protected]> * Address comments Signed-off-by: Kristen Tian <[email protected]> * Remove unnessacry check Signed-off-by: Kristen Tian <[email protected]> * Remove not needed check Signed-off-by: Kristen Tian <[email protected]> Signed-off-by: Kristen Tian <[email protected]> Signed-off-by: Zhongnan Su <[email protected]> Signed-off-by: Louis Chu <[email protected]> Signed-off-by: Yibo Wang <[email protected]> Signed-off-by: mpabba3003 <[email protected]> Signed-off-by: Su <[email protected]> Co-authored-by: Louis Chu <[email protected]> Co-authored-by: Zhongnan Su <[email protected]> Co-authored-by: Yan Zeng <[email protected]> Co-authored-by: Manideep Pabba <[email protected]> Co-authored-by: Yibo Wang <[email protected]>

* Instantiate credential management plugin code structure (opensearch-project#1996) Signed-off-by: Kristen Tian <[email protected]> * Data source inside stack management setup (opensearch-project#2017) (opensearch-project#2030) Signed-off-by: Kristen Tian <[email protected]> * enable CI for feature branch (opensearch-project#2010) Signed-off-by: Zhongnan Su <[email protected]> Signed-off-by: Kristen Tian <[email protected]> * Add empty data source plugin (opensearch-project#2052) Adds empty data source plugin. Signed-off-by: Kristen Tian <[email protected]> * [MD] Add initial credential management CRUD pages (opensearch-project#2040) * Add credential management CRUD pages 1. List all credentials 2. Create your saved credential 3. Edit your credential 4. Delete credentials Signed-off-by: Louis Chu <[email protected]> Signed-off-by: Kristen Tian <[email protected]> * Register Data source to savedObject & Update license header (opensearch-project#2037) Signed-off-by: Kristen Tian <[email protected]> * Move credential saved object to data source plugin (opensearch-project#2062) Signed-off-by: Louis Chu <[email protected]> Move credential saved object to data source plugin Resolve follow up comments on UI Signed-off-by: Kristen Tian <[email protected]> * breadcrumbfix for datasource management (opensearch-project#2066) * breadcrumbfix for datasource management Signed-off-by: mpabba3003 <[email protected]> * breadcrumbfix for datasource management - refactoring code Signed-off-by: mpabba3003 <[email protected]> * using services to update breadcrumb on data sources management page Signed-off-by: mpabba3003 <[email protected]> * Changing the license header on breadcrumbs.ts datasource management Signed-off-by: Kristen Tian <[email protected]> * Fix breadcrumb on listing page and update saved object mapping (opensearch-project#2069) Signed-off-by: Louis Chu <[email protected]> Signed-off-by: Kristen Tian <[email protected]> * Add data source step into IndexPattern with Mock switch (opensearch-project#2064) (opensearch-project#2086) Signed-off-by: Kristen Tian <[email protected]> * Add delete button for credential detailed page (opensearch-project#2067) Signed-off-by: Yibo Wang <[email protected]> Signed-off-by: Kristen Tian <[email protected]> * Link datasource to indexpattern (opensearch-project#2118) Signed-off-by: Kristen Tian <[email protected]> Signed-off-by: Kristen Tian <[email protected]> * Add encrypt/decrypt module on data source plugin (opensearch-project#2120) Signed-off-by: Louis Chu <[email protected]> 1. Add encrypt/decrypt module with UT 2. Add client factory wrapper for encrypt credential 3. Add encryption config support 4. Bugfix on Jest interpret Buffer Signed-off-by: Kristen Tian <[email protected]> * Integrate index pattern with new data client (opensearch-project#2146) Signed-off-by: Kristen Tian <[email protected]> * Add noAuth to dataSource attributes (opensearch-project#2154) Signed-off-by: Kristen Tian <[email protected]> Signed-off-by: Kristen Tian <[email protected]> * [MD] Datasource Management - creation & listing - UI only (opensearch-project#2128) * data source management - creation & Listing UI only * data source management - creation & Listing UI only * Create/edit data source feature * toggling default value * refactoring code as per review comments * toggling server flag to false Signed-off-by: mpabba3003 <[email protected]> Signed-off-by: Kristen Tian <[email protected]> * Apply dataSource plugin as dependent for cm and dsm plugins (opensearch-project#2150) Signed-off-by: Louis Chu <[email protected]> Apply dataSource plugin as dependent for cm and dsm plugins (opensearch-project#2150) Signed-off-by: Kristen Tian <[email protected]> * Leverage datasource enablement in index pattern management Signed-off-by: Kristen Tian <[email protected]> * [MD] Add client management module and register `getClient()` to route handler context (opensearch-project#2121) * Add client management module and register `getClient()` interface to route handler context Signed-off-by: Zhongnan Su <[email protected]> Signed-off-by: Kristen Tian <[email protected]> * Make step info in index pattern creation dynamic (opensearch-project#2164) Signed-off-by: Kristen Tian <[email protected]> Signed-off-by: Kristen Tian <[email protected]> * Refactor for credential listing page & add loading effect (opensearch-project#2142) Signed-off-by: Yibo Wang <[email protected]> Signed-off-by: Yibo Wang <[email protected]> Signed-off-by: Kristen Tian <[email protected]> * Integration sequal - replace data client placeholders (opensearch-project#2167) Signed-off-by: Kristen Tian <[email protected]> Signed-off-by: Kristen Tian <[email protected]> * Integrate with crypto module to decrpt password (opensearch-project#2170) Signed-off-by: Zhongnan Su <[email protected]> Signed-off-by: Kristen Tian <[email protected]> * [MD] noAuth integration, credential & endpoint validation (opensearch-project#2165) * noAuth integration, credential & endpoint validation Signed-off-by: mpabba3003 <[email protected]> * Refactoring validation message Signed-off-by: mpabba3003 <[email protected]> * Adding back accidentally deleted file home/tutorials/haproxy_metrics/index.ts Signed-off-by: mpabba3003 <[email protected]> Signed-off-by: mpabba3003 <[email protected]> Signed-off-by: Kristen Tian <[email protected]> * [MD] Refactor search strategy to conditionally user datasource client (opensearch-project#2171) Signed-off-by: Kristen Tian <[email protected]> * adding relation between credential selection and no auth checkbox (opensearch-project#2175) Signed-off-by: mpabba3003 <[email protected]> Signed-off-by: mpabba3003 <[email protected]> Signed-off-by: Kristen Tian <[email protected]> * Update getIndicesViaSearch with datasource (opensearch-project#2176) Signed-off-by: Kristen Tian <[email protected]> Signed-off-by: Kristen Tian <[email protected]> * Add null header to child client spawn (opensearch-project#2188) Signed-off-by: Su <[email protected]> Signed-off-by: Kristen Tian <[email protected]> * [MD][IP]Update button position & Add UT & Add skip option (opensearch-project#2195) * Add UT - index pattern related Signed-off-by: Kristen Tian <[email protected]> * Update button position Signed-off-by: Kristen Tian <[email protected]> * Add skip option to allow use default os data source Signed-off-by: Kristen Tian <[email protected]> Signed-off-by: Kristen Tian <[email protected]> * [MD]Refactor layout and validate input fields for listing and create pages (opensearch-project#2202) Signed-off-by: Louis Chu <[email protected]> Signed-off-by: Louis Chu <[email protected]> Signed-off-by: Kristen Tian <[email protected]> * Enable datasource link in saveObjectManagement (opensearch-project#2209) Signed-off-by: Kristen Tian <[email protected]> Signed-off-by: Kristen Tian <[email protected]> * Update configure data source per UX input (opensearch-project#2235) Signed-off-by: Kristen Tian <[email protected]> Signed-off-by: Kristen Tian <[email protected]> * Enable data source audit log to file (opensearch-project#2215) Signed-off-by: Kristen Tian <[email protected]> Signed-off-by: Kristen Tian <[email protected]> * [MD] Refactor of credential editing page layout & refactor backend field validation method (opensearch-project#2222) * Refactor of credential editing page layout & refactor backend field validation method * Resolved the comments & fix the multiple call for one operation Signed-off-by: Yibo Wang <[email protected]> Signed-off-by: Yibo Wang <[email protected]> Signed-off-by: Kristen Tian <[email protected]> * Tweak fetch data back to original (opensearch-project#2238) Signed-off-by: Kristen Tian <[email protected]> Signed-off-by: Kristen Tian <[email protected]> * [MD] Revamped UX for data source management (opensearch-project#2239) * revamped UX for data source management Signed-off-by: mpabba3003 <[email protected]> * refactored datasource screens as per PR comments Signed-off-by: mpabba3003 <[email protected]> Signed-off-by: mpabba3003 <[email protected]> Signed-off-by: Kristen Tian <[email protected]> * [MD] Credentials security redesign (opensearch-project#2253) 1. Data model changes for data source saved object 2. Server side changes for data source saved object a. Implement data_source_saved_objects_client_wrapper to integrate with CryptographyClient for password encryption / decryption. b. Change data_source_service to fetch credentials directly from data source (still decrypt via CryptographyClient) c. Fix unit tests accordingly Signed-off-by: Louis Chu <[email protected]> Signed-off-by: Kristen Tian <[email protected]> * Add step data source UI test (opensearch-project#2264) Signed-off-by: Kristen Tian <[email protected]> Signed-off-by: Kristen Tian <[email protected]> * [MD]Improve datasource server side error handling (opensearch-project#2236) Signed-off-by: Su <[email protected]> Signed-off-by: Kristen Tian <[email protected]> * [MD] Datasource management new API change integrations (opensearch-project#2282) * changing datasource management design to integrate with new API changes Signed-off-by: mpabba3003 <[email protected]> * Update edit_data_source_form.tsx moving masked password to constants Signed-off-by: mpabba3003 <[email protected]> Signed-off-by: Kristen Tian <[email protected]> * Update stream test to bypass CI env generate domain attribute Signed-off-by: Kristen Tian <[email protected]> * Delete credential management Signed-off-by: Kristen Tian <[email protected]> * Address comments Signed-off-by: Kristen Tian <[email protected]> * [MD] Datasource Management - Create data source - Unit tests (opensearch-project#2341) * Unit test cases for data source management - create Signed-off-by: mpabba3003 <[email protected]> * adding tests to utils.ts & changing it to test Signed-off-by: mpabba3003 <[email protected]> Signed-off-by: mpabba3003 <[email protected]> Signed-off-by: Kristen Tian <[email protected]> * Fix update data source & block update endpint (opensearch-project#2364) Signed-off-by: Kristen Tian <[email protected]> Signed-off-by: Kristen Tian <[email protected]> * MD datasource management-datasource table-UTs (opensearch-project#2350) Signed-off-by: Yibo Wang <[email protected]> Signed-off-by: Yibo Wang <[email protected]> Signed-off-by: Kristen Tian <[email protected]> * reafctor based on PR 2334 comments to merge to main (opensearch-project#2375) Signed-off-by: mpabba3003 <[email protected]> Signed-off-by: mpabba3003 <[email protected]> Signed-off-by: Kristen Tian <[email protected]> * removing invalid urls as CI fails on link checker for inavlid urls in git (opensearch-project#2376) Signed-off-by: mpabba3003 <[email protected]> Signed-off-by: mpabba3003 <[email protected]> Signed-off-by: Kristen Tian <[email protected]> * Address comments Signed-off-by: Kristen Tian <[email protected]> * Fix DS snapshot test Signed-off-by: Kristen Tian <[email protected]> * Add https://test.com/ to lychee exclude Signed-off-by: Kristen Tian <[email protected]> * Address comments Signed-off-by: Kristen Tian <[email protected]> * Remove unnessacry check Signed-off-by: Kristen Tian <[email protected]> * Remove not needed check Signed-off-by: Kristen Tian <[email protected]> Signed-off-by: Kristen Tian <[email protected]> Signed-off-by: Zhongnan Su <[email protected]> Signed-off-by: Louis Chu <[email protected]> Signed-off-by: Yibo Wang <[email protected]> Signed-off-by: mpabba3003 <[email protected]> Signed-off-by: Su <[email protected]> Co-authored-by: Louis Chu <[email protected]> Co-authored-by: Zhongnan Su <[email protected]> Co-authored-by: Yan Zeng <[email protected]> Co-authored-by: Manideep Pabba <[email protected]> Co-authored-by: Yibo Wang <[email protected]> Signed-off-by: Sergey V. Osipov <[email protected]>

zhongnansu force-pushed the ds-client-interface branch from c9bc463 to b236a55 Compare August 10, 2022 00:12

zhongnansu self-assigned this Aug 10, 2022

zhongnansu added the multiple datasource multiple datasource project label Aug 10, 2022

zhongnansu changed the title ~~register data source route handler context~~ [MD] Register data source route handler context Aug 10, 2022

zhongnansu marked this pull request as ready for review August 10, 2022 01:09

zhongnansu requested a review from a team as a code owner August 10, 2022 01:09

zhongnansu changed the title ~~[MD] Register data source route handler context~~ [MD] Register data source plugin interface to route handler context Aug 10, 2022

zhongnansu requested review from seraphjiang, zengyan-amazon, noCharger and kristenTian August 10, 2022 01:13

zhongnansu linked an issue Aug 10, 2022 that may be closed by this pull request

[MD] Register getClient interface from data_source plugin to request context as plugin context #1979

Closed

noCharger reviewed Aug 10, 2022

View reviewed changes

src/plugins/data_source/server/client/data_source_client.ts Outdated Show resolved Hide resolved

src/plugins/data_source/server/client/data_source_client.ts Outdated Show resolved Hide resolved

src/plugins/data_source/server/data_source_route_handler_context.ts Outdated Show resolved Hide resolved