Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Move DataSourceServiceImpl to core module #1084

Merged
merged 3 commits into from
Nov 18, 2022

Conversation

penghuo
Copy link
Collaborator

@penghuo penghuo commented Nov 17, 2022

Signed-off-by: Peng Huo [email protected]

Concepts

  • DataSourceService: manage DataSource.
    • DataSourceService is constructed by the list of DataSourceFactory at OpenSearch servcie bootstrap time. The list of DataSourceFactory is immutable.
    • Client could add DataSource defined by DataSourceMetaData at any time. DataSourceService use DataSourceFactory to create DataSource.
  • DataSourceFactory: Each data source (prometheus, opensearch) MUST define DataSourceFactory and register to DataSourceService. DataSourceFactory is one instance per JVM.
  • DataSource: Each user configured datasource mapping to one instance of DataSource per JVM.
  • DataSourceType: Pre-defined datasource type. Currently, PROMETHEUS and OPENSEARCH are supported.

Description

  1. In Catalog to Datasource changes #1027, we change the catalog to datasource in doc. In this PR. we change the major class name from catalog to datasource to make it sync with doc.
  2. Move DataSourceServiceImpl from Plugin module to Core module. And refactor the interface.
    • replace registerDefaultOpenSearchCatalog() with void addDataSource(DataSourceMetadata dataSourceMetadata);
    • add clear() interface.

To Reviewrs

  1. You can ignore most of files which only change names. and focus on
    • DataSourceServiceImpl and DataSourceServiceImplTest
    • SQLPlugin
    • DataSourceFactory

Check List

  • New functionality includes testing.
    • All tests pass, including unit test, integration test and doctest
  • New functionality has been documented.
    • New functionality has javadoc added
    • New functionality has user manual doc added
  • Commits are signed per the DCO using --signoff

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

@codecov-commenter
Copy link

codecov-commenter commented Nov 17, 2022

Codecov Report

Merging #1084 (457b2e2) into 2.x (fef20f8) will decrease coverage by 2.52%.
The diff coverage is 100.00%.

@@             Coverage Diff              @@
##                2.x    #1084      +/-   ##
============================================
- Coverage     98.27%   95.75%   -2.53%     
- Complexity     3434     3449      +15     
============================================
  Files           343      355      +12     
  Lines          8542     9233     +691     
  Branches        544      665     +121     
============================================
+ Hits           8395     8841     +446     
- Misses          142      334     +192     
- Partials          5       58      +53     
Flag Coverage Δ
query-workbench 62.76% <ø> (?)
sql-engine 98.28% <100.00%> (+<0.01%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files Coverage Δ
...ensearch/sql/datasource/DataSourceServiceImpl.java 100.00% <100.00%> (ø)
...pensearch/storage/OpenSearchDataSourceFactory.java 100.00% <100.00%> (ø)
...l/prometheus/storage/PrometheusStorageFactory.java 100.00% <100.00%> (ø)
...h/public/components/QueryLanguageSwitch/Switch.tsx 85.71% <0.00%> (ø)
workbench/public/utils/PanelWrapper.tsx 100.00% <0.00%> (ø)
workbench/public/components/app.tsx 0.00% <0.00%> (ø)
...ublic/components/QueryResults/QueryResultsBody.tsx 68.32% <0.00%> (ø)
workbench/public/components/Header/Header.tsx 100.00% <0.00%> (ø)
...ch/public/components/QueryResults/QueryResults.tsx 61.60% <0.00%> (ø)
workbench/public/components/PPLPage/PPLPage.tsx 56.52% <0.00%> (ø)
... and 3 more

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

@vmmusings
Copy link
Member

vmmusings commented Nov 17, 2022

@penghuo I have also made some changes in local for this PR. Have you removed all the references? Should I stop working on this?

@penghuo penghuo changed the title Change catalog to datasource Change CatalogService to DataSourceService Nov 17, 2022
@penghuo
Copy link
Collaborator Author

penghuo commented Nov 17, 2022

@penghuo I have also made some changes in local for this PR. Have you removed all the references? Should I stop working on this?

you can continue your work. I will merge this PR base on your code change.

@penghuo penghuo self-assigned this Nov 17, 2022
@penghuo penghuo added the maintenance Improves code quality, but not the product label Nov 17, 2022
@penghuo penghuo changed the title Change CatalogService to DataSourceService Move DataSourceServiceImpl to core module Nov 18, 2022
@penghuo penghuo marked this pull request as ready for review November 18, 2022 06:00
@penghuo penghuo requested a review from a team as a code owner November 18, 2022 06:00
Signed-off-by: Peng Huo <[email protected]>
Copy link
Member

@vmmusings vmmusings left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. Thanks for the changes.

Copy link
Collaborator

@dai-chen dai-chen left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the refactoring!

@penghuo penghuo merged commit e280866 into opensearch-project:2.x Nov 18, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
maintenance Improves code quality, but not the product
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants