Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add materlized views, manual refresh option and minor fixes #159

Merged
merged 5 commits into from
Oct 18, 2023
Merged

Add materlized views, manual refresh option and minor fixes #159

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
5 commits
Select commit Hold shift + click to select a range
09cac73
enable materialized views, update acc btn fill
ps48 Oct 17, 2023
16b8490
update acceleration links
ps48 Oct 17, 2023
0b295c7
add manual refresh option, update tests
ps48 Oct 17, 2023
b782d02
update copy for datasources
ps48 Oct 17, 2023
9a34f6d
fix redirection issue
ps48 Oct 18, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 5 additions & 3 deletions common/constants/index.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -5,8 +5,10 @@

export const PLUGIN_ID = 'queryWorkbenchDashboards';
export const PLUGIN_NAME = 'Query Workbench';
export const OPENSEARCH_ACC_DOCUMENTATION_URL = 'https://opensearch.org/docs/latest';
export const ACC_INDEX_TYPE_DOCUMENTATION_URL = 'https://opensearch.org/docs/latest';
export const OPENSEARCH_ACC_DOCUMENTATION_URL =
'https://opensearch.org/docs/latest/dashboards/management/accelerate-external-data/';
export const ACC_INDEX_TYPE_DOCUMENTATION_URL =
'https://github.com/opensearch-project/opensearch-spark/blob/main/docs/index.md';

export const SKIPPING_INDEX = `skipping_index`;
export const ON_LOAD_QUERY = `SHOW tables LIKE '%';`;
Expand Down Expand Up @@ -37,7 +39,7 @@ OPTIONS (
export const ACCELERATION_INDEX_TYPES = [
{ label: 'Skipping Index', value: 'skipping' },
{ label: 'Covering Index', value: 'covering' },
// { label: 'Materialized View', value: 'materialized' }, Hidden Option -> Until opensearch-spark feature is ready
{ label: 'Materialized View', value: 'materialized' },
];

export const ACCELERATION_AGGREGRATION_FUNCTIONS = [
Expand Down
6 changes: 4 additions & 2 deletions common/types/index.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -59,6 +59,8 @@ export interface FormErrorsType {
checkpointLocationError: string[];
}

export type AccelerationRefreshType = 'auto' | 'interval' | 'manual';

export interface CreateAccelerationForm {
dataSource: string;
database: string;
Expand All @@ -71,10 +73,10 @@ export interface CreateAccelerationForm {
accelerationIndexName: string;
primaryShardsCount: number;
replicaShardsCount: number;
refreshType: 'interval' | 'auto';
refreshType: AccelerationRefreshType;
checkpointLocation: string | undefined;
refreshIntervalOptions: RefreshIntervalType;
formErrors: FormErrorsType;
}

export type AsyncQueryLoadingStatus = "SUCCESS" | "FAILED" | "RUNNING" | "SCHEDULED" | "CANCELED"
export type AsyncQueryLoadingStatus = 'SUCCESS' | 'FAILED' | 'RUNNING' | 'SCHEDULED' | 'CANCELED';
14 changes: 13 additions & 1 deletion public/components/Main/main.tsx
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -109,6 +109,7 @@ interface MainState {
asyncLoading: boolean;
asyncLoadingStatus: AsyncQueryLoadingStatus;
asyncJobId: string;
isAccelerationFlyoutOpened: boolean;
}

const SUCCESS_MESSAGE = 'Success';
Expand Down Expand Up @@ -246,6 +247,7 @@ export class Main extends React.Component<MainProps, MainState> {
asyncLoading: false,
asyncLoadingStatus: 'SUCCESS',
asyncJobId: '',
isAccelerationFlyoutOpened: false,
};
this.httpClient = this.props.httpClient;
this.updateSQLQueries = _.debounce(this.updateSQLQueries, 250).bind(this);
Expand Down Expand Up @@ -789,7 +791,7 @@ export class Main extends React.Component<MainProps, MainState> {
});
}

handleDataSelect = (selectedItems: []) => {
handleDataSelect = (selectedItems: EuiComboBoxOptionOption[]) => {
if (selectedItems[0].label !== 'OpenSearch' && this.state.language === 'SQL') {
this.updateSQLQueries('');
}
Expand All @@ -798,6 +800,12 @@ export class Main extends React.Component<MainProps, MainState> {
});
};

setIsAccelerationFlyoutOpened = (value: boolean) => {
this.setState({
isAccelerationFlyoutOpened: value,
});
};

render() {
let page;
let link;
Expand All @@ -819,6 +827,10 @@ export class Main extends React.Component<MainProps, MainState> {
updateSQLQueries={this.updateSQLQueries}
selectedDatasource={this.state.selectedDatasource}
asyncLoading={this.state.asyncLoading}
openAccelerationFlyout={
this.props.isAccelerationFlyoutOpen && !this.state.isAccelerationFlyoutOpened
Comment on lines +830 to +831
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't quite get why these two booleans are used?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

One is for the prop which dictates if URL has a datasource that needs to be open with acceleration flyout. The other is to keep a local state. We need the local state to make sure, the URL flyout only opens once.

}
setIsAccelerationFlyoutOpened={this.setIsAccelerationFlyoutOpened}
/>
);
link = 'https://opensearch.org/docs/latest/search-plugins/sql/index/';
Expand Down
18 changes: 17 additions & 1 deletion public/components/SQLPage/SQLPage.tsx
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -37,6 +37,8 @@ interface SQLPageProps {
sqlTranslations: ResponseDetail<TranslateResult>[];
selectedDatasource: EuiComboBoxOptionOption[];
asyncLoading: boolean;
openAccelerationFlyout: boolean;
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we change this to something like isAccelerationFlyoutOpen

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

isAccelerationFlyoutOpen doesn't make much sense as a name here. This prop dictates should be acceleration flyout be opened or not by the SQL page component.
ex:

if parent props sends value openAccelerationFlyout={true} -> the child component opens the flyout

setIsAccelerationFlyoutOpened: (value: boolean) => void;
}

interface SQLPageState {
Expand Down Expand Up @@ -80,6 +82,21 @@ export class SQLPage extends React.Component<SQLPageProps, SQLPageState> {
});
};

componentDidUpdate(prevProps: SQLPageProps) {
const { selectedDatasource, openAccelerationFlyout } = this.props;
const prevDataSource = prevProps.selectedDatasource[0].label;
const currentDataSource = selectedDatasource[0].label;

if (
currentDataSource !== prevDataSource &&
currentDataSource !== 'OpenSearch' &&
openAccelerationFlyout
) {
this.setAccelerationFlyout();
this.props.setIsAccelerationFlyoutOpened(true);
}
}

render() {
const closeModal = () => this.setIsModalVisible(false);
const showModal = () => this.setIsModalVisible(true);
Expand Down Expand Up @@ -187,7 +204,6 @@ export class SQLPage extends React.Component<SQLPageProps, SQLPageState> {
this.props.selectedDatasource[0].label !== 'OpenSearch' && (
<EuiFlexItem grow={false}>
<EuiButton
fill={true}
className="sql-accelerate-button"
onClick={this.setAccelerationFlyout}
isDisabled={this.props.asyncLoading}
Expand Down
65 changes: 53 additions & 12 deletions .../components/acceleration/create/__tests__/__snapshots__/create_acceleration.test.tsx.snap
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -72,7 +72,7 @@ Array [

<a
class="euiLink euiLink--primary"
href="https://opensearch.org/docs/latest"
href="https://opensearch.org/docs/latest/dashboards/management/accelerate-external-data/"
rel="noopener noreferrer"
target="_blank"
>
Expand Down Expand Up @@ -132,7 +132,7 @@ Array [
<div
class="euiTextColor euiTextColor--subdued"
>
Select data connection where the data you want to accelerate resides.
Select the data source to accelerate data from. External data sources may take time to load.
</div>
</div>
<div
Expand Down Expand Up @@ -441,7 +441,7 @@ Array [
>
<a
class="euiLink euiLink--primary"
href="https://opensearch.org/docs/latest"
href="https://github.com/opensearch-project/opensearch-spark/blob/main/docs/index.md"
rel="noopener noreferrer"
target="_blank"
>
Expand Down Expand Up @@ -670,7 +670,7 @@ Array [
class="euiRadio__label"
for="refresh-option-1"
>
Auto Refresh
Auto refresh
</label>
</div>
<div
Expand All @@ -690,15 +690,35 @@ Array [
class="euiRadio__label"
for="refresh-option-2"
>
Refresh by interval
Auto refresh by interval
</label>
</div>
<div
class="euiRadio euiRadioGroup__item"
>
<input
class="euiRadio__input"
id="refresh-option-3"
name="refresh type radio group"
type="radio"
value=""
/>
<div
class="euiRadio__circle"
/>
<label
class="euiRadio__label"
for="refresh-option-3"
>
Manual refresh
</label>
</div>
</div>
<div
class="euiFormHelpText euiFormRow__text"
id="some_html_id-help-0"
>
Specify how often the index should refresh, which publishes the most recent changes and make them available for search. Default is set to auto refresh when data at the source changes.
Specify how often the index should refresh, which publishes the most recent changes and make them available for search.
</div>
</div>
</div>
Expand Down Expand Up @@ -1181,7 +1201,7 @@ Array [

<a
className="euiLink euiLink--primary"
href="https://opensearch.org/docs/latest"
href="https://opensearch.org/docs/latest/dashboards/management/accelerate-external-data/"
rel="noopener noreferrer"
target="_blank"
>
Expand Down Expand Up @@ -1243,7 +1263,7 @@ Array [
<div
className="euiTextColor euiTextColor--subdued"
>
Select data connection where the data you want to accelerate resides.
Select the data source to accelerate data from. External data sources may take time to load.
</div>
</div>,
<div
Expand Down Expand Up @@ -1635,7 +1655,7 @@ Array [
>
<a
className="euiLink euiLink--primary"
href="https://opensearch.org/docs/latest"
href="https://github.com/opensearch-project/opensearch-spark/blob/main/docs/index.md"
rel="noopener noreferrer"
target="_blank"
>
Expand Down Expand Up @@ -1900,7 +1920,7 @@ Array [
className="euiRadio__label"
htmlFor="refresh-option-1"
>
Auto Refresh
Auto refresh
</label>
</div>
<div
Expand All @@ -1921,15 +1941,36 @@ Array [
className="euiRadio__label"
htmlFor="refresh-option-2"
>
Refresh by interval
Auto refresh by interval
</label>
</div>
<div
className="euiRadio euiRadioGroup__item"
>
<input
checked={false}
className="euiRadio__input"
id="refresh-option-3"
name="refresh type radio group"
onChange={[Function]}
type="radio"
/>
<div
className="euiRadio__circle"
/>
<label
className="euiRadio__label"
htmlFor="refresh-option-3"
>
Manual refresh
</label>
</div>
</div>
<div
className="euiFormHelpText euiFormRow__text"
id="some_html_id-help-0"
>
Specify how often the index should refresh, which publishes the most recent changes and make them available for search. Default is set to auto refresh when data at the source changes.
Specify how often the index should refresh, which publishes the most recent changes and make them available for search.
</div>
</div>
</div>,
Expand Down
2 changes: 1 addition & 1 deletion ...ents/acceleration/create/__tests__/__snapshots__/create_acceleration_header.test.tsx.snap
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -29,7 +29,7 @@ exports[`Acceleration header renders acceleration flyout header 1`] = `

<a
className="euiLink euiLink--primary"
href="https://opensearch.org/docs/latest"
href="https://opensearch.org/docs/latest/dashboards/management/accelerate-external-data/"
rel="noopener noreferrer"
target="_blank"
>
Expand Down
Loading
Loading