Language with dataset service #70

kavilla · 2024-08-24T08:19:23Z

Description

Issues Resolved

Screenshot

Testing the changes

Check List

All tests pass
- yarn test:jest
- yarn test:jest_integration
New functionality includes testing.
New functionality has been documented.
Update CHANGELOG.md
Commits are signed per the DCO using --signoff

Signed-off-by: Kawika Avilla <[email protected]>

ashwin-pc · 2024-08-24T23:05:56Z

src/plugins/data/public/query/query_string/dataset_manager/lib/index_pattern_handler.ts

+      groupName: 'Index patterns',
      children: indexPatterns,
-      isLeaf: false,
+      hasNext: false,
    };


ashwin-pc · 2024-08-24T23:20:54Z

src/plugins/data/public/query/query_string/dataset_service/types.ts

+   * Retrieves the supported query languages for this dataset type.
+   * @returns {Promise<string[]>} A promise that resolves to an array of supported language names.
+   */
+  supportedLanguages: () => string[];


pass the dataset here. ppl and sql arent supported in opensearch serverless yet

ashwin-pc · 2024-08-24T23:24:08Z

src/plugins/data/public/query/query_string/query_string_manager.mock.ts

+/*
+ * SPDX-License-Identifier: Apache-2.0
+ *
+ * The OpenSearch Contributors require contributions made to
+ * this file be licensed under the Apache-2.0 license or a
+ * compatible open source license.
+ *
+ * Any modifications Copyright OpenSearch Contributors. See
+ * GitHub history for details.
+ */
+
+/*
+ * Licensed to Elasticsearch B.V. under one or more contributor
+ * license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright
+ * ownership. Elasticsearch B.V. licenses this file to you under
+ * the Apache License, Version 2.0 (the "License"); you may
+ * not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ *    http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied.  See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+


Suggested change

/*

* SPDX-License-Identifier: Apache-2.0

*

* The OpenSearch Contributors require contributions made to

* this file be licensed under the Apache-2.0 license or a

* compatible open source license.

*

* Any modifications Copyright OpenSearch Contributors. See

* GitHub history for details.

*/

/*

* Licensed to Elasticsearch B.V. under one or more contributor

* license agreements. See the NOTICE file distributed with

* this work for additional information regarding copyright

* ownership. Elasticsearch B.V. licenses this file to you under

* the Apache License, Version 2.0 (the "License"); you may

* not use this file except in compliance with the License.

* You may obtain a copy of the License at

*

* http://www.apache.org/licenses/LICENSE-2.0

*

* Unless required by applicable law or agreed to in writing,

* software distributed under the License is distributed on an

* "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY

* KIND, either express or implied. See the License for the

* specific language governing permissions and limitations

* under the License.

*/

ashwin-pc · 2024-08-24T23:37:57Z

src/plugins/data/public/query/query_string/query_string_manager.ts

+  public initDataset = async () => {
+    await this.datasetService.init();
+  };
+
+  public registerType = (type: DatasetTypeConfig) => {
+    this.datasetService.registerType(type);
+  };
+
+  public getDatasetType = (type: string) => {
+    return this.datasetService.getType(type);
+  };
+
+  public getDatasetTypes = () => {
+    return this.datasetService.getTypes();
+  };
+
+  public getLanguage = (language: string) => {
+    return this.languageService.getLanguage(language);
+  };
+
+  private getDefaultLanguage() {
+    return (
+      this.storage.get('userQueryLanguage') ||
+      this.uiSettings.get(UI_SETTINGS.SEARCH_QUERY_LANGUAGE)
+    );
+  }
+
+  public getLanguages = () => {
+    return this.languageService.getLanguages();
+  };
+
+  public registerLanguage = (languageConfig: LanguageConfig) => {
+    this.languageService.registerLanguage(languageConfig);
+  };


just export the services. Don't re-export each function like this. Its easier. Only export functions here if you need to combine logic between the two services or something else

ashwin-pc · 2024-08-24T23:43:31Z

src/plugins/data/public/search/search_service.ts

+        const existingIndexPattern = await indexPatterns.get(dataFrame.name!, true);
+        const dataset = await indexPatterns.create(
+          dataFrameToSpec(dataFrame, existingIndexPattern?.id),
          !existingIndexPattern?.id
        );
-        indexPatterns.saveToCache(dataSetName, dataSet);
+        indexPatterns.saveToCache(dataset.id!, dataset);


Why do we still need this?

ashwin-pc · 2024-08-25T00:00:14Z

src/plugins/data/public/ui/query_editor/query_editor.tsx

    };

-    const enhancement = this.props.settings.getQueryEnhancements(newQuery.language);
-    const fields = enhancement?.fields;
+    const fields = language?.fields;


Suggested change

const fields = language?.fields;

const fieldProps = language?.fields;

ashwin-pc · 2024-08-25T00:03:08Z

src/plugins/data/public/ui/query_editor/query_editor_top_row.tsx

@@ -111,14 +108,17 @@ export default function QueryEditorTopRow(props: QueryEditorTopRowProps) {
    return {
      from:
        props.dateRangeFrom ||
-        queryUiEnhancement?.dateRange?.initialFrom ||
+        queryLanguage?.searchBar?.dateRange?.initialFrom ||


nit: Do we need initialFrom and intiialTo? was this there in the legacy languages too?

ashwin-pc · 2024-08-25T03:43:04Z

src/plugins/query_enhancements/public/plugin.tsx

+          initialFrom: moment().subtract(2, 'days').toISOString(),
+          initialTo: moment().add(2, 'days').toISOString(),


Why do we need this?

ashwin-pc · 2024-08-25T04:18:12Z

src/plugins/data/public/ui/query_editor/language_selector.tsx

@@ -3,85 +3,105 @@
 * SPDX-License-Identifier: Apache-2.0
 */

+/*


ashwin-pc · 2024-08-25T04:18:40Z

src/plugins/data/public/ui/dataset_selector/dataset_explorer.tsx

@@ -151,7 +145,7 @@ export const DatasetExplorer = ({
              </div>
            );
          })}
-          {!!!path[path.length - 1]?.isLeaf && <LoadingEmptyColumn isLoading={loading} />}
+          {!!!path[path.length - 1]?.hasNext && <LoadingEmptyColumn isLoading={loading} />}


the boolean logic here needs to be flipped

Signed-off-by: Kawika Avilla <[email protected]>

kavilla added 3 commits August 24, 2024 04:13

update ppl interceptor

e5f22a6

Signed-off-by: Kawika Avilla <[email protected]>

add dataset service

42dab80

Signed-off-by: Kawika Avilla <[email protected]>

language service

6766938

Signed-off-by: Kawika Avilla <[email protected]>

github-actions bot added the valued-contributor label Aug 24, 2024

wired up but the dataset is off

8e13655

Signed-off-by: Kawika Avilla <[email protected]>

ashwin-pc reviewed Aug 25, 2024

View reviewed changes

kavilla added 17 commits August 25, 2024 07:32

Address review comments

906d4a2

Signed-off-by: Kawika Avilla <[email protected]>

ppl query

8fce6b1

Signed-off-by: Kawika Avilla <[email protected]>

dql and lucene working again

89e238c

Signed-off-by: Kawika Avilla <[email protected]>

fix issue that would deselect dataset

2323d10

Signed-off-by: Kawika Avilla <[email protected]>

language switches and sets the query correctly

910db55

Signed-off-by: Kawika Avilla <[email protected]>

upating query

97af277

Signed-off-by: Kawika Avilla <[email protected]>

fix the styling a little

a445672

Signed-off-by: Kawika Avilla <[email protected]>

little bit width

7dc216f

Signed-off-by: Kawika Avilla <[email protected]>

submit on update

42a6efd

Signed-off-by: Kawika Avilla <[email protected]>

fix sql

733a723

Signed-off-by: Kawika Avilla <[email protected]>

ppl working better

19a00be

Signed-off-by: Kawika Avilla <[email protected]>

need to still fix the aggregations

fef3163

Signed-off-by: Kawika Avilla <[email protected]>

gotta fix the aggs and still need to update dql lucene dataset

5a6186a

Signed-off-by: Kawika Avilla <[email protected]>

use patch

18751e4

Signed-off-by: Kawika Avilla <[email protected]>

Metadata slice

182792d

Signed-off-by: Kawika Avilla <[email protected]>

index pattern updates

b6f55d1

Signed-off-by: Kawika Avilla <[email protected]>

types and datasource prepend

2ed09f7

Signed-off-by: Kawika Avilla <[email protected]>

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Language with dataset service #70

Language with dataset service #70

kavilla commented Aug 24, 2024

ashwin-pc Aug 24, 2024

ashwin-pc Aug 24, 2024

ashwin-pc Aug 24, 2024

ashwin-pc Aug 24, 2024

ashwin-pc Aug 24, 2024

ashwin-pc Aug 25, 2024

ashwin-pc Aug 25, 2024

ashwin-pc Aug 25, 2024

ashwin-pc Aug 25, 2024

ashwin-pc Aug 25, 2024

	const fields = language?.fields;
	const fieldProps = language?.fields;

		initialFrom: moment().subtract(2, 'days').toISOString(),
		initialTo: moment().add(2, 'days').toISOString(),

Language with dataset service #70

Are you sure you want to change the base?

Language with dataset service #70

Conversation

kavilla commented Aug 24, 2024

Description

Issues Resolved

Screenshot

Testing the changes

Check List

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment