Skip to content
This repository has been archived by the owner on Aug 2, 2022. It is now read-only.

support remote cluster indices #244

Merged
merged 2 commits into from
Jun 25, 2020

Conversation

ylwu-amzn
Copy link
Contributor

Issue #, if available:

Description of changes:
Community has requirements to create detector on top of remote cluster’s indices, refer to Github issue (#215). User can do this with create detector public API, but can’t choose/input remote cluster indices and its fields on Kibana.

Currently Elasticsearch doesn’t support listing indices on remote cluster, nor get index mapping. So we just support remote cluster indices by allowing user input index and filed name manually.

Create/edit detector page

Screen Shot 2020-06-24 at 10 12 59 PM

Edit feature page

Screen Shot 2020-06-24 at 10 12 03 PM

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

@ohltyler
Copy link
Contributor

Do we want to get UX input on any of the changes and wording?

@ylwu-amzn
Copy link
Contributor Author

Do we want to get UX input on any of the changes and wording?

Ping UX and tech writer. Waiting for reply.

@ylwu-amzn
Copy link
Contributor Author

ylwu-amzn commented Jun 25, 2020

Do we want to get UX input on any of the changes and wording?

Ping UX and tech writer. Waiting for reply.

Add another callout based on UX designer's suggestion: "consider diaplaying something within the Add filter section as the user may scroll past the callout’s visibility."

Change callout message as tech writer's suggestion.

Screen Shot 2020-06-25 at 12 22 09 PM

Copy link
Contributor

@ohltyler ohltyler left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, thanks for adding this feature.

Comment on lines +163 to +167
onCreateOption={(createdOption: string) => {
const normalizedOptions = createdOption.trim();
if (!normalizedOptions) return;
form.setFieldValue('timeField', normalizedOptions);
}}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why allow user to create option? what if user inputs invalid timestamp field?

Copy link
Contributor Author

@ylwu-amzn ylwu-amzn Jun 25, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Similar questions: what if user input invalid filed in custom expression of filter and feature definition.
If user input invalid filed, like they input a filed which is string not timestamp, or the filed not exists, the detector just can't get any data to run.
A better way is we can pop up some warning message and block create detector with invalid parameters.
Amit is working on auto AD project. We can discuss that design, a common sanity checker component is useful. And I think that can be a part of Amit's project. Created a backlog issue to track this. opendistro-for-elasticsearch/anomaly-detection#179

Copy link
Contributor Author

@ylwu-amzn ylwu-amzn Jun 25, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why allow user to create option?

This change is not a final solution, just some workaround to support remote cluster indices.
For local indices, user still can select filed from dropdown list. For remote cluster indices, ES doesn't support listing remote cluster's indices, not get index mapping. We can't get remote cluster index's fields. So for remote cluster indices, user need to input filed manually.

Once ES supported listing remote indices and get remote index's mapping, we will query remote index mapping and user can choose filed like local indices.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks. Btw, why timestamp field from remote index is not visible?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks. Btw, why timestamp field from remote index is not visible?

For local index, we can get its index mapping and parse which filed is timestamp type. But for remote cluster index, we can't get its index mapping. So we don't know which filed is timestamp.

@ylwu-amzn ylwu-amzn linked an issue Jun 25, 2020 that may be closed by this pull request
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
feature new feature
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Support remote cluster index for anomaly detection
3 participants