Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

automatically create empty document on startup to avoid "no data" message spamming by Dashboards #527

Closed
mmguero opened this issue Jul 30, 2024 · 1 comment
Assignees
Labels
dashboards Relating to Malcolm's OpenSearch Dashboards interface elastic Related to issue with external ElasticSearch/Kibana output enhancement New feature or request opensearch Relating to Malcolm's use of OpenSearch UI Relating to general UI experience
Milestone

Comments

@mmguero
Copy link
Collaborator

mmguero commented Jul 30, 2024

Today, if you open Dashboards without any data in the indexes you get spammed with this message about a thousand times about how thiere's no data in the arkime-sessions3-* index.

It would be cool if on startup (probably right about here) we indexed a single log that's just like a "Malcolm started up" log (easily distinguishable from other network metadata logs, but existing in the same database) so that there would at least be one document at all times. This would be done via the OpenSearch/Elasticsearch REST API directly (probably using cURL or something).

To determine the index that it would need to be written into we'd have to take into account the customizability of the index and calculate the index name pretty much like we do here.

Idea suggested by @ee-hex-ee

@mmguero mmguero added enhancement New feature or request opensearch Relating to Malcolm's use of OpenSearch dashboards Relating to Malcolm's OpenSearch Dashboards interface UI Relating to general UI experience elastic Related to issue with external ElasticSearch/Kibana output labels Jul 30, 2024
@mmguero mmguero added this to the z.staging milestone Jul 30, 2024
@mmguero mmguero added this to Malcolm Jul 30, 2024
@mmguero mmguero modified the milestones: z.staging, v24.08.0 Jul 31, 2024
@mmguero mmguero moved this to Todo (develop) in Malcolm Jul 31, 2024
@mmguero mmguero modified the milestones: v24.08.0, v24.09.0 Aug 12, 2024
@mmguero mmguero modified the milestones: v24.09.0, z.staging Aug 20, 2024
@mmguero mmguero modified the milestones: v24.09.0, z.staging Sep 9, 2024
@piercema piercema self-assigned this Sep 11, 2024
@mmguero mmguero modified the milestones: z.staging, v24.09.0 Sep 17, 2024
@mmguero mmguero moved this from Todo (develop) to Done in Malcolm Sep 17, 2024
@mmguero
Copy link
Collaborator Author

mmguero commented Sep 17, 2024

Fixed via #567

@mmguero mmguero closed this as completed Sep 17, 2024
@mmguero mmguero changed the title automatically index one document on startup to avoid "no data" message spamming out of dashboards automatically create empty document on startup to avoid "no data" message spamming by Dashboards Sep 18, 2024
This was referenced Sep 18, 2024
@mmguero mmguero moved this from Done to Released in Malcolm Sep 19, 2024
mmguero added a commit to mmguero-dev/Malcolm that referenced this issue Oct 16, 2024
…reated

two issues were present:

1. opensearch_status.sh -w was no longer behaving as previously, as now an empty index is being created that doesn't have any events in it (see idaholab#527 and idaholab#567). It's been adjusted so that now it waits for an index with docs.count > 0.
2. The shared_object_creation.sh script needed to create the dummy detector if the .opendistro-anomaly-detection-state doesn't exist, so this check has been put in place
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
dashboards Relating to Malcolm's OpenSearch Dashboards interface elastic Related to issue with external ElasticSearch/Kibana output enhancement New feature or request opensearch Relating to Malcolm's use of OpenSearch UI Relating to general UI experience
Projects
Status: Released
Development

Successfully merging a pull request may close this issue.

2 participants