Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update multiple data sources and add TSVB and Vega viz types #7229

Merged
merged 56 commits into from
Jul 2, 2024
Merged
Show file tree
Hide file tree
Changes from 29 commits
Commits
Show all changes
56 commits
Select commit Hold shift + click to select a range
3444cd3
Update multiple data sources and add TSVB and Vega viz types
vagimeli May 22, 2024
da5a66b
add text
vagimeli May 28, 2024
e24f0d4
Merge branch 'main' into update-multi-data
vagimeli Jun 3, 2024
4d95088
Add tutorials
vagimeli Jun 3, 2024
dbeb1fc
Merge branch 'main' into update-multi-data
vagimeli Jun 3, 2024
6045594
Add tutorials
vagimeli Jun 3, 2024
0579355
Update _dashboards/visualize/vega.md
vagimeli Jun 4, 2024
6c5d0fc
Merge branch 'main' into update-multi-data
vagimeli Jun 4, 2024
d2cf069
Update tsvb.md
vagimeli Jun 4, 2024
4d461fd
Update vega.md
vagimeli Jun 4, 2024
1f62857
Update vega.md
vagimeli Jun 4, 2024
956c07b
Update tsvb.md
vagimeli Jun 4, 2024
9886080
Update vega.md
vagimeli Jun 4, 2024
9de6631
Merge branch 'main' into update-multi-data
vagimeli Jun 4, 2024
6db6392
Address SME feedback
vagimeli Jun 4, 2024
45a51ca
Address SME feedback
vagimeli Jun 4, 2024
55dbcbf
Address SME feedback
vagimeli Jun 4, 2024
ef450d0
Address SME feedback
vagimeli Jun 4, 2024
1569b9d
Merge branch 'main' into update-multi-data
vagimeli Jun 5, 2024
1cd6f7a
Update _dashboards/visualize/tsvb.md
vagimeli Jun 21, 2024
183de14
Update _dashboards/visualize/tsvb.md
vagimeli Jun 21, 2024
69dd8a0
Update _dashboards/visualize/vega.md
vagimeli Jun 21, 2024
64da49f
Update _dashboards/visualize/vega.md
vagimeli Jun 21, 2024
5edcceb
Update _dashboards/visualize/vega.md
vagimeli Jun 21, 2024
a3b8fd6
Update _dashboards/visualize/tsvb.md
vagimeli Jun 21, 2024
334e54d
Update _dashboards/visualize/tsvb.md
vagimeli Jun 21, 2024
c39f92d
Update _dashboards/visualize/tsvb.md
vagimeli Jun 21, 2024
a96c92e
Update _dashboards/visualize/tsvb.md
vagimeli Jul 2, 2024
dd185c5
Update _dashboards/visualize/tsvb.md
vagimeli Jul 2, 2024
b8f6a75
Update _dashboards/visualize/geojson-regionmaps.md
vagimeli Jul 2, 2024
c2acdee
Update _dashboards/visualize/selfhost-maps-server.md
vagimeli Jul 2, 2024
87ad874
Update _dashboards/visualize/vega.md
vagimeli Jul 2, 2024
d7be094
Update _dashboards/visualize/visbuilder.md
vagimeli Jul 2, 2024
4a7d9ae
Update _dashboards/visualize/vega.md
vagimeli Jul 2, 2024
0429a41
Update _dashboards/visualize/vega.md
vagimeli Jul 2, 2024
20f0bb0
Update _dashboards/visualize/vega.md
vagimeli Jul 2, 2024
81ff0fe
Update _dashboards/management/multi-data-sources.md
vagimeli Jul 2, 2024
b4e52ef
Update _dashboards/management/multi-data-sources.md
vagimeli Jul 2, 2024
6813cd0
Update _dashboards/management/multi-data-sources.md
vagimeli Jul 2, 2024
49aee03
Update _dashboards/management/multi-data-sources.md
vagimeli Jul 2, 2024
6e16e10
Update _dashboards/management/multi-data-sources.md
vagimeli Jul 2, 2024
fe6e4a6
Update _dashboards/management/multi-data-sources.md
vagimeli Jul 2, 2024
8b5bee4
Update _dashboards/visualize/selfhost-maps-server.md
vagimeli Jul 2, 2024
d4327af
Update _dashboards/visualize/tsvb.md
vagimeli Jul 2, 2024
dab6b7b
Update _dashboards/visualize/tsvb.md
vagimeli Jul 2, 2024
1836276
Update _dashboards/visualize/tsvb.md
vagimeli Jul 2, 2024
c18233d
Update _dashboards/visualize/tsvb.md
vagimeli Jul 2, 2024
4811ef4
Update _dashboards/visualize/tsvb.md
vagimeli Jul 2, 2024
6d84850
Update _dashboards/visualize/tsvb.md
vagimeli Jul 2, 2024
de5e9f3
Update _dashboards/visualize/tsvb.md
vagimeli Jul 2, 2024
eb98b31
Update _dashboards/visualize/tsvb.md
vagimeli Jul 2, 2024
582b89a
Update _dashboards/visualize/tsvb.md
vagimeli Jul 2, 2024
98ae5dd
Update _dashboards/visualize/vega.md
vagimeli Jul 2, 2024
dc11e05
Update _dashboards/visualize/vega.md
vagimeli Jul 2, 2024
442ef62
Update _dashboards/visualize/vega.md
vagimeli Jul 2, 2024
c52dc36
Merge branch 'main' into update-multi-data
vagimeli Jul 2, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
11 changes: 6 additions & 5 deletions _dashboards/management/multi-data-sources.md
Original file line number Diff line number Diff line change
Expand Up @@ -203,14 +203,15 @@ When the data source feature is enabled, you can navigate to **Search Relevance*

After configuring multiple data sources, you can analyze the data from each source. Refer to the following resources for more information:

- Learn about [managing index patterns]({{site.url}}{{site.baseurl}}/dashboards/management/index-patterns/) through OpenSearch Dashboards.
- Learn about [indexing data using Index Management]({{site.url}}{{site.baseurl}}/dashboards/im-dashboards/index/) through OpenSearch Dashboards.
- Learn about how to [connect OpenSearch and Amazon S3 through OpenSearch Dashboards]({{site.url}}{{site.baseurl}}/dashboards/management/S3-data-source/).
- Learn about the [Integrations tool]({{site.url}}{{site.baseurl}}/integrations/index/), which gives you the flexibility to use various data ingestion methods and connect data from the Dashboards UI.
- [Managing index patterns]({{site.url}}{{site.baseurl}}/dashboards/management/index-patterns/) through OpenSearch Dashboards.
vagimeli marked this conversation as resolved.
Show resolved Hide resolved
- [Indexing data using Index Management]({{site.url}}{{site.baseurl}}/dashboards/im-dashboards/index/) through OpenSearch Dashboards.
vagimeli marked this conversation as resolved.
Show resolved Hide resolved
- [Connect OpenSearch and Amazon S3 through OpenSearch Dashboards]({{site.url}}{{site.baseurl}}/dashboards/management/S3-data-source/).
vagimeli marked this conversation as resolved.
Show resolved Hide resolved
- [Integrations tool]({{site.url}}{{site.baseurl}}/integrations/index/), which gives you the flexibility to use various data ingestion methods and connect data from the OpenSearch Dashboards UI.
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

connect "to" data?

vagimeli marked this conversation as resolved.
Show resolved Hide resolved
- Creating [Vega]({{site.url}}{{site.baseurl}}/dashboards/visualize/vega/) and [TSVB]({{site.url}}{{site.baseurl}}/dashboards/visualize/tsvb/) visualizations.
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this list actually made more sense with the "Learn about" construct. As written, these are now not totally consistent, and it's not clear that the list items are all resources.

vagimeli marked this conversation as resolved.
Show resolved Hide resolved

## Limitations

The following features are not supported when using multiple data sources:

* Timeline visualization types
* Some external plugins, such as the `gantt-chart` plugin
* Certain external plugins, such as the `gantt-chart` plugin
2 changes: 1 addition & 1 deletion _dashboards/visualize/area.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
---
layout: default
title: Using area charts
title: Area charts
parent: Building data visualizations
nav_order: 5
---
Expand Down
4 changes: 2 additions & 2 deletions _dashboards/visualize/gantt.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
---
layout: default
title: Using Gantt charts
title: Gantt charts
parent: Building data visualizations
nav_order: 30
redirect_from:
Expand All @@ -18,7 +18,7 @@ To create a Gantt chart, perform the following steps:
1. In the visualizations menu, choose **Create visualization** and **Gantt Chart**.
1. Choose a source for the chart (e.g. some log data).
1. Under **Metrics**, choose **Event**. For log data, each log is an event.
1. Select the **Start Time** and **Duration** fields from your data set. The start time is the timestamp for the beginning of an event. The duration is the amount of time to add to the start time.
1. Select the **Start Time** and **Duration** fields from your dataset. The start time is the timestamp for the beginning of an event. The duration is the amount of time to add to the start time.
1. Under **Results**, choose the number of events to display on the chart. Gantt charts sequence events from earliest to latest based on start time.
1. Choose **Panel settings** to adjust axis labels, time format, and colors.
1. Choose **Update**.
Expand Down
6 changes: 3 additions & 3 deletions _dashboards/visualize/geojson-regionmaps.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
---
layout: default
title: Using coordinate and region maps
title: Coordinate and region maps
parent: Building data visualizations
has_children: true
nav_order: 15
Expand All @@ -12,7 +12,7 @@ redirect_from:

OpenSearch has a standard set of GeoJSON files that provide a vector map with each region map. OpenSearch Dashboards also provides basic map tiles with a standard vector map to create region maps. You can configure the base map tiles using [Web Map Service (WMS)](https://www.ogc.org/standards/wms). For more information, see [Configuring WMS in OpenSearch Dashboards]({{site.url}}{{site.baseurl}}/dashboards/maptiles/).

For air gapped environments, OpenSearch Dashboards provides a self-host maps server. For more information, see [Using the self-host maps server]({{site.url}}{{site.baseurl}}/dashboards/selfhost-maps-server/)
For air-gapped environments, OpenSearch Dashboards provides a self-host maps server. For more information, see [Using the self-host maps server]({{site.url}}{{site.baseurl}}/dashboards/selfhost-maps-server/)
vagimeli marked this conversation as resolved.
Show resolved Hide resolved
vagimeli marked this conversation as resolved.
Show resolved Hide resolved
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

With regard to the question in the resolved comment: Yes, it should be "self-hosted".


While you can't configure a server to support user-defined vector map layers, you can configure your own GeoJSON file and upload it for this purpose.
{: .note}
Expand All @@ -35,7 +35,7 @@ You can use [geojson.io](https://geojson.io/#map=2/20.0/0.0) to extract GeoJSON

To create your own custom vector map, upload a JSON file that contains GEO data for your customized regional maps. The JSON file contains vector layers for visualization.

1. Prepare a JSON file to upload. Make sure the file has either a .geojson or .json extension.
1. Prepare a JSON file to upload. Make sure the file has either a `.geojson` or `.json` extension.
1. On the top menu bar, go to **OpenSearch Dashboards > Visualize**.
1. Select the **Create Visualization** button.
1. Select **Region Map**.
Expand Down
2 changes: 1 addition & 1 deletion _dashboards/visualize/maps-stats-api.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@ layout: default
title: Maps Stats API
nav_order: 20
grand_parent: Building data visualizations
parent: Using coordinate and region maps
parent: Coordinate and region maps
has_children: false
---

Expand Down
2 changes: 1 addition & 1 deletion _dashboards/visualize/maps.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@
layout: default
title: Using maps
grand_parent: Building data visualizations
parent: Using coordinate and region maps
parent: Coordinate and region maps
nav_order: 10
redirect_from:
- /dashboards/maps/
Expand Down
2 changes: 1 addition & 1 deletion _dashboards/visualize/maptiles.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@
layout: default
title: Configuring a Web Map Service (WMS)
grand_parent: Building data visualizations
parent: Using coordinate and region maps
parent: Coordinate and region maps
nav_order: 30
redirect_from:
- /dashboards/maptiles/
Expand Down
6 changes: 3 additions & 3 deletions _dashboards/visualize/selfhost-maps-server.md
Original file line number Diff line number Diff line change
@@ -1,14 +1,14 @@
---
layout: default
title: Using the self-host maps server
title: Using self-host maps servers
vagimeli marked this conversation as resolved.
Show resolved Hide resolved
grand_parent: Building data visualizations
parent: Using coordinate and region maps
parent: Coordinate and region maps
nav_order: 40
redirect_from:
- /dashboards/selfhost-maps-server/
---

# Using the self-host maps server
# Using self-host maps servers
vagimeli marked this conversation as resolved.
Show resolved Hide resolved

The self-host maps server for OpenSearch Dashboards allows users to access the default maps service in air-gapped environments. OpenSearch-compatible map URLs include a map manifest with map tiles and vectors, the map tiles, and the map vectors.
vagimeli marked this conversation as resolved.
Show resolved Hide resolved

Expand Down
71 changes: 71 additions & 0 deletions _dashboards/visualize/tsvb.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,71 @@
---
layout: default
title: TSVB
parent: Building data visualizations
nav_order: 45
---

# TSVB

The Time-Series Visual Builder (TSVB) is a powerful data visualization tool in OpenSearch Dashboards that allows you to create detailed time-series visualizations. One of its key features is the ability to add annotations or markers at specific time points based on index data. This feature is particularly useful for making connections between multiple indexes and building visualizations that display data over time, such as flight status, delays by type, and more. TSVB currently supports the following visualization types: Area, Line, Metric, Gauge, Markdown, and Data Table.

## Creating TSVB visualizations from multiple data sources
Introduced 2.14
{: .label .label-purple }

Before proceeding, ensure that the following configuration settings are enabled in the `config/opensearch_dasboards.yaml` file:

```yaml
data_source.enabled: true
vis_type_timeseries.enabled: true
```
{% include copy-curl.html %}

Once you have configured [multiple data sources]({{site.url}}{{site.baseurl}}/dashboards/management/multi-data-sources/) in OpenSearch Dashboards, you can use Vega to query those data sources. The following GIF shows the process of creating TSVB visualizations in OpenSearch Dashboards.

![Process of creating TSVB visualizations in OpenSearch Dashboards]({{site.url}}{{site.baseurl}}/images/dashboards/configure-tsvb.gif)

**Step 1: Set up and connect data sources**

Open OpenSearch Dashboards and follow these steps:

1. Select **Dashboards Management** from the main menu on the left.
2. Select **Data sources** and then select the **Create data source** button.
3. From the **Create data source** page, enter the connection details and endpoint URL.
vagimeli marked this conversation as resolved.
Show resolved Hide resolved
4. From the **Home** page, select **Add sample data** and then select the **Add data** button for the **Sample web logs** dataset.
vagimeli marked this conversation as resolved.
Show resolved Hide resolved

The following GIF shows the steps required to set up and connect a data source.

![Create data source]({{site.url}}{{site.baseurl}}/images/dashboards/create-datasource.gif)

**Step 2: Create the visualization**

Follow these steps to create the visualization:

1. From the menu on the left, select **Visualize**.
2. From the **Visualizations** page, select **Create Visualization** and then select **TSVB** in the pop-up window.
vagimeli marked this conversation as resolved.
Show resolved Hide resolved
3. Proceed with the steps under Specify data sources.
vagimeli marked this conversation as resolved.
Show resolved Hide resolved
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This step doesn't appear to be necessary.


**Step 3: Specify data sources**

After creating a TSVB visualization, data may appear based on your default index pattern. To change the index pattern or configure additional settings, follow these steps to customize your visualization:
vagimeli marked this conversation as resolved.
Show resolved Hide resolved

1. From the **Create** window, select **Panel options**.
vagimeli marked this conversation as resolved.
Show resolved Hide resolved
2. From **Data source**, select the OpenSearch cluster from which to pull data. In this case, choose your newly created data source.
vagimeli marked this conversation as resolved.
Show resolved Hide resolved
3. From **Index name**, enter `opensearch_dashboards_sample_data_logs`.
vagimeli marked this conversation as resolved.
Show resolved Hide resolved
4. Under **Time field**, select `@timestamp`. This setting specifies the time range for rendering the visualization.

**(Optional) Step 4: Add annotations**

Annotations are markers that can be added to time-series visualizations. Follow these steps to add annotations:

1. On the upper-left corner of the page, select **Time Series**.
2. Select the **Annotations** tab and then **Add data source**.
3. In the **Index** name field, specify the appropriate index. In this case, continue using the same index from the previous steps, that is, `opensearch_dashboards_sample_data_logs`.
4. From **Time** field, select **@timestamp**.
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is all of the formatting on the last word intentional/correct?

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
4. From **Time** field, select **@timestamp**.
4. In the **Time** field, select **@timestamp**.

vagimeli marked this conversation as resolved.
Show resolved Hide resolved
5. In the **Fields** field, enter `timestamp`.
6. In the **Row template** field, enter `timestamp`.

The visualization automatically updates to display your annotations, as shown in the following image.

<img src="{{site.url}}{{site.baseurl}}/images/dashboards/tsvb-with-annotations.png" alt="TSVB visualization with annotations" width="700"/>
Loading