From 7c6ea2a8d15390cfc3f5243445eccd95ada12568 Mon Sep 17 00:00:00 2001 From: Fanit Kolchina Date: Thu, 14 Mar 2024 17:07:52 -0400 Subject: [PATCH 01/23] First iteration Signed-off-by: Fanit Kolchina --- _config.yml | 7 + _getting-started/communicate.md | 52 +++++++ _getting-started/index.md | 38 +++++ _getting-started/ingest-data.md | 7 + {_about => _getting-started}/intro.md | 133 +++++++++++------- {_about => _getting-started}/quickstart.md | 5 +- _getting-started/search-data.md | 7 + _ml-commons-plugin/tutorials/rag-chatbot.md | 8 ++ .../tutorials/rag-conversational-agent.md | 10 ++ 9 files changed, 213 insertions(+), 54 deletions(-) create mode 100644 _getting-started/communicate.md create mode 100644 _getting-started/index.md create mode 100644 _getting-started/ingest-data.md rename {_about => _getting-started}/intro.md (53%) rename {_about => _getting-started}/quickstart.md (99%) create mode 100644 _getting-started/search-data.md create mode 100644 _ml-commons-plugin/tutorials/rag-chatbot.md create mode 100644 _ml-commons-plugin/tutorials/rag-conversational-agent.md diff --git a/_config.yml b/_config.yml index 57dbf8c641..50ef431ba4 100644 --- a/_config.yml +++ b/_config.yml @@ -118,6 +118,9 @@ collections: dashboards-assistant: permalink: /:collection/:path/ output: true + getting-started: + permalink: /:collection/:path/ + output: true opensearch_collection: # Define the collections used in the theme @@ -125,6 +128,9 @@ opensearch_collection: about: name: About OpenSearch nav_fold: true + getting-started: + name: Getting started + nav_fold: true install-and-configure: name: Install and upgrade nav_fold: true @@ -196,6 +202,7 @@ opensearch_collection: developer-documentation: name: Developer documentation nav_fold: true + clients_collection: collections: diff --git a/_getting-started/communicate.md b/_getting-started/communicate.md new file mode 100644 index 0000000000..351dbd840e --- /dev/null +++ b/_getting-started/communicate.md @@ -0,0 +1,52 @@ +--- +layout: default +title: Communicate with OpenSearch +nav_order: 30 +--- + +# Communicate with OpenSearch + +You interact with OpenSearch clusters using the REST API, which offers a lot of flexibility. You can use clients like [curl](https://curl.se/) or any programming language that can send HTTP requests. To add a JSON document to an OpenSearch index (i.e. index a document), you send an HTTP request: + +```json +PUT https://://_doc/ +{ + "title": "The Wind Rises", + "release_date": "2013-07-20" +} +``` + +To run a search for the document: + +```json +GET https://://_search?q=wind +``` + +To delete the document: + +```json +DELETE https://://_doc/ +``` + +You can change most OpenSearch settings using the REST API, modify indexes, check the health of the cluster, get statistics---almost everything. + + +When you add the document to an index, OpenSearch adds some metadata, such as the unique document *ID*: + +```json +{ + "_index": "", + "_type": "_doc", + "_id": "", + "_version": 1, + "_source": { + "title": "The Wind Rises", + "release_date": "2013-07-20" + } +} +``` + +Indexes also contain mappings and settings: + +- A *mapping* is the collection of *fields* that documents in the index have. In this case, those fields are `title` and `release_date`. +- Settings include data like the index name, creation date, and number of shards. \ No newline at end of file diff --git a/_getting-started/index.md b/_getting-started/index.md new file mode 100644 index 0000000000..885b7f3789 --- /dev/null +++ b/_getting-started/index.md @@ -0,0 +1,38 @@ +--- +layout: default +title: Getting started +nav_order: 1 +has_children: true +has_toc: false +nav_exclude: true +permalink: /getting-started/ +--- + +# Getting started + +OpenSearch is a distributed search and analytics engine based on [Apache Lucene](https://lucene.apache.org/). After adding your data to OpenSearch, you can perform full-text searches on it with all of the features you might expect: search by field, search multiple indexes, boost fields, rank results by score, sort results by field, and aggregate results. + +Unsurprisingly, people often use search engines like OpenSearch as the backend for a search application---think [Wikipedia](https://en.wikipedia.org/wiki/Wikipedia:FAQ/Technical#What_software_is_used_to_run_Wikipedia?) or an online store. It offers excellent performance and can scale up and down as the needs of the application grow or shrink. + +An equally popular, but less obvious use case is log analytics, in which you take the logs from an application, feed them into OpenSearch, and use the rich search and visualization functionality to identify issues. For example, a malfunctioning web server might throw a 500 error 0.5% of the time, which can be hard to notice unless you have a real-time graph of all HTTP status codes that the server has thrown in the past four hours. You can use [OpenSearch Dashboards]({{site.url}}{{site.baseurl}}/dashboards/index/) to build these sorts of visualizations from data in OpenSearch. + +## Components + +Though this section focuses on the core product, OpenSearch is more than the core engine. It also includes the following components: + +- [OpenSearch Dashboards]({{site.url}}{{site.baseurl}}/dashboards/index/): The OpenSearch data visualization UI. +- [Data Prepper]({{site.url}}{{site.baseurl}}/data-prepper/): A server-side data collector capable of filtering, enriching, transforming, normalizing, and aggregating data for downstream analytics and visualization. +- [Clients]({{site.url}}{{site.baseurl}}/clients/): Language APIs that let you communicate with OpenSearch in several popular programming languages. + +## Use cases + +OpenSearch supports a wide variety of use cases, for example: + +- [Observability]({{site.url}}{{site.baseurl}}/observing-your-data/): Visualize data-driven events by using Piped Processing Language to explore, discover, and query data stored in OpenSearch. +- [Search]({{site.url}}{{site.baseurl}}/search-plugins/): Choose the search method best for your application, from regular lexical search to conversational search powered by machine learning (ML). +- [Machine learning]({{site.url}}{{site.baseurl}}/ml-commons-plugin/): Integrate ML models into your OpenSearch application. +- [Security analytics]({{site.url}}{{site.baseurl}}/security-analytics/): Investigate, detect, analyze, and respond to security threats that can jeopardize the success of businesses and organizations and their online operations. + +## Next steps + +- Learn the essential OpenSearch concepts in [Introduction to OpenSearch]({{site.url}}{{site.baseurl}}/getting-started/intro/) \ No newline at end of file diff --git a/_getting-started/ingest-data.md b/_getting-started/ingest-data.md new file mode 100644 index 0000000000..ff9a626286 --- /dev/null +++ b/_getting-started/ingest-data.md @@ -0,0 +1,7 @@ +--- +layout: default +title: Ingest data +nav_order: 40 +--- + +# Ingest your data into OpenSearch \ No newline at end of file diff --git a/_about/intro.md b/_getting-started/intro.md similarity index 53% rename from _about/intro.md rename to _getting-started/intro.md index ef1dc4977f..2a08965453 100644 --- a/_about/intro.md +++ b/_getting-started/intro.md @@ -2,92 +2,117 @@ layout: default title: Intro to OpenSearch nav_order: 2 -permalink: /intro/ +has_math: true +redirect_from: + - /intro/ --- # Introduction to OpenSearch -OpenSearch is a distributed search and analytics engine based on [Apache Lucene](https://lucene.apache.org/). After adding your data to OpenSearch, you can perform full-text searches on it with all of the features you might expect: search by field, search multiple indexes, boost fields, rank results by score, sort results by field, and aggregate results. +OpenSearch is a distributed search and analytics engine, which supports various use cases, from implementing a search box on a website to analyzing security data for threat detection. The term _distributed_ means you can run OpenSearch on multiple computers. _Search and analytics_ means you can search and analyze your data once you ingest it into OpenSearch. Whether your data is geographic or genetic, you can store and analyze it using OpenSearch. -Unsurprisingly, people often use search engines like OpenSearch as the backend for a search application---think [Wikipedia](https://en.wikipedia.org/wiki/Wikipedia:FAQ/Technical#What_software_is_used_to_run_Wikipedia?) or an online store. It offers excellent performance and can scale up and down as the needs of the application grow or shrink. +## Document -An equally popular, but less obvious use case is log analytics, in which you take the logs from an application, feed them into OpenSearch, and use the rich search and visualization functionality to identify issues. For example, a malfunctioning web server might throw a 500 error 0.5% of the time, which can be hard to notice unless you have a real-time graph of all HTTP status codes that the server has thrown in the past four hours. You can use [OpenSearch Dashboards]({{site.url}}{{site.baseurl}}/dashboards/index/) to build these sorts of visualizations from data in OpenSearch. +A _document_ is a unit that stores information, such as text or structured data. In OpenSearch, documents are stored in [JSON](https://www.json.org/) format. For example, a document may represent a line in a Shakespeare play: +```json +{ + "type": "line", + "line_id": 5, + "play_name": "Henry IV", + "speech_number": 1, + "line_number": "1.1.2", + "speaker": "KING HENRY IV", + "text_entry": "Find we a time for frighted peace to pant," +} +``` -## Clusters and nodes - -Its distributed design means that you interact with OpenSearch *clusters*. Each cluster is a collection of one or more *nodes*, servers that store your data and process search requests. +You can use the preceding document for full-text search. -You can run OpenSearch locally on a laptop---its system requirements are minimal---but you can also scale a single cluster to hundreds of powerful machines in a data center. +You can think of a document in several ways: -In a single node cluster, such as a laptop, one machine has to do everything: manage the state of the cluster, index and search data, and perform any preprocessing of data prior to indexing it. As a cluster grows, however, you can subdivide responsibilities. Nodes with fast disks and plenty of RAM might be great at indexing and searching data, whereas a node with plenty of CPU power and a tiny disk could manage cluster state. For more information on setting node types, see [Cluster formation]({{site.url}}{{site.baseurl}}/opensearch/cluster/). +- If you have a collection of encyclopedia articles, a document might represent one article. +- When you search for information, OpenSearch returns documents related to your search. +- If you're familiar with traditional databases, a document represents a row. +For example, in a school database, a document might represent one student and contain structured data like the student ID and name: -## Indexes and documents +ID | First name | Last name | Address | +:--- | :--- | :--- | :--- | +123456 | John | Doe | 121 Main St. | -OpenSearch organizes data into *indexes*. Each index is a collection of JSON *documents*. If you have a set of raw encyclopedia articles or log lines that you want to add to OpenSearch, you must first convert them to [JSON](https://www.json.org/). A simple JSON document for a movie might look like this: +Here is how this document looks in JSON format: ```json { - "title": "The Wind Rises", - "release_date": "2013-07-20" + "id": "123456", + "first_name": John, + "last_name": "Doe", + "address": "121 Main St." } ``` -When you add the document to an index, OpenSearch adds some metadata, such as the unique document *ID*: +## Index -```json -{ - "_index": "", - "_type": "_doc", - "_id": "", - "_version": 1, - "_source": { - "title": "The Wind Rises", - "release_date": "2013-07-20" - } -} -``` +An index is a collection of documents. -Indexes also contain mappings and settings: +You can think of an index in several ways: -- A *mapping* is the collection of *fields* that documents in the index have. In this case, those fields are `title` and `release_date`. -- Settings include data like the index name, creation date, and number of shards. +- If you have a collection of encyclopedia articles, an index represents the whole collection. +- When you search for information, you query data contained in an index. +- If you're familiar with traditional databases, a document represents a database table. -## Primary and replica shards +For example, in a school database, an index might contain all students in the school: -OpenSearch splits indexes into *shards* for even distribution across nodes in a cluster. For example, a 400 GB index might be too large for any single node in your cluster to handle, but split into ten shards, each one 40 GB, OpenSearch can distribute the shards across ten nodes and work with each shard individually. +ID | First name | Last name | Address +:--- | :--- | :--- | :--- +123456 | John | Doe | 121 Main St. +123457 | Jane | Smith | 1 Chestnut St. -By default, OpenSearch creates a *replica* shard for each *primary* shard. If you split your index into ten shards, for example, OpenSearch also creates ten replica shards. These replica shards act as backups in the event of a node failure---OpenSearch distributes replica shards to different nodes than their corresponding primary shards---but they also improve the speed and rate at which the cluster can process search requests. You might specify more than one replica per index for a search-heavy workload. +An OpenSearch index is represented as an _inverted index_. An inverted index maps words to the documents that they occur in. For example, consider an index containing the following two documents: -Despite being a piece of an OpenSearch index, each shard is actually a full Lucene index---confusing, we know. This detail is important, though, because each instance of Lucene is a running process that consumes CPU and memory. More shards is not necessarily better. Splitting a 400 GB index into 1,000 shards, for example, would place needless strain on your cluster. A good rule of thumb is to keep shard size between 10--50 GB. +- Document 1 : "Beauty is in the eye of the beholder" +- Document 2: "Beauty and the beast" +An inverted index for such an index maps the words to the documents where they occur: -## REST API +Word | Document +:--- | :--- +beauty | 1, 2 +is | 1 +in | 1 +the | 1, 2 +eye | 1 +of | 1 +the | 1 +beholder | 1 +and | 2 +beast | 2 -You interact with OpenSearch clusters using the REST API, which offers a lot of flexibility. You can use clients like [curl](https://curl.se/) or any programming language that can send HTTP requests. To add a JSON document to an OpenSearch index (i.e. index a document), you send an HTTP request: +In addition to the document ID, OpenSearch stores the position of the word within that document for phrase queries, where words must appear next to each other. -```json -PUT https://://_doc/ -{ - "title": "The Wind Rises", - "release_date": "2013-07-20" -} -``` +When searching for documents, you need to make sure that the word is _relevant_. For example, the word `the` appears in most English phrases but searching for this word is meaningless. OpenSearch determines the relevance of a term by calculating two values: -To run a search for the document: +- _Term frequency_: how often a word appears in the document +- _Document frequency_: how often a word appears in all documents -```json -GET https://://_search?q=wind -``` +Then, the relevance of a word is calculated as $$ relevance = { \text {term frequency} \over \text {document frequency} }. $$ -To delete the document: +## Clusters and nodes -```json -DELETE https://://_doc/ -``` +Its distributed design means that you interact with OpenSearch *clusters*. Each cluster is a collection of one or more *nodes*, servers that store your data and process search requests. -You can change most OpenSearch settings using the REST API, modify indexes, check the health of the cluster, get statistics---almost everything. +You can run OpenSearch locally on a laptop---its system requirements are minimal---but you can also scale a single cluster to hundreds of powerful machines in a data center. + +In a single node cluster, such as a laptop, one machine has to do everything: manage the state of the cluster, index and search data, and perform any preprocessing of data prior to indexing it. As a cluster grows, however, you can subdivide responsibilities. Nodes with fast disks and plenty of RAM might be great at indexing and searching data, whereas a node with plenty of CPU power and a tiny disk could manage cluster state. For more information about setting node types, see [Cluster formation]({{site.url}}{{site.baseurl}}/opensearch/cluster/). + +## Primary and replica shards + +OpenSearch splits indexes into *shards* for even distribution across nodes in a cluster. For example, a 400 GB index might be too large for any single node in your cluster to handle, but split into ten shards, each one 40 GB, OpenSearch can distribute the shards across ten nodes and work with each shard individually. + +By default, OpenSearch creates a *replica* shard for each *primary* shard. If you split your index into ten shards, for example, OpenSearch also creates ten replica shards. These replica shards act as backups in the event of a node failure---OpenSearch distributes replica shards to different nodes than their corresponding primary shards---but they also improve the speed and rate at which the cluster can process search requests. You might specify more than one replica per index for a search-heavy workload. + +Despite being a piece of an OpenSearch index, each shard is actually a full Lucene index---confusing, we know. This detail is important, though, because each instance of Lucene is a running process that consumes CPU and memory. More shards is not necessarily better. Splitting a 400 GB index into 1,000 shards, for example, would place needless strain on your cluster. A good rule of thumb is to keep shard size between 10--50 GB. ## Advanced concepts @@ -109,4 +134,8 @@ A _flush_ operation persists the files to disk using `fsync`, ensuring durabilit ### Merge -In OpenSearch, a shard is a Lucene index, which consists of _segments_ (or segment files). Segments store the indexed data and are immutable. Periodically, smaller segments are merged into larger ones. Merging reduces the overall number of segments on each shard, frees up disk space, and improves search performance. Eventually, segments reach a maximum size specified in the merge policy and are no longer merged into larger segments. The merge policy also specifies how often merges are performed. \ No newline at end of file +In OpenSearch, a shard is a Lucene index, which consists of _segments_ (or segment files). Segments store the indexed data and are immutable. Periodically, smaller segments are merged into larger ones. Merging reduces the overall number of segments on each shard, frees up disk space, and improves search performance. Eventually, segments reach a maximum size specified in the merge policy and are no longer merged into larger segments. The merge policy also specifies how often merges are performed. + +## Next steps + +- Learn how to quickly install OpenSearch in [Installation quickstart]({{site.url}}{{site.baseurl}}/getting-started/quickstart/) \ No newline at end of file diff --git a/_about/quickstart.md b/_getting-started/quickstart.md similarity index 99% rename from _about/quickstart.md rename to _getting-started/quickstart.md index 5c7da2950e..528a3cd00c 100644 --- a/_about/quickstart.md +++ b/_getting-started/quickstart.md @@ -1,13 +1,14 @@ --- layout: default -title: Quickstart +title: Installation quickstart nav_order: 3 permalink: /quickstart/ redirect_from: - /opensearch/install/quickstart/ + - /quickstart/ --- -# Quickstart +# Installation quickstart Get started using OpenSearch and OpenSearch Dashboards by deploying your containers with [Docker](https://www.docker.com/). Before proceeding, you need to [get Docker](https://docs.docker.com/get-docker/) and [Docker Compose](https://github.com/docker/compose) installed on your local machine. diff --git a/_getting-started/search-data.md b/_getting-started/search-data.md new file mode 100644 index 0000000000..ee4e2500ac --- /dev/null +++ b/_getting-started/search-data.md @@ -0,0 +1,7 @@ +--- +layout: default +title: Search your data +nav_order: 50 +--- + +# Search your data \ No newline at end of file diff --git a/_ml-commons-plugin/tutorials/rag-chatbot.md b/_ml-commons-plugin/tutorials/rag-chatbot.md new file mode 100644 index 0000000000..3c1b8e5ca9 --- /dev/null +++ b/_ml-commons-plugin/tutorials/rag-chatbot.md @@ -0,0 +1,8 @@ +--- +layout: default +title: RAG chatbot +parent: Tutorials +nav_order: 50 +--- + +# Retrieval-augmented generation chatbot \ No newline at end of file diff --git a/_ml-commons-plugin/tutorials/rag-conversational-agent.md b/_ml-commons-plugin/tutorials/rag-conversational-agent.md new file mode 100644 index 0000000000..7fae32fe3c --- /dev/null +++ b/_ml-commons-plugin/tutorials/rag-conversational-agent.md @@ -0,0 +1,10 @@ +--- +layout: default +title: RAG with a conversational flow agent +parent: Tutorials +nav_order: 40 +--- + +# Retrieval-augmented generation with a conversational flow agent + +Retrieval-augmented generation (RAG) \ No newline at end of file From a20ed08838ff74627059e06ac77cbd77f62d5702 Mon Sep 17 00:00:00 2001 From: Fanit Kolchina Date: Tue, 26 Mar 2024 17:40:56 -0400 Subject: [PATCH 02/23] Add shard and node info Signed-off-by: Fanit Kolchina --- _getting-started/intro.md | 28 +++++++++++++++++++++++----- images/intro/cluster-replicas.png | Bin 0 -> 38382 bytes images/intro/cluster.png | Bin 0 -> 30067 bytes images/intro/index-shard.png | Bin 0 -> 17682 bytes 4 files changed, 23 insertions(+), 5 deletions(-) create mode 100644 images/intro/cluster-replicas.png create mode 100644 images/intro/cluster.png create mode 100644 images/intro/index-shard.png diff --git a/_getting-started/intro.md b/_getting-started/intro.md index 2a08965453..45a65e5ef4 100644 --- a/_getting-started/intro.md +++ b/_getting-started/intro.md @@ -98,21 +98,39 @@ When searching for documents, you need to make sure that the word is _relevant_. Then, the relevance of a word is calculated as $$ relevance = { \text {term frequency} \over \text {document frequency} }. $$ +OpenSearch scores the results in terms of relevance and returns them sorted by relevance. + ## Clusters and nodes -Its distributed design means that you interact with OpenSearch *clusters*. Each cluster is a collection of one or more *nodes*, servers that store your data and process search requests. +OpenSearch is designed to be a distributed search engine. OpenSearch can run on one or more *nodes*, servers that store your data and process search requests. An OpenSearch *cluster* is a collection of nodes. You can run OpenSearch locally on a laptop---its system requirements are minimal---but you can also scale a single cluster to hundreds of powerful machines in a data center. -In a single node cluster, such as a laptop, one machine has to do everything: manage the state of the cluster, index and search data, and perform any preprocessing of data prior to indexing it. As a cluster grows, however, you can subdivide responsibilities. Nodes with fast disks and plenty of RAM might be great at indexing and searching data, whereas a node with plenty of CPU power and a tiny disk could manage cluster state. For more information about setting node types, see [Cluster formation]({{site.url}}{{site.baseurl}}/opensearch/cluster/). +In a single-node cluster, such as a laptop, one machine has to do everything: manage the state of the cluster, index and search data, and perform any preprocessing of data prior to indexing it. As a cluster grows, however, you can subdivide responsibilities. Nodes with fast disks and plenty of RAM might be great at indexing and searching data, whereas a node with plenty of CPU power and a tiny disk could manage cluster state. + +In each cluster, there is an elected _cluster manager_ node, which orchestrates cluster-level operations, such as creating an index. Nodes communicate with each other, so if your request is routed to a node, that node sends requests to appropriate nodes, gathers the nodes' responses, and returns the final response. + +For more information about setting node types, see [Cluster formation]({{site.url}}{{site.baseurl}}/opensearch/cluster/). + +## Shards + +OpenSearch splits indexes into *shards*. Each shard stores a subset of all documents in an index, as shown in the following image. + +An index is split into shards + +Shards are used for even distribution across nodes in a cluster. For example, a 400-GB index might be too large for any single node in your cluster to handle, but split into ten shards, each one 40 GB, OpenSearch can distribute the shards across ten nodes and work with each shard individually. When you index documents into OpenSearch, the documents are directed to a particular shard, and shards can reside on different nodes in a cluster. For example, consider a cluster with two indexes: index 1 and index 2. Index 1 is split into 2 shards, and index 2 is split into 4 shards. The shards are distributed across nodes 1 and 2, as shown in the following image. + +A cluster containing two indexes and two nodes + +Despite being a piece of an OpenSearch index, each shard is actually a full Lucene index---confusing, we know. This detail is important, though, because each instance of Lucene is a running process that consumes CPU and memory. More shards is not necessarily better. Splitting a 400 GB index into 1,000 shards, for example, would place needless strain on your cluster. A good rule of thumb is to keep shard size as 10--50 GB. ## Primary and replica shards -OpenSearch splits indexes into *shards* for even distribution across nodes in a cluster. For example, a 400 GB index might be too large for any single node in your cluster to handle, but split into ten shards, each one 40 GB, OpenSearch can distribute the shards across ten nodes and work with each shard individually. +In OpenSearch, shards may be _primary_ (the original) or _replica_ (a copy). By default, OpenSearch creates a replica shard for each primary shard. Thus, if you split your index into ten shards, OpenSearch creates ten replica shards. For example, consider a cluster described in the previous section. If you add one replica for each shard of each index in the cluster, your cluster will contain a total of 2 shards and 2 replicas for index 1 and 4 shards and 4 replicas for index 2, as shown in the following image. -By default, OpenSearch creates a *replica* shard for each *primary* shard. If you split your index into ten shards, for example, OpenSearch also creates ten replica shards. These replica shards act as backups in the event of a node failure---OpenSearch distributes replica shards to different nodes than their corresponding primary shards---but they also improve the speed and rate at which the cluster can process search requests. You might specify more than one replica per index for a search-heavy workload. +A cluster containing two indexes with one replica shard for each shard in the index -Despite being a piece of an OpenSearch index, each shard is actually a full Lucene index---confusing, we know. This detail is important, though, because each instance of Lucene is a running process that consumes CPU and memory. More shards is not necessarily better. Splitting a 400 GB index into 1,000 shards, for example, would place needless strain on your cluster. A good rule of thumb is to keep shard size between 10--50 GB. +These replica shards act as backups in the event of a node failure---OpenSearch distributes replica shards to different nodes than their corresponding primary shards---but they also improve the speed at which the cluster processes search requests. You might specify more than one replica per index for a search-heavy workload. ## Advanced concepts diff --git a/images/intro/cluster-replicas.png b/images/intro/cluster-replicas.png new file mode 100644 index 0000000000000000000000000000000000000000..3462406b985cdc0aeac97c7f9e07a9f1c55bf6b9 GIT binary patch literal 38382 zcmeFZ1yohr+CQv_N+{ha-QC^YAtl{NZ@NX0?v#=iq>&QoP6-w1kdW@~Z*Afn@AbU* z{=e~!`+n#A&mCjCfw|_I>zU8*>1QqjYl!(hSOxpU{Sgt)NcojVZTckbMUhk5{v z@Z7}<1->QiMbzvqosG>bjP8&y30?n2!bopwZD&uyBuv7{Xb94CGP9;Num;)C>RIZ! zT3Z3XTLKq=S(es@X2xblhSxJ0=^2@6=$U93S(O>sNtguaS%EJGHd=NT294|adL~9z zH_OUcyP8>8=#emrve424OOeay8JJnwTicnEFbM*GOIX<(SpYwQVc=U%3HVY2{xQ(g zGO%3F06Bt}XJn$KrvNU985x*c1FM0j03QZmNYo5uWNhsW3|`H;Cc(&F&*Yk18B-b= zYc(2UF$V{I1!)mUH8C@>>p6}_AUiW_tD9Xh(6Z4oUth3yu`#+H)dyKS*%^Vr!%QM1 z^a8-+z#k?dLo+=SkRHGvdMiE4o5}LPzG(o8z$`PnVU`#@jggQlqa&M+rGbjNf(eU; z+0{dSUhQ^GOJ^~Vo{g!Dwc!;@44toO;9vn31g~l6ay`JxbTeQAx_!*eQ^5mDX0A8% zFaWfI?=x{QGc>ZhzV~LLy|uN4y_wBF4jNcnSs5AJ0CzJ{4+Q4GKiq3@?n>jYX zzW-qcctFWO&*IlnRWn0-0C}*$v9sKe2c9i%WM*P|`(yywYr-w{ZYSRi*qQ1XT033e ze>Lnb|I2b>%3`(> zYR-%j!i?%nwgN7a^iK_MrOQ#z!r_Lw*PODmce&*@$lAfm@Jd$!ITds=HM2KTve7dD zPj>>u3m7xCx3mEMVgP;x41$G_wS_h4ilfxB!;^$_E>`SeOCk48-*BZJGYH;giMVl?%G3M1sMNi0F8}}nGL|tx{;j!r~zcW^?RQXHI(DOfPw*K{->DeuM6mH_2s81R|1G+ zfjZSy)k?`!4`g`#J2Ee=SV?5!g2(@wc?%zXaTWU9f*GTKy5&HzM)31pAj@ z{I3i4uLUhxpw0R>kH7yZd1ko%`>*Gvf30ub!25T0%uN3jasP$!{k1wGXJcfgWTXc& zFa?y@z{0@}obg^AclcYH>fgHOx2g1P`QayBX1J}b{9LR4^QBkd7CtzS`YD(G1=#<; zx=H_=%?HL?2yc_9+v4?4kp5A^eCwqDJk=4#U#la3PegAW>|Y?^F~Lts(I3P8cP@DRqTPR8xc|f% z)BolX+20P8-*uP*ZC?E6$$@X*+4%zn2C)8b(-;Ua|M@rw3{ATgTemYIsculHRG+4-bUAdUATWukN-6J(*LHlX1TS_zgg>FkNEsWto5(`!rO4` z9~Jnsu^;Q{Z3WzSZ%co%3xh`=4~qOhUjre*eQepjX%a;bq&O-vYe4di#3s z^@W=oz;91qzZN=4)Mt6;&XYS5!UD?9+UqIzlb?!Ux8w6?W!FSTY|!5IiBge(j*Z)} zBF@TIz=F(z%+9Leg3JOwyGfY!HMBjGlXe?Cc83#gsk`SJ*5@M}cE%UGeQv29CQ}^W zVbCcq^5i4+XG9=~`ENfnr;wFaQ&8{Dn=3ZIouANSR;UrxbxemJq+W8X=Jk! z~;?>B-%j&z2A{;K{zU#`xP}(DIOle;Vt&LWf1f_u=-){ad1Cn+B`+Y)BJPN!;lpv(NZ;;? zA9!y0Efv68|C@xg3wRx@QP66!`?K55&DFgwW#(OO$V=t+^sRBa5O^sp!M($M91%~a zoon(c9HGgef8OUeeoPPnd&T3+apH5g!LQ@;@?VhpasP~0J?JY2y#*m?fhOyFVB=5^ zI>XuN#@&ADSW;X{uWi(NtIwi6yAY};b0wZ19QPSd9^tKiE+Y-aeEB6OZ0;0MNqYKn zZyA|d=M1*nUFC6PrZ9^KpY<(^y-4^GO}9H6T=!vgo}OYb5)L~pIcHryf!(-tgOBcj zYm2dMp?6vqHSBsbKAQtk$H~&sFw3Fk3NMl+9$D)7X>;l8ig`$CC58y^*~reKIt2t- z7+%O1vT(hD+*Fw=B~cz&6FI*ME;#~$=GM2Hi;r)Ro+*YThoHebwoDrSKCZD)YWNAS zHg&`2#Tf1^y+V}X(u6sb23ygqa7oohIwrDw#`T)V!rO{QU%l|h8nA^vfct#STNU1@cOoKW@yWw1>m0GACfXhHH7mIHRWL{Hz{{$wr> zLf+2g#~dK^#!uM8^d+ipYh#s&n2bguNqy%L40euB9oO@WUbC9TxN4{CXAitp(y2(U zHJy4?n(Uz>ay;d-h-<&fg@Y^#-#mdin_xRT+rv4XVLaI$5!p!Oe?TDl)ilp@w8)z< zk8B+0$aVXy`pYRUVaf2Jfqa82W=P^*Fp@oE##oxDddR6u`bs9sVDJx_j!ZH5ZIQy_ z!4j8;%m@okmeam_m}R=%1q;sGJFmWY;&FlQ>~M^NezaK!|Iqy&go_r=olkTl;hThD z1f9>h^?Z>WsaSD6^Np+5sx7uPrISlhvSG4R5t|&fP7He9@-dt)+Kzy>GRFEf7CBIZ+7o-eb%KN>2jcxeoo& z>L>F^<+WLN3KdG(^=r|QA7fQDh0&}E8}2-`;IdbzP8St;ceXs3_0+_$Cla0L!J+Wj zUV`ryPL}YO-1NFwg!7LQ&yvqii9Hc!?K@waeQ-Jq3t}#saEA{nhtsdRB8p?5{q^UqcIYy4wiS6>lF60d( ze0&?P)aZ0qS$sa@<5lIx&Okf)^41(mnAL(*FI>KmkEvb|)zaDx3RKF4+WoVUCJwiP zsC6ZX#d`N@^l}3Hx{RjN;zJBSyGLoZY!wD~sc(D{Zb70Jvf$Vh~*)ZKk;%TpRv zp_*k}_KD^P8%gI_y2yL0PM4uN!jQ7xXzr)iio!HG9?!Z|gq+NG8Bm`YVSPI*h2*NV zl5e0>D<}2~c|+Qr+=9Lkth*4GDV|83?fM2Kaq={rzy4*L2(e{mfTT(*UzHUl16igq zlc!du@Jw%NN1lm4JU-qK6^hcJq*%G_8-yA&UyR98Bc96O@LDl#kaY%%r9ixsPmR}R zUtxMsqeo3uy5$$n*Nlr!^IfWHF?>ZCKAZK}6uKNoKlbyR4wOh2#SxEA{d~kDS3H8n zQidY0&FhOp#5AaK=U!tOTz_w#alO;X`~W(A*?%7=X%UUBAiMv zw`EpMX{}5`6cbOEB6f*Sn&%ad*x&N-FmFk7hU&v9Zcmc{k@w|9{i_TPrCf2{T4(5i zIw53gb$Zyr3=#D83z$N#1GL#gi5F1zP$DMh?tC7i%!R7=xx%!|oLkj#Z@iN;+1()i z;0_s>NY1Bx&Eb4BP$P3>_youzAd>+9&9Qy{?B^A+P>*mDs3JHJ$AYr;sxPg2+F4$o z^uP?gU)0B+>_Qg%`neQE%Dc=4(0GR&UP=w}_%vLp#+@lV+{D>~LhR?$Z!{SvOBO?r z&`4ww zMN#q)RaESV7!&8uSPNW;kd^|zbp^~X8o^Q^9-SRCq3%bg0@gIc>yXuSus+9BF8dCm z^GfTW?&iO#GLfjTbz8U#A3DjcPk z@Ywb zpdsoJ4y__YBO(YtA~ z0W>N|0?~kfjGTpWp+f8qm@zdH=gvwAMDonEq;JKFJ>!E0wf-kMaqVOal$nCDiO;9g z)}%BMX&x1&I&Dt8h5D+yS`c*=$uznF@yBxvt_0@6G6SZ93j?nJqb8WM_L2(~@d7atIx`F_uh-u;Xk zXQT=l_Zf3z+wP^vsiD3EJOPZ+p2aJK#P!7C$EI>UaCNazb+mD*MvQHIg!<%k33}Ag zqONc_pqGRfwTqC6pO^M43RewZHWJ9%4GTzwhi(e4{meWFR4W-+5t_C}BXR?tHZ$ zB9W+jzKr)cKAE;VDuew()H(dJ=drs>SAfop6y76`?KAbE1Ltkz`fV@rZ&h5xc6>%uXDMXb(R9AdD z$5~X%7Z{pJt-RGKgCwE?cx)5>BGywT7%C=qW2zL+1s2V|EWb7d%p6Uk67zA^1)s$7 zr8$lJ0vj2>-Qx2Ct^tik(b?C0`rT>%M;XrQ!SYM1*gi(hqYl%t`g)T6QY0`|^isNO z1Er1moq~`DEN=N_KZ$)QiW^OIzkB-mEyLtbvf?jXdD{MAF<0nri?B zm>^1PDOP~|$VOV51YGokv}czJ;>U(#P7;Rj6z2EW$%NV?x~H!y(WnEpWD|$teWA~@ zMg~0+4Qxjg(X_wRI3L*d#FMlZbw$XJ3{PrffmJLaIM1*|(#BhwVStNkqWu14Sy1KU zGdkCVUcM&?KRD=$Ci~#JOt~>t?cWiO7uY{HoZ1n--MKwIn^Q|D78ueQNpyZ` zVkoS5spFm@EV%Z;HiBG2P&f#sD|7uY&OdT@oG2UoVAq}yT`q}6w(FhNn^EDd4jnTs z+Mabn6C4!H1f6J}U-nZS9d7UtOD1BJ&b2DPq<#K%SdcCV!C4`*{(^#UCu(+UaE{D* zZ;Jts*OkyWgw<&uh&OC-8k*9|G;rt!J9ES3lTnbM__I`3gM@o16B+e-e%0 zlknbHIYa(JI)OCXsQ^!k!faP* zyTAX@`%;dV;w6j`%oEl2tV#rT(TCrQmC#F<4sq4$K~#%YA)$2tLc^uK1bcrAj>1muGCzW8mzqGQzRfuv#ENnbQoPmp-GGW?K?FEgccrL zLv1AVvp1NIUb6*Y>340HwLFcP9CtJbL+~7Dvl5$pFf_0?-Yp(a{Zq$b5&XpTted0 zWFgm}D=X7&u~p>ux?~|?o6uyKeHCnLib7uvPwvlL++!{pf^^wgXbiZRegc!})fA@J z@g};&tix7Xdnzfb3u2%h~S=yeLNarXD?T2oT@|-IJm9B7DKh31q!vy_o-s15! z!%olQmeJ%eneNwAZMNsV@*%$?Ge9N(9$GSnj>uP1OUNm*j6m8Nm9b~E^NJLHqWQYa zN63}h;n5quG@#13YdCRvdkR(1`VR=u=#CLzm66)!<9XG%wB_L*3ijRMl+Q;!iD!wW0{a_^@b#;@MtSn-yGNIxddbB%C3sXulzcwX~xnQw( zy)T861F8;R8asp8=fJ*J7q0Xu5u2%)YO-G9Ar6-flox*Tx`Snvkk1#ReD4Adrp)PZ zKHm`>kfU{Z2#ECr<;rN0d$`3KG{>3G>9OhsC-2*R1A90vY_{TRDXdRlZIwz5&YcbW zUXPiNS^E3oaX3D>FUc$zZR{Xl>=2BQB-~dN{u;ASG&KXPfLPhq#USYskR^97sYPU& z=e2a3&;4L=lE~yJcVo*C3wp880C$k+IG&!qkmuO=C8s}vNrzU5Fd|tCn}sPklve<( zB+Qc2ybFLpPpM(Zj1Zdi+gwp4gjs{$uk)mt3vmMbJALglESaxR{kArTo`(H#Xg(*b z&@JDb`!l&5@%`^TluOsWpmyGx#RV_@DZE%6pb_wpCs9y)jt$0N;a-;*#l-A_M3F1O z2E^frB^B9QYVp#qZ`wihGV)x0xkvIY(U){HF|fla{8-M+tw9!^Dx3{ZHP<=R;`|oE@c|!~HrpQ*8zZ!5?TJZT! zqr)|oC^WoA@}9StAlCeDQSOM_$4D)bhI=;FJ*}cYWSze?yC4;luZ$+bN9hN+UQFC5 zdI|vK!xHS@_~0%JNt;lm98gRffX0KRvaS)RR%-BWFp`ko_H(Q0WJT=vL68U}ujd{{ zetdD$=VPf7hZa_gOLH!u^fS4|R7kso;O$d0+{EW}(dK07KR3S|jKT4N1ZKX+Vt=mo zWBoc;c$$%ar1+sHW5`81i+)Ft_tZU4RuYIV4tr!9f4ilM2HNx%b-!$=N9GNcdY^!- zd=kY0dsyUc0Vad%7Dh~=@&g(!?90rtr4e;?K+rb^nG;Jy`rdp8B=k57Dw!QbsNWG9 zx`+8Cy(10JII_jGotDN{U!m4$PEnf$ps^Vm`z6 zXd=Tcsef+U<&9C2FRVcc$?{MVYVU3tF zKGz~&Tvs>$__Wsub)S04p|Lf+u3Lp;Y&k%W| zox;U1s2EE^lSaut(T-7c@Lnsvc%_K$VGKsn+X}KkrX)Hweus$Qpz2R=M-}Va?F-OnyNu?A@fyiwRlNoh0eN?K}_X8TjCFd2+X|;*{zr@v0L3ABg zbXEqHDeUe%Ma1~#xqi`b$e?yKq+>$YS}<))D5kx*+rGrUOxw}QJ`9y~r9HN9p_mCE zF6*4|jbLL%ln@KYtIQjayd|Q^ntHp+V#Mt=E2zFUkuC5c_Oyt#Uep~9IPjzI=li(9 zXNZm_NmrPz z03+!#S!V~&W<4SN!y>_55R#MA_B7QiRz3=!B5~)($xOu)Y9$2xEIG5O&n;VFb6zPe z@Ak-l5~nqLZQy zv738-!GX5zVqIWgcYtFO#2-y4A5{A>c15p0jOXY_jrEq6{}1w*Qu?ud)8IUoeErq- zLeudQ+)CW(pgFOyOwXYQ7)+MTp98a^;AUgZ8LNeY8V7wu4cb1(V(^(KYlv=)7R7Vo z=>?3qEv@ZTitE~_#Y+erb(jkrDdIeiEX% zMfgf^nCwOc+FHZEssV>Ly|sli05pn_-Lqkt+F4LBVy&IMm7Xc5R1Ake^tFdOh5q!p zvI=32OiQI&okfMKWY=esQpYw85um#nO z$>XZT*US>z)byt`ocMJ{2~ug4lRF=*an$E3LUcuw2nQKLM{rl4B@BSP$zM$!2M^nP zpLIxXc;cnqQu}HZfnk=*d22@D1-(PS*BA<<>d;9gz~E6p2r+(ZbW6cq;9ef?weq)h zn^jj9u(=3_e)bLU*>+nG%#HMu!}zQ`Xt_pmHzP7OR$}wT-!L9O^XJg{XagNdC?Z%3 zxjmIDd$I3vJZrR1S*mQNNC?!_Me)&%7Yb0k`#1^8g5hyENm@s}nx$bXcF~LG!r0^G zVVXbKP6>S_k39;{bARa%k9&E3vRF8NXd@6% zuQoeTkJ5!q?)?rj2Pp|Ug2!AVb~1ks1{P7gZzDVl&IB_~tQCV~md^7NdHBE?KJwga z43n>My>VY+Uv!G6jYe+m?xb@Fq&Ne{HjmJ0Ct^hm{?&)mhJk24oy*JNmvY)b?n!}k zPdM|)$;Gq`#WRwqA-}9zcC|7X>e0Koi`LQHyGQv@%-tP2-I)7c(jj(Mh$^RiAjecD zk+cS9y5JMDuh1HM?5X+f&&ihTUQn+@DD;Wqivbm#80&bZ)yvlyhi`c>QsU>=sv(7g z_@@ul;|}}Q=F4mE3Q)$J64frtKi854VxQCZ4BbFlK~6DEyI6Dy#ABs(AOGQ=$u3My zA1qHcwOYLeinnsE=U5b(!)}4RZEuJ8@;uyCG(o99sy;o;)H{dij3>)PeOj*|rWVW( za%IfVny)c%R63h zveF@LGB1-SlQ@j)O@Jzi!Mc;vYGSf?TbD0|l+ALtOfrWXX8y%QcG&X-)>5k3j^I}< zVwa*l2Sm2ppC2n<)}ov|N)9?7_f&~&OsNS^*NVrtot<}aKT+U)gujQIAr>4Iq~j>~ z4t_VyBOmJn^~!mMEd1We7=N8B^%hB{q0=TWLPu-CK3F)~bLd3UY*p$t72_|NntM}$ zczrZL1X9P3aDd0NmJ1T%BdLil;z-#g3dx46(>YcG<6FV9Z!H4_dZ>F#n(&(@=Sa(~ z+cY}EG~tnaeI0S@{xML}G4yaq+3x~2u~bDWx=HArkEkbV4iT_bLz{Z5^W_(l>T}_7 z@ovuouva>fx*nWD$J0(X`o$W&^pNrnp}kyDRN+3~9HrM8*w z4M4`FBA7oTy^HJl$+{nU4}I?Q0;Eq+qXYH<#a*&u+B#Cfjnv4&n!Qo;nf7X<0CYCe z16=k%%~#muh##!xw9n|D)Ar$CK2512&-BKhFL*_j6CZGh+98_Iy>2rX@Sa)*y%Z3c z$?jph*P18<$!e{Pn%!N|TE?|HR7%gt`I_>}q9i1(S}h*c@LKB$pC|=Pf9oIv=u&q| z6MtsXi6}#HzfY~klkTGK62T;_S1d24sR5!dRXDA|j+GOT)zGM&b4EO1uXFD&<|b@UfSRaS#6as^@gJP?5(#!BQkH{7 z?+o(smowN+EbWY2@F-vBe$XO%nc*4##M7b|F@wS6z+U`VfF%}lqE3kihszc&0W_@A z|4!>1Ij0ry!%=1C(Y}NOELQc7@wAsf4Af8kFdDE~#mB_qhhp)_%wHmtsy1df>Q2R@ zDZ8Ex)?%}5=H}{-F?Ti3Pd29Swplr8BH77rU~!o!lf7cMx1@3H@9p2MA(#psgM<^xk1B`&F%l_d-%i z%2+zt8yme(e`o85#>(Noxw^k?N=1#b2jgm$()bv;&Rc}}%@sDh7qDb#8avctB$~@&ixW%w*v)T-54LpCd{P7crb>~hH zKT(W(LkMnbAe#~^k?aSuvHeKamV`+DVUE-Nlu&bK;i)yVEujWcD_i#cLy@c=m3!u1 z_<=}mpm!hVUA(8^lLAE_Q_e1Jm*ozng$H$hdR-=%a>=Hr>^6`1P_RczG~BpPD;rX{ z`tyB;EH9@0&HGZUqS4lrdP|)gYG*zC+%7vPG~?mQn{aUoZ+Y+KwI4QqVdo)K%qjGpbUs4^^`vA>b=`Bsmka^W$ zMT;uC{k%1^8HaMpIC)39b-49c1|CIacnn6Z2`2;>JFhffhe5$&k#}<+J?HR=rL_cb z*HVaWxhu}_xy&#_u*P8apT{#8=@fzB)(L<-4{;}aS_x=J^clB8i=?dLL|Jn${mdpAE2H$&JUaDsphz1@|)n}tcMvu!a4md!l zRxj)~$&~a#^WnQgHNGi<{2ghY3OW87j1`1PBkww#Ev6hXq8CHjdO8_oMnm(HaS7$| zO0;Ul;X3PLJ1#l>trOuCGi=rZDvu0?9AgSUYX=sFU@GPXtXAz)rZuT9L~skC5{xmz zC`_UN!ARiXoafWX>h+p1NMVSD^eILx^5dLdHJLzRim~}|jV!hJP1C-J%(ogZFDj=@ zUf0aX?f16Sylt?p&qPU6y{m`RH&@a zuU(kr<(+;nQE%O^P7-pep%M!?6rn;m+f(eCw;+Ce5bm{c7JTNy7zj1(;7;rNtXd8> zuQLY4n=gDbL%6H!3wk84+v5rLsz=+i?XC6q$YROA(m+&ga=(nj6%7hd1KDGZpdx0p zIRu7NC8w-I+jF0NO|N$5b5K-1%20Zt7Ts`ab^@w+L>C{F4K-2eSc|1Z@#Kqfevz{u ztbdiTZfHNBqW(_P{?Kf!d!!C*OKQBn?xv{ZQS5p73nUMzI7`tK>yz2RPkcT+ozH?i zF^^*vT8v;HV9y$%E=J(JZBb`rXdT!Z%Zupqr6FxUXZy;@G2WfV%+hYEpA0VB=bFi{ zYpzFezz!6zJ|U=INt3GKe4BW3w5`PLwGA8Y<2?X&LIXyj9qd`39$ zEG5Wpqm>IGW8KC^R&v#ISZ;<=NE{q~rsR8@(Dr~#+$|^RsT8IuUE`7!hA@iR09q(h z&dKxS)e?x#yJw88p}U+Q3K8u=!BKf(cN{ke+(`a}P9mM*#To1m&85+3@9w={h$;># zxbXP#UI~$$s2NnP9fv=VpFW0dN=&5r(8T1-0o?IZDr#nZH7PLy`UcHK(ypG6hFV^1 zj9fYKea$k^7EVQlm^cw!K~sPnYk%reNr2v`)i|(07<{zFLE?gOj1&(cL#oFv z{3dfq(N|m`oHcY>nZP|&!r#m<*Z8KI6!_anfxU+{g7s z)z*qIt$4F3-8)eVPbr#_NdtgWPYFc3k(5S+sL&w^AIin2%wF~;C^nEl3^CRt#KYCc z7f-YnOR?rc>5df)F<9!Wx+`R7AtjH(ezl{7$R6Nx-?Q#b8YKH#petts$BA}bzYng` z!E{2Ze0k7r6=|A(!wBP@6`e6LER8G%3GMwYGb%T(M^Mlz6Lq=-J)b%@+9GbjNeL2oA5i z+-{iK*rx;R56K|J!cvOkJQh5;FfCIKS2fP+O#_0mx|zx|gu}9;%(I}@s53r#eDo-c zvtM8=n&=^m7JVrSu`tEg_X(N4dT=l}p z;31Ab`K5y+AIU6k{DdmTQEz9-%$jL1xC4}H>1tX$GUQvB_P9(5%oPoR)8*&Qarw!0 z6b!QO-%}C{QeSl|y1&Jr7Yy|oIzvV`C0!dpX^ zt_7*;LwV7bmr~pjr*I29*%QM*)}l@^;tNOo7N{!o>a3>SwNP1By#j?dO&Wt$_2-iP zi)Pq6g0W#YTd9kD?Ym}+%Ze6AnBHZlMJv>+m1YDpU4G0He^+Zr*a=~q6~bNeET~bF z<1>{l&3*l zSU?5CH;65&yGN6d<{)-KWCVk#*FB8~{&Sz6PSx3=k6&u~%b2t34&{C7qzZFXcv&Ld z#MT!d-9zY=T2}z+uQ|rz%Hy*02z0CfSe^NM)j1+>XwV6o#}z(Az^SwBa3@rJTnis= zwjQ3EBd0o1SeeTpSIdJWY!3)l6{Q?hZYN(%3y+toQ|#>8z4)9XQ-slNV4nq5Yv*!* zW2!FvXj`8kh@G9QTq^AQq#HZ}^A6RaRX0OqAP)$B$-q`quHf~cPhnqJGT6j44p4Vb zH8_#z^{9uh0o7f}mnsAG>aisYFecVAA$B5=A6u@QV^z~IyWcEN?PyqG1dbN-WSmLj zTzL|hopsB`rmIeeZ9zO8fV83mDHs;34dzNhA+S9~V}1JC%lM}IW- zgZ#^cAgAiyLu|GMysdzpYE}ow=U<;|;N?$9H>(VYj?ysn4M7&BoqvujG4d;`B5CJ1 zN1AKe*!Zz5>Cef-26tyL-!rC(=x zd@;vd_!B4$j|Zk*Rx>MX+2wFDz>2L0oqZMD0=G(vv!tX?EI}=Aa+C^TiUy>C?v3~e zWB2Gd(p+xP$ZJ~Cluk4kv*|tUeP>WdrJ&>LGzFaX;0!QbS@89Q+M{w(0!MJQK|fp61!T- zB;E^)M=>KJk;sJDComG~b$`X19A8`>Bs8&Wy>c3M1~ZSdg`@z9eF#V_rpy4$n16}* z2ux@h8%g{9ADD{7@(-`ul-S&{@s-=lz;O@TV@AA*8lMiqULLnI`7`%|*2!qG@UIPB zmvQkZuM~+z(Ndi#>x zXLIM69O;R771Y(S9%g2M>{Q?s56}TUNlqvcBA$7AeZ;1i@h)o|a2f~d`q5A>alp}~ z0TJLx#jZLcxsIycX}je%XX!&{o9S(r0RP#`2G{$-!FZ?&nIfl~Cgr)}5$ps}q;eVN zXBG-{)$I5WK-)C5oXOd^!cW$}R;?)?#zU;PRjr8t>$xI2A2%DVjaR^%$fmX>D4E!( zxc4P<27lO|BNS*p3LtIOdPH$B-=C|{)^FNPiO206@DLtRG2MJ3i2*Y_S;9=!p)ttDT=s)dbEZq!30puB1Ay ztbn)6w+`Me7UBno78UB`GAOFJ0{-;oV}){E-k9zr`1`z_0RjD5@@}Li(|rwL2J0J> zBPHhXNUunw;`y^zVin2_5u;hupCxN1?-HY5{s8v0g5X`QbXloXHvItDQzdv$-w5S5 z7NVQ22PZ@V5C|ahR0{(*@x6jtBQujyxZ>ix&gP*#HR>OIiNElBD2E5jpavcL>PZMD z*W2uV=psgtt*oDBXPW?U=&D>2a%@sI2fE=P!jndqad7$}vcueAni4%HcpGJfP{~~T z=piSbC_#bZ&(^z90ViQqyDR9`+FuYf{=qsG!M8zgbfv;{wEF&Y)?Bkn7^Q`-HV#@4 zL3|_loEQ}#2r-V&+rNL&Hl_P`p13ZYCcT5fVuLaTjemDigH>qTJ^|r-_Iaz0{Fe#4 zHH?p;*ukg$Z)R&~X@Kq#Et4J3*Z1?YIEd8|!Aua2$N(S4%yMOU3X8ZXHt-DSfpkr$p>g- zGDLB%?3Bb~!nC>pZJ^CyK?LEo#P!InaK!$~YZ(Lt7E8nQTCW4uZB692Ecr)hs7{)NE!N%$)SmVBqxM4?>$0 z-{p}?Tdue;9PduKI_qP$a)JD@N*YG=E}xH|R{T}=LZ<=I)0`Ki6(x;=GchA&@O6~r zV$shDV+7NJ=r;I$xLFxeyWSad0Vckq`GqKLL zWBO>=YK#;yKw}lISb|ldBoTBV=*IAzj`K|8bxP3-uojqfx;CH+Ux z?2hM*C!zR;xm^G>$N)5hE?F(=Sg2rh-YIa!W|`8Jqh}suv=qNefO6lSz5kx9$2AAKBLhKpZI$Vl`tt}B?`H0b`iuduFLDgIV-7t;s{q4$LiYOXqV00v%}N{6*7`8*KL zlatI;ey6|0_+BpHoaEm>eCBnN0lnuD>ZV^5_4jqk?d39)0QEEk4YpG>9Vc@{ zeW%Vg4m=P6I@ABSg|xNav1yg`6@QY!LIok~VHbkRhH2kGwY^1+7>ZNG6(=)1eRV)j z`~mB*#@(FXY5F;%n55olE#NNwHA~7IyEmJvEd6%Lk6a8ZbYPVbkJi3vNhh)3Cz(hK zeY0Ptw*BUqKR?hFO|FufDspWVZt&5+S_ea< zmXubYHCL_1;2~N#mKoc*v4k6Cg=t{a@$THz`KdDwMJ)lMIBj1F#FeNRR+KSQUTfhk zO!?;;uY(&sXDtanox(Sc_QY@Z3=jmMkpz*3uO94)X06PEplkB}Lm^yAfkGU3 z7SL3y{@yqC4b%RFmOp&HW6$B8Zb$Uqiyy1sClI~oXEh;lU#hCziK&xF zTBaiCxSc`=Z1(;1*bqC9Pp98#n;^g5jo)QkI&&Zm;|OnWUL^ANe*AsOU8xYjV^PSHvbrPS6&ec^^Ga*gmo`m#LzI+iggU(YOmaVZ@eqA$ z!g0JqP724XmCg$ex;k>7l|t}R$~O+|7NU61BXNSpg6%{vTHBtREU&z2j3^jNZ70g= zl$95qZ$6fCa~XeNe`aewzA!U`v)IxRelM^z<%8`hK1xqC_Oep8pkEBdQ2*XL#gFb^ zL%7Elipv&T7pY~F-OlaiHh2Q?$qp`GEt+p*vVi6XV#U*tJ_@JJAo2#`)igYgP8M4pegMsp7ERZ`+N02c?Rs>H9&=wf zO02szP$sZG?Ue)2a%EuUE=ONXc8?4m>QfXWrf1LJZ`-z6kV^3k_56f!_k=~2y=z#{ zwGQf0(VPjbHp zU@ZmjB9_o*rx@1u_{1T(+8%R<`<*5xUtL(cgqwdPvFye z(MF?Q1()I%+E_Xrm+Qr4zeBv^seVr^j=K4@E83qD(m@2`9@l-)u{A~~iM`44>VbYm z8Blxm@Zi=&<(e04KpR6Z;5&B9KTlK*YOJ3lBexx{vn4W=j($MXfbZL`;~rM@xIv`7 zpOVlr2qjnb9btQKf{*1XJUQZPax3nvqa$SI!i#K;_H(^4`-z%Meri;?=HzKFv2!aP z8{O{^d$??O^6nWpy8j}ph1vl`!sX}yk;6fa`!$YKKnEJnz_ZoGR$I49^l@ai zBC4iF%>!G5w&sgDGWj%P8WIS$wW&}~oMn(Pj5bSo*J+FI>P+A>SyD4R|} zh+Xm)9-QK89_4eA4UFysCmlNvH7oI$F@(k{nw(h7{V4Ak-IWUjd?9D(pn;gA*B)W;!N6fb_XGTRIb!3vUsxi$KnlkGV!Oq#md54k74pnIvhjxr#U% zF1^w2J=EXNlCZ92V>gOw4cObLNrw@Xqj|_>2QyE2;KF&F>8cs8hhsT zW^{n5N=5NDCJoBz9ut6c%_}y3BShP078!gBzC(`&;!`h`Tx@pAN9!D5xg3keUOQ8| zjq%_i@M{Ah_iGg5+BfUBzo^%LvH{*Q0^(xuZ7AN>c5ihub(nT5#NBMC!9oT+gL)+N zJ+tRwXsW(H;Xp6<`!S%v035PDhoLvf*)!iWHwH&KKb{f-3SRR(TxacY3brIswOISz z?wy55dKK+{o!XLeStcKgZ-Csy5?P0Tv>yxRab41&>a_uL3duXUq0iD=Q!|OkvA(DG zjOCQabkWoQ8!MA0Sjq_?hBrpu!FgZUox-lP7d>&W4LqxL$qEY@Z}@7PSYW!*nL z5(y4ui;`B^#Bp@&lGbyH3E|cp>ohWs2*qkh`OxW`)Wmll;^UPf9I44M)wkoNs zy#cbtfx`RHhcTFup|-(|F;-tF2?tsj`{>07M38sUzZEE2yK%T3Gm;)|EJ!}Jo*My^o57gBviU|DpMb0xT%xFgvPsRzfXghxAQHVL{&e&v;=_bb@H zUDJ8Ab;oJbdP)s9m<0zK&4?xScAEcXyEp!~JBsS=M1&Div`!7nPS@u+wDjNa=sI=# zXRdZ5QW0QCkaps8R;0M6Y5&r9-k|C z6Rb;XV+}C=%sm*7@n3^R(^`3wO9E|umy)#+NoU#hO=t#CET+-F{zCcW;t6wr~eK^=7jpL=06 ze!?@Pt>$=iLJ9Vr@BX$lbjwL0xdXoz%>pl#?ICXl?Y{j6rxOH;ODhXJ4Xq`IDtvi; z_|8*<=PzQ68bTjs(0s{?deIIJKg}oUq~3x^P@TS?`=O+O?=xX-R`Q74D=@)pf64R( zb2#&c-^W|+=Q0DA2NS95#aG(Ix(nv4l8KsX-W2ckQxEVmw9|B_7vGdK$X7<6s$GZ#}W&^FOOIb zt}!rsU3xm7>1C#m7YV&7wVq-xXfv{H?#t=1ecMpfP$jZyzjTD=JpNaj6<^1fT@-*Q ztZ`s2=Vm@RiluavAWaZ?@gAM~1@z-Jzg!TIcO7?s8@LS3mo_rN^f?JD*Y$bY>8%~e z?Rd;ze``7hTNf?v|8?VjQ$)dJw-$Ip1VjtF{6)gMQylG$l>M zjDp8IrDCSP?)%Z02!oeI71y=Z+FW;z()n;TzZd3QV5=vmrJQ?_Ytf?cV%wnFR+4gS z-mNK2(8AKO748ldht6xusqbRDf`j#stn#`!v&iUT&nJ0zXJ$AQzll?&|jz=21~OdP`5clrb_8l{^b$fjrgQ%rh@JAN$>y z7OssI&=1tg&?Kamxy!LlH|j-Wg;EmUxpsVnco9YHT01|7k`Z<>CmJ3+cFjMv)6p<7 zt^8VEpOL(PO#M9z(Ar!aX!Yv&&XhcDDTB!`mx#(Au*Ev;!B($Jp${u6Jk8g&#uVq{ z!?`;cZ^o04gkgTb!y!f+UY{eDbLj;fGw$@sB~5H(P(%5NwFU0IH+okJBqf!M#3j9_ z_Ss4@Tnh%^`0^Lu+cT@6-Bk#-X%&1dn1qYHDM9^p^t2NFe2U7z`nYd|-DawL257-j zUM;Kk*gGB25kq6BK~^nw4Ih!HW*u3m%LYY6CrXB(9BQ^?q}X9qe%VP@FLJONiKINH zT?_S^^PTmy-T%&b=1UrN;||L}Z|W}t209Znk^k|&?aV{skw`%~GpB?6?X7W|Z@lC1 zHBDnnT*E=lL8;380lR35@pc~dt2$6M>};9@)R#GVIo`9|a{7(Pmz-Bn0U-b4-+P@4 z=%YChpoFB}&0{NwdNW!Q+!!$;*qSmT2eCYg=9%xWN!f{mR%(Bq4R++RMwMO*vqbTP z@)H)+8{F!~<+L|SO=n*r9?0~HOfMcmkQvxEp)&^{bU%)L{!4rVYR8H5G6hc-3CQhR zf^&$J$?Wi%;B1*s2B}w;5ZKP1$AM9E_`dj*yKg!Ic-;>!R#|XefCcR0}yqA;RR53 zUKp~cc<&-HZS&zFig#6VjJ?b@N3V*jw299qo%{+VELs%2b2PPAe2_Ha*Scc#iytOr zD5@0TS6OW>$;4i9q@5frcawfriY;_z zXNAn~d*%kU2S3+*q}fT2fy=ch=isWqwd0l$ONx_BHWV}S zUJYj=!q8YN7q7~p|g7wbakz&!p>gEKIeKU zb3Xyg^%DKC-VsT^P!WyztfW?lcIo*;Uxeh0skE+Jjk*;*4o}Ba}dvF#;i+W8!yAjzTi9X#) z4EFvT%v6w{I-~YqQ?+$5cGfLYDi9Ym&6xwdBb=*XErGnhVt}|0G|{g<2Q+R^P8x{3 zo%dQO(xG(Fy`>D3HG}3Gk+Yc-?7PYAg%HOQR3y)2xi26=M8!5c_N%(>-qZ6uG?W{^ zFUq=MF}6Pn>E+kp!|Aj#VGYr1Ohu%3t@OCJ)3~qq z+xu_}xcmgip3% zPI;JbA-39+-#GqpVXQ4OZ*!g65#(3&LdPEa*%Fb4w<(ykiQ0(Oci16L)XcIS{ka6{ zGgM+I)th-?1vRah`&d|%*K5qN8&NPm>=E43vDID$`%(t;8g>LX)T6)EVK^ajMO2%P zqn2E+H;@qCv#^n?1ViQ_T`8q*OAk_#Ej$PJHw6)Uobi6XSKT26L_YNmo6CdH>*Zgl zcudggrkCdSiXGX#jI;L(HCOoZS(0!wmdl!C>z{y6X~+uc%rSUe&*&`sd4b)2pi+!c zTls+`YNN-g^Cnxaae+8o7m$@Z7qIyD>?3vOk#4)yeJw3* z*xO@>XjK4y&j00rXr5l`I%4qQQJGDNyYF`nZjp;7r(`5~4R%6Y!QG-b>a}%!I%mm{2k59Wl?!<_6BmR18m)Q7iV*Vp_hK` z45*l{51juLA~`>jRstDFpvv+<3K8rM%s@)VOAEC#L>K!Vs;YX3*2p9fGn-fhVha2d zvnU1W+27=5t-6uwp_@51lnvom@EP8_quuLK?w4@UD!U4M?_KT4#_j8#8e~Vl zdxO{^nhu9eyw^vJLI-O>ud&_lQND3*#k69SFoUHq4L(BWVVeFjo=)-yhQn|+n;0IntOv5V*1QcW;OT`oKMVb$fw12A``&_`Z%3L7q9%E%d{8lOO^Darju-<^X zAzQ%5N^OAd??lj(m@dFWOD)iIa0upYrIm;zHO;7F z*%c=%JSU8nUtxb3&#fgT+@HvT`~D)YuQP2u4p3B%pb^X#&^Qj9+Rmcz8BkGe27nU~ z_ez|Du}#z=*#N{3hFrZ%g3*sOS!mdka0Yx3ElEB z%Z{lUU=-LoTubTIP@ef3>gzp{=~5GT)FgYyf*>;Frb1R~*`)%Ohii|WH)dTYcp&F! zf-h}GHCDF+PZ?~wA++<-` zjDMpiPH6DF4Fx?Ek8vs^`Y&)4W`TaDJ8DH$ZY^f#=Mh0 zK%dCMfA?irpHO)b0isB4(=zmRYSLI-6hOCNaZwhdUU^6Ys^)Ki3%x>Q3%2Y8f4P;9}V2>{at zXS*|N8TmFGyIPl7{17yftN=*OPsFI4<$kb*xGdZIDthmZ%~YwQR$QPBg~S?1lipB$ znas`6IrEM>3k2?A1&a?9vd;eUS{wOET|Na9mrDq4fyTTD13ziVX8!Bs+J}NMczX+T z9>;>%{C%_l2+_%1&dcCO|2iDd078U2C{PYiG=JllPVi@PcJEE@`4`t2A`0kB{oZDM z(*J~lwi|@NO_UtWjsFg%$%X@9DwhwYS6JEn>(EpJsG^n-tF!+yj{4#NFi8|0?g9VL z!2Xl5e~_^MU*?!*2&b%V_GgTmjvWWMp}Zs*x^gA`HSJ}kr}=-w jib*tJ&JopZLC~zBxQ~2Y8*=Gtw>8cDVarq+x-?I_C z=iYO^`+ooNjrTw9eeXNQJ_zes&s=NG{LMMnK7sPG5|0p{BHp@n>(O&b(U-Sw!GUhw zx=nKb9x!tI6S~K(TcqRmVrupl&W0xD2DivqM1K87#>{AJWoJ*uB1*>047SyEGO?o9 zx3aaS*R{}fwXy_$w*Vdhvn;H@CWa;k;9oPD8JSt>7+L6;*_D_X$ykI~Ie;IG9Q2&* zEb71J>lztYUXzuvay2nG*Ck^XXQO8XNKwk@>YG^FTiF?tu?PcyKex0uFbDnxhJl}Q ziog#w;6EltdM37CGi)6J@=VMu^o&%%0|^6tGb?}^YzpvU0*1s*Yz+*poPoisS--Ym zV6SWRYrFb7%HqP-#%gwsdK`=rGR)4P=g)u5aWt^CGqJL~J{1!^2R-Yr2lg)32ERu2 zY^|K^3~XV;EMjDgLcnsc4~q!cMAyhx7vK-0rLM*GWO?A+bij^aEHk-gmZ}U$-@!pd zmBm8cR8`2;)tEyRIOE?_yP;{}EMco_Z7gF2zG4a3`PUw}*Z_hsnqZe-18khv`!uq> zS>}3G*npym>-Bz^fW5-rGjcEi8`%AN_j;nem6f@@iS^$P>RVY_8t7jO?s}rGEsO(y z_pYIp`Sszh=U4;h{<|5l0Y!aX^FNQOnt<&A$-@+mlkIwYu-TFZCPv0LD+7}KwQ&pG zo5|M$cE-A3E2m%YUk&|HD8JZv!%SN%D}de~WcJ6$HUYD(>vaHK`agg7 zH#cm5YybZrau$xtBFbVM%F@<`62{V2?9O%^x;NV8sB7+U&D>v{va@%&;kK=ngC+RN zRslT~b}}}xH&C?J)rU=Y0@Mo_Gq$%d2mWFLegzzYxrmjymF*QzS-`9YOoqSg4yHJO zf!G^dYvEsHjaX1Lj>DT*t3}=EeY5e-(uFUCd1Ye`d?_ zUpq6sU!L!U-i>$D)73XKy7Gr|4)y>Um=A<`J-c5a@(N-6g9kJ;G-TC>t#z$A|E(U7 zndKi$?MB3ZFtxw(fd6`i#`a$u+JAyTS-JkF2$cO_SD>8w2KxH{mn@X&2Bcm)U1rul z2g4g{{9`P`pFzNoUWivDX2|3?D;(tHO z{&N-S#*P0`*w-rYziu4xaKB5m|9-;$b7|_Ygng|N|B=G}6XO2&wd_Bat^P{b*DCQJ zDeOPR?0;Wj|G5Gp3v^ol!vXj|hH0jodh@@Z=l*j+spHvX;`ZEIuhYq5D z-AVrZ^D4FXGiXI`b}0xqCI0Gk=&RJjN4OKc-v*F^Y3pCd z887z_{%Ka@vQPj47q3`sIZo^z(G4rUNxvqRX7De51grP2hlRH(PoL=5KfJ|B^IlF zv7U6pt}oIL6oDg{8YY-r5rc^`iRD{?`=F)|Wb@lF`ix!^1JFv>E>=EbZutGyLOT;ccQsbyT1 zK8@azaUa9AYHo-W2*rDyz)_GkUG(5_DWk)gbt0b~nuxeudG$q!kzBFu3S9Jg6?eL5 zsG*aG%8go20t+Ik&+M-a8_Uiw|f(z{mMK+Tu%O@gFUc9eF7Zaeds3Ag-;--Oq48ip)$&!1uJfgF{ zsb#nkv>@rXU)}pmsIl2`*a-gmtlwk+PNl*-noqv|=s@#ow8}#=;i5lP{mmSWLL_*3 z$cb~TG&`7~Q>odTt6wQzmJcCA^b>S-TH^6$OTe2+n~QY;%9nEw?N|7aQkuhjeC@L4 zW81N{D&237b6kefj@kyUk0>>V;Uc%#WeIQieVy$u(~wW8?|JVnR|1yk;f1$ZI*>uy z9Dm^EXfEE`J^HO~{1&(~QY1#J-VokU7C-ldMzaQe%xR<0l7k+^sWGuxlL}QrB@q|x zy~tp6nUFu@*7%7-L0|W28r|V(*=uxATNs|;fN`Jlsja4un2J;w*U|oNOEAhBLQr0v zNNa{dJ`|hB;SJggW|EN#N3IuV2F33)&_79I_B2LOTx5gFAG|N!3~5b2OHR^=AW~F$ zky4Gupi_%vlS~nY@4ZgpR^Wc5-{DTHUD}l@1`4@WTuqq5J)71c4_9e48H~-?8jL7! zQK3D@P&8za_~ebFB%Z5(_-3SR(W?8@fVd35F8x3;rakJZL;b$cifzo!-Uo$kx3l{! zSosQihI8*4j0fwo(5&ycmOVsMVUF4#Khu$~dnrTIEfqW->HEw|p;H>MW-LbP*^jp3 zZxv#0>tkg?;33cEFp}YBy0UCa3QNq0*?`$Bw0cpV2z_*@1#=!tY=$T~vN=tGcE|FsM0VwNSH4 z-T>eU=L-cxv9GPXA{H=Up*j+T!)8m!S4cVh zbS;?NaH{+HyK=A(8ptlrQZP*OIOaaJ1~2Ztfbn38FoI?zh_ZV5JbI(=!>qS5zL8wT zHd|b@C^)2}W$2o_Pf70fmD0owSd?YS-6?Wzc#(H%hT~{4{lQZwdt?cl-|{1+9V#0Q zddsRs8-x`kHJE~KTYE1|{6d&al-#(aJx95*-f4@nn6!ky#7iejbSP1WJiD&RA30nd zBRJPw<5ih?8cLMO+mZ273lhhmxI7YuN{$yn=}uOc9@!Tclj9aad3W5LQboE>-tk|` zEqv7xjTZO>)gbgh=UR50P14}t=af5Myh8=On6SsDdYQ>)EQ3jM?G}fhq_#7+J8y;r zVozj|I1*I9!DiB{Q_gkqk#aux`Dtv1PHbq%C(g(6(=ltF_k=L-M4gM)gJP#bGP8qM zUpreWJU5^y`cZ?0nZ`WsuaL!M5`Ef;$1#o`wcfJfk(q*vPT0yFHp#PuJb$yhIj8{+ zqaMx+@6h+x>**C}kef)Be#c1_Nh*U%3iaNKp^o4pI(d{5P}r&JkxLb?TkW`ycJG2y z>|E1uJ&a(g5GvhJh&+f~qjh9SzBcXVq(G~R7}F7i#c~xwc|m6!9+OAS z(YVxsjXdqN9@;s$Mg_s(Em2~WCbmZ@f4q2+Wsxwf+v=^H(}tx-*uSFKHjpnyCGPDN z%peagT^o}#s9vD!{G#y|YPr=zXS20wQEDYM^>Y04Rt*~EZe;v3s*eX-lVN3FP4#Dk z@{d^U980&PJ!zO3qcAkV9WUfppfU1=HZ+5+RmxovoX>4bJNS<(_(`GsN?zk19jpwX za3wO#87di?x@1u88s`G2%5JItnk&-!Ab|zPjh|T6gkbHESVY^bJR=E>#5|0YmPdmv~rBxv|Yk!wQ)MBVuoL0M8 zPB(DRk&B%Q3dXeRYGqqJFba7`n9;{!ux)Tmovl_HHdaoygr*9v=4b6iINVYIkIpvH zN?qZL8Gbe|zNKDtk8)zT%w$8*%lE+_SAOj0N`X64@Uh&Az>c3>5(@Mb zU3ru}k6v4@j(hG#x&XNPWW8A8Jd#{3nce$0J)Gr9czBU6FqiC3AFW#Sf{ zXk7P29-450{+gfAyvg{eYG94V+`)$l+Ka+==MHI45e3VjEf#0I&ERo|hTg(R5EE4+ zTpKNmIy1}RW(AmBIuY*!s&J)Ug*x61NE?Gg%UMb*qfMoLB zKq+C+M@R1LT-0xa7%gZX$2&!kfum8U&RNzXDEwKNX~dGNSP=S3EfYMUSEBQda%&ev zx<}yi3DnD_BSd*ynt<1J@jE=*f%DFjx_urh^E*pF(_FZjRfcX5rYg)4L98R%5q0fq z0w;^8euF(^06!a~L%@F01n6Tk)kBYsd|u!^PWT)Ck_4bBiMylB4T7A3!NZ#2vP7KU zz)lO-Y_a<{G-ZWB)B7n}e}kqlc(_$cXC!_@I28hf4)LS)_csXg z3(T-w3N5%;eiIG_)@E%q8vczSr(n_A&7v#*H#9XS1)wP^e>M4UXbMBlQ)U`HHxXV^ zkOWvebbFNT_X!SR&{TWe*6=ssr@-1KZ+i>=22Eke`EV?j<;J!>L4mL?*wqtiKk9!TDfo_vQMdW- zi~~9m^rYIMhxxv3QZ)t_&%>s>_W5>N?Nm@osQod)DcPIy`^$)R=pcJhG;6}!T5A&? zEbi45L9{`%T6Jxn&tG5VfVv^K#8j5=As?!1mbo_$4`X@WX6XBtm7AX)d%-6#t3Jky zml%io4a2>%r?wlyi7$P%D}rImlz9$u%w zF5l2-NjKzmfkVMV78YQ_$}Zo5G+Z~h4?*%-2x@A(9zZ9Lv22-3%M1lrB)$Xccg_`5 zBqTV@JuDU%PiqJqC0X_0^yiA=M1|`eOcxD2 zn|4DNRk_3}%!Z)ySmKdNMWR4RBo3rxiDaZyw=>%jS?GTp1TquUgli6SJ5}#ou0;?B z6HCW!xtQ0-_2@Kq=x4osR~5Fl>M&n_xaRzGh6s&JCS=4$kE#KkZVZrBw>YH7xkv7jWs>cthRmB_87lyr>V_)y~r8sGVCcGH4r-A z6w<_q?}8sdh+W`5pcg82%#pZ6at5Gp#+a1UxzFop*0>{J6;8A+=9XI@chV)+TVqH( zPHN7O@N5Tm@WVVldO327s{aFPzLgWl2bv@1s6|U@H%qS&gr0F(fB~KhVWnSQV6uS1 zT*9!1$38ngVb6b3Semf2Iw%e9g5XrzdIs@UG820h{FIC*WhZ`H;-Z<@^+2osn8A^d z)29|eZvIPyBB%QiX)?cC*qlzZU?4+Ku<8TR0PKdvdB)s3S`Bf;U9=Bfjd(&3evBy( zb8|RhwrXUwr?)e2(rK&jHk=~}D_OJ}5*4rqsg`Mib{6Br6pGD-7J8xw(7Zb-PrQ0G zf%MTycX4%Nq;&pSP+Y9uTIfs=u0TPb;pgan40kH%=Xc}KBxZt&A=av)N;9MV*`Z&` zpM2oidtrxJ-=p?SyFOhhKJsWampGgDt@3+PLK3{6MmdS__J>mHi4nVc&>6sq$tJiit|~p<27c+v0NCepLOKBt~_bM;~?R(5bcM6CMe#s=+i7 z8jV}w!5D(D8Hxaq9mij*e|){|@zYxO9cI6ucyPxSsjV_O84e{<0T`ouEGh@ycq|57 zF`YBqDJ<6kx<l9jjILK|Ccwt_Hi&P9#ES7%S&aA$^ z_Y~j9_lxP^CJ)l`OI%1(y}j9`%KCcSPJNnm_|7u@+t`}W?9=Zse%fnHou9K1)bvhh z?En_&R3BHn0{LYnm6_B2a}zmlCK2YoR|aGAf5w->>Un1Qa2JuZ=<=J%XLEsjJ7^T* zszhaT0V47aT37s|53a7)Y4!OQUrj==tiH5)$R0$S(2e76K@7FOUEk(utT2q=vbWT{ z{AQYnxuNT)e^368gtrYUQ8ewCuE ziGd8b$WucrZ4;Oq+^gvjrjsEQz;|=nY4Ok6*3se7r4(x#n z+FhIyqoK=d-OXJP0F#;i-NZsMh!|wC1`AZ=i#96I^xZ%pO|py>6EDfZ|Fh{=dRAlp z`?%rej;GHY;B|s%%M->n@Ruapw-(}{<~!KFpI`{J~TT!d?JkM^_s z$MU_4Uo3`YkEOB)c^>;}_yflK5?ndhZMhY)^Mz}{ueP*R-45_7 zcb7i4Lkr?LnR)Ir5{%qlOG*a_po7xJNhZu}Mw#Oa3mMMfwA0h%R5irt&QFU@9vm(P z#|5A;d1qQY8Jk%k4#S*#ffb_O#%M+V)hE+w#;$1aUb+UrL3H5v}Pw#5Y2vs@0~Hs-;hY z(7XMJ&eh)y)~J^CmmZRukGtIUq9+K0=wr!)LEk!6)RuBD)k4|4@*28S3+3xc z0c>r9uzle^GmRf7%J#x8JDn zRd-PtuD3Q#tr?n&P~-8hNB*4nYBZ&KufYI zBK7p=HfP&5A7ta6YMnZuo?6;|vNco5YZzeaN8uNCMyFK2wTp`WCO$9oVXWSd1<>~4 zUfb|sy`Nk)=GLTCjl(>q-vI&Q@msgtz&t^H*_2A*fWYGAzEAvB4tZ9*tIjqZ4J3y8 zGuxxW?qxzzei+_{QV;`~3~5NFT~Zg@h35M38rO#`IM&O!<9FNhqUKzy7g={k^wVaD zgLh_{7A7!|VRp{@68v13RqwmC(s2#-kNm`7JQv%$22gbe(?DkUPQ=@>x4(|Dx0HpB zGrFzf<&7O{rC?3G+5iiNymieR)6W*aTYubvz2U&7`Z ztd;N>k>)0akd43My=C7F(`?#s3{V#bL(@0@rXss8*|Rc3)T%&@fL%x3I$@W-8rg^I z{3X(pFQZj2?BTyTITP}FoS>I9KUk3-Vo(uCIo#t#TIZJx%u{)!PGVWZxy%7gIhmY0 zmR5?;sk9Ia(AIN+U~n$WXdqKViY=R_o)yP%DrmatmrKOdd(K-%RGg%Pwb{1N#O(Yy z5rSyV-m?RhaP6gVLcUf;R3p!na%{WHcRq>7!UrU|@@En4dc4kix?Phy0P-d7$`BjM z@DEogu;%)VKKk*F+hb}-466%TcRV{5Gbe)fKJ`#r97Y{_IdXX5xgudkTk7Vjs{bLO z_=c@RbUU@q2RQn~+U-~gNeFq1x@25dK;D;PLID>c`1l8kap^C8(`LDV(-&UrjkA z(whSr&FADx_rI*h(Lq|AXQrG5VVqU}JfAo{grE`cl8HU9n@Fic)iiiG+mA$*f5EYkzms&fM67bQFRr#R z>h&^EoD1tX=n2oqv`Tp}PUFIXaD!NEn1kn!0UV4c>YuIN_dS&5)Ha1LpD)X1K3#*4|K6cFENa|lcjP&1 z&JgyzQJ9Epu}NKtGim5V4ZYY9OgH_dE)dhU@*sg(p=9*2RZ3AG-RPL4>yZ z7%#2Q_r9tiVODU~xZ}M2LeqEbMINY9XdT4V*PD0@uLlQSC zG`NfnBrI}VIg475#Z3qL;mhOxrCFZu9PJ-a-_~bJCN*XUmPVJTf@^ssTkqJ_TjZG~ znc497QOUU@`_^RL)8Weel7ao9JyL?R-3so}GP`^r8Gvw>1#$#(h6>U}MNbX!iyT|j z#pd>yQ}oYs_NBmluG{x563$+0X-rQLbiZfG<-Q|5<&Ct0xNbMRl4R=Iw37@MHuh#_}TW+lARb~K!BbB3{e)69F%ZG?6`$ckkol0NnxW!W$TFsi< z5Mr^nDbrEUX0a&Y6hBHXusQ--hFbOEjgwex^>AhFID#fz!y9oNWW3jyc|oIqQP9}C zZ8tkDohh4HP`$U*tbemW6;l!Z5R18;t9z#wBt2R44 z7U$+*$@UvaIdhy#BQbh=bX>m;5nlejs8PYtvl!`3W0@F^N@ofd#qfuA9vb(h2Y3Y^ zUV4&`dl2_v?Ro7g)d=|3d(G5O*G$(QqQv=TpNJ-E!~kgz*-Uh*q8|NiTd;Ii9sY7q z*Ltd{eMpseVV3(l=1!)(Ji82DY~+wg#LC3cD^wNcJzpB%f#{o54zJTT*<4~;#hvXZ zPV#IfzLU2@L_X!{;m*&Lh{!Fgi%W6&4jT2(rU<;Cz|jW-l_BcwEL}_loJJgffrwUe zQpeU$b?4rm8NFlM|Eez*qD@F~f~YZc@!>!r-loID7VwH9CE^$m0V{Hr_wKomU^}_J z3d{G!=@#M_v5^x?68^>(Sb~9fNDw?^uPlFFFeq;$e8JM4Jk(VXpFjmw3DPH9Q0=6U z(FhX2LY5e1csRy}N#gtUagO*&v;kliLA3w$bX6qTvdMKws)f@H#}9{#er)lrQu@X# zx<2!)k+c4J#PxZD^`2_Mlz9Hr%x_lPNO%)F!JXiZDK|L$>dUu^Sq!te47>UR-?mUm zTv?S)KGH~e#9}3RW`~-kPrl!G%diS>U4>vZFvQEaZGVx&n=eWx?jpU+XlH)#K`a44 zW}G_nvMQ4{LfQ_Mm6TB+wqxNKO+geUrY?|0tqtvwyMED(@gPx0Zu z0}U>#OAI=7!p&-A z(e3Zq_2FCPF3$P%BpiE1hq_IaT634N=#=l7kI88XlSxAilIMYQwtWwxgJ~?rKc3Td ze=O-46FMbSoukn2<>L64)-imCKN1J%<_1`Mt_>d)&~5wxCb;%z)TwCX^3 z>Ge~ARHMGUiP$Y@oyyLXSKeW? z{Jv$}Z}XX0@xv>2JbqV9jbRzVgFKVIv~lMUymG;FdZDwCcMPJS0=yN-ZvX0Pf^Th= zXYi9OdSbg|*PImB(mEsvqgbpO>z%`=2rDEI@VbYv zeu}J#3$5~4s_Qh(l#Y7QRr0{%YZKrNbL+oMuW<|b6LL8$cE)6VE0}50js)v-?L09Tq;X-oU0-e>l1Y- z5pa}ly+0kLkxig$2WRM<32~Gqi`gK0jeT9&KcqmbUZRwuo^7O5ginc51AEsgR2`{t zQ88F%g{BBq135N!S}z2X4a!|JkSCqYc`4@WIO6%!iq{3FFVFB%X$CA4#bFMYz}o%c z{H$J*#ktuWkK6WMOYnkHuk@o06VwCj?^k(#xzUtiY$mMa)t^G!KlF2_-v)e0p9flZ zXYZO8xYhS0y0z#wT_4m01Atv=+|yFYUJhHGDI-oZ(Hdey7e1E(A~H^&B#!M(bg)ED zN*P1lj!lykvaRWrET+DHW)Ee*stw?jMl8kd{cw)RGJevY=afab`s%Vl5u44EXj5A| zkh568aqQKvbh)1q&-@Df3Jn)~mSUDB5b1=m9@z-mW!-c8F@f=+R~vWj63-{H#V)sO z!388D-XpZv8e@I706$9f?IU0fSBi!7f3yx)(%9gd;I40y|+5SGFR&H1~w>KxEC8PQFGS6KQL!;#=e#rG5c- zg~w?f8e2+SyU(1M-QA6@$nuBMUXzcNsWDZvYQmq^-zwCuZN0a1lt|Ln6F6=q@T#+- zCjg&^+;m{g_qjgz+?uTz%1Tr(3X3?ay}1Uig~p&fa-?hudpw$%zNh+!WCQHTcEk_b zC7xCvo+mYy-}B% zq8}&rt&NwE9FB@J;%Kg_B$N-0)~@2onn-NH<#vMz(hAe9;U1kYBeNFrj9!@jtu)BU zNo-c7?UMN_LNeML>ch->1UZ9sZJs%M*fuw8yVajsy38j?%P8WUyQ6Xuff_(VzX2)-PpWsSe(l@IbQQ z2}ec9AeR_!;6ovD)$~7lr|=wh80IF|FiiDFqL*-^)Zr|?Ro?Db|4j@n-thU)B`1tkcPVI?Y z6SGYMgZgJm>WWF;dd+jswXh~il`0Hvfhq4URm(L0Ep!`Po6=78j$Dxhbs^%lqdKnk zXeP3cy_#w4Ln+*xaxn_iDj~{^nZ(jlt$3X7?Drm7hVPb|&VBOJc-!zO4W?S7rFb>z zMI0@6;!Wblk;q?Yc?)?Ye5wV7xzmhZRX>LpD}GgbdpFLt>zr{&Tm!9cYgdr67pbZw zjD+4Lp?bYQz9)!^!MVMFlS|zY&1Uz$oV^YK3$#%c($vG(_KXlOiCK)eUBey3ocS0H zL~+pDkcIOy$6W2xuEoyTg*e;C;5f@?%0Hfd45r12Fknok)HZ3!@6f(e^~~^Pn-$GF z6VWRN_kzY%R$~ecc>$p&qpeqWcHPs_D_36Ps?zI1{;Sg4KeY8mtGd{@)!lsbGmoS> zne5Nmaj>nvTWrlfldBqTse)EeDiDJ+GzVpm+!nEmFINW<#8iA&lOWs%Xd$ON2s>X> zytYB{TRT1#Uq>`}@2gMIBcX$OlldfmMcVkW;X$i*r2P%C5=$)YdK(T&O^*YR=YW~g z6A&VL^PcK#>FDfa*cBM=s(2my#@Uwl{u$%JY0d4`CXE@gdiN)1wDnF6t0AxgB<2!Y z(n|m-0SzI5y__DH^Xn;U;t&dJL~#wh56+Kh-g4 z1nLXnTgg!vyiuh;wkET+%i~NLJkR9hH@k!*rXNpK8$8{`Jbg@o-2Ld}Csl84WEM@A zbaE5fp{Iow$E**o+F~6;<&H;@ReQq8B0m%(*YVY2&W2i<9!;Iu%SR4RJpd@kGPk&9 zJ$9@J2KC(4brnp;M)uSxd)^$EyfwO!rLdvwj_3&@BajFC)^#@q_UG%2s|VKA)onj? z4)o*KvcM?)xpn+t`1=ng3u~ZX^`)nt<5gFcj9Q5y&2C@@R`d+DhU8@D!y3TBF41c$ zy5%n93uFufuF_`X1pk9n{QcQRUmspx`DEQEM2)AQ4?qQNrlF}#GKxBo+H8xgUhDH* zAxEQW??n_P#SeuXdnR;hWi*IhtMAtqGb?}c*kM@JT!u9rixnsusFSR!FBK#W`fC`G z)1dDn)m2Trw5sR1&<_79Y<*s9_)t4o9VWcw%)&-5XhDJS<*U`4LX8irK{O$P;T@$7A#TmS*3SHnr2O6;$^sZ#GW*r5q*6Zg)4lA89`*baT zpSg09P#E>dLHHyL!Kj6DB?e+OAD-{bl{y@NM(3HY5DcFR;t3EeApEqFDMtuRo{ z`-3RQ{c_6)Z90nL_BIwpTUY=5VtzZvrRSa}&?C~sOkaJn^URcLAu>=k5*!#oYC=pZ z^&WeL#Yng@`V(Dcd8fiK(7B|7;+0~rJ%wf2k6YFBS#2glL$gEsv2D0iePJ?B+@^q2!35K)Ab^@nzq|hQV_3)OAl_F z8_|92C-M`{cR-JY<^?A=g^9-%<=A}>ACjL0aDeU7d;fBb;pdUJtgyacw{qt;8lS

uNiLV(CC3NV zu`TO02WkZ6L9{0+9v4fi-#@qEHUgW@-8fApWIZsTd~yTlMO z!f7Z_Cfym!oZ6bK^@oGsBl1{{ZhuFO!7l@OS*s@z6k<)RZ?Lt`IrBhPv+iw_D*gP= zmS9eo3*NGg5uXto{S3jrvpMBk$H5B{-QoIUh&k{N!fd7ef6e>pnNy`jrc%(3W@Z`UTi z=1vzxi;MxXWr#snk@nQ`zK=?rs!}SIt#d*{8iv;7QfQs-du!+QLW*j^m%K%Ss+uhN+eYL6KPd$QH11{S20Rbbx9gZDpXx`b=>EHb0zenT0Rf3JXai z&01it6J(Re29A$H_RIWI@3zNbS-ax1^5bN4z2F6N;uk#c0ZHeRLzlfAIYV)Sz7zZq z9xCnmn&i7P1XTM)!7?~}@FiJ2LU9NAed&4zqMjF>6)|)#ZC44qzFAG&b>G|I*j^d= zPMqQDhP{y2OA9-`DCW6b%BAlB%Z6?pAAp_qjP&xv0N$d21P%xTJ$MfFHrr7pu~@>G zy3{`pOz1L|2);% zM*T)mP~s1G*eV>~DQ2-)U$LRQj*=q%Jy}{pmZu8IE>jyq8nRmXA_!or_gIfK*r3V- zne`M#adT`F6$LZ78UP)n>HrPI?*s)cK%}U4R3WrXBJN5jUK@)sqXg>cDh%&^ggd@3 z?;^(?xUF}C&~r49X`J!~98Cf74KzEl70?{5qJw@4RBJXb`kElek^JBSv7`6{Uxm;x z+}_{!)%haL`g6rJH#^g-kmbnnU?t=y)yr4g<KFh;O1Vbq8yAIdmT{2tJu0UJ_N8+K}`&I5u zbmw&_cA@e0&mL=zg-b$+D>Hk)$hME>S6DcH1a^1AD0W%0f@Vo4=vIMH4lpZuw08QT zSS-BQ5af=;w?ajM^#nHDS$h{h+*3NGa{emhjlk|>yNY7f*NKxsh`(A54glBFC>;K`*O3IWc3Pz%iEGsJP&hN^EZa5B1$xD#@An{ZZC3|n>**el# zCM2yO)>Q#I*!xksqV(3rEWrd|d~SwqFJiS49pI<=@lRguy_c!@>>8srzdz9UsgBm? zWoA&u={rx$Bsi{WjPkENfch2KC46Mc1TOqo+HZ#-h(G62h21u>90(gWB=7Q$&)aEg z)&HK|;ap_L!*9E1Xx*mOQT)C)SwJGs+qbVkS&?;Rw6$0qFHfxJvA3N9n^uErTzmRu zWjh4VJodmEB{69bI24b+iMmY??ZY72mA7k`YCfhZ9%0E}6XFfcY1mm`?V{5jyYY3# zp2CGzl{>DXbo`bR8lXi9^s_^7pZ(pe*MJKN@sp>uK*LlC%vXPUi)D$vacGnULg(SC zbus(hGdz($G<_d89kjn1fVZVjM;x4}WvW;G6aZKbu;`*iKli=y`=v2*9I;9$K&>DJ z)XsNBO$;Nv?>P$9)1j>~-%S0sE7jJhl+ke4PQ69I?-}yCeYP}!*4FV;;D)KD>h>TN z%#TzCNI#Um%;3`udPm%KOFQ|<@>XPt7#!B=Et@g7wKqQ2vtw+u*i3kR8R8CQCej0k zZo7AVWBf+*t!Qfz{& zix>PR-5AY|ZS%&eWCC*y*>e0xss}|exvKk~dpl$T&QJ#C43BjriGvRqhY0|)s4QWe zdJ2n4v|I}ko}TY2%`4HEP(=-M(@-?ny4-y^i}i5I>mEu?xgDWAyj1s%*VBa6T79H+ za*JC+2+njrkZIHA8;`ljN_4M zor%bwUVEEc{%hr1Gefo2?Gd}7@?A$MuV^UQi?d!Nj!?<{1b|SFZ2YND41me!I~_K$ z%bm=)sG-=%>+^+XN#h(+ds2#Em*V!5Zv#C#mBe@Z>xBi6sfhL^?^99bEmYUQ`NpB? z3cv0+yi2SR!wq8I<7H5jPNoOAh*u6EeLlJdGufb7zvA2A=> z5q*GS{2@~onH2`;$2X=z46Ve_9zg&*sXz>(tE)R*ye~(`jyCC4HkT#hU_suA@zE#b zpgcxe=t8CPZa3XzTu;kpSnAUuF5GpuI@)YMguiQ7Ezm~0TK%ycJ(DQ1+dGTcGJE>0oVmU zoqNluAdbxmp&W*;)R!HRt9kS;j>D6uhs6#Dv?2tnRDfMSitnElzL9!UfkAdUOzMnp zV~EXuJ`)YcBWSvZdwSY!2gTc?sW#;*8Ob`Y!ydMrY&^~#e%9ODnY5z#VrwZED%IAL zDCR#)V!b~kGjO&_gOtdOQSFtK#S8)rUu0Mzqoo{~A>BUB8@sRAhkG^6wfga@n%}`A z+yE==De)wFt-f3rJ9q78Uf&X9Ek<4!kVwoo6Lt;sh5`QEeR`m^*g!gh-NiP+?obra z6!x#CPPc>dzCqwGGh{DI#%s-0UQLt+wnxJ<9V#EER8&J~4Q5Y4;l$XxJbM!AdeEBIQlEC&+)?3ria8{c5OjaG2>}u{w>IVD-yTVye=-`9+4XIu zjnKxT$yGE6yYHihYdYYK6AVkMfy8Pp5V&)jGP!9`;>`kZxG0b#8y0Zg-~NdG(7r1S zDYfaKDFdZ@bUhTu`kWzzO85NpI-LRA4qdpzA~NWH$@^b z>~2mbkx(d8Q-Gws4V(O&hl)mr9J?|2c-e`>lJg!J6R7K=J~1ziXSmo$E2MNOM&S? zha5wjBX8IgvnZeT>ePI(DfXg~i|sz5d`UWBJbfGYa9sf*E?K=8k1C4PC3sB4Cc>l= zlV381l*)5;@J_K_b5R!=AI!mP8J2;7JD-c(z?Whz`*$vAX7YerD2(Fjz;#%5?mcM* zurDfv&8__z>3I$MNvUN%!4$9L`gkPVi=aFWr>!*M+#$WfL3E-#=&KJSQEbog+N?eI zb<}nbr?aO^tE*&|`@LSf($DVh=nq=7d@;L7@k;wT)1b!Vu&U8Z_adH@iP-BWM5b_` z{7TW!8-OIckRuRK9=i*p-b$A(Lrf7g(C4;OSe}4?h_qz-5@z&~OQpOIoO8mgBg#>y zwAN3<@SDzNn;2(@;f)2%E8g3rC30CSxj(c%@9l?yC3_q$MExJ#%Jsnv2fAMgcu}~R z4bevn5RHdZZ&z`gKc?tSk?1jHQxYicig`#(F&=GE@3sMA)G4)C+XYg}o-*2%i7fyR z$e{x2vvk&_V&TRD#6G%oqDyT6W0-&L83yjD!9fhdaohKD33xq22A)UN#e_y$OdKET zOrV4EKT}1L1ie9u<~#RO8B!FD<=Z1Oax<=_%Dr-MP=CtvUIO*YLjgBt(7{hZx3iR3 zz+A#>yJ@8%7_FkQ@jdz~8dE?wP*2~Gr-~Cd$p^4805MS7ImPbXy|jyIx1})3@K}KY zd5~pv@aQ7!N>}#*RrPp(Za~G$($avjMiVUwrO0Lk$D!+&50hHtj7Bj zY3(NX?PepmIDdk4JrRwK;p}3dL^?gj!n;^v1)yPOvdD3o;*rnZaLZ@_ayw9oE2IdZ zr8!P^A@~zQ|Cy@1XR76&0^?+eREaj!;V^QXY^}?O9BBA56FcW zmgR%PXYAHk^**50?^YyJXY!YGS!luF*d(iaUYN|VL*47X7SDrS1Mnv$_*@D(;Xrp6 z<+k1a?f@ObnQBvIo4E-W*7d$8p`zj@o1HO|C_FL`_&RRgLg9h^e*qr19pNVb2D+Sz0C3n$h)C>qEt+{s=f z`UrfvYazp>Y(wBRAjdS_mcm+bFeq?u!|gEQESO;3D~a01OM_gJTs0&Z=%pBCLL3n5wZe5xvA4;&<u@*PQaqsM9^{6e-mzW8I$ z!TpXwI?$l}VqZ#!CVNjDeh$ha^7qUC7y$WcT>d0!8 zhB!ElV~U19#n|natu4^9aqBX4a6LWu507BbP|>V)QJm~zAc2Ck>dn$cW8J@e?hZ+; zw0{8dji|2j#Gmur4ZNS1C0`^v$w&sx>jVWE$A9XkNoJ}9cN-@NH9fg#2TGt&ocyw$ zJM*vc@_RYhm}>M95_F1o?q0GGL-Wv)KP=}@72H7t_xkmO;u2}ZUYGsy~-O}Uz z!Z?bf6jX=-ncH1{4#$fk-2ikZ;-`n3bY=cZn5T=K+11EtT2W*fBtU^zw!UQTYYx#o!SBj(KKPe`GU=`J4O zC}yIK7V%N`Pf!>mw`Q&lx9lO+KUO5}GEJ<R+Q-Q?U~)2{9W?C{if;k!&`7?I4B8<(d!CQf^(cCh6nm3nrWvDk z_<>&jQx8qmuq%(HDPNg0+jJ zD2nC?9WuQ5Lp^8Pr4F2(ERX9x7owM3#MC&?)*2tw;a+{{sFTEg_T_%i3&d z#$C(Rj0oxJ;Cn`Ew-~gBDm1*yOcY#9WU5Qk!FWrQNTK zJ1@1KlO)ztZIv}Mopa<3Wvb~adh+OK8KJ9e{vPXJA&p;G+qj#r(ycTpJ$AlAhdGsY zpIAk3g&kXOdh48*Fm2E$bYeX2r`9y3chYe3nh@K=R<#*dto`&_PujNQ*1Bbt?{I1U zF59vjRrO?Zv~8kSY>qxBwpQiuMeZKT_2#d2gx9V?3M=re%AmBe)Nmy4OaTZdVczO|;7}b> z0;k({pV)G26`s^-RS}>*OYI)r<^5K#^R2E+y}D7cTaxo+YF2MCaXJ~9t_w0?T& zby4T&8Gda0Y^yBgX{qcnFuM#qDR0mROs1crNz+ZF?&IIy`wfz+-Z}6f8V$X+#=gHJ=sv}W1rxJAyw2gTif%58Ewmt#?S1zt;(X8oDwRHciv zjm4wwy$8LR73Z$_9`Q}!M9r!NOf{>sdr_vGnD$9ThygXLzxpX-B=svrna7QHz?0f} zNq`4WYCnwk6@Ne`{4tKsn$O^BTsXj#`y^ovHC@c5}YblH63 zHXikInAKytHBO*KitAz%9k8#<@3ah=RdQkou90sVLYiO$IhPO>j&Fwmz z_ka@8hAgbi*@-P+YbyrA0xj(_a|6d0bP%rG>RfU6!B~}n@(}HcbDS9F(1(+(`H^pB zE@5KmI>+zuLC7+r$VFRRdYLD!z~ZgO5GnHyLT2|i(UjJ(z8WCf zVB3lBGPF?LiP$>GMT98hjFW2Y0Q^%2fs9vXZtbK9nB$Er02})t950Aj3q~96 zp#dT$D1hqz2R!GW>*O=^^Zv`O={#C>%}8d=_$j0l2HB9TiFwq$OflOd_VXv}+Fc8& zRfTZMK&x7czn3zm-wnfH^_H`=2}Nvnh@u80T=O12MsU^|3gSxd#4W7aAn6I9`?8Hq z*T016--vYLT~_Q7o`m*Wk9g8ur?lE_v`)`M?koPQIW-_{x_Wyrq1Tb z_E+OC+~2t&A_<^pbD%KiUS)`A{33+LFAp&_718)a7-T>Ssa^!lyCNlsik`H$ifH@^ zG|>1o3&Ca)jqeMzwd)yIOwvlj`xSu3CyuZUMKnHAtzgb~-rrx;aE&D3G3ZVpzdv79 zdmU(eY<5Q47aHFh3Kb}X0HXc*DsC@QsNh-c4P{Zo2GDm+%Y8S|3`(?M3*9?5IwC4m z5CCL%w{Ut0WpYTs9>-%@b4NVAW8^g=|@Bh mJ{g*zioTB!|5KXahWpy&nZ{Bl$?#{YVu!3TR*VDgmwyMDcW4Cw literal 0 HcmV?d00001 diff --git a/images/intro/index-shard.png b/images/intro/index-shard.png new file mode 100644 index 0000000000000000000000000000000000000000..f2663d2e95fb7b7b5f6ad554e0bee5cf722dfc76 GIT binary patch literal 17682 zcmeHv2UL^Kwyy;h5v3_bL?odKgc5oc2uzOlD+76Oh4%RMa2rJVwbWn-oUv!-8=16-7Iw*{elhXuc z=!`%@jFBi?h@rKi8`1{&Z4JBtT3I7a5M~HdljD}0?3`Sz>`+!t9yJa=I;a>2JMhH~ zh4Aw8=^VE=gqzx&3@eXxLs(fE(s4?0L)d|#808I(5jGA;dviLdIPffM<6vq9{0oqQ zZ$(w$OAGkR!4BczK5l_>!t&sRLfDyr7t*H2mPlYUY!l$a0g$8+C{r_}3qU+-bvy-A z2SfPrbY)De;GF7E4K>*Zy!UN+9W7+VosQc$nWF3wNSl*YaX@$>T*ogQTy0H{sYWQI zv%M(_ONL6)v5Nuhupg*|3BnMLG6dwqZewVD(pU*tH!CnBtjG{2!s6oO;uSa1RzfOi z+pDAG3{(_RW`8@{>6q3o(kMe)b9to6kw{Ejj_1J74Gf4K)5P_dz|D6uPdMt7%?Vd5 zK^5V4G9Lh#qkeElgo&yBao>}M4oIYx1H$&NiN;798&l&G;7%GEqOcP9tG;GPtCPi@ zw6O)&{Z}op1XW{0t3OgT5he}*@>qrA<35=lwzZ5Y0&afF89?@U;?{KPqgswtnP^P z&qe$nf|fQ#TAMncT!CRvM&vn>&WToYKu?~X|3gQhd?z&X-}U2^Xn3NR@V^X=o%H_= zc0ijG=>Pfjv7Yj8H01BXa$=yr!6FM_0JtRVgnq*YkS3s%zXM?JfI?cDo9b+x&`%SJp%-Vk=8%4IKV3%#S3FsD+B;L3i=OF_|5K&j*V2#=+qF642><}M=&Wm zIsn69%@u2)_Q&4*BpCkJ8|B~P#{Wm$IQ|}RfAQJ>I>>SVxsc;Lh2_-b{*9sI`2%#S z=7uN}z?q(ep+BWh<$#{3^gluPR0;mIoht7?#P=kU{weW=0$%L3}hZL|G~V=`F$mD>QZpJ9gK28Z`f>Gu*!f#S=g|Pt%*< zQry#B(C-F&nk@b6#8QqEzx!tgl=mbZ`u7L)k6V^MqlI$*O}_hYY@z>&jmrOGmkT}h zr~m#IdUDA3o6ChEv4^C8<`kI!uQMdh(~S7Hfco!|0iULNe?a%ooB(t1{I7xck2~07 z<2~`xzvaPyo-;vC-v2f5{&53)4DX50{!_vW<@(o!7b*c9`Tv6}p`&+yb@%rBrNGhK z)7!q|7bhLCm#N3MM5fKw{b$bHIU@@bQ*+T@OugiCqiOV`!JTx5^)5@_NB3a3&wU8b zQa&bty_8wWW%Lb3ot3P)uS|5EjkM@a3&S@CTP9x|oG4fx@-O@wX_&ON4eIJaX) zs$heE*?qHUzj{|_Z9&$lbK%qSVMn!KRjK=`Y48PI86O<%r~cBp9dnR~(uC$Q;rcl) zBD}}Pui(J%wCpZQt*6FbdEtY?z~%lBLzfG)(g zT);!dhvtU=enxlqpqvGuJI{T}mB){lmWi|X5r(E-7lG(k`_INmTjELDzS7XDvcg-c zoxdhOHqmh;W&&)pUp4Cxh+d^H#aI_?PwY!R7?2Y`&faHO?SYd>_-)08gY(Tr9#hoj z!$pwVUAOCPajXGUf+RoN<|phX%|UJsp5z(LUVf}~`)QoSgGC(IB*uQ>xQ`Qze}A)O z)bNFdQ9mD7V1wNEO_V(QLR>AMlMuxS`et|(a$UOWQc@C$As^UqV~goSHfUKJCsF)8=dcx{7zb1*xp%s)dD%{i)Ab zQcG@vR@0fu)gRfid@eJ5QnfomOYgF-L~j+|8|0)bcAdY?RaqBr1j_6GI6lz{AMffg zpTxu~QiSzd@9~8pMxp z<`nLQ{dbqgzK<0P5^mT&;Aq&s=FZcudEmV<$Bh$fk(!s7h-p!!iY>n1x z##NakWhOeCB=aV!B_Cm|^hWAoD8&vjdUj2;&65Pd{uxFbqx#c!I=-iVOwRrP187_qqtn3Vwa+rUSzW?drusiRY#lYsuDb?=hUe@@o_U8`xSKLh>%2a&b z{ywy2*E!;-BwX#s37uaY*fktL@8cp@{ZT2SBSTf%g87tKVPp9`$X7%Q`1O{RW~*nc z%4Y;k#$-h$#QGNLs4Ks-6=NMl85~(E6=13J;%#_QSeM`mc!3|qKccWsquvpCD7eDJ z3_F(f^Pq(0Bxw-VB@7Tf4|)fAnNNu2l^_kcgxi+4gRqLL3p|WhMwW;lN%1x?wrjpC z?f+-=Pes=*egA>rQvj*^m9{Iy1-h;EaiRDo!&=PV1{H2=hm9g=YYJWwjNc`ki+9+c zpE(!jc;*}o5C6J1rjz%8;js4LaDLb20HUAIcpryM!q(Of=hBTHT{5r7IN5M#3jm!* zx>%pcF&itxBqJvh5Zi-jAnc%7e8S;^58~daRCbAmyGPFCKL}K)g4q%VUuTMaT}p60 zwMwoeqpr4t^Re#*!f>b%P)3zEO z*GQkJfMm#nb;sCA5$trZ#2Xo{Z)$W0RTvElxWvU_#PbaScsFHIwJ9Zi+-eIf=XMJ5 zGJ>|hj4Ql)IsP+>gwKOZIbX7}Sj21mr4jjOK7}+{Y1eMMX9P@O742AvIv(A9oFFxE zhwAsZ5T*f8-1S2;?|4RS9jlo65+gn09Xy=i&r@5){vSPAp=)sSl`_n zuv|tJ)2Q6%_xtW#!5@Wr7nrgP`%Dwf;iH%+szR46RoteqPvtdka|B=PbdfXc&W<|4_J8yCMa zs5FEDtOd`SK8U|grN!+Z(0|X|oYg-dfQD<~ntxwG$herPzt|UK>?1iShv+~E8QUEi zFJ9u!B{6aFYMILopSD6A2zLo?iPcn=6MDh3OA4H)H`D~nl6EeXe<#X;?_OwXciR*s z;g515E>3nf%#cAl9_;PF7_LA0`BNZUlTG(!l~Un>@Hk49|7O7cS1=G=15*Wgz&6c1 zjf=zKsdiNXJBL9%S*k>KovDH5*TX-eSXDN1Z&+)c-w%cf&o_V0O-U($4mSaoalN8kOLS*%|`eo^S2es@Io%{H%s5 zFN?A!MKpzE6!U&RBY+2WT@8}%S~_coGRz2Yd!x*5rF&g)X0V$cb+9Mt!R-h!`=p;M zX zsOraajoHB{dJuIW<>4Sconx@unHu*k@#LohJM9i%RkIXmybtym>Cl2JEQyJU@7y+W zSIa8UAs*v;%R(D0Y5J~Bjrkwf4^w>y?Mc6tzxQ1Eny1Lr^R8VB(0FZRYGMw&Ik6=o zMAP`&Sn~_LWKwbg6ZV_KJt_&(XKRcfVfHlgwH_0Q?h)4Qt%j)7lCtVN+Wz@{rYrsY zd#M7B_+d2^iOK13>BN?w zL}|wT!&BSiK=d5TDKyY45%4|`3ckUdnb1=I_)aJsfi$^CV|~bmn6Obp*Bd9#K3*uYtL+8ic+r`XVJP?6&aDhi|MZ5VV|j z<1kbI61NSDrs`ocP;?mAbhCgntodoO(aiJFN(&#iHxcNV=;%gD`nHLg?u_v;lDJq* z!>Y;(0Kk*F{Y7Nu$C0*Ap`S~^a+CGx6S&r0m)*zv!TANyOw&15(!m|f_wrj#7=UkQ5yPJ#V0|iv{is<}>!$uOtZ+V5a z;?f-M^0h4-jEuMvPp_$e%23Xb3f7i|{#x%*Tp2>vO<91@zZ~cLua;Z%psDPD>(ntpoXHn)F){*WDy9{LGQOOd;&l1>$n-jmG~;KOiC<)V z%`=Ez@X|+|nW@|V`r*^vdX)&Lj z5VSDEgS8*+s%?|nlasLvokp|79#^0oDxJ8v4T0SgHMx2scCNIl)}4R?2U*h0R`LjF zw_J%oA6tsxpNvh-s$*SUcx_PUrQKHm^7;T%*z%kR9psfmGF{r(zSLLK$t68AS zACQ`&cY~5L@6%{BFR`%w_^WRGn_M!3nj%j@%QY?4tCjB=m#bPR=Ya_)Kcjo_jL~We zzZ@hPtkiiCX8_{%?8kQm^XtCv}O3Os$bOT;E9F?5A(Q-%NMP> zMtSaTGHvlpzG6a>@jGq-ey(jQ$75%eiMMm=*7mszHeO0=ld>z80ppsNU-7qi?jxcy znHcW2Q0A|iyt1F}V4r&#=yow1Zu~HQ&}}o5=UF&RNP(en5c32Agq76TK`5$?nT+^n zoKH11Exe{x7H@SDM(>)V2$4z({FLUl$!)QxuaA@8TdJdGWe7rxx0RCbRQshk(!w=m zptGM}g{-EWCnHt@-^t?YOcOQWAaISyQA`tY`5r(Pn`kpgmLc!&rV(=et-D0Mjax_F zr#GBY3jwi;ikOaj9g77@>AtJC{N+{$aUrzdXo?I!T$BWNcXzjOA^2zSp*>n=yIfh# z5^*==QZ4AdT^EKI##_tN!q3O1`Y*6d#%k@DIM4ULj&uC*CU5)!ai zf<;)#v%>`nqO3j0pgBY;D}K)$8>y%nRG`L9J1KJ z`YUzHEd-tq?hs3x)1*-tW{==!s<@K-0p@5PJFKL6hoIbJ~{cDOGh3Hx~Fv zTA65EFN5LIVDDs!4g%nhWx+J8agYOGXSN z>1ZTx(&lb$-R$5BeRQ(_%L({$tWesNpL2S8dYkGhYH^30 zHNcL40ELHgD&6wnv4A1qz$mHd&T~6;c7VT~r2DYHyTzQU)e$ZylBJxjMAM=f-a^Y3 z{!my(o;-S!F{B3YxRhtY-mIVmg zDpTOOkiDd^;AD^^GA)oxLVWd-CCbTTb$OyW0)F{^E&qTqBZ`%JiCOstOd~oTQ*xNP zR;ow69>c~ONdW#4H~CX&muIK)f+9G^a_8=4f_Na5n^6=s2ToPWC32eicjpH?1&vYEIm~0mlx@!k70?QUVD8I?`g1eUJK2cnU%#nr2-Y}bV8s9+j|j6>5WSy zD^k0?Ya1~lz(&O)J;!7->eWi-9NT-vFsEYCN6nhw!SQfl`&r7V{LbwOECvKK+ z;&e`fS3t@SXPx#@IT`_SlZtWW_r*3Oiqw*6KJMKt6L6s~1RyFDy0KFeN8;r7L*@Mi z)&;SB>&|!JQQkiq5dSJ0^ey;o9%U+3Vv$@2-O-30eb*i=9T6j%ehkRm(LARyKX`*4 zRlQuCDzqmj?sCzJ9BWb^Gsd3TO% zxSjnE8CX)D=v*TGi~bi#dv_<2DD6Z9t-@?*N=K7{eB-QLReR=sd|_2is%0O*S&=R$ z6eg71&PmLW7??;EJ*r#ABf#A{t0z)Effz)WmO7w@%X%ML8}Qn7W0@xj*x0+O{8lPH%XY zso+MwC+)ZsplfAicc?ex!fe##Sdzxvw^-{^i7V^wa6)qO=lziCQFLNr(6qi6 zL8s`*1$oMWwK;cy*RvebHC{hV=Ig4JCgAWcDnoYWdf!bw!MBnXC9isO&oWnJlPWAc zGTr$UIJ6q){jF-5Y~u42#^t4+y~F(K9bdoToELV2WCT9KE0u%0oHSY7Alz-DCTG*ti zI&Iim-5Fb5qn&T%AZ5?*HBNhXWCi#3{Pr0)Cr|PoGe|?LD!j;W!$F-(1EtKGLJ+3F zs>u2S=`hTL9PyLd+G==C=JtHb^D2kRC!Pg}++^za&K&g*D)!t-Az7>OYzAU6Ew$sk ze&cpRUKDp@*^DhAxdCfpon8n~iCFcv$6FTks@(kd4mQKFP04KS8e&q`p!bwu2JD%P zc~9kK`fzV5^}A2+fG6&b?wa4NP2nUn#l$xHcyo6;H|+sz^9wmN7U0}cf$-5fWH?zRpY zd3RJ*+!R!&6WAp9z2gi44pWO+A^P1Ohw#cr>krwoU)Vv(w0HU}dB+FIq>XnPWO1vi zpz@7r^00nk16!OQ-iJg7rHCxlkMe^71YY!r_wD*rB0W3XX(NZl{hI2}wR4=y6zV|05qhi92?-kg_R}`#&w)xE0O^E;-Z8ehkI?&SPPHr9rULxU&m#@+!s>07Jl}ee) z9W1cM;zE|f&~cu_{WHCHy{h8j8=*sEW7`aOHSyj@#=A2&g%6bzY$Q8=IG_qq%Fv;c zP7FlrVlRFAkzWdWMZw_n-b=#%I=p2gJS92S71@Z+_{|^j^qJ;3{+UsI9`&R4&q$@R8Sl580u=Eh! z5Vo6rLpn+dcHFs1-8nV-u&?RBIr#83ofH$0lsyI40@&kVV$)iA!zhz ze~k{ECvx>R?z5>k_{uL2+f2F}LaLlMHHER2WmT-fFhnrGNnWK1M|35r6A~(YW0Q;| zzy!oH2l!q%-xr*TsEP~g81jwAw7(p_&C4$_ADOVE&(2O5*SIf^BztSW@=T&*f1yLt za%XP|^{ntQ?*!IqYtVx~sV^o-S-lk`tBUUoOmK-|sF+k}UD7Rk7C}EQZ^@3Je*IpR znN*%OLTcd!rqtqQ5lRqzN00jBhb!1b0@Ey8VF)~Y&sZe5{~;!B9nRCE*4Lp2X(=Exr`>A z<^~gghg<(p9))Ms&VzLLfNqCQjRij_ zd4M-pI3ap@&XJOcf+C10qj?@Bs?gH{`vwQNloKxH?7&<%`*H;~OH(i?e)75NL06jgT5>Qb3aPapMa z?7T1&(f78pdk1XaKgfqodx=RB1w3gq3P!@0;jHZ8EcWz|g>cvB`mR{(@O~^6emLI{ z=+=J$3|)!x8FlLy80sP>;kTXP6Q^`&0M1zmNlAZz1-!y=%1FLcIkV06=J@%)Gf>Vz zMQ-Awb}nw@Uy(8!KYtdy4xIQz<$v_v6efigeozeYmI8LYp7qUzkv=@zWmw0UDcwLCZ{H4B_Bg7~je^AnZ+I0$B|-NRcPdb>qT|v%1w1yHdFI^_nhpQZ#c3>7k1s6P z0ei`by^Y%LoPM;IHTFU^t7B#)i{NV+LZUm;rxQn`KsP863OD-RK+WOs^B%ru1;e>D zgLQsUR)d%5@~-jPQ9^dr&TuXwg2uIJw(&ef2)ps7iiwHI3+Lt0na{7TSPT>_UWG4B zOydBjYaHGO9!)~$&YkO;Zf#y^=%G?Bi?jnrJfz7&@9Rds)%MI=Uoawwtu{^clc`m-$F#d5O)JqIB<^Po0)k zrWn%3#>PkON!)Ffz|qM}*MS&J__GRmGnG+8+`d6Wg_H2c!Zc?`ZY|Zu!hMof$UAVe zBF&F2D&Rr`YSeFiPi0k>%jVYOz(Cw$oBaGd^*pJtN}u|4iKm2=lp(-@wApJ3@wRkz zjJu>osKyrey}01_MvZoT#pSQiRFnek7qIsjNDx2R8N_s<_#ttT9p-kU%_x&V%Ig+` z=V#{n@*;lzv@I@(kMs4DeEOlu3Fq_Y&xB-T@o`Q|!#N(cP8+#=9Oi+6QqJjiA1VXQ zwJ2|h2Vd#wTHd)<_5L|}k(6k0%B~Wdt)i~1q-QV?y`9D#cUmN45_{>=&9o>!SYk?a z^JcCP9oqo9b9H&VNpiZeiTxbB6CL1D$wNc)J#(WqhM|zs(}k+Hs`QNvw{oG4qgtL? z+l@LR`zocJ=^?u8!>y5gcv1`NddZ;@G4ZvOdkqH@U#gaIi8rZ;=g)s(P)QdLP%5`j zr&G^Sp_{jVbw4f^ICO|()>QE3uDsI>Tmje>-c^=u4yXDM1(vo2t{3WFK8S~;YrNLM zz4DsrQB5H13`<6(O?}z4S zyX`<3X_BU2eBo&_#HkaqwDfe}*JpdQpMc~gYYSe%f{3+c^&yx8=Z?;u)v2{&hmT&Y zP$)@GPL3-*zffnbUaif0Y>q}tH`eM$bfy{%nTBfR(;L|INd9zsEjFF`I{5e>QkzT=(A7$&Ei0`mbT zW#7Xr-}v!0AL~A`N>9MbON07=D-CC2J47WC>eW~HRd+n-ByRd`r>$z!uN+eD?hoJ; zhzgOBlSj7u`%}2$=bRzD^QLw2@!8MM3%sx4wy+&MV>%-le4Y5ootw%^hB%j6llwN) zZdy)s{W9$auA!f#(vNq%E73Tr3%~>F0+H zgjC>RDU{h#5?g?fNeC1m_$FSW!4@Dm0|iR&pauCijtUUOfh%r2v)lgI0)$~%pa9{2 zoY4n4la1V|Kr8*R)Lt!S>@x414XFQ*2%o<{||vMK&k)$ literal 0 HcmV?d00001 From 3587e5a53d685e2119bc1152767c93c78ba93a80 Mon Sep 17 00:00:00 2001 From: Fanit Kolchina Date: Mon, 1 Apr 2024 13:24:35 -0400 Subject: [PATCH 03/23] Communicate section additions Signed-off-by: Fanit Kolchina --- _getting-started/communicate.md | 325 ++++++++++++++++++++++++++++++-- _getting-started/ingest-data.md | 50 ++++- _getting-started/intro.md | 4 +- _getting-started/quickstart.md | 48 ----- 4 files changed, 356 insertions(+), 71 deletions(-) diff --git a/_getting-started/communicate.md b/_getting-started/communicate.md index 351dbd840e..45bfd48073 100644 --- a/_getting-started/communicate.md +++ b/_getting-started/communicate.md @@ -6,47 +6,334 @@ nav_order: 30 # Communicate with OpenSearch -You interact with OpenSearch clusters using the REST API, which offers a lot of flexibility. You can use clients like [curl](https://curl.se/) or any programming language that can send HTTP requests. To add a JSON document to an OpenSearch index (i.e. index a document), you send an HTTP request: +You can communicate with OpenSearch using REST API or one of the OpenSearch language clients. This page introduces the OpenSearch REST API. If you need to communicate with OpenSearch in your programming language, see the [Clients]({{site.url}}{{site.baseurl}}/clients/) section for a list of available clients. + +## OpenSearch REST API + +You interact with OpenSearch clusters using the REST API, which offers a lot of flexibility. You can change most OpenSearch settings using the REST API, modify indexes, check the health of the cluster, get statistics---almost everything. You can use clients like [cURL](https://curl.se/) or any programming language that can send HTTP requests. + +You can send HTTP requests in your terminal or in the Dev Tools console in OpenSearch Dashboards. + +### Sending requests in the terminal + +To send a cURL request in your terminal, enter the request in cURL format. For example, to view the indexes in your cluster, send a CAT indices request. + +If you're not using the Security plugin, the CAT indices request is as follows: + +```json +curl -XGET "http://localhost:9200/_cat/indices" +``` +{% include copy.html %} + +If you're using the Security plugin, provide the user name and password in the request: + +```json +curl -H 'Content-Type: application/json' -X GET "https://localhost:9200/_cat/indices" -ku admin: +``` +{% include copy.html %} + +The default username is `admin` and the password is set in your `docker-compose.yml` file in the `OPENSEARCH_INITIAL_ADMIN_PASSWORD=` setting. + +### Sending requests in Dev Tools + +The Dev Tools console in OpenSearch Dashboards uses a simplified syntax to format REST requests compared to the cURL command. To send requests in Dev Tools, use the following steps: + +1. Access OpenSearch Dashboards by opening `http://localhost:5601/` in a web browser on the same host that is running your OpenSearch cluster. The default username is `admin` and the password is set in your `docker-compose.yml` file in the `OPENSEARCH_INITIAL_ADMIN_PASSWORD=` setting. +1. On the top menu bar, go to **Management > Dev Tools**. +1. In the left pane of the console, enter the following request: + ```json + GET _cat/indices + ``` + {% include copy-curl.html %} +1. Choose the triangle icon at the upper right of the request to submit the query. You can also submit the request by pressing `Ctrl+Enter` (or `Cmd+Enter` for Mac users). To learn more about using the OpenSearch Dashboards console for submitting queries, see [Running queries in the console]({{site.url}}{{site.baseurl}}/dashboards/run-queries/). + +## Indexing documents + +To add a JSON document to an OpenSearch index (that is, to _index_ a document), you send an HTTP request that has the following header: ```json PUT https://://_doc/ +``` + +For example, to index a document representing a student, you can send the following request: + +```json +PUT /students/_doc/123456 { - "title": "The Wind Rises", - "release_date": "2013-07-20" + "name": "John Doe", + "grade": 12, + "gpa": 3.89, + "grad_year": 2022, + "future_plans": "John plans to be a computer science major" } ``` +{% include copy-curl.html %} + +Once you send the preceding request, OpenSearch creates an index called `students` and stores the ingested document in the index. If you don't provide an ID for your document, OpenSearch generates a document ID. For the preceding request, you have specified the document ID to be the student ID (`123456`). + +### Dynamic mapping + +When you index a document, OpenSearch infers the field types from the JSON types submitted in the document. This process is called _dynamic mapping_. For more information, see [Dynamic mapping]({{site.url}}{{site.baseurl}}/field-types/#dynamic-mapping). + +To view the inferred field data types, send a request to the `_mapping` endpoint: + +```json +GET students/_mapping +``` +{% include copy-curl.html %} -To run a search for the document: +OpenSearch responds with the field `type` for each field: ```json -GET https://://_search?q=wind +{ + "students": { + "mappings": { + "properties": { + "future_plans": { + "type": "text", + "fields": { + "keyword": { + "type": "keyword", + "ignore_above": 256 + } + } + }, + "gpa": { + "type": "float" + }, + "grad_year": { + "type": "long" + }, + "grade": { + "type": "long" + }, + "name": { + "type": "text", + "fields": { + "keyword": { + "type": "keyword", + "ignore_above": 256 + } + } + } + } + } + } +} ``` -To delete the document: +OpenSearch mapped the numeric fields to the `float` and `long` types. Notice that OpenSearch mapped the text fields to `text` and added a `name.keyword` subfield mapped to `keyword`. Fields mapped to `text` are used for full-text search, while fields mapped to `keyword` are used for exact term search. + +OpenSearch mapped the `grad_year` field to `long`. If you want to map it to the `date` type instead, you need to [delete the index](#deleting-the-index) and recreate it, specifying the mappings you want explicitly. For steps to specify explicit mappings, see [Index settings and mappings](#index-settings-and-mappings). + +## Searching for documents + +To run a search for the document, specify the index that you're searching and a query that will be used to match documents. For example, the simplest query is the `match_all` query that matches all documents in the index: ```json -DELETE https://://_doc/ +GET /students/_search +{ + "query": { + "match_all": {} + } +} ``` +{% include copy-curl.html %} + +Note that when you run the preceding query in a terminal, the response is not formatted because queries submitted to OpenSearch generally return a flat JSON by default. For a human-readable response body, provide the `pretty` query parameter: -You can change most OpenSearch settings using the REST API, modify indexes, check the health of the cluster, get statistics---almost everything. +```json +curl -XGET "http://localhost:9200/students/_search?pretty" -H 'Content-Type: application/json' -d' +{ + "query": { + "match_all": {} + } +}' +``` +{% include copy.html %} +For more information about `pretty` and other useful query parameters, see [Common REST parameters]({{site.url}}{{site.baseurl}}/opensearch/common-parameters/). -When you add the document to an index, OpenSearch adds some metadata, such as the unique document *ID*: +OpenSearch returns the document that you indexed: ```json { - "_index": "", - "_type": "_doc", - "_id": "", - "_version": 1, - "_source": { - "title": "The Wind Rises", - "release_date": "2013-07-20" + "took": 12, + "timed_out": false, + "_shards": { + "total": 1, + "successful": 1, + "skipped": 0, + "failed": 0 + }, + "hits": { + "total": { + "value": 1, + "relation": "eq" + }, + "max_score": 1, + "hits": [ + { + "_index": "students", + "_id": "123456", + "_score": 1, + "_source": { + "name": "John Doe", + "grade": 12, + "gpa": 3.89, + "grad_year": 2022, + "future_plans": "John plans to be a computer science major" + } + } + ] } } ``` -Indexes also contain mappings and settings: +## Updating documents + +In OpenSearch, documents are immutable. However, you can update a document by retrieving it, updating its information, and reindexing it. You can update the whole document using the Index Document API, providing values for all existing and added fields in the document. For example, to update the `gpa` field and add an `address` field to the previously indexed document, send the following request: + +```json +PUT /students/_doc/123456 +{ + "name": "John Doe", + "grade": 12, + "gpa": 3.91, + "grad_year": 2022, + "future_plans": "John plans to be a computer science major", + "address": "123 Main St." +} +``` +{% include copy.html %} + +Alternatively, you can update parts of a document by calling the Update Document API: + +```json +POST /students/_update/123456/ +{ + "doc": { + "gpa": 3.91, + "address": "123 Main St." + } +} +``` +{% include copy.html %} + +For more information about partial document updates, see [Update Document API]({{site.url}}{{site.baseurl}}/api-reference/document-apis/update-document/). + +## Deleting documents + +To delete the document, send a delete request and provide the document ID: + +```json +DELETE /students/_doc/123456 +``` +{% include copy.html %} + +## Deleting the index + +To delete the index, send the following delete request: + +```json +DELETE /students +``` +{% include copy.html %} + +## Index settings and mappings + +OpenSearch indexes contain mappings and settings: + +- A _mapping_ is the collection of fields and the types of those fields. For more information, see [Mappings and field types]({{site.url}{{site.baseurl}}/field-types). +- _Settings_ include index data like the index name, creation date, and number of shards. For more information, see [Configuring OpenSearch]({{site.url}{{site.baseurl}}/install-and-configure/configuring-opensearch/index/). + +You can specify the settings and mappings in one request. For example, the following request specifies the number of shards for the index and maps the `name` field to `text` and the `grad_year` field to `date`: + +```json +PUT /students +{ + "settings": { + "index.number_of_shards": 1 + }, + "mappings": { + "properties": { + "name": { + "type": "text" + }, + "grad_year": { + "type": "date" + } + } + } +} +``` +{% include copy.html %} + +Now you can index the same document as you indexed in the previous section: + +```json +PUT /students/_doc/123456 +{ + "name": "John Doe", + "grade": 12, + "gpa": 3.89, + "grad_year": 2022, + "future_plans": "John plans to be a computer science major" +} +``` +{% include copy.html %} + +To view the mappings for the index fields, send the following request: + +```json +GET students/_mapping +``` +{% include copy-curl.html %} + +OpenSearch mapped the `name` and `grad_year` fields according to the types you specified and inferred the field types for the other fields: + +```json +{ + "students": { + "mappings": { + "properties": { + "future_plans": { + "type": "text", + "fields": { + "keyword": { + "type": "keyword", + "ignore_above": 256 + } + } + }, + "gpa": { + "type": "float" + }, + "grad_year": { + "type": "date" + }, + "grade": { + "type": "long" + }, + "name": { + "type": "text" + } + } + } + } +} +``` + +You cannot change the mappings once the index is created. +{: .note} + +## Next steps + +- Learn about ingestion options in [Ingest data into OpenSearch]({{site.url}}{{site.baseurl}}/getting-started/ingest-data/) + +## Further reading -- A *mapping* is the collection of *fields* that documents in the index have. In this case, those fields are `title` and `release_date`. -- Settings include data like the index name, creation date, and number of shards. \ No newline at end of file +- For information about OpenSearch REST API, see the [REST API reference]({{site.url}{{site.baseurl}}/api-reference/) +- For information about OpenSearch language clients, see [Clients]({{site.url}{{site.baseurl}}/clients/) +- For information about mappings, see [Mappings and field types]({{site.url}{{site.baseurl}}/field-types) +- For information about settings, see [Configuring OpenSearch]({{site.url}{{site.baseurl}}/install-and-configure/configuring-opensearch/index/) diff --git a/_getting-started/ingest-data.md b/_getting-started/ingest-data.md index ff9a626286..027bce8219 100644 --- a/_getting-started/ingest-data.md +++ b/_getting-started/ingest-data.md @@ -4,4 +4,52 @@ title: Ingest data nav_order: 40 --- -# Ingest your data into OpenSearch \ No newline at end of file +# Ingest your data into OpenSearch + +## Create an index and field mappings using sample data + +Create an index and define field mappings using a dataset provided by the OpenSearch Project. The same fictitious e-commerce data is also used for sample visualizations in OpenSearch Dashboards. To learn more, see [Getting started with OpenSearch Dashboards]({{site.url}}{{site.baseurl}}/dashboards/index/). + +1. Download [ecommerce-field_mappings.json](https://github.com/opensearch-project/documentation-website/blob/{{site.opensearch_major_minor_version}}/assets/examples/ecommerce-field_mappings.json). This file defines a [mapping]({{site.url}}{{site.baseurl}}/opensearch/mappings/) for the sample data you will use. + ```bash + # Using cURL: + curl -O https://raw.githubusercontent.com/opensearch-project/documentation-website/{{site.opensearch_major_minor_version}}/assets/examples/ecommerce-field_mappings.json + + # Using wget: + wget https://raw.githubusercontent.com/opensearch-project/documentation-website/{{site.opensearch_major_minor_version}}/assets/examples/ecommerce-field_mappings.json + ``` +1. Download [ecommerce.json](https://github.com/opensearch-project/documentation-website/blob/{{site.opensearch_major_minor_version}}/assets/examples/ecommerce.json). This file contains the index data formatted so that it can be ingested by the bulk API. To learn more, see [index data]({{site.url}}{{site.baseurl}}/opensearch/index-data/) and [Bulk]({{site.url}}{{site.baseurl}}/api-reference/document-apis/bulk/). + ```bash + # Using cURL: + curl -O https://raw.githubusercontent.com/opensearch-project/documentation-website/{{site.opensearch_major_minor_version}}/assets/examples/ecommerce.json + + # Using wget: + wget https://raw.githubusercontent.com/opensearch-project/documentation-website/{{site.opensearch_major_minor_version}}/assets/examples/ecommerce.json + ``` +1. Define the field mappings with the mapping file. + ```bash + curl -H "Content-Type: application/x-ndjson" -X PUT "https://localhost:9200/ecommerce" -ku admin: --data-binary "@ecommerce-field_mappings.json" + ``` +1. Upload the index to the bulk API. + ```bash + curl -H "Content-Type: application/x-ndjson" -X PUT "https://localhost:9200/ecommerce/_bulk" -ku admin: --data-binary "@ecommerce.json" + ``` +1. Query the data using the search API. The following command submits a query that will return documents where `customer_first_name` is `Sonya`. + ```bash + curl -H 'Content-Type: application/json' -X GET "https://localhost:9200/ecommerce/_search?pretty=true" -ku admin: -d' {"query":{"match":{"customer_first_name":"Sonya"}}}' + ``` + +1. Access OpenSearch Dashboards by opening `http://localhost:5601/` in a web browser on the same host that is running your OpenSearch cluster. The default username is `admin` and the password is set in your `docker-compose.yml` file in the `OPENSEARCH_INITIAL_ADMIN_PASSWORD=` setting. +1. On the top menu bar, go to **Management > Dev Tools**. +1. In the left pane of the console, enter the following: + ```json + GET ecommerce/_search + { + "query": { + "match": { + "customer_first_name": "Sonya" + } + } + } + ``` +1. Choose the triangle icon at the top right of the request to submit the query. You can also submit the request by pressing `Ctrl+Enter` (or `Cmd+Enter` for Mac users). To learn more about using the OpenSearch Dashboards console for submitting queries, see [Running queries in the console]({{site.url}}{{site.baseurl}}/dashboards/run-queries/). \ No newline at end of file diff --git a/_getting-started/intro.md b/_getting-started/intro.md index 45a65e5ef4..fcf33786ba 100644 --- a/_getting-started/intro.md +++ b/_getting-started/intro.md @@ -27,8 +27,6 @@ A _document_ is a unit that stores information, such as text or structured data. } ``` -You can use the preceding document for full-text search. - You can think of a document in several ways: - If you have a collection of encyclopedia articles, a document might represent one article. @@ -54,7 +52,7 @@ Here is how this document looks in JSON format: ## Index -An index is a collection of documents. +An _index_ is a collection of documents. You can think of an index in several ways: diff --git a/_getting-started/quickstart.md b/_getting-started/quickstart.md index 528a3cd00c..9e67e01b24 100644 --- a/_getting-started/quickstart.md +++ b/_getting-started/quickstart.md @@ -79,54 +79,6 @@ You'll need a special file, called a Compose file, that Docker Compose uses to d ``` 1. Explore OpenSearch Dashboards by opening `http://localhost:5601/` in a web browser on the same host that is running your OpenSearch cluster. The default username is `admin` and the default password is set in your `docker-compose.yml` file in the `OPENSEARCH_INITIAL_ADMIN_PASSWORD=` setting. -## Create an index and field mappings using sample data - -Create an index and define field mappings using a dataset provided by the OpenSearch Project. The same fictitious e-commerce data is also used for sample visualizations in OpenSearch Dashboards. To learn more, see [Getting started with OpenSearch Dashboards]({{site.url}}{{site.baseurl}}/dashboards/index/). - -1. Download [ecommerce-field_mappings.json](https://github.com/opensearch-project/documentation-website/blob/{{site.opensearch_major_minor_version}}/assets/examples/ecommerce-field_mappings.json). This file defines a [mapping]({{site.url}}{{site.baseurl}}/opensearch/mappings/) for the sample data you will use. - ```bash - # Using cURL: - curl -O https://raw.githubusercontent.com/opensearch-project/documentation-website/{{site.opensearch_major_minor_version}}/assets/examples/ecommerce-field_mappings.json - - # Using wget: - wget https://raw.githubusercontent.com/opensearch-project/documentation-website/{{site.opensearch_major_minor_version}}/assets/examples/ecommerce-field_mappings.json - ``` -1. Download [ecommerce.json](https://github.com/opensearch-project/documentation-website/blob/{{site.opensearch_major_minor_version}}/assets/examples/ecommerce.json). This file contains the index data formatted so that it can be ingested by the bulk API. To learn more, see [index data]({{site.url}}{{site.baseurl}}/opensearch/index-data/) and [Bulk]({{site.url}}{{site.baseurl}}/api-reference/document-apis/bulk/). - ```bash - # Using cURL: - curl -O https://raw.githubusercontent.com/opensearch-project/documentation-website/{{site.opensearch_major_minor_version}}/assets/examples/ecommerce.json - - # Using wget: - wget https://raw.githubusercontent.com/opensearch-project/documentation-website/{{site.opensearch_major_minor_version}}/assets/examples/ecommerce.json - ``` -1. Define the field mappings with the mapping file. - ```bash - curl -H "Content-Type: application/x-ndjson" -X PUT "https://localhost:9200/ecommerce" -ku admin: --data-binary "@ecommerce-field_mappings.json" - ``` -1. Upload the index to the bulk API. - ```bash - curl -H "Content-Type: application/x-ndjson" -X PUT "https://localhost:9200/ecommerce/_bulk" -ku admin: --data-binary "@ecommerce.json" - ``` -1. Query the data using the search API. The following command submits a query that will return documents where `customer_first_name` is `Sonya`. - ```bash - curl -H 'Content-Type: application/json' -X GET "https://localhost:9200/ecommerce/_search?pretty=true" -ku admin: -d' {"query":{"match":{"customer_first_name":"Sonya"}}}' - ``` - Queries submitted to the OpenSearch REST API will generally return a flat JSON by default. For a human readable response body, use the query parameter `pretty=true`. For more information about `pretty` and other useful query parameters, see [Common REST parameters]({{site.url}}{{site.baseurl}}/opensearch/common-parameters/). -1. Access OpenSearch Dashboards by opening `http://localhost:5601/` in a web browser on the same host that is running your OpenSearch cluster. The default username is `admin` and the password is set in your `docker-compose.yml` file in the `OPENSEARCH_INITIAL_ADMIN_PASSWORD=` setting. -1. On the top menu bar, go to **Management > Dev Tools**. -1. In the left pane of the console, enter the following: - ```json - GET ecommerce/_search - { - "query": { - "match": { - "customer_first_name": "Sonya" - } - } - } - ``` -1. Choose the triangle icon at the top right of the request to submit the query. You can also submit the request by pressing `Ctrl+Enter` (or `Cmd+Enter` for Mac users). To learn more about using the OpenSearch Dashboards console for submitting queries, see [Running queries in the console]({{site.url}}{{site.baseurl}}/dashboards/run-queries/). - ## Next steps You successfully deployed your own OpenSearch cluster with OpenSearch Dashboards and added some sample data. Now you're ready to learn about configuration and functionality in more detail. Here are a few recommendations on where to begin: From 5b04bdce0ca8310c7d019e11b7a5b474ea976eb1 Mon Sep 17 00:00:00 2001 From: Fanit Kolchina Date: Tue, 2 Apr 2024 14:32:25 -0400 Subject: [PATCH 04/23] Change examples Signed-off-by: Fanit Kolchina --- _getting-started/communicate.md | 153 +++++------ _getting-started/ingest-data.md | 110 ++++++-- _getting-started/intro.md | 58 ++--- _getting-started/quickstart.md | 80 ++++-- _getting-started/search-data.md | 439 +++++++++++++++++++++++++++++++- 5 files changed, 673 insertions(+), 167 deletions(-) diff --git a/_getting-started/communicate.md b/_getting-started/communicate.md index 45bfd48073..501cca65d3 100644 --- a/_getting-started/communicate.md +++ b/_getting-started/communicate.md @@ -6,34 +6,55 @@ nav_order: 30 # Communicate with OpenSearch -You can communicate with OpenSearch using REST API or one of the OpenSearch language clients. This page introduces the OpenSearch REST API. If you need to communicate with OpenSearch in your programming language, see the [Clients]({{site.url}}{{site.baseurl}}/clients/) section for a list of available clients. +You can communicate with OpenSearch using the REST API or one of the OpenSearch language clients. This page introduces the OpenSearch REST API. If you need to communicate with OpenSearch in your programming language, see the [Clients]({{site.url}}{{site.baseurl}}/clients/) section for a list of available clients. ## OpenSearch REST API -You interact with OpenSearch clusters using the REST API, which offers a lot of flexibility. You can change most OpenSearch settings using the REST API, modify indexes, check the health of the cluster, get statistics---almost everything. You can use clients like [cURL](https://curl.se/) or any programming language that can send HTTP requests. +You interact with OpenSearch clusters using the REST API, which offers a lot of flexibility. Through the REST API, you can change most OpenSearch settings, modify indexes, check the health of the cluster, get statistics---almost everything. You can use clients like [cURL](https://curl.se/) or any programming language that can send HTTP requests. You can send HTTP requests in your terminal or in the Dev Tools console in OpenSearch Dashboards. ### Sending requests in the terminal -To send a cURL request in your terminal, enter the request in cURL format. For example, to view the indexes in your cluster, send a CAT indices request. +When sending cURL requests in a terminal, the request format varies depending on whether you're using the Security plugin. As an example, consider a request to the Cluster Health API. -If you're not using the Security plugin, the CAT indices request is as follows: +If you're not using the Security plugin, send the following request: -```json -curl -XGET "http://localhost:9200/_cat/indices" +```bash +curl -XGET "http://localhost:9200/_cluster/health" ``` {% include copy.html %} If you're using the Security plugin, provide the user name and password in the request: -```json -curl -H 'Content-Type: application/json' -X GET "https://localhost:9200/_cat/indices" -ku admin: +```bash +curl -X GET "http://localhost:9200/_cluster/health" -ku admin: ``` {% include copy.html %} The default username is `admin` and the password is set in your `docker-compose.yml` file in the `OPENSEARCH_INITIAL_ADMIN_PASSWORD=` setting. +Queries submitted to OpenSearch generally return a flat JSON by default. For a human-readable response body, provide the `pretty` query parameter: + +```bash +curl -XGET "http://localhost:9200/_cluster/health?pretty" +``` +{% include copy.html %} + +For more information about `pretty` and other useful query parameters, see [Common REST parameters]({{site.url}}{{site.baseurl}}/opensearch/common-parameters/). + +For requests that contain a body, specify the `Content-Type` header and provide the request payload in the `-d` (data) oprion: + +```json +curl -XGET "http://localhost:9200/students/_search?pretty" -H 'Content-Type: application/json' -d' +{ + "query": { + "match_all": {} + } +}' +``` +{% include copy.html %} + ### Sending requests in Dev Tools The Dev Tools console in OpenSearch Dashboards uses a simplified syntax to format REST requests compared to the cURL command. To send requests in Dev Tools, use the following steps: @@ -42,11 +63,13 @@ The Dev Tools console in OpenSearch Dashboards uses a simplified syntax to forma 1. On the top menu bar, go to **Management > Dev Tools**. 1. In the left pane of the console, enter the following request: ```json - GET _cat/indices + GET _cluster/health ``` {% include copy-curl.html %} 1. Choose the triangle icon at the upper right of the request to submit the query. You can also submit the request by pressing `Ctrl+Enter` (or `Cmd+Enter` for Mac users). To learn more about using the OpenSearch Dashboards console for submitting queries, see [Running queries in the console]({{site.url}}{{site.baseurl}}/dashboards/run-queries/). +In the following sections, and in most of OpenSearch documentation, requests are presented in the Dev Tools console format. + ## Indexing documents To add a JSON document to an OpenSearch index (that is, to _index_ a document), you send an HTTP request that has the following header: @@ -58,27 +81,27 @@ PUT https://://_doc/ For example, to index a document representing a student, you can send the following request: ```json -PUT /students/_doc/123456 +PUT /students/_doc/1 { "name": "John Doe", - "grade": 12, "gpa": 3.89, - "grad_year": 2022, - "future_plans": "John plans to be a computer science major" + "grad_year": 2022 } ``` {% include copy-curl.html %} -Once you send the preceding request, OpenSearch creates an index called `students` and stores the ingested document in the index. If you don't provide an ID for your document, OpenSearch generates a document ID. For the preceding request, you have specified the document ID to be the student ID (`123456`). +Once you send the preceding request, OpenSearch creates an index called `students` and stores the ingested document in the index. If you don't provide an ID for your document, OpenSearch generates a document ID. For the preceding request, you have specified the document ID to be the student ID (`1`). -### Dynamic mapping +To learn more about indexing, see [Managing indexes]({{site.url}}{{site.baseurl}}/im-plugin/). + +## Dynamic mapping When you index a document, OpenSearch infers the field types from the JSON types submitted in the document. This process is called _dynamic mapping_. For more information, see [Dynamic mapping]({{site.url}}{{site.baseurl}}/field-types/#dynamic-mapping). To view the inferred field data types, send a request to the `_mapping` endpoint: ```json -GET students/_mapping +GET /students/_mapping ``` {% include copy-curl.html %} @@ -89,24 +112,12 @@ OpenSearch responds with the field `type` for each field: "students": { "mappings": { "properties": { - "future_plans": { - "type": "text", - "fields": { - "keyword": { - "type": "keyword", - "ignore_above": 256 - } - } - }, "gpa": { "type": "float" }, "grad_year": { "type": "long" }, - "grade": { - "type": "long" - }, "name": { "type": "text", "fields": { @@ -122,9 +133,9 @@ OpenSearch responds with the field `type` for each field: } ``` -OpenSearch mapped the numeric fields to the `float` and `long` types. Notice that OpenSearch mapped the text fields to `text` and added a `name.keyword` subfield mapped to `keyword`. Fields mapped to `text` are used for full-text search, while fields mapped to `keyword` are used for exact term search. +OpenSearch mapped the numeric fields to the `float` and `long` types. Notice that OpenSearch mapped the `name` text field to `text` and added a `name.keyword` subfield mapped to `keyword`. Fields mapped to `text` are analyzed (lowercased and split into terms) and can be used for full-text search. Fields mapped to `keyword` are used for exact term search. -OpenSearch mapped the `grad_year` field to `long`. If you want to map it to the `date` type instead, you need to [delete the index](#deleting-the-index) and recreate it, specifying the mappings you want explicitly. For steps to specify explicit mappings, see [Index settings and mappings](#index-settings-and-mappings). +OpenSearch mapped the `grad_year` field to `long`. If you want to map it to the `date` type instead, you need to [delete the index](#deleting-the-index) and then recreate it, specifying the mappings you want explicitly. For steps to specify explicit mappings, see [Index settings and mappings](#index-settings-and-mappings). ## Searching for documents @@ -140,20 +151,6 @@ GET /students/_search ``` {% include copy-curl.html %} -Note that when you run the preceding query in a terminal, the response is not formatted because queries submitted to OpenSearch generally return a flat JSON by default. For a human-readable response body, provide the `pretty` query parameter: - -```json -curl -XGET "http://localhost:9200/students/_search?pretty" -H 'Content-Type: application/json' -d' -{ - "query": { - "match_all": {} - } -}' -``` -{% include copy.html %} - -For more information about `pretty` and other useful query parameters, see [Common REST parameters]({{site.url}}{{site.baseurl}}/opensearch/common-parameters/). - OpenSearch returns the document that you indexed: ```json @@ -175,14 +172,12 @@ OpenSearch returns the document that you indexed: "hits": [ { "_index": "students", - "_id": "123456", + "_id": "1", "_score": 1, "_source": { "name": "John Doe", - "grade": 12, "gpa": 3.89, - "grad_year": 2022, - "future_plans": "John plans to be a computer science major" + "grad_year": 2022 } } ] @@ -190,18 +185,18 @@ OpenSearch returns the document that you indexed: } ``` +For more information about searching, see [Search your data]({{site.url}}{{site.baseurl}}/getting-started/search-data/). + ## Updating documents In OpenSearch, documents are immutable. However, you can update a document by retrieving it, updating its information, and reindexing it. You can update the whole document using the Index Document API, providing values for all existing and added fields in the document. For example, to update the `gpa` field and add an `address` field to the previously indexed document, send the following request: ```json -PUT /students/_doc/123456 +PUT /students/_doc/1 { "name": "John Doe", - "grade": 12, "gpa": 3.91, "grad_year": 2022, - "future_plans": "John plans to be a computer science major", "address": "123 Main St." } ``` @@ -210,7 +205,7 @@ PUT /students/_doc/123456 Alternatively, you can update parts of a document by calling the Update Document API: ```json -POST /students/_update/123456/ +POST /students/_update/1/ { "doc": { "gpa": 3.91, @@ -218,7 +213,7 @@ POST /students/_update/123456/ } } ``` -{% include copy.html %} +{% include copy-curl.html %} For more information about partial document updates, see [Update Document API]({{site.url}}{{site.baseurl}}/api-reference/document-apis/update-document/). @@ -227,25 +222,25 @@ For more information about partial document updates, see [Update Document API]({ To delete the document, send a delete request and provide the document ID: ```json -DELETE /students/_doc/123456 +DELETE /students/_doc/1 ``` -{% include copy.html %} +{% include copy-curl.html %} ## Deleting the index -To delete the index, send the following delete request: +To delete the index, send the following request: ```json DELETE /students ``` -{% include copy.html %} +{% include copy-curl.html %} ## Index settings and mappings -OpenSearch indexes contain mappings and settings: +OpenSearch indexes are configured with mappings and settings: -- A _mapping_ is the collection of fields and the types of those fields. For more information, see [Mappings and field types]({{site.url}{{site.baseurl}}/field-types). -- _Settings_ include index data like the index name, creation date, and number of shards. For more information, see [Configuring OpenSearch]({{site.url}{{site.baseurl}}/install-and-configure/configuring-opensearch/index/). +- A _mapping_ is the collection of fields and the types of those fields. For more information, see [Mappings and field types]({{site.url}}{{site.baseurl}}/field-types/). +- _Settings_ include index data like the index name, creation date, and number of shards. For more information, see [Configuring OpenSearch]({{site.url}}{{site.baseurl}}/install-and-configure/configuring-opensearch/index/). You can specify the settings and mappings in one request. For example, the following request specifies the number of shards for the index and maps the `name` field to `text` and the `grad_year` field to `date`: @@ -267,26 +262,24 @@ PUT /students } } ``` -{% include copy.html %} +{% include copy-curl.html %} Now you can index the same document as you indexed in the previous section: ```json -PUT /students/_doc/123456 +PUT /students/_doc/1 { "name": "John Doe", - "grade": 12, "gpa": 3.89, - "grad_year": 2022, - "future_plans": "John plans to be a computer science major" + "grad_year": 2022 } ``` -{% include copy.html %} +{% include copy-curl.html %} To view the mappings for the index fields, send the following request: ```json -GET students/_mapping +GET /students/_mapping ``` {% include copy-curl.html %} @@ -297,24 +290,12 @@ OpenSearch mapped the `name` and `grad_year` fields according to the types you s "students": { "mappings": { "properties": { - "future_plans": { - "type": "text", - "fields": { - "keyword": { - "type": "keyword", - "ignore_above": 256 - } - } - }, "gpa": { "type": "float" }, "grad_year": { "type": "date" }, - "grade": { - "type": "long" - }, "name": { "type": "text" } @@ -327,13 +308,13 @@ OpenSearch mapped the `name` and `grad_year` fields according to the types you s You cannot change the mappings once the index is created. {: .note} -## Next steps +## Further reading -- Learn about ingestion options in [Ingest data into OpenSearch]({{site.url}}{{site.baseurl}}/getting-started/ingest-data/) +- For information about OpenSearch REST API, see the [REST API reference]({{site.url}}{{site.baseurl}}/api-reference/) +- For information about OpenSearch language clients, see [Clients]({{site.url}}{{site.baseurl}}/clients/) +- For information about mappings, see [Mappings and field types]({{site.url}}{{site.baseurl}}/field-types) +- For information about settings, see [Configuring OpenSearch]({{site.url}}{{site.baseurl}}/install-and-configure/configuring-opensearch/index/) -## Further reading +## Next steps -- For information about OpenSearch REST API, see the [REST API reference]({{site.url}{{site.baseurl}}/api-reference/) -- For information about OpenSearch language clients, see [Clients]({{site.url}{{site.baseurl}}/clients/) -- For information about mappings, see [Mappings and field types]({{site.url}{{site.baseurl}}/field-types) -- For information about settings, see [Configuring OpenSearch]({{site.url}{{site.baseurl}}/install-and-configure/configuring-opensearch/index/) +- Learn about ingestion options in [Ingest data into OpenSearch]({{site.url}}{{site.baseurl}}/getting-started/ingest-data/) \ No newline at end of file diff --git a/_getting-started/ingest-data.md b/_getting-started/ingest-data.md index 027bce8219..22572739b3 100644 --- a/_getting-started/ingest-data.md +++ b/_getting-started/ingest-data.md @@ -6,50 +6,106 @@ nav_order: 40 # Ingest your data into OpenSearch -## Create an index and field mappings using sample data +There are several ways to ingest data into OpenSearch: -Create an index and define field mappings using a dataset provided by the OpenSearch Project. The same fictitious e-commerce data is also used for sample visualizations in OpenSearch Dashboards. To learn more, see [Getting started with OpenSearch Dashboards]({{site.url}}{{site.baseurl}}/dashboards/index/). +- Ingest individual documents. For more information, see [Indexing documents]({{site.url}}{{site.baseurl}}/getting-started/communicate/#indexing-documents). +- Index multiple documents in bulk. For more information, see [Bulk indexing](#bulk-indexing). +- Use Data Prepper---an OpenSearch server-side data collector that can enrich data for downstream analytics and visualization. For more information, see [Data Prepper]({{site.url}}{{site.baseurl}}/data-prepper/). +- Use other ingestion tools. For more information, see [OpenSearch tools]({{site.url}}{{site.baseurl}}/tools/). + +## Bulk indexing + +To index documents in bulk, you can use the [Bulk API]({{site.url}}{{site.baseurl}}/api-reference/document-apis/bulk/). For example, if you want to index several documents into the `students` index, send the following request: + +```json +POST _bulk +{ "create": { "_index": "students", "_id": "2" } } +{ "name": "Jonathan Powers", "gpa": 3.85, "grad_year": 2025 } +{ "create": { "_index": "students", "_id": "3" } } +{ "name": "Jane Doe", "gpa": 3.52, "grad_year": 2024 } +``` +{% include copy-curl.html %} + +## Experiment with sample data + +OpenSearch provides a fictitious e-commerce dataset that you can use to experiment with REST API requests and OpenSearch Dashboards visualizations. Create an index and define field mappings using a dataset provided by the OpenSearch Project. + +### Create a sample index + +Use the following steps to create a sample index and define field mappings for the document fields: 1. Download [ecommerce-field_mappings.json](https://github.com/opensearch-project/documentation-website/blob/{{site.opensearch_major_minor_version}}/assets/examples/ecommerce-field_mappings.json). This file defines a [mapping]({{site.url}}{{site.baseurl}}/opensearch/mappings/) for the sample data you will use. + + To use cURL, send the following request: + ```bash - # Using cURL: curl -O https://raw.githubusercontent.com/opensearch-project/documentation-website/{{site.opensearch_major_minor_version}}/assets/examples/ecommerce-field_mappings.json + ``` + {% include copy.html %} + + To use wget, send the following request: - # Using wget: + ``` wget https://raw.githubusercontent.com/opensearch-project/documentation-website/{{site.opensearch_major_minor_version}}/assets/examples/ecommerce-field_mappings.json ``` -1. Download [ecommerce.json](https://github.com/opensearch-project/documentation-website/blob/{{site.opensearch_major_minor_version}}/assets/examples/ecommerce.json). This file contains the index data formatted so that it can be ingested by the bulk API. To learn more, see [index data]({{site.url}}{{site.baseurl}}/opensearch/index-data/) and [Bulk]({{site.url}}{{site.baseurl}}/api-reference/document-apis/bulk/). + {% include copy.html %} + +1. Download [ecommerce.json](https://github.com/opensearch-project/documentation-website/blob/{{site.opensearch_major_minor_version}}/assets/examples/ecommerce.json). This file contains the index data formatted so that it can be ingested by the Bulk API: + + To use cURL, send the following request: + ```bash - # Using cURL: curl -O https://raw.githubusercontent.com/opensearch-project/documentation-website/{{site.opensearch_major_minor_version}}/assets/examples/ecommerce.json + ``` + {% include copy.html %} + + To use wget, send the following request: - # Using wget: + ``` wget https://raw.githubusercontent.com/opensearch-project/documentation-website/{{site.opensearch_major_minor_version}}/assets/examples/ecommerce.json ``` -1. Define the field mappings with the mapping file. + {% include copy.html %} + +1. Define the field mappings provided in the mapping file: ```bash curl -H "Content-Type: application/x-ndjson" -X PUT "https://localhost:9200/ecommerce" -ku admin: --data-binary "@ecommerce-field_mappings.json" ``` -1. Upload the index to the bulk API. + {% include copy.html %} + +1. Upload the documents using the Bulk API: + ```bash curl -H "Content-Type: application/x-ndjson" -X PUT "https://localhost:9200/ecommerce/_bulk" -ku admin: --data-binary "@ecommerce.json" ``` -1. Query the data using the search API. The following command submits a query that will return documents where `customer_first_name` is `Sonya`. - ```bash - curl -H 'Content-Type: application/json' -X GET "https://localhost:9200/ecommerce/_search?pretty=true" -ku admin: -d' {"query":{"match":{"customer_first_name":"Sonya"}}}' - ``` - -1. Access OpenSearch Dashboards by opening `http://localhost:5601/` in a web browser on the same host that is running your OpenSearch cluster. The default username is `admin` and the password is set in your `docker-compose.yml` file in the `OPENSEARCH_INITIAL_ADMIN_PASSWORD=` setting. -1. On the top menu bar, go to **Management > Dev Tools**. -1. In the left pane of the console, enter the following: - ```json - GET ecommerce/_search - { - "query": { - "match": { - "customer_first_name": "Sonya" - } - } + {% include copy.html %} + +### Query the data + +Query the data using the Search API. The following query searches for documents where `customer_first_name` is `Sonya`: + +```json +GET ecommerce/_search +{ + "query": { + "match": { + "customer_first_name": "Sonya" } - ``` -1. Choose the triangle icon at the top right of the request to submit the query. You can also submit the request by pressing `Ctrl+Enter` (or `Cmd+Enter` for Mac users). To learn more about using the OpenSearch Dashboards console for submitting queries, see [Running queries in the console]({{site.url}}{{site.baseurl}}/dashboards/run-queries/). \ No newline at end of file + } +} +``` +{% include copy-curl.html %} + +### Visualize the data + +To learn how to use OpenSearch Dashboards to visualize the data, see the [OpenSearch Dashboards quickstart guide]({{site.url}}{{site.baseurl}}/dashboards/quickstart/). + +## Further reading + +- For information about Data Prepper, see [Data Prepper]({{site.url}}{{site.baseurl}}/data-prepper/) +- For information about ingestion tools, see [OpenSearch tools]({{site.url}}{{site.baseurl}}/tools/) +- For information about OpenSearch Dashboards, see [OpenSearch Dashboards quickstart guide]({{site.url}}{{site.baseurl}}/dashboards/quickstart/) +- For information about bulk indexing, see [Bulk API]({{site.url}}{{site.baseurl}}/api-reference/document-apis/bulk/) + +## Next steps + +- Learn about search options in [Search your data]({{site.url}}{{site.baseurl}}/getting-started/search-data/) \ No newline at end of file diff --git a/_getting-started/intro.md b/_getting-started/intro.md index fcf33786ba..92bd36e495 100644 --- a/_getting-started/intro.md +++ b/_getting-started/intro.md @@ -13,43 +13,32 @@ OpenSearch is a distributed search and analytics engine, which supports various ## Document -A _document_ is a unit that stores information, such as text or structured data. In OpenSearch, documents are stored in [JSON](https://www.json.org/) format. For example, a document may represent a line in a Shakespeare play: - -```json -{ - "type": "line", - "line_id": 5, - "play_name": "Henry IV", - "speech_number": 1, - "line_number": "1.1.2", - "speaker": "KING HENRY IV", - "text_entry": "Find we a time for frighted peace to pant," -} -``` +A _document_ is a unit that stores information (text or structured data). In OpenSearch, documents are stored in [JSON](https://www.json.org/) format. You can think of a document in several ways: -- If you have a collection of encyclopedia articles, a document might represent one article. +- In a database of students, a document might represent one student. - When you search for information, OpenSearch returns documents related to your search. - If you're familiar with traditional databases, a document represents a row. -For example, in a school database, a document might represent one student and contain structured data like the student ID and name: +For example, in a school database, a document might represent one student and contain the following data. -ID | First name | Last name | Address | +ID | Name | GPA | Graduation year | :--- | :--- | :--- | :--- | -123456 | John | Doe | 121 Main St. | +1 | John Doe | 3.89 | 2022 | -Here is how this document looks in JSON format: +Here is what this document looks like in JSON format: ```json { - "id": "123456", - "first_name": John, - "last_name": "Doe", - "address": "121 Main St." + "name": "John Doe", + "gpa": 3.89, + "grad_year": 2022 } ``` +You'll learn about how document IDs are assigned in [Indexing documents]({{site.url}}{{site.baseurl}}/getting-started/communicate/#indexing-documents). + ## Index An _index_ is a collection of documents. @@ -60,14 +49,17 @@ You can think of an index in several ways: - When you search for information, you query data contained in an index. - If you're familiar with traditional databases, a document represents a database table. -For example, in a school database, an index might contain all students in the school: +For example, in a school database, an index might contain all students in the school. -ID | First name | Last name | Address +ID | Name | GPA | Graduation year :--- | :--- | :--- | :--- -123456 | John | Doe | 121 Main St. -123457 | Jane | Smith | 1 Chestnut St. +1 | John Doe | 3.89 | 2022 +2 | Jonathan Powers | 3.85 | 2025 +3 | Jane Doe | 3.52 | 2024 -An OpenSearch index is represented as an _inverted index_. An inverted index maps words to the documents that they occur in. For example, consider an index containing the following two documents: +## Inverted index + +An OpenSearch index uses a data structure called an _inverted index_. An inverted index maps words to the documents that they occur in. For example, consider an index containing the following two documents: - Document 1 : "Beauty is in the eye of the beholder" - Document 2: "Beauty and the beast" @@ -89,18 +81,20 @@ beast | 2 In addition to the document ID, OpenSearch stores the position of the word within that document for phrase queries, where words must appear next to each other. -When searching for documents, you need to make sure that the word is _relevant_. For example, the word `the` appears in most English phrases but searching for this word is meaningless. OpenSearch determines the relevance of a term by calculating two values: +## Relevance + +When searching for documents using a phrase, you need to make sure that the words you are searching for are _relevant_. For example, the word `the` appears in most English phrases but searching for this word is meaningless. OpenSearch determines the relevance of a term by calculating two values: - _Term frequency_: how often a word appears in the document - _Document frequency_: how often a word appears in all documents -Then, the relevance of a word is calculated as $$ relevance = { \text {term frequency} \over \text {document frequency} }. $$ +The relevance of a word is calculated as $$ relevance = { \text {term frequency} \over \text {document frequency} }. $$ This score is called term frequency/inverse document frequency (TF/IDF). For more information, see [TF/IDF](https://en.wikipedia.org/wiki/Tf%E2%80%93idf). -OpenSearch scores the results in terms of relevance and returns them sorted by relevance. +OpenSearch uses the BM25 ranking algorithm to calculate relevance scores and returns the results sorted by relevance. To learn more, see [Okapi BM25](https://en.wikipedia.org/wiki/Okapi_BM25). ## Clusters and nodes -OpenSearch is designed to be a distributed search engine. OpenSearch can run on one or more *nodes*, servers that store your data and process search requests. An OpenSearch *cluster* is a collection of nodes. +OpenSearch is designed to be a distributed search engine. OpenSearch can run on one or more _nodes_---servers that store your data and process search requests. An OpenSearch *cluster* is a collection of nodes. You can run OpenSearch locally on a laptop---its system requirements are minimal---but you can also scale a single cluster to hundreds of powerful machines in a data center. @@ -116,7 +110,7 @@ OpenSearch splits indexes into *shards*. Each shard stores a subset of all docum An index is split into shards -Shards are used for even distribution across nodes in a cluster. For example, a 400-GB index might be too large for any single node in your cluster to handle, but split into ten shards, each one 40 GB, OpenSearch can distribute the shards across ten nodes and work with each shard individually. When you index documents into OpenSearch, the documents are directed to a particular shard, and shards can reside on different nodes in a cluster. For example, consider a cluster with two indexes: index 1 and index 2. Index 1 is split into 2 shards, and index 2 is split into 4 shards. The shards are distributed across nodes 1 and 2, as shown in the following image. +Shards are used for even distribution across nodes in a cluster. For example, a 400-GB index might be too large for any single node in your cluster to handle, but split into ten shards, each one 40 GB, OpenSearch can distribute the shards across ten nodes and work with each shard individually. For example, consider a cluster with two indexes: index 1 and index 2. Index 1 is split into 2 shards, and index 2 is split into 4 shards. The shards are distributed across nodes 1 and 2, as shown in the following image. A cluster containing two indexes and two nodes diff --git a/_getting-started/quickstart.md b/_getting-started/quickstart.md index 9e67e01b24..b8ae44bd32 100644 --- a/_getting-started/quickstart.md +++ b/_getting-started/quickstart.md @@ -2,7 +2,6 @@ layout: default title: Installation quickstart nav_order: 3 -permalink: /quickstart/ redirect_from: - /opensearch/install/quickstart/ - /quickstart/ @@ -19,33 +18,63 @@ The Docker Compose commands used in this guide are written with a hyphen (for ex You'll need a special file, called a Compose file, that Docker Compose uses to define and create the containers in your cluster. The OpenSearch Project provides a sample Compose file that you can use to get started. Learn more about working with Compose files by reviewing the official [Compose specification](https://docs.docker.com/compose/compose-file/). -1. Before running OpenSearch on your machine, you should disable memory paging and swapping performance on the host to improve performance and increase the number of memory maps available to OpenSearch. See [important system settings]({{site.url}}{{site.baseurl}}/opensearch/install/important-settings/) for more information. +1. Before running OpenSearch on your machine, you should disable memory paging and swapping performance on the host to improve performance and increase the number of memory maps available to OpenSearch. + + Disable memory paging and swapping: + ```bash - # Disable memory paging and swapping. sudo swapoff -a + ``` + {% include copy.html %} + + Edit the sysctl config file that defines the host's max map count: - # Edit the sysctl config file that defines the host's max map count. + ```bash sudo vi /etc/sysctl.conf + ``` + {% include copy.html %} - # Set max map count to the recommended value of 262144. + Set max map count to the recommended value of `262144`: + + ```bash vm.max_map_count=262144 + ``` + {% include copy.html %} - # Reload the kernel parameters. + Reload the kernel parameters: + + ``` sudo sysctl -p ``` + {% include copy.html %} + + For more information, see [important system settings]({{site.url}}{{site.baseurl}}/opensearch/install/important-settings/). + 1. Download the sample Compose file to your host. You can download the file with command line utilities like `curl` and `wget`, or you can manually copy [docker-compose.yml](https://github.com/opensearch-project/documentation-website/blob/{{site.opensearch_major_minor_version}}/assets/examples/docker-compose.yml) from the OpenSearch Project documentation-website repository using a web browser. + + To use cURL, send the following request: + ```bash - # Using cURL: curl -O https://raw.githubusercontent.com/opensearch-project/documentation-website/{{site.opensearch_major_minor_version}}/assets/examples/docker-compose.yml + ``` + {% include copy.html %} + + To use wget, send the following request: - # Using wget: + ``` wget https://raw.githubusercontent.com/opensearch-project/documentation-website/{{site.opensearch_major_minor_version}}/assets/examples/docker-compose.yml ``` -1. In your terminal application, navigate to the directory containing the `docker-compose.yml` file you just downloaded, and run the following command to create and start the cluster as a background process. + {% include copy.html %} + +1. In your terminal application, navigate to the directory containing the `docker-compose.yml` file you downloaded, and run the following command to create and start the cluster as a background process: + ```bash docker-compose up -d ``` + {% include copy.html %} + 1. Confirm that the containers are running with the command `docker-compose ps`. You should see an output like the following: + ```bash $ docker-compose ps NAME COMMAND SERVICE STATUS PORTS @@ -53,11 +82,16 @@ You'll need a special file, called a Compose file, that Docker Compose uses to d opensearch-node1 "./opensearch-docker…" opensearch-node1 running 0.0.0.0:9200->9200/tcp, 9300/tcp, 0.0.0.0:9600->9600/tcp, 9650/tcp opensearch-node2 "./opensearch-docker…" opensearch-node2 running 9200/tcp, 9300/tcp, 9600/tcp, 9650/tcp ``` -1. Query the OpenSearch REST API to verify that the service is running. You should use `-k` (also written as `--insecure`) to disable hostname checking because the default security configuration uses demo certificates. Use `-u` to pass the default username and password (`admin:`). + +1. Query the OpenSearch REST API to verify that the service is running. You should use `-k` (also written as `--insecure`) to disable hostname checking because the default security configuration uses demo certificates. Use `-u` to pass the default username and password (`admin:`): + ```bash curl https://localhost:9200 -ku admin: ``` - Sample response: + {% include copy.html %} + + The response confirms that the installation is successful: + ```json { "name" : "opensearch-node1", @@ -79,16 +113,6 @@ You'll need a special file, called a Compose file, that Docker Compose uses to d ``` 1. Explore OpenSearch Dashboards by opening `http://localhost:5601/` in a web browser on the same host that is running your OpenSearch cluster. The default username is `admin` and the default password is set in your `docker-compose.yml` file in the `OPENSEARCH_INITIAL_ADMIN_PASSWORD=` setting. -## Next steps - -You successfully deployed your own OpenSearch cluster with OpenSearch Dashboards and added some sample data. Now you're ready to learn about configuration and functionality in more detail. Here are a few recommendations on where to begin: -- [About the Security plugin]({{site.url}}{{site.baseurl}}/security/index/) -- [OpenSearch configuration]({{site.url}}{{site.baseurl}}/install-and-configure/configuring-opensearch/) -- [OpenSearch plugin installation]({{site.url}}{{site.baseurl}}/opensearch/install/plugins/) -- [Getting started with OpenSearch Dashboards]({{site.url}}{{site.baseurl}}/dashboards/index/) -- [OpenSearch tools]({{site.url}}{{site.baseurl}}/tools/index/) -- [Index APIs]({{site.url}}{{site.baseurl}}/api-reference/index-apis/index/) - ## Common issues Review these common issues and suggested solutions if your containers fail to start or exit unexpectedly. @@ -116,3 +140,17 @@ opensearch-node1 | ERROR: [1] bootstrap checks failed opensearch-node1 | [1]: max virtual memory areas vm.max_map_count [65530] is too low, increase to at least [262144] opensearch-node1 | ERROR: OpenSearch did not exit normally - check the logs at /usr/share/opensearch/logs/opensearch-cluster.log ``` + +## Further reading + +You successfully deployed your own OpenSearch cluster with OpenSearch Dashboards and added some sample data. Now you're ready to learn about configuration and functionality in more detail. Here are a few recommendations on where to begin: +- [About the Security plugin]({{site.url}}{{site.baseurl}}/security/index/) +- [OpenSearch configuration]({{site.url}}{{site.baseurl}}/install-and-configure/configuring-opensearch/) +- [OpenSearch plugin installation]({{site.url}}{{site.baseurl}}/opensearch/install/plugins/) +- [Getting started with OpenSearch Dashboards]({{site.url}}{{site.baseurl}}/dashboards/index/) +- [OpenSearch tools]({{site.url}}{{site.baseurl}}/tools/index/) +- [Index APIs]({{site.url}}{{site.baseurl}}/api-reference/index-apis/index/) + +## Next steps + +- Learn about how to send requests to OpenSearch in [Communicate with OpenSearch]({{site.url}}{{site.baseurl}}/getting-started/communicate/) diff --git a/_getting-started/search-data.md b/_getting-started/search-data.md index ee4e2500ac..758196b16c 100644 --- a/_getting-started/search-data.md +++ b/_getting-started/search-data.md @@ -4,4 +4,441 @@ title: Search your data nav_order: 50 --- -# Search your data \ No newline at end of file +# Search your data + +In OpenSearch, there are several ways to search data: + +- [Query domain-specific language (DSL)]({{site.url}}{{site.baseurl}}/query-dsl/index/): The primary OpenSearch query language that supports creating complex, fully customizable queries. +- [Query string query language]({{site.url}}{{site.baseurl}}/query-dsl/full-text/query-string/): A scaled-down query language that you can use in a query parameter of a search request or in OpenSearch Dashboards. +- [SQL]({{site.url}}{{site.baseurl}}/search-plugins/sql/sql/index/): A traditional query language that bridges the gap between traditional relational database concepts and the flexibility of OpenSearch’s document-oriented data storage. +- [Piped Processing Language (PPL)]({{site.url}}{{site.baseurl}}/search-plugins/sql/ppl/index/): The primary language used with observability in OpenSearch. PPL uses a pipe syntax that chains commands into a query. +- [Dashboards Query Language (DQL)]({{site.url}}{{site.baseurl}}/dashboards/dql/): A simple text-based query language for filtering data in OpenSearch Dashboards. + +## Prepare the data + +For this tutorial, you'll need to index student data if you haven't done so. You can start by deleting the `students` index (`DELETE /students`) and then sending the following bulk request: + +```json +POST _bulk +{ "create": { "_index": "students", "_id": "1" } } +{ "name": "John Doe", "gpa": 3.89, "grad_year": 2022} +{ "create": { "_index": "students", "_id": "2" } } +{ "name": "Jonathan Powers", "gpa": 3.85, "grad_year": 2025 } +{ "create": { "_index": "students", "_id": "3" } } +{ "name": "Jane Doe", "gpa": 3.52, "grad_year": 2024 } +``` +{% include copy-curl.html %} + +## Retrieve all documents in an index + +To retrieve all documents in an index, send the following request: + +```json +GET /students/_search +``` +{% include copy-curl.html %} + +The preceding request is an equivalent of the `match_all` query, which matches all documents in the index: + +```json +GET /students/_search +{ + "query": { + "match_all": {} + } +} +``` +{% include copy-curl.html %} + +OpenSearch returns the matching documents: + +```json +{ + "took": 12, + "timed_out": false, + "_shards": { + "total": 1, + "successful": 1, + "skipped": 0, + "failed": 0 + }, + "hits": { + "total": { + "value": 3, + "relation": "eq" + }, + "max_score": 1, + "hits": [ + { + "_index": "students", + "_id": "1", + "_score": 1, + "_source": { + "name": "John Doe", + "gpa": 3.89, + "grad_year": 2022 + } + }, + { + "_index": "students", + "_id": "2", + "_score": 1, + "_source": { + "name": "Jonathan Powers", + "gpa": 3.85, + "grad_year": 2025 + } + }, + { + "_index": "students", + "_id": "3", + "_score": 1, + "_source": { + "name": "Jane Doe", + "gpa": 3.52, + "grad_year": 2024 + } + } + ] + } +} +``` + +## Response fields + +The preceding response contains the following fields. + + +### took + + +The `took` field contains the amount of time the query took to run, in milliseconds. + + +### timed_out + + +This field specifies whether the request timed out. If a request timed out, OpenSearch returns those results that were gathered before the timeout. You can set the desired timeout value by providing the `timeout` query parameter: + +```json +GET /students/_search?timeout=20ms +``` +{% include copy-curl.html %} + + +### _shards + + +The `_shards` object specifies the total number of shards the query ran on, and the number of shards that were successful or failed. A shard may fail if the shard itself and all its replicas are unavailable. If any of the involved shards failed, OpenSearch continues to run the query on the remaining shards. + + +### hits + + +The `hits` object contains the total number of matching documents and the documents themselves (listed in the `hits` array). Each matching document contains the `_index` and `_id` fields and the `_source` field, which contains the originally indexed complete document. + +Each document is given a relevance score in the `_score` field. Because you ran a `match_all` search, all document scores are set to `1` (there is no difference in their relevance). The `max_score` field contains the highest score of any matching document. + +## Query string queries + +Query string queries are lightweight but powerful. You can send a query string query as a `q` query parameter. For example, the following query searches for students with the name `john`: + +```json +GET /students/_search?q=name:john +``` +{% include copy-curl.html %} + +OpenSearch returns the matching document: + +```json +{ + "took": 18, + "timed_out": false, + "_shards": { + "total": 1, + "successful": 1, + "skipped": 0, + "failed": 0 + }, + "hits": { + "total": { + "value": 1, + "relation": "eq" + }, + "max_score": 0.9808291, + "hits": [ + { + "_index": "students", + "_id": "1", + "_score": 0.9808291, + "_source": { + "name": "John Doe", + "grade": 12, + "gpa": 3.89, + "grad_year": 2022, + "future_plans": "John plans to be a computer science major" + } + } + ] + } +} +``` + +For more information about query string syntax, see [Query string query language]({{site.url}}{{site.baseurl}}/query-dsl/full-text/query-string/). + +## Query DSL + +Using Query DSL, you can create more complex and customized queries. + +### Full-text search + +You can run a full-text search on fields mapped as `text`. By default, text fields are analyzed by the `default` analyzer. The analyzer splits text into terms and makes it lowercase. For more information about OpenSearch analyzers, see [Analyzers]({{site.url}}{{site.baseurl}}/analyzers/). + +For example, the following query searches for students with the name `john`: + +```json +GET /students/_search +{ + "query": { + "match": { + "name": "john" + } + } +} +``` +{% include copy-curl.html %} + +The response contains the matching document: + +```json +{ + "took": 13, + "timed_out": false, + "_shards": { + "total": 1, + "successful": 1, + "skipped": 0, + "failed": 0 + }, + "hits": { + "total": { + "value": 1, + "relation": "eq" + }, + "max_score": 0.9808291, + "hits": [ + { + "_index": "students", + "_id": "1", + "_score": 0.9808291, + "_source": { + "name": "John Doe", + "gpa": 3.89, + "grad_year": 2022 + } + } + ] + } +} +``` + +Notice that the query text is lowercase while the text in the field is not, but the query still returns the matching document. + +You can reorder the terms in the search string. For example, the following query searches for `doe john`: + +```json +GET /students/_search +{ + "query": { + "match": { + "name": "doe john" + } + } +} +``` +{% include copy-curl.html %} + +The response contains two matching documents: + +```json +{ + "took": 14, + "timed_out": false, + "_shards": { + "total": 1, + "successful": 1, + "skipped": 0, + "failed": 0 + }, + "hits": { + "total": { + "value": 2, + "relation": "eq" + }, + "max_score": 1.4508327, + "hits": [ + { + "_index": "students", + "_id": "1", + "_score": 1.4508327, + "_source": { + "name": "John Doe", + "gpa": 3.89, + "grad_year": 2022 + } + }, + { + "_index": "students", + "_id": "3", + "_score": 0.4700036, + "_source": { + "name": "Jane Doe", + "gpa": 3.52, + "grad_year": 2024 + } + } + ] + } +} +``` + +Both `John Doe` and `Jane Doe` matched the word `doe`, but `John Doe` is scored higher because it also matched `john`. + +### Keyword search + +The `name` field also contains the `name.keyword` subfield, which was added by OpenSearch automatically. You can try to search the `name.keyword` field in a manner similar to the previous request: + +```json +GET /students/_search +{ + "query": { + "match": { + "name.keyword": "john" + } + } +} +``` +{% include copy-curl.html %} + +This request returns no hits because the `keyword` fields must be matched exactly. + +However, you can search for the exact text `John Doe`: + +```json +GET /students/_search +{ + "query": { + "match": { + "name.keyword": "John Doe" + } + } +} +``` +{% include copy-curl.html %} + +OpenSearch returns the matching document: + +```json +{ + "took": 19, + "timed_out": false, + "_shards": { + "total": 1, + "successful": 1, + "skipped": 0, + "failed": 0 + }, + "hits": { + "total": { + "value": 1, + "relation": "eq" + }, + "max_score": 0.9808291, + "hits": [ + { + "_index": "students", + "_id": "1", + "_score": 0.9808291, + "_source": { + "name": "John Doe", + "gpa": 3.89, + "grad_year": 2022 + } + } + ] + } +} +``` + +### Filters + +You can add a filter clause to your query for fields with exact values using a Boolean query. + +Term filters match specific terms. For example, the following Boolean query searches for students whose graduation year is 2022: + +```json +GET students/_search +{ + "query": { + "bool": { + "filter": [ + { "term": { "grad_year": 2022 }} + ] + } + } +} +``` +{% include copy-curl.html %} + +Range filters support specifying a range of values. For example, the following Boolean query searches for students whose GPA is greater than 3.6: + +```json +GET students/_search +{ + "query": { + "bool": { + "filter": [ + { "range": { "gpa": { "gt": 3.6 }}} + ] + } + } +} +``` +{% include copy-curl.html %} + +For more information about filters, see [Query and filter context]({{site.url}}{{site.baseurl}}/query-dsl/query-filter-context/). + +### Compound queries + +A compound query lets you combine multiple query or filter clauses. A Boolean query is an example of a compound query. + +For example, to search for students whose name matches `doe` and filter by graduation year and GPA, use the following request: + +```json +GET students/_search +{ + "query": { + "bool": { + "must": [ + { + "match": { + "name": "doe" + } + }, + { "range": { "gpa": { "gte": 3.6, "lte": 3.9 } } }, + { "term": { "grad_year": 2022 }} + ] + } + } +} +``` +{% include copy-curl.html %} + +## Search methods + +Along with the traditional BM25 search described in this tutorial, OpenSearch supports a range of machine learning (ML)-powered search methods, including k-NN, semantic, multimodal, sparse, hybrid, and conversational search. For information about all search methods, see [Search]({{site.url}}{{site.baseurl}}/search-plugins/). + +## Next steps + +- For information about available query types, see [Query DSL]({{site.url}}{{site.baseurl}}/query-dsl/index/) +- For information about available search methods, see [Search]({{site.url}}{{site.baseurl}}/search-plugins/) \ No newline at end of file From 69f41d96122c72d96ef47d97d51394a65f7903b8 Mon Sep 17 00:00:00 2001 From: Fanit Kolchina Date: Tue, 2 Apr 2024 14:43:17 -0400 Subject: [PATCH 05/23] Remove extraneous files Signed-off-by: Fanit Kolchina --- _ml-commons-plugin/tutorials/rag-chatbot.md | 8 -------- .../tutorials/rag-conversational-agent.md | 10 ---------- 2 files changed, 18 deletions(-) delete mode 100644 _ml-commons-plugin/tutorials/rag-chatbot.md delete mode 100644 _ml-commons-plugin/tutorials/rag-conversational-agent.md diff --git a/_ml-commons-plugin/tutorials/rag-chatbot.md b/_ml-commons-plugin/tutorials/rag-chatbot.md deleted file mode 100644 index 3c1b8e5ca9..0000000000 --- a/_ml-commons-plugin/tutorials/rag-chatbot.md +++ /dev/null @@ -1,8 +0,0 @@ ---- -layout: default -title: RAG chatbot -parent: Tutorials -nav_order: 50 ---- - -# Retrieval-augmented generation chatbot \ No newline at end of file diff --git a/_ml-commons-plugin/tutorials/rag-conversational-agent.md b/_ml-commons-plugin/tutorials/rag-conversational-agent.md deleted file mode 100644 index 7fae32fe3c..0000000000 --- a/_ml-commons-plugin/tutorials/rag-conversational-agent.md +++ /dev/null @@ -1,10 +0,0 @@ ---- -layout: default -title: RAG with a conversational flow agent -parent: Tutorials -nav_order: 40 ---- - -# Retrieval-augmented generation with a conversational flow agent - -Retrieval-augmented generation (RAG) \ No newline at end of file From 6aa0729d949c0e6e1337be80a366a2853b8efd7c Mon Sep 17 00:00:00 2001 From: kolchfa-aws <105444904+kolchfa-aws@users.noreply.github.com> Date: Wed, 3 Apr 2024 14:47:39 -0400 Subject: [PATCH 06/23] Update _getting-started/communicate.md Signed-off-by: kolchfa-aws <105444904+kolchfa-aws@users.noreply.github.com> --- _getting-started/communicate.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/_getting-started/communicate.md b/_getting-started/communicate.md index 501cca65d3..b16a6fc6d7 100644 --- a/_getting-started/communicate.md +++ b/_getting-started/communicate.md @@ -43,7 +43,7 @@ curl -XGET "http://localhost:9200/_cluster/health?pretty" For more information about `pretty` and other useful query parameters, see [Common REST parameters]({{site.url}}{{site.baseurl}}/opensearch/common-parameters/). -For requests that contain a body, specify the `Content-Type` header and provide the request payload in the `-d` (data) oprion: +For requests that contain a body, specify the `Content-Type` header and provide the request payload in the `-d` (data) option: ```json curl -XGET "http://localhost:9200/students/_search?pretty" -H 'Content-Type: application/json' -d' From 1abbab9b9191daf12c445785387d96ac514703ed Mon Sep 17 00:00:00 2001 From: kolchfa-aws <105444904+kolchfa-aws@users.noreply.github.com> Date: Wed, 3 Apr 2024 14:54:09 -0400 Subject: [PATCH 07/23] Update _getting-started/intro.md Signed-off-by: kolchfa-aws <105444904+kolchfa-aws@users.noreply.github.com> --- _getting-started/intro.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/_getting-started/intro.md b/_getting-started/intro.md index 92bd36e495..65973f0aa7 100644 --- a/_getting-started/intro.md +++ b/_getting-started/intro.md @@ -90,7 +90,7 @@ When searching for documents using a phrase, you need to make sure that the word The relevance of a word is calculated as $$ relevance = { \text {term frequency} \over \text {document frequency} }. $$ This score is called term frequency/inverse document frequency (TF/IDF). For more information, see [TF/IDF](https://en.wikipedia.org/wiki/Tf%E2%80%93idf). -OpenSearch uses the BM25 ranking algorithm to calculate relevance scores and returns the results sorted by relevance. To learn more, see [Okapi BM25](https://en.wikipedia.org/wiki/Okapi_BM25). +OpenSearch uses the BM25 ranking algorithm to calculate document relevance scores and returns the results sorted by relevance. To learn more, see [Okapi BM25](https://en.wikipedia.org/wiki/Okapi_BM25). ## Clusters and nodes From 0e82f23702bf41f1821843074cf557dbd6e00bf7 Mon Sep 17 00:00:00 2001 From: kolchfa-aws <105444904+kolchfa-aws@users.noreply.github.com> Date: Wed, 3 Apr 2024 15:09:18 -0400 Subject: [PATCH 08/23] Update _getting-started/intro.md Signed-off-by: kolchfa-aws <105444904+kolchfa-aws@users.noreply.github.com> --- _getting-started/intro.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/_getting-started/intro.md b/_getting-started/intro.md index 65973f0aa7..3a424b5d9d 100644 --- a/_getting-started/intro.md +++ b/_getting-started/intro.md @@ -102,7 +102,7 @@ In a single-node cluster, such as a laptop, one machine has to do everything: ma In each cluster, there is an elected _cluster manager_ node, which orchestrates cluster-level operations, such as creating an index. Nodes communicate with each other, so if your request is routed to a node, that node sends requests to appropriate nodes, gathers the nodes' responses, and returns the final response. -For more information about setting node types, see [Cluster formation]({{site.url}}{{site.baseurl}}/opensearch/cluster/). +For more information about other node types, see [Cluster formation]({{site.url}}{{site.baseurl}}/opensearch/cluster/). ## Shards From 07b9882880ecd2dfa886294d1eb02eed3faa62db Mon Sep 17 00:00:00 2001 From: kolchfa-aws <105444904+kolchfa-aws@users.noreply.github.com> Date: Wed, 3 Apr 2024 15:10:03 -0400 Subject: [PATCH 09/23] Update _getting-started/intro.md Signed-off-by: kolchfa-aws <105444904+kolchfa-aws@users.noreply.github.com> --- _getting-started/intro.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/_getting-started/intro.md b/_getting-started/intro.md index 3a424b5d9d..91db47f62e 100644 --- a/_getting-started/intro.md +++ b/_getting-started/intro.md @@ -110,7 +110,7 @@ OpenSearch splits indexes into *shards*. Each shard stores a subset of all docum An index is split into shards -Shards are used for even distribution across nodes in a cluster. For example, a 400-GB index might be too large for any single node in your cluster to handle, but split into ten shards, each one 40 GB, OpenSearch can distribute the shards across ten nodes and work with each shard individually. For example, consider a cluster with two indexes: index 1 and index 2. Index 1 is split into 2 shards, and index 2 is split into 4 shards. The shards are distributed across nodes 1 and 2, as shown in the following image. +Shards are used for even distribution across nodes in a cluster. For example, a 400-GB index might be too large for any single node in your cluster to handle, but split into ten shards, each one 40 GB, OpenSearch can distribute the shards across ten nodes and work with each shard individually. Consider a cluster with two indexes: index 1 and index 2. Index 1 is split into 2 shards, and index 2 is split into 4 shards. The shards are distributed across nodes 1 and 2, as shown in the following image. A cluster containing two indexes and two nodes From 9b66b24c3e3fc126c7cdaeea89c66cb0178b47a7 Mon Sep 17 00:00:00 2001 From: kolchfa-aws <105444904+kolchfa-aws@users.noreply.github.com> Date: Wed, 3 Apr 2024 15:13:46 -0400 Subject: [PATCH 10/23] Update _getting-started/search-data.md Signed-off-by: kolchfa-aws <105444904+kolchfa-aws@users.noreply.github.com> --- _getting-started/search-data.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/_getting-started/search-data.md b/_getting-started/search-data.md index 758196b16c..3eca3cfec6 100644 --- a/_getting-started/search-data.md +++ b/_getting-started/search-data.md @@ -306,7 +306,7 @@ Both `John Doe` and `Jane Doe` matched the word `doe`, but `John Doe` is scored ### Keyword search -The `name` field also contains the `name.keyword` subfield, which was added by OpenSearch automatically. You can try to search the `name.keyword` field in a manner similar to the previous request: +The `name` field contains the `name.keyword` subfield, which was added by OpenSearch automatically. You can try to search the `name.keyword` field in a manner similar to the previous request: ```json GET /students/_search From 02fccc4d39670264b0f3a03c87fe1d0209e59b40 Mon Sep 17 00:00:00 2001 From: kolchfa-aws <105444904+kolchfa-aws@users.noreply.github.com> Date: Wed, 3 Apr 2024 15:32:50 -0400 Subject: [PATCH 11/23] Apply suggestions from code review Co-authored-by: Melissa Vagi Signed-off-by: kolchfa-aws <105444904+kolchfa-aws@users.noreply.github.com> --- _getting-started/communicate.md | 16 ++++++++-------- _getting-started/index.md | 8 ++++---- _getting-started/ingest-data.md | 10 +++++----- _getting-started/intro.md | 10 +++++----- _getting-started/quickstart.md | 2 +- 5 files changed, 23 insertions(+), 23 deletions(-) diff --git a/_getting-started/communicate.md b/_getting-started/communicate.md index b16a6fc6d7..11cf5008c4 100644 --- a/_getting-started/communicate.md +++ b/_getting-started/communicate.md @@ -10,9 +10,9 @@ You can communicate with OpenSearch using the REST API or one of the OpenSearch ## OpenSearch REST API -You interact with OpenSearch clusters using the REST API, which offers a lot of flexibility. Through the REST API, you can change most OpenSearch settings, modify indexes, check the health of the cluster, get statistics---almost everything. You can use clients like [cURL](https://curl.se/) or any programming language that can send HTTP requests. +You interact with OpenSearch clusters using the REST API, which offers a lot of flexibility. Through the REST API, you can change most OpenSearch settings, modify indexes, check cluster health, get statistics---almost everything. You can use clients like [cURL](https://curl.se/) or any programming language that can send HTTP requests. -You can send HTTP requests in your terminal or in the Dev Tools console in OpenSearch Dashboards. +You can send HTTP requests in your terminal or in the [Dev Tools console]({{site.url}}{{site.baseurl}}/dashboards/dev-tools/index-dev/) in OpenSearch Dashboards. ### Sending requests in the terminal @@ -25,7 +25,7 @@ curl -XGET "http://localhost:9200/_cluster/health" ``` {% include copy.html %} -If you're using the Security plugin, provide the user name and password in the request: +If you're using the Security plugin, provide the username and password in the request: ```bash curl -X GET "http://localhost:9200/_cluster/health" -ku admin: @@ -310,11 +310,11 @@ You cannot change the mappings once the index is created. ## Further reading -- For information about OpenSearch REST API, see the [REST API reference]({{site.url}}{{site.baseurl}}/api-reference/) -- For information about OpenSearch language clients, see [Clients]({{site.url}}{{site.baseurl}}/clients/) -- For information about mappings, see [Mappings and field types]({{site.url}}{{site.baseurl}}/field-types) -- For information about settings, see [Configuring OpenSearch]({{site.url}}{{site.baseurl}}/install-and-configure/configuring-opensearch/index/) +- For information about OpenSearch REST API, see the [REST API reference]({{site.url}}{{site.baseurl}}/api-reference/). +- For information about OpenSearch language clients, see [Clients]({{site.url}}{{site.baseurl}}/clients/). +- For information about mappings, see [Mappings and field types]({{site.url}}{{site.baseurl}}/field-types/). +- For information about settings, see [Configuring OpenSearch]({{site.url}}{{site.baseurl}}/install-and-configure/configuring-opensearch/index/). ## Next steps -- Learn about ingestion options in [Ingest data into OpenSearch]({{site.url}}{{site.baseurl}}/getting-started/ingest-data/) \ No newline at end of file +- Learn about ingestion options in [Ingest data into OpenSearch]({{site.url}}{{site.baseurl}}/getting-started/ingest-data/). \ No newline at end of file diff --git a/_getting-started/index.md b/_getting-started/index.md index 885b7f3789..33134925df 100644 --- a/_getting-started/index.md +++ b/_getting-started/index.md @@ -20,19 +20,19 @@ An equally popular, but less obvious use case is log analytics, in which you tak Though this section focuses on the core product, OpenSearch is more than the core engine. It also includes the following components: -- [OpenSearch Dashboards]({{site.url}}{{site.baseurl}}/dashboards/index/): The OpenSearch data visualization UI. +- [OpenSearch Dashboards]({{site.url}}{{site.baseurl}}/dashboards/index/): The OpenSearch data visualization user interface (UI). - [Data Prepper]({{site.url}}{{site.baseurl}}/data-prepper/): A server-side data collector capable of filtering, enriching, transforming, normalizing, and aggregating data for downstream analytics and visualization. - [Clients]({{site.url}}{{site.baseurl}}/clients/): Language APIs that let you communicate with OpenSearch in several popular programming languages. ## Use cases -OpenSearch supports a wide variety of use cases, for example: +OpenSearch supports a variety of use cases, for example: -- [Observability]({{site.url}}{{site.baseurl}}/observing-your-data/): Visualize data-driven events by using Piped Processing Language to explore, discover, and query data stored in OpenSearch. +- [Observability]({{site.url}}{{site.baseurl}}/observing-your-data/): Visualize data-driven events by using Piped Processing Language (PPL) to explore, discover, and query data stored in OpenSearch. - [Search]({{site.url}}{{site.baseurl}}/search-plugins/): Choose the search method best for your application, from regular lexical search to conversational search powered by machine learning (ML). - [Machine learning]({{site.url}}{{site.baseurl}}/ml-commons-plugin/): Integrate ML models into your OpenSearch application. - [Security analytics]({{site.url}}{{site.baseurl}}/security-analytics/): Investigate, detect, analyze, and respond to security threats that can jeopardize the success of businesses and organizations and their online operations. ## Next steps -- Learn the essential OpenSearch concepts in [Introduction to OpenSearch]({{site.url}}{{site.baseurl}}/getting-started/intro/) \ No newline at end of file +- Learn the essential OpenSearch concepts in [Introduction to OpenSearch]({{site.url}}{{site.baseurl}}/getting-started/intro/). \ No newline at end of file diff --git a/_getting-started/ingest-data.md b/_getting-started/ingest-data.md index 22572739b3..6667da9f16 100644 --- a/_getting-started/ingest-data.md +++ b/_getting-started/ingest-data.md @@ -101,11 +101,11 @@ To learn how to use OpenSearch Dashboards to visualize the data, see the [OpenSe ## Further reading -- For information about Data Prepper, see [Data Prepper]({{site.url}}{{site.baseurl}}/data-prepper/) -- For information about ingestion tools, see [OpenSearch tools]({{site.url}}{{site.baseurl}}/tools/) -- For information about OpenSearch Dashboards, see [OpenSearch Dashboards quickstart guide]({{site.url}}{{site.baseurl}}/dashboards/quickstart/) -- For information about bulk indexing, see [Bulk API]({{site.url}}{{site.baseurl}}/api-reference/document-apis/bulk/) +- For information about Data Prepper, see [Data Prepper]({{site.url}}{{site.baseurl}}/data-prepper/). +- For information about ingestion tools, see [OpenSearch tools]({{site.url}}{{site.baseurl}}/tools/). +- For information about OpenSearch Dashboards, see [OpenSearch Dashboards quickstart guide]({{site.url}}{{site.baseurl}}/dashboards/quickstart/). +- For information about bulk indexing, see [Bulk API]({{site.url}}{{site.baseurl}}/api-reference/document-apis/bulk/). ## Next steps -- Learn about search options in [Search your data]({{site.url}}{{site.baseurl}}/getting-started/search-data/) \ No newline at end of file +- Learn about search options in [Search your data]({{site.url}}{{site.baseurl}}/getting-started/search-data/). \ No newline at end of file diff --git a/_getting-started/intro.md b/_getting-started/intro.md index 91db47f62e..cd40bd03a8 100644 --- a/_getting-started/intro.md +++ b/_getting-started/intro.md @@ -9,7 +9,7 @@ redirect_from: # Introduction to OpenSearch -OpenSearch is a distributed search and analytics engine, which supports various use cases, from implementing a search box on a website to analyzing security data for threat detection. The term _distributed_ means you can run OpenSearch on multiple computers. _Search and analytics_ means you can search and analyze your data once you ingest it into OpenSearch. Whether your data is geographic or genetic, you can store and analyze it using OpenSearch. +OpenSearch is a distributed search and analytics engine that supports various use cases, from implementing a search box on a website to analyzing security data for threat detection. The term _distributed_ means you can run OpenSearch on multiple computers. _Search and analytics_ means you can search and analyze your data once you ingest it into OpenSearch. Whether your data is geographic or genetic, you can store and analyze it using OpenSearch. ## Document @@ -86,7 +86,7 @@ In addition to the document ID, OpenSearch stores the position of the word withi When searching for documents using a phrase, you need to make sure that the words you are searching for are _relevant_. For example, the word `the` appears in most English phrases but searching for this word is meaningless. OpenSearch determines the relevance of a term by calculating two values: - _Term frequency_: how often a word appears in the document -- _Document frequency_: how often a word appears in all documents +- _Document frequency_: how often a word appears in all documents The relevance of a word is calculated as $$ relevance = { \text {term frequency} \over \text {document frequency} }. $$ This score is called term frequency/inverse document frequency (TF/IDF). For more information, see [TF/IDF](https://en.wikipedia.org/wiki/Tf%E2%80%93idf). @@ -106,7 +106,7 @@ For more information about other node types, see [Cluster formation]({{site.url} ## Shards -OpenSearch splits indexes into *shards*. Each shard stores a subset of all documents in an index, as shown in the following image. +OpenSearch splits indexes into _shards_. Each shard stores a subset of all documents in an index, as shown in the following image. An index is split into shards @@ -118,7 +118,7 @@ Despite being a piece of an OpenSearch index, each shard is actually a full Luce ## Primary and replica shards -In OpenSearch, shards may be _primary_ (the original) or _replica_ (a copy). By default, OpenSearch creates a replica shard for each primary shard. Thus, if you split your index into ten shards, OpenSearch creates ten replica shards. For example, consider a cluster described in the previous section. If you add one replica for each shard of each index in the cluster, your cluster will contain a total of 2 shards and 2 replicas for index 1 and 4 shards and 4 replicas for index 2, as shown in the following image. +In OpenSearch, shards may be _primary_ (the original) or _replica_ (a copy). By default, OpenSearch creates a replica shard for each primary shard. Thus, if you split your index into 10 shards, OpenSearch creates 10 replica shards. For example, consider a cluster described in the previous section. If you add one replica for each shard of each index in the cluster, your cluster will contain a total of 2 shards and 2 replicas for index 1 and 4 shards and 4 replicas for index 2, as shown in the following image. A cluster containing two indexes with one replica shard for each shard in the index @@ -148,4 +148,4 @@ In OpenSearch, a shard is a Lucene index, which consists of _segments_ (or segme ## Next steps -- Learn how to quickly install OpenSearch in [Installation quickstart]({{site.url}}{{site.baseurl}}/getting-started/quickstart/) \ No newline at end of file +- Learn how to install OpenSearch within minutes in [Installation quickstart]({{site.url}}{{site.baseurl}}/getting-started/quickstart/). \ No newline at end of file diff --git a/_getting-started/quickstart.md b/_getting-started/quickstart.md index b8ae44bd32..412f9de181 100644 --- a/_getting-started/quickstart.md +++ b/_getting-started/quickstart.md @@ -153,4 +153,4 @@ You successfully deployed your own OpenSearch cluster with OpenSearch Dashboards ## Next steps -- Learn about how to send requests to OpenSearch in [Communicate with OpenSearch]({{site.url}}{{site.baseurl}}/getting-started/communicate/) +- Learn about how to send requests to OpenSearch in [Communicate with OpenSearch]({{site.url}}{{site.baseurl}}/getting-started/communicate/). From 08dcd8ee6634c8f747631ecfa49765a702f82026 Mon Sep 17 00:00:00 2001 From: kolchfa-aws <105444904+kolchfa-aws@users.noreply.github.com> Date: Wed, 3 Apr 2024 15:36:08 -0400 Subject: [PATCH 12/23] Apply suggestions from code review Co-authored-by: Melissa Vagi Signed-off-by: kolchfa-aws <105444904+kolchfa-aws@users.noreply.github.com> --- _getting-started/search-data.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/_getting-started/search-data.md b/_getting-started/search-data.md index 3eca3cfec6..4c193f96c0 100644 --- a/_getting-started/search-data.md +++ b/_getting-started/search-data.md @@ -440,5 +440,5 @@ Along with the traditional BM25 search described in this tutorial, OpenSearch su ## Next steps -- For information about available query types, see [Query DSL]({{site.url}}{{site.baseurl}}/query-dsl/index/) -- For information about available search methods, see [Search]({{site.url}}{{site.baseurl}}/search-plugins/) \ No newline at end of file +- For information about available query types, see [Query DSL]({{site.url}}{{site.baseurl}}/query-dsl/index/). +- For information about available search methods, see [Search]({{site.url}}{{site.baseurl}}/search-plugins/). \ No newline at end of file From 586fed2b7ebf3ade3beccf784f7a09f39d8d6956 Mon Sep 17 00:00:00 2001 From: Fanit Kolchina Date: Wed, 3 Apr 2024 16:33:32 -0400 Subject: [PATCH 13/23] Tech review comments Signed-off-by: Fanit Kolchina --- _getting-started/communicate.md | 2 +- _getting-started/intro.md | 22 +++++++++++++++------- _getting-started/search-data.md | 4 ++-- 3 files changed, 18 insertions(+), 10 deletions(-) diff --git a/_getting-started/communicate.md b/_getting-started/communicate.md index 11cf5008c4..32d9f42a63 100644 --- a/_getting-started/communicate.md +++ b/_getting-started/communicate.md @@ -305,7 +305,7 @@ OpenSearch mapped the `name` and `grad_year` fields according to the types you s } ``` -You cannot change the mappings once the index is created. +You cannot change the type of a field once it is created. Changing a field type requires deleting the index and recreating it with the new mappings. {: .note} ## Further reading diff --git a/_getting-started/intro.md b/_getting-started/intro.md index cd40bd03a8..fb8009057e 100644 --- a/_getting-started/intro.md +++ b/_getting-started/intro.md @@ -83,12 +83,13 @@ In addition to the document ID, OpenSearch stores the position of the word withi ## Relevance -When searching for documents using a phrase, you need to make sure that the words you are searching for are _relevant_. For example, the word `the` appears in most English phrases but searching for this word is meaningless. OpenSearch determines the relevance of a term by calculating two values: +Individual words in a search query are called search _terms_. Each search term is scored according to the following rules: -- _Term frequency_: how often a word appears in the document -- _Document frequency_: how often a word appears in all documents +1. A search term that occurs more frequently in a document will tend to score higher. A document about dogs that uses the word `dog` many times is likely more relevant than a document that contains the word `dog` fewer times. This is the _term frequency_ component of the score. -The relevance of a word is calculated as $$ relevance = { \text {term frequency} \over \text {document frequency} }. $$ This score is called term frequency/inverse document frequency (TF/IDF). For more information, see [TF/IDF](https://en.wikipedia.org/wiki/Tf%E2%80%93idf). +1. A search term that occurs in more documents will tend to score lower. A query for the terms `blue` and `axolotl` should prefer documents that contain `axolotl` over the likely more common word `blue`. This is the _inverse document frequency_ component of the score. + +1. A match on a longer document should tend to score lower than a match on a shorter document. A document that contains a full dictionary would match on any word, but is not very relevant to any particular word. This corresponds to the _length normalization_ component of the score. OpenSearch uses the BM25 ranking algorithm to calculate document relevance scores and returns the results sorted by relevance. To learn more, see [Okapi BM25](https://en.wikipedia.org/wiki/Okapi_BM25). @@ -128,11 +129,18 @@ These replica shards act as backups in the event of a node failure---OpenSearch The following section describes more advanced OpenSearch concepts. -### Translog +### Update lifecycle + +The lifecycle of an update operation consists of the following steps: -Any index changes, such as document indexing or deletion, are written to disk during a Lucene commit. However, Lucene commits are expensive operations, so they cannot be performed after every change to the index. Instead, each shard records every indexing operation in a transaction log called _translog_. When a document is indexed, it is added to the memory buffer and recorded in the translog. After a process or host restart, any data in the in-memory buffer is lost. Recording the document in the translog ensures durability because the translog is written to disk. +1. An update is received by a primary shard and is written to the shard's transaction log ([translog](#translog)). The translog is flushed to disk (followed by an fsync) before the update is acknowledged. This guarantees durability. +1. The update is also passed to the Lucene index writer, which adds it to an in-memory buffer. +1. On a [refresh operation](#refresh), the Lucene index writer flushes the in-memory buffers to disk (with each buffer becoming a new Lucene segment), and a new index reader is opened over the resulting segment files. The updates are now visible for search. +1. On a [flush operation](#flush), the shard fsyncs the Lucene segments. Because the segment files are a durable representation of the updates, the translog is no longer needed to provide durability, do the updates can be purged from the translog. + +### Translog -Frequent refresh operations write the documents in the memory buffer to a segment and then clear the memory buffer. Periodically, a [flush](#flush) performs a Lucene commit, which includes writing the segments to disk using `fsync`, purging the old translog, and starting a new translog. Thus, a translog contains all operations that have not yet been flushed. +An indexing or bulk call responds when the documents have been written to the translog and the translog is flushed to disk, so the updates are durable. The updates will be visible from search requests until after a [refresh operation](#refresh). ### Refresh diff --git a/_getting-started/search-data.md b/_getting-started/search-data.md index 4c193f96c0..4a7e6e2e2c 100644 --- a/_getting-started/search-data.md +++ b/_getting-started/search-data.md @@ -302,7 +302,7 @@ The response contains two matching documents: } ``` -Both `John Doe` and `Jane Doe` matched the word `doe`, but `John Doe` is scored higher because it also matched `john`. +The match query type uses `OR` as an operator by default, so the query is functionally `doe OR john`. Both `John Doe` and `Jane Doe` matched the word `doe`, but `John Doe` is scored higher because it also matched `john`. ### Keyword search @@ -436,7 +436,7 @@ GET students/_search ## Search methods -Along with the traditional BM25 search described in this tutorial, OpenSearch supports a range of machine learning (ML)-powered search methods, including k-NN, semantic, multimodal, sparse, hybrid, and conversational search. For information about all search methods, see [Search]({{site.url}}{{site.baseurl}}/search-plugins/). +Along with the traditional full-text search described in this tutorial, OpenSearch supports a range of machine learning (ML)-powered search methods, including k-NN, semantic, multimodal, sparse, hybrid, and conversational search. For information about all search methods, see [Search]({{site.url}}{{site.baseurl}}/search-plugins/). ## Next steps From 98563cb404e0e0bddfe2d40c1973258dfd5b6d50 Mon Sep 17 00:00:00 2001 From: Fanit Kolchina Date: Wed, 3 Apr 2024 16:41:08 -0400 Subject: [PATCH 14/23] Add link to compound query section Signed-off-by: Fanit Kolchina --- _getting-started/search-data.md | 2 ++ 1 file changed, 2 insertions(+) diff --git a/_getting-started/search-data.md b/_getting-started/search-data.md index 4a7e6e2e2c..cf035935c8 100644 --- a/_getting-started/search-data.md +++ b/_getting-started/search-data.md @@ -434,6 +434,8 @@ GET students/_search ``` {% include copy-curl.html %} +For more information about Boolean and other compound queries, see [Compound queries]({{site.url}}{{site.baseurl}}/query-dsl/compound/index/). + ## Search methods Along with the traditional full-text search described in this tutorial, OpenSearch supports a range of machine learning (ML)-powered search methods, including k-NN, semantic, multimodal, sparse, hybrid, and conversational search. For information about all search methods, see [Search]({{site.url}}{{site.baseurl}}/search-plugins/). From 15a8227c1061cc3275e49848bc0f696d4c56b8f1 Mon Sep 17 00:00:00 2001 From: Fanit Kolchina Date: Wed, 3 Apr 2024 16:56:22 -0400 Subject: [PATCH 15/23] Added install types section Signed-off-by: Fanit Kolchina --- _getting-started/quickstart.md | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/_getting-started/quickstart.md b/_getting-started/quickstart.md index 412f9de181..b491c25a53 100644 --- a/_getting-started/quickstart.md +++ b/_getting-started/quickstart.md @@ -141,6 +141,10 @@ opensearch-node1 | [1]: max virtual memory areas vm.max_map_count [65530 opensearch-node1 | ERROR: OpenSearch did not exit normally - check the logs at /usr/share/opensearch/logs/opensearch-cluster.log ``` +## Other installation types + +In addition to Docker, you can install OpenSearch on various Linux distributions and on Windows. For all available installation guides, see [Install and upgrade OpenSearch]({{site.url}}{{site.baseurl}}/install-and-configure/). + ## Further reading You successfully deployed your own OpenSearch cluster with OpenSearch Dashboards and added some sample data. Now you're ready to learn about configuration and functionality in more detail. Here are a few recommendations on where to begin: From 9a2bea835626bdbd344d51f1bb05f60d73808ff6 Mon Sep 17 00:00:00 2001 From: Fanit Kolchina Date: Wed, 3 Apr 2024 16:57:31 -0400 Subject: [PATCH 16/23] Remove further reading suggestions Signed-off-by: Fanit Kolchina --- _getting-started/quickstart.md | 3 --- 1 file changed, 3 deletions(-) diff --git a/_getting-started/quickstart.md b/_getting-started/quickstart.md index b491c25a53..01350d3b2c 100644 --- a/_getting-started/quickstart.md +++ b/_getting-started/quickstart.md @@ -151,9 +151,6 @@ You successfully deployed your own OpenSearch cluster with OpenSearch Dashboards - [About the Security plugin]({{site.url}}{{site.baseurl}}/security/index/) - [OpenSearch configuration]({{site.url}}{{site.baseurl}}/install-and-configure/configuring-opensearch/) - [OpenSearch plugin installation]({{site.url}}{{site.baseurl}}/opensearch/install/plugins/) -- [Getting started with OpenSearch Dashboards]({{site.url}}{{site.baseurl}}/dashboards/index/) -- [OpenSearch tools]({{site.url}}{{site.baseurl}}/tools/index/) -- [Index APIs]({{site.url}}{{site.baseurl}}/api-reference/index-apis/index/) ## Next steps From 2b0ba5a005be9534fb99ce0c1186c124734e1d59 Mon Sep 17 00:00:00 2001 From: Fanit Kolchina Date: Wed, 3 Apr 2024 17:17:39 -0400 Subject: [PATCH 17/23] Reorder sections Signed-off-by: Fanit Kolchina --- _getting-started/intro.md | 66 ++++++++++++++++++++------------------- 1 file changed, 34 insertions(+), 32 deletions(-) diff --git a/_getting-started/intro.md b/_getting-started/intro.md index fb8009057e..3f93b91240 100644 --- a/_getting-started/intro.md +++ b/_getting-started/intro.md @@ -57,6 +57,38 @@ ID | Name | GPA | Graduation year 2 | Jonathan Powers | 3.85 | 2025 3 | Jane Doe | 3.52 | 2024 +## Clusters and nodes + +OpenSearch is designed to be a distributed search engine. OpenSearch can run on one or more _nodes_---servers that store your data and process search requests. An OpenSearch *cluster* is a collection of nodes. + +You can run OpenSearch locally on a laptop---its system requirements are minimal---but you can also scale a single cluster to hundreds of powerful machines in a data center. + +In a single-node cluster, such as a laptop, one machine has to do everything: manage the state of the cluster, index and search data, and perform any preprocessing of data prior to indexing it. As a cluster grows, however, you can subdivide responsibilities. Nodes with fast disks and plenty of RAM might be great at indexing and searching data, whereas a node with plenty of CPU power and a tiny disk could manage cluster state. + +In each cluster, there is an elected _cluster manager_ node, which orchestrates cluster-level operations, such as creating an index. Nodes communicate with each other, so if your request is routed to a node, that node sends requests to appropriate nodes, gathers the nodes' responses, and returns the final response. + +For more information about other node types, see [Cluster formation]({{site.url}}{{site.baseurl}}/opensearch/cluster/). + +## Shards + +OpenSearch splits indexes into _shards_. Each shard stores a subset of all documents in an index, as shown in the following image. + +An index is split into shards + +Shards are used for even distribution across nodes in a cluster. For example, a 400-GB index might be too large for any single node in your cluster to handle, but split into ten shards, each one 40 GB, OpenSearch can distribute the shards across ten nodes and work with each shard individually. Consider a cluster with two indexes: index 1 and index 2. Index 1 is split into 2 shards, and index 2 is split into 4 shards. The shards are distributed across nodes 1 and 2, as shown in the following image. + +A cluster containing two indexes and two nodes + +Despite being a piece of an OpenSearch index, each shard is actually a full Lucene index---confusing, we know. This detail is important, though, because each instance of Lucene is a running process that consumes CPU and memory. More shards is not necessarily better. Splitting a 400 GB index into 1,000 shards, for example, would place needless strain on your cluster. A good rule of thumb is to keep shard size as 10--50 GB. + +## Primary and replica shards + +In OpenSearch, shards may be _primary_ (the original) or _replica_ (a copy). By default, OpenSearch creates a replica shard for each primary shard. Thus, if you split your index into 10 shards, OpenSearch creates 10 replica shards. For example, consider a cluster described in the previous section. If you add one replica for each shard of each index in the cluster, your cluster will contain a total of 2 shards and 2 replicas for index 1 and 4 shards and 4 replicas for index 2, as shown in the following image. + +A cluster containing two indexes with one replica shard for each shard in the index + +These replica shards act as backups in the event of a node failure---OpenSearch distributes replica shards to different nodes than their corresponding primary shards---but they also improve the speed at which the cluster processes search requests. You might specify more than one replica per index for a search-heavy workload. + ## Inverted index An OpenSearch index uses a data structure called an _inverted index_. An inverted index maps words to the documents that they occur in. For example, consider an index containing the following two documents: @@ -83,6 +115,8 @@ In addition to the document ID, OpenSearch stores the position of the word withi ## Relevance +When you search for a document, OpenSearch matches the words in the query to the words in the documents. For example, if you search the index described in the previous section for the word `beauty`, OpenSearch will return documents 1 and 2. Each document is assigned a _relevance score_ that tells you how well the document matched the query. + Individual words in a search query are called search _terms_. Each search term is scored according to the following rules: 1. A search term that occurs more frequently in a document will tend to score higher. A document about dogs that uses the word `dog` many times is likely more relevant than a document that contains the word `dog` fewer times. This is the _term frequency_ component of the score. @@ -93,38 +127,6 @@ Individual words in a search query are called search _terms_. Each search term i OpenSearch uses the BM25 ranking algorithm to calculate document relevance scores and returns the results sorted by relevance. To learn more, see [Okapi BM25](https://en.wikipedia.org/wiki/Okapi_BM25). -## Clusters and nodes - -OpenSearch is designed to be a distributed search engine. OpenSearch can run on one or more _nodes_---servers that store your data and process search requests. An OpenSearch *cluster* is a collection of nodes. - -You can run OpenSearch locally on a laptop---its system requirements are minimal---but you can also scale a single cluster to hundreds of powerful machines in a data center. - -In a single-node cluster, such as a laptop, one machine has to do everything: manage the state of the cluster, index and search data, and perform any preprocessing of data prior to indexing it. As a cluster grows, however, you can subdivide responsibilities. Nodes with fast disks and plenty of RAM might be great at indexing and searching data, whereas a node with plenty of CPU power and a tiny disk could manage cluster state. - -In each cluster, there is an elected _cluster manager_ node, which orchestrates cluster-level operations, such as creating an index. Nodes communicate with each other, so if your request is routed to a node, that node sends requests to appropriate nodes, gathers the nodes' responses, and returns the final response. - -For more information about other node types, see [Cluster formation]({{site.url}}{{site.baseurl}}/opensearch/cluster/). - -## Shards - -OpenSearch splits indexes into _shards_. Each shard stores a subset of all documents in an index, as shown in the following image. - -An index is split into shards - -Shards are used for even distribution across nodes in a cluster. For example, a 400-GB index might be too large for any single node in your cluster to handle, but split into ten shards, each one 40 GB, OpenSearch can distribute the shards across ten nodes and work with each shard individually. Consider a cluster with two indexes: index 1 and index 2. Index 1 is split into 2 shards, and index 2 is split into 4 shards. The shards are distributed across nodes 1 and 2, as shown in the following image. - -A cluster containing two indexes and two nodes - -Despite being a piece of an OpenSearch index, each shard is actually a full Lucene index---confusing, we know. This detail is important, though, because each instance of Lucene is a running process that consumes CPU and memory. More shards is not necessarily better. Splitting a 400 GB index into 1,000 shards, for example, would place needless strain on your cluster. A good rule of thumb is to keep shard size as 10--50 GB. - -## Primary and replica shards - -In OpenSearch, shards may be _primary_ (the original) or _replica_ (a copy). By default, OpenSearch creates a replica shard for each primary shard. Thus, if you split your index into 10 shards, OpenSearch creates 10 replica shards. For example, consider a cluster described in the previous section. If you add one replica for each shard of each index in the cluster, your cluster will contain a total of 2 shards and 2 replicas for index 1 and 4 shards and 4 replicas for index 2, as shown in the following image. - -A cluster containing two indexes with one replica shard for each shard in the index - -These replica shards act as backups in the event of a node failure---OpenSearch distributes replica shards to different nodes than their corresponding primary shards---but they also improve the speed at which the cluster processes search requests. You might specify more than one replica per index for a search-heavy workload. - ## Advanced concepts The following section describes more advanced OpenSearch concepts. From 812f5006d0af94d22a1dcc6f2b02ea46b478525d Mon Sep 17 00:00:00 2001 From: kolchfa-aws <105444904+kolchfa-aws@users.noreply.github.com> Date: Thu, 4 Apr 2024 10:27:22 -0400 Subject: [PATCH 18/23] Apply suggestions from code review Co-authored-by: Nathan Bower Signed-off-by: kolchfa-aws <105444904+kolchfa-aws@users.noreply.github.com> --- _getting-started/communicate.md | 54 ++++++++++++++++----------------- _getting-started/index.md | 14 ++++----- _getting-started/ingest-data.md | 8 ++--- _getting-started/intro.md | 36 +++++++++++----------- _getting-started/quickstart.md | 6 ++-- _getting-started/search-data.md | 28 ++++++++--------- 6 files changed, 73 insertions(+), 73 deletions(-) diff --git a/_getting-started/communicate.md b/_getting-started/communicate.md index 32d9f42a63..3feb4f3348 100644 --- a/_getting-started/communicate.md +++ b/_getting-started/communicate.md @@ -14,7 +14,7 @@ You interact with OpenSearch clusters using the REST API, which offers a lot of You can send HTTP requests in your terminal or in the [Dev Tools console]({{site.url}}{{site.baseurl}}/dashboards/dev-tools/index-dev/) in OpenSearch Dashboards. -### Sending requests in the terminal +### Sending requests in a terminal When sending cURL requests in a terminal, the request format varies depending on whether you're using the Security plugin. As an example, consider a request to the Cluster Health API. @@ -32,9 +32,9 @@ curl -X GET "http://localhost:9200/_cluster/health" -ku admin:` setting. +The default username is `admin`, and the password is set in your `docker-compose.yml` file in the `OPENSEARCH_INITIAL_ADMIN_PASSWORD=` setting. -Queries submitted to OpenSearch generally return a flat JSON by default. For a human-readable response body, provide the `pretty` query parameter: +OpenSearch generally returns responses in a flat JSON format by default. For a human-readable response body, provide the `pretty` query parameter: ```bash curl -XGET "http://localhost:9200/_cluster/health?pretty" @@ -57,22 +57,22 @@ curl -XGET "http://localhost:9200/students/_search?pretty" -H 'Content-Type: app ### Sending requests in Dev Tools -The Dev Tools console in OpenSearch Dashboards uses a simplified syntax to format REST requests compared to the cURL command. To send requests in Dev Tools, use the following steps: +The Dev Tools console in OpenSearch Dashboards uses a simpler syntax to format REST requests as compared to the cURL command. To send requests in Dev Tools, use the following steps: -1. Access OpenSearch Dashboards by opening `http://localhost:5601/` in a web browser on the same host that is running your OpenSearch cluster. The default username is `admin` and the password is set in your `docker-compose.yml` file in the `OPENSEARCH_INITIAL_ADMIN_PASSWORD=` setting. +1. Access OpenSearch Dashboards by opening `http://localhost:5601/` in a web browser on the same host that is running your OpenSearch cluster. The default username is `admin`, and the password is set in your `docker-compose.yml` file in the `OPENSEARCH_INITIAL_ADMIN_PASSWORD=` setting. 1. On the top menu bar, go to **Management > Dev Tools**. 1. In the left pane of the console, enter the following request: ```json GET _cluster/health ``` {% include copy-curl.html %} -1. Choose the triangle icon at the upper right of the request to submit the query. You can also submit the request by pressing `Ctrl+Enter` (or `Cmd+Enter` for Mac users). To learn more about using the OpenSearch Dashboards console for submitting queries, see [Running queries in the console]({{site.url}}{{site.baseurl}}/dashboards/run-queries/). +1. Choose the triangle icon on the upper right of the request to submit the query. You can also submit the request by pressing `Ctrl+Enter` (or `Cmd+Enter` for Mac users). To learn more about using the OpenSearch Dashboards console for submitting queries, see [Running queries in the console]({{site.url}}{{site.baseurl}}/dashboards/run-queries/). -In the following sections, and in most of OpenSearch documentation, requests are presented in the Dev Tools console format. +In the following sections, and in most of the OpenSearch documentation, requests are presented in the Dev Tools console format. ## Indexing documents -To add a JSON document to an OpenSearch index (that is, to _index_ a document), you send an HTTP request that has the following header: +To add a JSON document to an OpenSearch index (that is, to _index_ a document), you send an HTTP request with the following header: ```json PUT https://://_doc/ @@ -90,7 +90,7 @@ PUT /students/_doc/1 ``` {% include copy-curl.html %} -Once you send the preceding request, OpenSearch creates an index called `students` and stores the ingested document in the index. If you don't provide an ID for your document, OpenSearch generates a document ID. For the preceding request, you have specified the document ID to be the student ID (`1`). +Once you send the preceding request, OpenSearch creates an index called `students` and stores the ingested document in the index. If you don't provide an ID for your document, OpenSearch generates a document ID. In the preceding request, the document ID is specified as the student ID (`1`). To learn more about indexing, see [Managing indexes]({{site.url}}{{site.baseurl}}/im-plugin/). @@ -98,7 +98,7 @@ To learn more about indexing, see [Managing indexes]({{site.url}}{{site.baseurl} When you index a document, OpenSearch infers the field types from the JSON types submitted in the document. This process is called _dynamic mapping_. For more information, see [Dynamic mapping]({{site.url}}{{site.baseurl}}/field-types/#dynamic-mapping). -To view the inferred field data types, send a request to the `_mapping` endpoint: +To view the inferred field types, send a request to the `_mapping` endpoint: ```json GET /students/_mapping @@ -135,11 +135,11 @@ OpenSearch responds with the field `type` for each field: OpenSearch mapped the numeric fields to the `float` and `long` types. Notice that OpenSearch mapped the `name` text field to `text` and added a `name.keyword` subfield mapped to `keyword`. Fields mapped to `text` are analyzed (lowercased and split into terms) and can be used for full-text search. Fields mapped to `keyword` are used for exact term search. -OpenSearch mapped the `grad_year` field to `long`. If you want to map it to the `date` type instead, you need to [delete the index](#deleting-the-index) and then recreate it, specifying the mappings you want explicitly. For steps to specify explicit mappings, see [Index settings and mappings](#index-settings-and-mappings). +OpenSearch mapped the `grad_year` field to `long`. If you want to map it to the `date` type instead, you need to [delete the index](#deleting-the-index) and then recreate it, explicitly specifying the mappings. For instructions on how to explicitly specify mappings, see [Index settings and mappings](#index-settings-and-mappings). ## Searching for documents -To run a search for the document, specify the index that you're searching and a query that will be used to match documents. For example, the simplest query is the `match_all` query that matches all documents in the index: +To run a search for the document, specify the index that you're searching and a query that will be used to match documents. The simplest query is the `match_all` query, which matches all documents in an index: ```json GET /students/_search @@ -151,7 +151,7 @@ GET /students/_search ``` {% include copy-curl.html %} -OpenSearch returns the document that you indexed: +OpenSearch returns the indexed document: ```json { @@ -185,11 +185,11 @@ OpenSearch returns the document that you indexed: } ``` -For more information about searching, see [Search your data]({{site.url}}{{site.baseurl}}/getting-started/search-data/). +For more information about search methods, see [Search your data]({{site.url}}{{site.baseurl}}/getting-started/search-data/). ## Updating documents -In OpenSearch, documents are immutable. However, you can update a document by retrieving it, updating its information, and reindexing it. You can update the whole document using the Index Document API, providing values for all existing and added fields in the document. For example, to update the `gpa` field and add an `address` field to the previously indexed document, send the following request: +In OpenSearch, documents are immutable. However, you can update a document by retrieving it, updating its information, and reindexing it. You can update an entire document using the Index Document API, providing values for all existing and added fields in the document. For example, to update the `gpa` field and add an `address` field to the previously indexed document, send the following request: ```json PUT /students/_doc/1 @@ -217,32 +217,32 @@ POST /students/_update/1/ For more information about partial document updates, see [Update Document API]({{site.url}}{{site.baseurl}}/api-reference/document-apis/update-document/). -## Deleting documents +## Deleting a document -To delete the document, send a delete request and provide the document ID: +To delete a document, send a delete request and provide the document ID: ```json DELETE /students/_doc/1 ``` {% include copy-curl.html %} -## Deleting the index +## Deleting an index -To delete the index, send the following request: +To delete an index, send the following request: ```json DELETE /students ``` {% include copy-curl.html %} -## Index settings and mappings +## Index mappings and settings OpenSearch indexes are configured with mappings and settings: -- A _mapping_ is the collection of fields and the types of those fields. For more information, see [Mappings and field types]({{site.url}}{{site.baseurl}}/field-types/). +- A _mapping_ is a collection of fields and the types of those fields. For more information, see [Mappings and field types]({{site.url}}{{site.baseurl}}/field-types/). - _Settings_ include index data like the index name, creation date, and number of shards. For more information, see [Configuring OpenSearch]({{site.url}}{{site.baseurl}}/install-and-configure/configuring-opensearch/index/). -You can specify the settings and mappings in one request. For example, the following request specifies the number of shards for the index and maps the `name` field to `text` and the `grad_year` field to `date`: +You can specify the mappings and settings in one request. For example, the following request specifies the number of index shards and maps the `name` field to `text` and the `grad_year` field to `date`: ```json PUT /students @@ -264,7 +264,7 @@ PUT /students ``` {% include copy-curl.html %} -Now you can index the same document as you indexed in the previous section: +Now you can index the same document that you indexed in the previous section: ```json PUT /students/_doc/1 @@ -283,7 +283,7 @@ GET /students/_mapping ``` {% include copy-curl.html %} -OpenSearch mapped the `name` and `grad_year` fields according to the types you specified and inferred the field types for the other fields: +OpenSearch mapped the `name` and `grad_year` fields according to the specified types and inferred the field types for the other fields: ```json { @@ -305,16 +305,16 @@ OpenSearch mapped the `name` and `grad_year` fields according to the types you s } ``` -You cannot change the type of a field once it is created. Changing a field type requires deleting the index and recreating it with the new mappings. +Once a field is created, you cannot change its type. Changing a field type requires deleting the index and recreating it with the new mappings. {: .note} ## Further reading -- For information about OpenSearch REST API, see the [REST API reference]({{site.url}}{{site.baseurl}}/api-reference/). +- For information about the OpenSearch REST API, see the [REST API reference]({{site.url}}{{site.baseurl}}/api-reference/). - For information about OpenSearch language clients, see [Clients]({{site.url}}{{site.baseurl}}/clients/). - For information about mappings, see [Mappings and field types]({{site.url}}{{site.baseurl}}/field-types/). - For information about settings, see [Configuring OpenSearch]({{site.url}}{{site.baseurl}}/install-and-configure/configuring-opensearch/index/). ## Next steps -- Learn about ingestion options in [Ingest data into OpenSearch]({{site.url}}{{site.baseurl}}/getting-started/ingest-data/). \ No newline at end of file +- See [Ingest data into OpenSearch]({{site.url}}{{site.baseurl}}/getting-started/ingest-data/) to learn about ingestion options. \ No newline at end of file diff --git a/_getting-started/index.md b/_getting-started/index.md index 33134925df..b25587c522 100644 --- a/_getting-started/index.md +++ b/_getting-started/index.md @@ -12,16 +12,16 @@ permalink: /getting-started/ OpenSearch is a distributed search and analytics engine based on [Apache Lucene](https://lucene.apache.org/). After adding your data to OpenSearch, you can perform full-text searches on it with all of the features you might expect: search by field, search multiple indexes, boost fields, rank results by score, sort results by field, and aggregate results. -Unsurprisingly, people often use search engines like OpenSearch as the backend for a search application---think [Wikipedia](https://en.wikipedia.org/wiki/Wikipedia:FAQ/Technical#What_software_is_used_to_run_Wikipedia?) or an online store. It offers excellent performance and can scale up and down as the needs of the application grow or shrink. +Unsurprisingly, builders often use a search engine like OpenSearch as the backend for a search application---think [Wikipedia](https://en.wikipedia.org/wiki/Wikipedia:FAQ/Technical#What_software_is_used_to_run_Wikipedia?) or an online store. It offers excellent performance and can scale up or down as the needs of the application grow or shrink. An equally popular, but less obvious use case is log analytics, in which you take the logs from an application, feed them into OpenSearch, and use the rich search and visualization functionality to identify issues. For example, a malfunctioning web server might throw a 500 error 0.5% of the time, which can be hard to notice unless you have a real-time graph of all HTTP status codes that the server has thrown in the past four hours. You can use [OpenSearch Dashboards]({{site.url}}{{site.baseurl}}/dashboards/index/) to build these sorts of visualizations from data in OpenSearch. ## Components -Though this section focuses on the core product, OpenSearch is more than the core engine. It also includes the following components: +OpenSearch is more than just the core engine. It also includes the following components: -- [OpenSearch Dashboards]({{site.url}}{{site.baseurl}}/dashboards/index/): The OpenSearch data visualization user interface (UI). -- [Data Prepper]({{site.url}}{{site.baseurl}}/data-prepper/): A server-side data collector capable of filtering, enriching, transforming, normalizing, and aggregating data for downstream analytics and visualization. +- [OpenSearch Dashboards]({{site.url}}{{site.baseurl}}/dashboards/index/): The OpenSearch data visualization UI. +- [Data Prepper]({{site.url}}{{site.baseurl}}/data-prepper/): A server-side data collector capable of filtering, enriching, transforming, normalizing, and aggregating data for downstream analysis and visualization. - [Clients]({{site.url}}{{site.baseurl}}/clients/): Language APIs that let you communicate with OpenSearch in several popular programming languages. ## Use cases @@ -29,10 +29,10 @@ Though this section focuses on the core product, OpenSearch is more than the cor OpenSearch supports a variety of use cases, for example: - [Observability]({{site.url}}{{site.baseurl}}/observing-your-data/): Visualize data-driven events by using Piped Processing Language (PPL) to explore, discover, and query data stored in OpenSearch. -- [Search]({{site.url}}{{site.baseurl}}/search-plugins/): Choose the search method best for your application, from regular lexical search to conversational search powered by machine learning (ML). +- [Search]({{site.url}}{{site.baseurl}}/search-plugins/): Choose the best search method for your application, from regular lexical search to conversational search powered by machine learning (ML). - [Machine learning]({{site.url}}{{site.baseurl}}/ml-commons-plugin/): Integrate ML models into your OpenSearch application. -- [Security analytics]({{site.url}}{{site.baseurl}}/security-analytics/): Investigate, detect, analyze, and respond to security threats that can jeopardize the success of businesses and organizations and their online operations. +- [Security analytics]({{site.url}}{{site.baseurl}}/security-analytics/): Investigate, detect, analyze, and respond to security threats that can jeopardize organizational success and online operations. ## Next steps -- Learn the essential OpenSearch concepts in [Introduction to OpenSearch]({{site.url}}{{site.baseurl}}/getting-started/intro/). \ No newline at end of file +- See [Introduction to OpenSearch]({{site.url}}{{site.baseurl}}/getting-started/intro/) to learn about essential OpenSearch concepts. \ No newline at end of file diff --git a/_getting-started/ingest-data.md b/_getting-started/ingest-data.md index 6667da9f16..73cf1502f7 100644 --- a/_getting-started/ingest-data.md +++ b/_getting-started/ingest-data.md @@ -10,7 +10,7 @@ There are several ways to ingest data into OpenSearch: - Ingest individual documents. For more information, see [Indexing documents]({{site.url}}{{site.baseurl}}/getting-started/communicate/#indexing-documents). - Index multiple documents in bulk. For more information, see [Bulk indexing](#bulk-indexing). -- Use Data Prepper---an OpenSearch server-side data collector that can enrich data for downstream analytics and visualization. For more information, see [Data Prepper]({{site.url}}{{site.baseurl}}/data-prepper/). +- Use Data Prepper---an OpenSearch server-side data collector that can enrich data for downstream analysis and visualization. For more information, see [Data Prepper]({{site.url}}{{site.baseurl}}/data-prepper/). - Use other ingestion tools. For more information, see [OpenSearch tools]({{site.url}}{{site.baseurl}}/tools/). ## Bulk indexing @@ -28,7 +28,7 @@ POST _bulk ## Experiment with sample data -OpenSearch provides a fictitious e-commerce dataset that you can use to experiment with REST API requests and OpenSearch Dashboards visualizations. Create an index and define field mappings using a dataset provided by the OpenSearch Project. +OpenSearch provides a fictitious e-commerce dataset that you can use to experiment with REST API requests and OpenSearch Dashboards visualizations. You can create an index and define field mappings by downloading the corresponding dataset and mapping files. ### Create a sample index @@ -81,7 +81,7 @@ Use the following steps to create a sample index and define field mappings for t ### Query the data -Query the data using the Search API. The following query searches for documents where `customer_first_name` is `Sonya`: +Query the data using the Search API. The following query searches for documents in which `customer_first_name` is `Sonya`: ```json GET ecommerce/_search @@ -108,4 +108,4 @@ To learn how to use OpenSearch Dashboards to visualize the data, see the [OpenSe ## Next steps -- Learn about search options in [Search your data]({{site.url}}{{site.baseurl}}/getting-started/search-data/). \ No newline at end of file +- See [Search your data]({{site.url}}{{site.baseurl}}/getting-started/search-data/) to learn about search options. \ No newline at end of file diff --git a/_getting-started/intro.md b/_getting-started/intro.md index 3f93b91240..41ecfecb44 100644 --- a/_getting-started/intro.md +++ b/_getting-started/intro.md @@ -9,7 +9,7 @@ redirect_from: # Introduction to OpenSearch -OpenSearch is a distributed search and analytics engine that supports various use cases, from implementing a search box on a website to analyzing security data for threat detection. The term _distributed_ means you can run OpenSearch on multiple computers. _Search and analytics_ means you can search and analyze your data once you ingest it into OpenSearch. Whether your data is geographic or genetic, you can store and analyze it using OpenSearch. +OpenSearch is a distributed search and analytics engine that supports various use cases, from implementing a search box on a website to analyzing security data for threat detection. The term _distributed_ means that you can run OpenSearch on multiple computers. _Search and analytics_ means that you can search and analyze your data once you ingest it into OpenSearch. No matter your type of data, you can store and analyze it using OpenSearch. ## Document @@ -19,7 +19,7 @@ You can think of a document in several ways: - In a database of students, a document might represent one student. - When you search for information, OpenSearch returns documents related to your search. -- If you're familiar with traditional databases, a document represents a row. +- A document represents a row in a traditional database. For example, in a school database, a document might represent one student and contain the following data. @@ -45,9 +45,9 @@ An _index_ is a collection of documents. You can think of an index in several ways: -- If you have a collection of encyclopedia articles, an index represents the whole collection. +- In a database of students, an index represents all students in the database. - When you search for information, you query data contained in an index. -- If you're familiar with traditional databases, a document represents a database table. +- An index represents a database table in a traditional database. For example, in a school database, an index might contain all students in the school. @@ -59,13 +59,13 @@ ID | Name | GPA | Graduation year ## Clusters and nodes -OpenSearch is designed to be a distributed search engine. OpenSearch can run on one or more _nodes_---servers that store your data and process search requests. An OpenSearch *cluster* is a collection of nodes. +OpenSearch is designed to be a distributed search engine, meaning that it can run on one or more _nodes_---servers that store your data and process search requests. An OpenSearch *cluster* is a collection of nodes. You can run OpenSearch locally on a laptop---its system requirements are minimal---but you can also scale a single cluster to hundreds of powerful machines in a data center. -In a single-node cluster, such as a laptop, one machine has to do everything: manage the state of the cluster, index and search data, and perform any preprocessing of data prior to indexing it. As a cluster grows, however, you can subdivide responsibilities. Nodes with fast disks and plenty of RAM might be great at indexing and searching data, whereas a node with plenty of CPU power and a tiny disk could manage cluster state. +In a single-node cluster, such as one deployed on a laptop, one machine has to perform every task: manage the state of the cluster, index and search data, and perform any preprocessing of data prior to indexing it. As a cluster grows, however, you can subdivide responsibilities. Nodes with fast disks and plenty of RAM might perform well when indexing and searching data, whereas a node with plenty of CPU power and a tiny disk could manage cluster state. -In each cluster, there is an elected _cluster manager_ node, which orchestrates cluster-level operations, such as creating an index. Nodes communicate with each other, so if your request is routed to a node, that node sends requests to appropriate nodes, gathers the nodes' responses, and returns the final response. +In each cluster, there is an elected _cluster manager_ node, which orchestrates cluster-level operations, such as creating an index. Nodes communicate with each other, so if your request is routed to a node, that node sends requests to other nodes, gathers the nodes' responses, and returns the final response. For more information about other node types, see [Cluster formation]({{site.url}}{{site.baseurl}}/opensearch/cluster/). @@ -75,15 +75,15 @@ OpenSearch splits indexes into _shards_. Each shard stores a subset of all docum An index is split into shards -Shards are used for even distribution across nodes in a cluster. For example, a 400-GB index might be too large for any single node in your cluster to handle, but split into ten shards, each one 40 GB, OpenSearch can distribute the shards across ten nodes and work with each shard individually. Consider a cluster with two indexes: index 1 and index 2. Index 1 is split into 2 shards, and index 2 is split into 4 shards. The shards are distributed across nodes 1 and 2, as shown in the following image. +Shards are used for even distribution across nodes in a cluster. For example, a 400 GB index might be too large for any single node in your cluster to handle, but split into 10 shards of 40 GB each, OpenSearch can distribute the shards across 10 nodes and manage each shard individually. Consider a cluster with 2 indexes: index 1 and index 2. Index 1 is split into 2 shards, and index 2 is split into 4 shards. The shards are distributed across nodes 1 and 2, as shown in the following image. A cluster containing two indexes and two nodes -Despite being a piece of an OpenSearch index, each shard is actually a full Lucene index---confusing, we know. This detail is important, though, because each instance of Lucene is a running process that consumes CPU and memory. More shards is not necessarily better. Splitting a 400 GB index into 1,000 shards, for example, would place needless strain on your cluster. A good rule of thumb is to keep shard size as 10--50 GB. +Despite being one piece of an OpenSearch index, each shard is actually a full Lucene index. This detail is important because each instance of Lucene is a running process that consumes CPU and memory. More shards is not necessarily better. Splitting a 400 GB index into 1,000 shards, for example, would unnecessarily strain your cluster. A good rule of thumb is to limit shard size to 10--50 GB. ## Primary and replica shards -In OpenSearch, shards may be _primary_ (the original) or _replica_ (a copy). By default, OpenSearch creates a replica shard for each primary shard. Thus, if you split your index into 10 shards, OpenSearch creates 10 replica shards. For example, consider a cluster described in the previous section. If you add one replica for each shard of each index in the cluster, your cluster will contain a total of 2 shards and 2 replicas for index 1 and 4 shards and 4 replicas for index 2, as shown in the following image. +In OpenSearch, a shard may be either a _primary_ (original) shard or a _replica_ (copy) shard. By default, OpenSearch creates a replica shard for each primary shard. Thus, if you split your index into 10 shards, OpenSearch creates 10 replica shards. For example, consider the cluster described in the previous section. If you add 1 replica for each shard of each index in the cluster, your cluster will contain a total of 2 shards and 2 replicas for index 1 and 4 shards and 4 replicas for index 2, as shown in the following image. A cluster containing two indexes with one replica shard for each shard in the index @@ -91,12 +91,12 @@ These replica shards act as backups in the event of a node failure---OpenSearch ## Inverted index -An OpenSearch index uses a data structure called an _inverted index_. An inverted index maps words to the documents that they occur in. For example, consider an index containing the following two documents: +An OpenSearch index uses a data structure called an _inverted index_. An inverted index maps words to the documents in which they occur. For example, consider an index containing the following two documents: -- Document 1 : "Beauty is in the eye of the beholder" +- Document 1: "Beauty is in the eye of the beholder" - Document 2: "Beauty and the beast" -An inverted index for such an index maps the words to the documents where they occur: +An inverted index for such an index maps the words to the documents in which they occur: Word | Document :--- | :--- @@ -111,7 +111,7 @@ beholder | 1 and | 2 beast | 2 -In addition to the document ID, OpenSearch stores the position of the word within that document for phrase queries, where words must appear next to each other. +In addition to the document ID, OpenSearch stores the position of the word within the document for running phrase queries, where words must appear next to each other. ## Relevance @@ -119,13 +119,13 @@ When you search for a document, OpenSearch matches the words in the query to the Individual words in a search query are called search _terms_. Each search term is scored according to the following rules: -1. A search term that occurs more frequently in a document will tend to score higher. A document about dogs that uses the word `dog` many times is likely more relevant than a document that contains the word `dog` fewer times. This is the _term frequency_ component of the score. +1. A search term that occurs more frequently in a document will tend to be scored higher. A document about dogs that uses the word `dog` many times is likely more relevant than a document that contains the word `dog` fewer times. This is the _term frequency_ component of the score. -1. A search term that occurs in more documents will tend to score lower. A query for the terms `blue` and `axolotl` should prefer documents that contain `axolotl` over the likely more common word `blue`. This is the _inverse document frequency_ component of the score. +1. A search term that occurs in more documents will tend to be scored lower. A query for the terms `blue` and `axolotl` should prefer documents that contain `axolotl` over the likely more common word `blue`. This is the _inverse document frequency_ component of the score. -1. A match on a longer document should tend to score lower than a match on a shorter document. A document that contains a full dictionary would match on any word, but is not very relevant to any particular word. This corresponds to the _length normalization_ component of the score. +1. A match on a longer document should tend to be scored lower than a match on a shorter document. A document that contains a full dictionary would match on any word but is not very relevant to any particular word. This corresponds to the _length normalization_ component of the score. -OpenSearch uses the BM25 ranking algorithm to calculate document relevance scores and returns the results sorted by relevance. To learn more, see [Okapi BM25](https://en.wikipedia.org/wiki/Okapi_BM25). +OpenSearch uses the BM25 ranking algorithm to calculate document relevance scores and then returns the results sorted by relevance. To learn more, see [Okapi BM25](https://en.wikipedia.org/wiki/Okapi_BM25). ## Advanced concepts diff --git a/_getting-started/quickstart.md b/_getting-started/quickstart.md index 01350d3b2c..5ef783959a 100644 --- a/_getting-started/quickstart.md +++ b/_getting-started/quickstart.md @@ -66,7 +66,7 @@ You'll need a special file, called a Compose file, that Docker Compose uses to d ``` {% include copy.html %} -1. In your terminal application, navigate to the directory containing the `docker-compose.yml` file you downloaded, and run the following command to create and start the cluster as a background process: +1. In your terminal application, navigate to the directory containing the `docker-compose.yml` file you downloaded and run the following command to create and start the cluster as a background process: ```bash docker-compose up -d @@ -90,7 +90,7 @@ You'll need a special file, called a Compose file, that Docker Compose uses to d ``` {% include copy.html %} - The response confirms that the installation is successful: + The response confirms that the installation was successful: ```json { @@ -154,4 +154,4 @@ You successfully deployed your own OpenSearch cluster with OpenSearch Dashboards ## Next steps -- Learn about how to send requests to OpenSearch in [Communicate with OpenSearch]({{site.url}}{{site.baseurl}}/getting-started/communicate/). +- See [Communicate with OpenSearch]({{site.url}}{{site.baseurl}}/getting-started/communicate/) to learn about how to send requests to OpenSearch. diff --git a/_getting-started/search-data.md b/_getting-started/search-data.md index cf035935c8..c6970e7e7b 100644 --- a/_getting-started/search-data.md +++ b/_getting-started/search-data.md @@ -8,15 +8,15 @@ nav_order: 50 In OpenSearch, there are several ways to search data: -- [Query domain-specific language (DSL)]({{site.url}}{{site.baseurl}}/query-dsl/index/): The primary OpenSearch query language that supports creating complex, fully customizable queries. +- [Query domain-specific language (DSL)]({{site.url}}{{site.baseurl}}/query-dsl/index/): The primary OpenSearch query language, which you can use to create complex, fully customizable queries. - [Query string query language]({{site.url}}{{site.baseurl}}/query-dsl/full-text/query-string/): A scaled-down query language that you can use in a query parameter of a search request or in OpenSearch Dashboards. - [SQL]({{site.url}}{{site.baseurl}}/search-plugins/sql/sql/index/): A traditional query language that bridges the gap between traditional relational database concepts and the flexibility of OpenSearch’s document-oriented data storage. -- [Piped Processing Language (PPL)]({{site.url}}{{site.baseurl}}/search-plugins/sql/ppl/index/): The primary language used with observability in OpenSearch. PPL uses a pipe syntax that chains commands into a query. +- [Piped Processing Language (PPL)]({{site.url}}{{site.baseurl}}/search-plugins/sql/ppl/index/): The primary language used for observability in OpenSearch. PPL uses a pipe syntax that chains commands into a query. - [Dashboards Query Language (DQL)]({{site.url}}{{site.baseurl}}/dashboards/dql/): A simple text-based query language for filtering data in OpenSearch Dashboards. ## Prepare the data -For this tutorial, you'll need to index student data if you haven't done so. You can start by deleting the `students` index (`DELETE /students`) and then sending the following bulk request: +For this tutorial, you'll need to index student data if you haven't done so already. You can start by deleting the `students` index (`DELETE /students`) and then sending the following bulk request: ```json POST _bulk @@ -38,7 +38,7 @@ GET /students/_search ``` {% include copy-curl.html %} -The preceding request is an equivalent of the `match_all` query, which matches all documents in the index: +The preceding request is equivalent to the `match_all` query, which matches all documents in an index: ```json GET /students/_search @@ -118,7 +118,7 @@ The `took` field contains the amount of time the query took to run, in milliseco ### timed_out -This field specifies whether the request timed out. If a request timed out, OpenSearch returns those results that were gathered before the timeout. You can set the desired timeout value by providing the `timeout` query parameter: +This field indicates whether the request timed out. If a request timed out, then OpenSearch returns the results that were gathered before the timeout. You can set the desired timeout value by providing the `timeout` query parameter: ```json GET /students/_search?timeout=20ms @@ -129,13 +129,13 @@ GET /students/_search?timeout=20ms ### _shards -The `_shards` object specifies the total number of shards the query ran on, and the number of shards that were successful or failed. A shard may fail if the shard itself and all its replicas are unavailable. If any of the involved shards failed, OpenSearch continues to run the query on the remaining shards. +The `_shards` object specifies the total number of shards on which the query ran as well as the number of shards that succeeded or failed. A shard may fail if the shard itself and all its replicas are unavailable. If any of the involved shards fail, OpenSearch continues to run the query on the remaining shards. ### hits -The `hits` object contains the total number of matching documents and the documents themselves (listed in the `hits` array). Each matching document contains the `_index` and `_id` fields and the `_source` field, which contains the originally indexed complete document. +The `hits` object contains the total number of matching documents and the documents themselves (listed in the `hits` array). Each matching document contains the `_index` and `_id` fields as well as the `_source` field, which contains the complete originally indexed document. Each document is given a relevance score in the `_score` field. Because you ran a `match_all` search, all document scores are set to `1` (there is no difference in their relevance). The `max_score` field contains the highest score of any matching document. @@ -192,7 +192,7 @@ Using Query DSL, you can create more complex and customized queries. ### Full-text search -You can run a full-text search on fields mapped as `text`. By default, text fields are analyzed by the `default` analyzer. The analyzer splits text into terms and makes it lowercase. For more information about OpenSearch analyzers, see [Analyzers]({{site.url}}{{site.baseurl}}/analyzers/). +You can run a full-text search on fields mapped as `text`. By default, text fields are analyzed by the `default` analyzer. The analyzer splits text into terms and changes it to lowercase. For more information about OpenSearch analyzers, see [Analyzers]({{site.url}}{{site.baseurl}}/analyzers/). For example, the following query searches for students with the name `john`: @@ -306,7 +306,7 @@ The match query type uses `OR` as an operator by default, so the query is functi ### Keyword search -The `name` field contains the `name.keyword` subfield, which was added by OpenSearch automatically. You can try to search the `name.keyword` field in a manner similar to the previous request: +The `name` field contains the `name.keyword` subfield, which is added by OpenSearch automatically. If you search the `name.keyword` field in a manner similar to the previous request: ```json GET /students/_search @@ -320,9 +320,9 @@ GET /students/_search ``` {% include copy-curl.html %} -This request returns no hits because the `keyword` fields must be matched exactly. +Then the request returns no hits because the `keyword` fields must exactly match. -However, you can search for the exact text `John Doe`: +However, if you search for the exact text `John Doe`: ```json GET /students/_search @@ -372,7 +372,7 @@ OpenSearch returns the matching document: ### Filters -You can add a filter clause to your query for fields with exact values using a Boolean query. +Using a Boolean query, you can add a filter clause to your query for fields with exact values Term filters match specific terms. For example, the following Boolean query searches for students whose graduation year is 2022: @@ -390,7 +390,7 @@ GET students/_search ``` {% include copy-curl.html %} -Range filters support specifying a range of values. For example, the following Boolean query searches for students whose GPA is greater than 3.6: +With range filters, you can specify a range of values. For example, the following Boolean query searches for students whose GPA is greater than 3.6: ```json GET students/_search @@ -438,7 +438,7 @@ For more information about Boolean and other compound queries, see [Compound que ## Search methods -Along with the traditional full-text search described in this tutorial, OpenSearch supports a range of machine learning (ML)-powered search methods, including k-NN, semantic, multimodal, sparse, hybrid, and conversational search. For information about all search methods, see [Search]({{site.url}}{{site.baseurl}}/search-plugins/). +Along with the traditional full-text search described in this tutorial, OpenSearch supports a range of machine learning (ML)-powered search methods, including k-NN, semantic, multimodal, sparse, hybrid, and conversational search. For information about all OpenSearch-supported search methods, see [Search]({{site.url}}{{site.baseurl}}/search-plugins/). ## Next steps From c552f74cff6e4669f77ee7b27b1d1384febe5c65 Mon Sep 17 00:00:00 2001 From: kolchfa-aws <105444904+kolchfa-aws@users.noreply.github.com> Date: Thu, 4 Apr 2024 10:46:18 -0400 Subject: [PATCH 19/23] Update _getting-started/intro.md Co-authored-by: Nathan Bower Signed-off-by: kolchfa-aws <105444904+kolchfa-aws@users.noreply.github.com> --- _getting-started/intro.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/_getting-started/intro.md b/_getting-started/intro.md index 41ecfecb44..ce80bfebf7 100644 --- a/_getting-started/intro.md +++ b/_getting-started/intro.md @@ -138,7 +138,7 @@ The lifecycle of an update operation consists of the following steps: 1. An update is received by a primary shard and is written to the shard's transaction log ([translog](#translog)). The translog is flushed to disk (followed by an fsync) before the update is acknowledged. This guarantees durability. 1. The update is also passed to the Lucene index writer, which adds it to an in-memory buffer. 1. On a [refresh operation](#refresh), the Lucene index writer flushes the in-memory buffers to disk (with each buffer becoming a new Lucene segment), and a new index reader is opened over the resulting segment files. The updates are now visible for search. -1. On a [flush operation](#flush), the shard fsyncs the Lucene segments. Because the segment files are a durable representation of the updates, the translog is no longer needed to provide durability, do the updates can be purged from the translog. +1. On a [flush operation](#flush), the shard fsyncs the Lucene segments. Because the segment files are a durable representation of the updates, the translog is no longer needed to provide durability, so the updates can be purged from the translog. ### Translog From ad6bfaa96926f5f06e15f73bfd01112ac3198621 Mon Sep 17 00:00:00 2001 From: Fanit Kolchina Date: Thu, 4 Apr 2024 10:55:52 -0400 Subject: [PATCH 20/23] Fix links Signed-off-by: Fanit Kolchina --- _getting-started/communicate.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/_getting-started/communicate.md b/_getting-started/communicate.md index 3feb4f3348..47e5b257ab 100644 --- a/_getting-started/communicate.md +++ b/_getting-started/communicate.md @@ -135,7 +135,7 @@ OpenSearch responds with the field `type` for each field: OpenSearch mapped the numeric fields to the `float` and `long` types. Notice that OpenSearch mapped the `name` text field to `text` and added a `name.keyword` subfield mapped to `keyword`. Fields mapped to `text` are analyzed (lowercased and split into terms) and can be used for full-text search. Fields mapped to `keyword` are used for exact term search. -OpenSearch mapped the `grad_year` field to `long`. If you want to map it to the `date` type instead, you need to [delete the index](#deleting-the-index) and then recreate it, explicitly specifying the mappings. For instructions on how to explicitly specify mappings, see [Index settings and mappings](#index-settings-and-mappings). +OpenSearch mapped the `grad_year` field to `long`. If you want to map it to the `date` type instead, you need to [delete the index](#deleting-an-index) and then recreate it, explicitly specifying the mappings. For instructions on how to explicitly specify mappings, see [Index settings and mappings](#index-mappings-and-settings). ## Searching for documents From 5c66faf3938968478852e2a8aea2dcce8bcba9f0 Mon Sep 17 00:00:00 2001 From: Fanit Kolchina Date: Thu, 4 Apr 2024 11:09:31 -0400 Subject: [PATCH 21/23] Reword Signed-off-by: Fanit Kolchina --- _getting-started/communicate.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/_getting-started/communicate.md b/_getting-started/communicate.md index 47e5b257ab..7d62d0f5b2 100644 --- a/_getting-started/communicate.md +++ b/_getting-started/communicate.md @@ -185,7 +185,7 @@ OpenSearch returns the indexed document: } ``` -For more information about search methods, see [Search your data]({{site.url}}{{site.baseurl}}/getting-started/search-data/). +For more information about search, see [Search your data]({{site.url}}{{site.baseurl}}/getting-started/search-data/). ## Updating documents From 901897d01aad90c149319d15a3077b25bfb1b994 Mon Sep 17 00:00:00 2001 From: Fanit Kolchina Date: Thu, 4 Apr 2024 11:12:43 -0400 Subject: [PATCH 22/23] Reword Signed-off-by: Fanit Kolchina --- _getting-started/communicate.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/_getting-started/communicate.md b/_getting-started/communicate.md index 7d62d0f5b2..391bc9bef0 100644 --- a/_getting-started/communicate.md +++ b/_getting-started/communicate.md @@ -283,7 +283,7 @@ GET /students/_mapping ``` {% include copy-curl.html %} -OpenSearch mapped the `name` and `grad_year` fields according to the specified types and inferred the field types for the other fields: +OpenSearch mapped the `name` and `grad_year` fields according to the specified types and inferred the field type for the `gpa` field: ```json { From 8793d8c9b837a1d647ca6134073a435526679ce6 Mon Sep 17 00:00:00 2001 From: kolchfa-aws <105444904+kolchfa-aws@users.noreply.github.com> Date: Thu, 4 Apr 2024 16:39:40 -0400 Subject: [PATCH 23/23] Update _getting-started/intro.md Signed-off-by: kolchfa-aws <105444904+kolchfa-aws@users.noreply.github.com> --- _getting-started/intro.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/_getting-started/intro.md b/_getting-started/intro.md index ce80bfebf7..272d8d6981 100644 --- a/_getting-started/intro.md +++ b/_getting-started/intro.md @@ -142,7 +142,7 @@ The lifecycle of an update operation consists of the following steps: ### Translog -An indexing or bulk call responds when the documents have been written to the translog and the translog is flushed to disk, so the updates are durable. The updates will be visible from search requests until after a [refresh operation](#refresh). +An indexing or bulk call responds when the documents have been written to the translog and the translog is flushed to disk, so the updates are durable. The updates will not be visible to search requests until after a [refresh operation](#refresh). ### Refresh