[DOC] Add new documentation alerting monitors and anomaly detectors

.github/ISSUE_TEMPLATE/broken_links.md

@@ -0,0 +1,7 @@
+---
+title: '[AUTOCUT] Broken links'
+labels: 'bug'
+---
+
+Links checker has failed on push of your commit.
+Please examine the workflow log {{ env.WORKFLOW_URL }}.

.github/workflows/jekyll-build.yml

@@ -0,0 +1,16 @@
+name: Jekyll Build Verification
+
+on: [pull_request]
+
+jobs:
+  check:
+    runs-on: ubuntu-latest
+
+    steps:
+    - uses: actions/checkout@v3
+    - uses: ruby/setup-ruby@v1
+      with:
+        ruby-version: '3.0'
+        bundler-cache: true
+    - run: |
+        JEKYLL_FATAL_LINK_CHECKER=internal bundle exec jekyll build --future

.github/workflows/link-checker.yml

@@ -0,0 +1,26 @@
+name: Check Links
+on:
+  workflow_dispatch:
+  schedule:
+    - cron: "30 11 * * 0"
+jobs:
+  check:
+    if: github.repository == ‘opensearch-project/documentation-website’
+    runs-on: ubuntu-latest
+    steps:
+    - uses: actions/checkout@v3
+    - uses: ruby/setup-ruby@v1
+      with:
+        ruby-version: '3.0'
+        bundler-cache: true
+    - run: |
+        JEKYLL_FATAL_LINK_CHECKER=all bundle exec jekyll build --future
+    - name: Create Issue On Build Failure
+      if: ${{ failure() }}
+      uses: dblock/create-a-github-issue@v3
+      env:
+        GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
+        WORKFLOW_URL: "${{ github.server_url }}/${{ github.repository }}/actions/runs/${{ github.run_id }}"
+      with:
+        update_existing: true
+        filename: .github/ISSUE_TEMPLATE/broken_links.md

Gemfile

@@ -32,4 +32,8 @@ gem "tzinfo-data", platforms: [:mingw, :mswin, :x64_mingw, :jruby]
 gem "wdm", "~> 0.1.0" if Gem.win_platform?
 
 # Installs webrick dependency for building locally
-gem "webrick", "~> 1.7"
+gem "webrick", "~> 1.7"
+
+# Link checker
+gem "typhoeus"
+gem "ruby-link-checker"

_api-reference/explain.md

@@ -10,7 +10,7 @@ Introduced 1.0
 
 Wondering why a specific document ranks higher (or lower) for a query? You can use the explain API for an explanation of how the relevance score (`_score`) is calculated for every result.
 
-OpenSearch uses a probabilistic ranking framework called [Okapi BM25](https://en.wikipedia.org/wiki/Okapi_BM25) to calculate relevance scores. Okapi BM25 is based on the original [TF/IDF](http://lucene.apache.org/core/{{site.lucene_version}}/core/org/apache/lucene/search/package-summary.html#scoring) framework used by Apache Lucene.
+OpenSearch uses a probabilistic ranking framework called [Okapi BM25](https://en.wikipedia.org/wiki/Okapi_BM25) to calculate relevance scores. Okapi BM25 is based on the original [TF/IDF](https://lucene.apache.org/core/{{site.lucene_version}}/core/org/apache/lucene/search/package-summary.html#scoring) framework used by Apache Lucene.
 
 The explain API is an expensive operation in terms of both resources and time. On production clusters, we recommend using it sparingly for the purpose of troubleshooting.
 {: .warning }

_clients/OSC-dot-net.md

@@ -15,7 +15,7 @@ This getting started guide illustrates how to connect to OpenSearch, index docum
 
 ## Installing OpenSearch.Client
 
-To install OpenSearch.Client, download the [OpenSearch.Client NuGet package](https://www.nuget.org/packages/OpenSearch.Client) and add it to your project in an IDE of your choice. In Microsoft Visual Studio, follow the steps below: 
+To install OpenSearch.Client, download the [OpenSearch.Client NuGet package](https://www.nuget.org/packages/OpenSearch.Client/) and add it to your project in an IDE of your choice. In Microsoft Visual Studio, follow the steps below: 
 - In the **Solution Explorer** panel, right-click on your solution or project and select **Manage NuGet Packages for Solution**.
 - Search for the OpenSearch.Client NuGet package, and select **Install**.
 

_clients/java.md

@@ -10,30 +10,55 @@ The OpenSearch Java client allows you to interact with your OpenSearch clusters
 
 This getting started guide illustrates how to connect to OpenSearch, index documents, and run queries. For the client source code, see the [opensearch-java repo](https://github.com/opensearch-project/opensearch-java).
 
-## Installing the client
+## Installing the client using Apache HttpClient 5 Transport
 
-To start using the OpenSearch Java client, ensure that you have the following dependencies in your project's `pom.xml` file:
+To start using the OpenSearch Java client, you need to provide a transport. The default `ApacheHttpClient5TransportBuilder` transport comes with the Java client. To use the OpenSearch Java client with the default transport, add it to your `pom.xml` file as a dependency:
+
+```xml
+<dependency>
+  <groupId>org.opensearch.client</groupId>
+  <artifactId>opensearch-java</artifactId>
+  <version>2.3.0</version>
+</dependency>
+```
+{% include copy.html %}
+
+If you're using Gradle, add the following dependencies to your project:
+
+```
+dependencies {
+  implementation 'org.opensearch.client:opensearch-java:2.3.0'
+}
+```
+{% include copy.html %}
+
+You can now start your OpenSearch cluster.
+
+## Installing the client using RestClient Transport
+
+Alternatively, you can create a Java client by using the `RestClient`-based transport. In this case, make sure that you have the following dependencies in your project's `pom.xml` file:
 
 ```xml
 <dependency>
   <groupId>org.opensearch.client</groupId>
   <artifactId>opensearch-rest-client</artifactId>
   <version>{{site.opensearch_version}}</version>
 </dependency>
+
 <dependency>
   <groupId>org.opensearch.client</groupId>
   <artifactId>opensearch-java</artifactId>
-  <version>2.2.0</version>
+  <version>2.3.0</version>
 </dependency>
 ```
 {% include copy.html %}
 
-If you're using Gradle, add the following dependencies to your project.
+If you're using Gradle, add the following dependencies to your project"
 
 ```
 dependencies {
-    implementation 'org.opensearch.client:opensearch-rest-client: {{site.opensearch_version}}'
-    implementation 'org.opensearch.client:opensearch-java:2.0.0'
+  implementation 'org.opensearch.client:opensearch-rest-client: {{site.opensearch_version}}'
+  implementation 'org.opensearch.client:opensearch-java:2.3.0'
 }
 ```
 {% include copy.html %}
@@ -95,57 +120,118 @@ static class IndexData {
 ```
 {% include copy.html %}
 
-## Initializing the client with SSL and TLS enabled
+## Initializing the client with SSL and TLS enabled using Apache HttpClient 5 Transport
 
 This code example uses basic credentials that come with the default OpenSearch configuration. If you’re using the Java client with your own OpenSearch cluster, be sure to change the code so that it uses your own credentials.
 
 The following sample code initializes a client with SSL and TLS enabled:
 
+
 ```java
-import org.apache.http.HttpHost;
-import org.apache.http.auth.AuthScope;
-import org.apache.http.auth.UsernamePasswordCredentials;
-import org.apache.http.client.CredentialsProvider;
-import org.apache.http.impl.client.BasicCredentialsProvider;
-import org.apache.http.impl.nio.client.HttpAsyncClientBuilder;
+import javax.net.ssl.SSLContext;
+import javax.net.ssl.SSLEngine;
+
+import org.apache.hc.client5.http.auth.AuthScope;
+import org.apache.hc.client5.http.auth.UsernamePasswordCredentials;
+import org.apache.hc.client5.http.impl.auth.BasicCredentialsProvider;
+import org.apache.hc.client5.http.impl.nio.PoolingAsyncClientConnectionManager;
+import org.apache.hc.client5.http.impl.nio.PoolingAsyncClientConnectionManagerBuilder;
+import org.apache.hc.client5.http.ssl.ClientTlsStrategyBuilder;
+import org.apache.hc.core5.function.Factory;
+import org.apache.hc.core5.http.HttpHost;
+import org.apache.hc.core5.http.nio.ssl.TlsStrategy;
+import org.apache.hc.core5.reactor.ssl.TlsDetails;
+import org.apache.hc.core5.ssl.SSLContextBuilder;
+import org.opensearch.client.opensearch.OpenSearchClient;
+import org.opensearch.client.transport.OpenSearchTransport;
+import org.opensearch.client.transport.httpclient5.ApacheHttpClient5TransportBuilder;
+
+public class OpenSearchClientExample {
+  public static void main(String[] args) throws Exception {
+    System.setProperty("javax.net.ssl.trustStore", "/full/path/to/keystore");
+    System.setProperty("javax.net.ssl.trustStorePassword", "password-to-keystore");
+
+    final HttpHost host = new HttpHost("https", "localhost", 9200);
+    final BasicCredentialsProvider credentialsProvider = new BasicCredentialsProvider();
+    // Only for demo purposes. Don't specify your credentials in code.
+    credentialsProvider.setCredentials(new AuthScope(host), new UsernamePasswordCredentials("admin", "admin".toCharArray()));
+
+    final SSLContext sslcontext = SSLContextBuilder
+      .create()
+      .loadTrustMaterial(null, (chains, authType) -> true)
+      .build();
+
+    final ApacheHttpClient5TransportBuilder builder = ApacheHttpClient5TransportBuilder.builder(host);
+    builder.setHttpClientConfigCallback(httpClientBuilder -> {
+      final TlsStrategy tlsStrategy = ClientTlsStrategyBuilder.create()
+        .setSslContext(SSLContextBuilder.create().build())
+        // See https://issues.apache.org/jira/browse/HTTPCLIENT-2219
+        .setTlsDetailsFactory(new Factory<SSLEngine, TlsDetails>() {
+          @Override
+          public TlsDetails create(final SSLEngine sslEngine) {
+            return new TlsDetails(sslEngine.getSession(), sslEngine.getApplicationProtocol());
+          }
+        })
+        .build();
+
+      final PoolingAsyncClientConnectionManager connectionManager = PoolingAsyncClientConnectionManagerBuilder
+        .create()
+        .setTlsStrategy(tlsStrategy)
+        .build();
+
+      return httpClientBuilder
+        .setDefaultCredentialsProvider(credentialsProvider)
+        .setConnectionManager(connectionManager);
+    });
+
+    final OpenSearchTransport transport = ApacheHttpClient5TransportBuilder.builder(host).build();
+    OpenSearchClient client = new OpenSearchClient(transport);
+  }
+}
+
+```
+
+## Initializing the client with SSL and TLS enabled using RestClient Transport
+
+This code example uses basic credentials that come with the default OpenSearch configuration. If you’re using the Java client with your own OpenSearch cluster, be sure to change the code so that it uses your own credentials.
+
+The following sample code initializes a client with SSL and TLS enabled:
+
+```java
+import org.apache.hc.client5.http.auth.AuthScope;
+import org.apache.hc.client5.http.auth.UsernamePasswordCredentials;
+import org.apache.hc.client5.http.impl.async.HttpAsyncClientBuilder;
+import org.apache.hc.client5.http.impl.auth.BasicCredentialsProvider;
+import org.apache.hc.core5.http.HttpHost;
 import org.opensearch.client.RestClient;
 import org.opensearch.client.RestClientBuilder;
-import org.opensearch.client.base.RestClientTransport;
-import org.opensearch.client.base.Transport;
 import org.opensearch.client.json.jackson.JacksonJsonpMapper;
 import org.opensearch.client.opensearch.OpenSearchClient;
-import org.opensearch.client.opensearch._global.IndexRequest;
-import org.opensearch.client.opensearch._global.IndexResponse;
-import org.opensearch.client.opensearch._global.SearchResponse;
-import org.opensearch.client.opensearch.indices.*;
-import org.opensearch.client.opensearch.indices.put_settings.IndexSettingsBody;
-
-import java.io.IOException;
+import org.opensearch.client.transport.OpenSearchTransport;
+import org.opensearch.client.transport.rest_client.RestClientTransport;
 
 public class OpenSearchClientExample {
-  public static void main(String[] args) {
-    RestClient restClient = null;
-    try{
+  public static void main(String[] args) throws Exception {
     System.setProperty("javax.net.ssl.trustStore", "/full/path/to/keystore");
     System.setProperty("javax.net.ssl.trustStorePassword", "password-to-keystore");
 
+    final HttpHost host = new HttpHost("https", "localhost", 9200);
+    final BasicCredentialsProvider credentialsProvider = new BasicCredentialsProvider();
     //Only for demo purposes. Don't specify your credentials in code.
-    final CredentialsProvider credentialsProvider = new BasicCredentialsProvider();
-    credentialsProvider.setCredentials(AuthScope.ANY,
-        new UsernamePasswordCredentials("admin", "admin"));
+    credentialsProvider.setCredentials(new AuthScope(host), new UsernamePasswordCredentials("admin", "admin".toCharArray()));
 
     //Initialize the client with SSL and TLS enabled
-    restClient = RestClient.builder(new HttpHost("localhost", 9200, "https")).
+    final RestClient restClient = RestClient.builder(host).
       setHttpClientConfigCallback(new RestClientBuilder.HttpClientConfigCallback() {
         @Override
         public HttpAsyncClientBuilder customizeHttpClient(HttpAsyncClientBuilder httpClientBuilder) {
         return httpClientBuilder.setDefaultCredentialsProvider(credentialsProvider);
         }
       }).build();
-    Transport transport = new RestClientTransport(restClient, new JacksonJsonpMapper());
-    OpenSearchClient client = new OpenSearchClient(transport);
+
+    final OpenSearchTransport transport = new RestClientTransport(restClient, new JacksonJsonpMapper());
+    final OpenSearchClient client = new OpenSearchClient(transport);
   }
- }
 }
 ```
 {% include copy.html %}

_config.yml

@@ -5,10 +5,10 @@ baseurl: "/docs/latest" # the subpath of your site, e.g. /blog
 url: "https://opensearch.org" # the base hostname & protocol for your site, e.g. http://example.com
 permalink: /:path/
 
-opensearch_version: 2.6.0
-opensearch_dashboards_version: 2.6.0
-opensearch_major_minor_version: 2.6
-lucene_version: 9_5_0
+opensearch_version: '2.6.0'
+opensearch_dashboards_version: '2.6.0'
+opensearch_major_minor_version: '2.6'
+lucene_version: '9_5_0'
 
 # Build settings
 markdown: kramdown

_dashboards/reporting.md

@@ -54,7 +54,7 @@ This problem can occur for two reasons:
 
 - You don't have the correct version of `headless-chrome` to match the operating system on which OpenSearch Dashboards is running. Download the [correct version](https://github.com/opensearch-project/reporting/releases/tag/chromium-1.12.0.0).
 
-- You're missing additional dependencies. Install the required dependencies for your operating system from the [additional libraries](https://github.com/opensearch-project/dashboards-reports/blob/main/dashboards-reports/rendering-engine/headless-chrome/README.md#additional-libaries) section.
+- You're missing additional dependencies. Install the required dependencies for your operating system from the [additional libraries](https://github.com/opensearch-project/dashboards-reports/blob/1.x/dashboards-reports/rendering-engine/headless-chrome/README.md#additional-libaries) section.
 
 ### Characters not loading in reports
 

_data-prepper/common-use-cases/trace-analytics.md

@@ -39,7 +39,7 @@ The [OpenTelemetry source]({{site.url}}{{site.baseurl}}/data-prepper/pipelines/c
 There are three processors for the trace analytics feature:
 
 * *otel_trace_raw* - The *otel_trace_raw* processor receives a collection of [span](https://github.com/opensearch-project/data-prepper/blob/fa65e9efb3f8d6a404a1ab1875f21ce85e5c5a6d/data-prepper-api/src/main/java/org/opensearch/dataprepper/model/trace/Span.java) records from [*otel-trace-source*]({{site.url}}{{site.baseurl}}/data-prepper/pipelines/configuration/sources/otel-trace/), and performs stateful processing, extraction, and completion of trace-group-related fields.
-* *otel_trace_group* - The *otel_trace_group* processor fills in the missing trace-group-related fields in the collection of [span](https://github.com/opensearch-project/data-prepper/blob/fa65e9efb3f8d6a404a1ab1875f21ce85e5c5a6d/data-prepper-api/src/main/java/com/amazon/dataprepper/model/trace/Span.java) records by looking up the OpenSearch backend.
+* *otel_trace_group* - The *otel_trace_group* processor fills in the missing trace-group-related fields in the collection of [span](https://github.com/opensearch-project/data-prepper/blob/298e7931aa3b26130048ac3bde260e066857df54/data-prepper-api/src/main/java/org/opensearch/dataprepper/model/trace/Span.java) records by looking up the OpenSearch backend.
 * *service_map_stateful* – The *service_map_stateful* processor performs the required preprocessing for trace data and builds metadata to display the `service-map` dashboards.
 
 

_data-prepper/managing-data-prepper/configuring-log4j.md

@@ -11,7 +11,7 @@ You can configure logging using Log4j in Data Prepper.
 
 ## Logging 
 
-Data Prepper uses [SLF4J](http://www.slf4j.org/) with a [Log4j 2 binding](http://logging.apache.org/log4j/2.x/log4j-slf4j-impl/). 
+Data Prepper uses [SLF4J](https://www.slf4j.org/) with a [Log4j 2 binding](https://logging.apache.org/log4j/2.x/log4j-slf4j-impl.html).
 
 For Data Prepper versions 2.0 and later, the Log4j 2 configuration file can be found and edited in `config/log4j2.properties` in the application's home directory. The default properties for Log4j 2 can be found in `log4j2-rolling.properties` in the *shared-config* directory.
 

_data-prepper/managing-data-prepper/monitoring.md

@@ -11,11 +11,11 @@ You can monitor Data Prepper with metrics using [Micrometer](https://micrometer.
 
 ## JVM and system metrics
 
-JVM and system metrics are runtime metrics that are used to monitor Data Prepper instances. They include metrics for classloaders, memory, garbage collection, threads, and others. For more information, see [JVM and system metrics](https://micrometer.io/docs/ref/jvm). 
+JVM and system metrics are runtime metrics that are used to monitor Data Prepper instances. They include metrics for classloaders, memory, garbage collection, threads, and others. For more information, see [JVM and system metrics](https://micrometer.io/?/docs/ref/jvm).
 
 ### Naming
 
-JVM and system metrics follow predefined names in [Micrometer](https://micrometer.io/docs/concepts#_naming_meters). For example, the Micrometer metrics name for memory usage is `jvm.memory.used`. Micrometer changes the name to match the metrics system. Following the same example, `jvm.memory.used` is reported to Prometheus as `jvm_memory_used`, and is reported to Amazon CloudWatch as `jvm.memory.used.value`.
+JVM and system metrics follow predefined names in [Micrometer](https://micrometer.io/?/docs/concepts#_naming_meters). For example, the Micrometer metrics name for memory usage is `jvm.memory.used`. Micrometer changes the name to match the metrics system. Following the same example, `jvm.memory.used` is reported to Prometheus as `jvm_memory_used`, and is reported to Amazon CloudWatch as `jvm.memory.used.value`.
 
 ### Serving
 

_data-prepper/pipelines/configuration/sources/http-source.md

@@ -20,7 +20,7 @@ request_timeout | No | Integer | The request timeout, in milliseconds. Default v
 thread_count | No | Integer | The number of threads to keep in the ScheduledThreadPool. Default value is `200`.
 max_connection_count | No | Integer | The maximum allowed number of open connections. Default value is `500`.
 max_pending_requests | No | Integer | The maximum allowed number of tasks in the `ScheduledThreadPool` work queue. Default value is `1024`.
-authentication | No | Object | An authentication configuration. By default, this creates an unauthenticated server for the pipeline. This uses pluggable authentication for HTTPS. To use basic authentication define the `http_basic` plugin with a `username` and `password`. To provide customer authentication, use or create a plugin that implements [ArmeriaHttpAuthenticationProvider](https://github.com/opensearch-project/data-prepper/blob/main/data-prepper-plugins/armeria-common/src/main/java/com/amazon/dataprepper/armeria/authentication/ArmeriaHttpAuthenticationProvider.java).
+authentication | No | Object | An authentication configuration. By default, this creates an unauthenticated server for the pipeline. This uses pluggable authentication for HTTPS. To use basic authentication define the `http_basic` plugin with a `username` and `password`. To provide customer authentication, use or create a plugin that implements [ArmeriaHttpAuthenticationProvider](https://github.com/opensearch-project/data-prepper/blob/1.2.0/data-prepper-plugins/armeria-common/src/main/java/com/amazon/dataprepper/armeria/authentication/ArmeriaHttpAuthenticationProvider.java).
 ssl | No | Boolean | Enables TLS/SSL. Default value is false.
 ssl_certificate_file | Conditionally | String | SSL certificate chain file path or Amazon Simple Storage Service (Amazon S3) path. Amazon S3 path example `s3://<bucketName>/<path>`. Required if `ssl` is set to true and `use_acm_certificate_for_ssl` is set to false.
 ssl_key_file | Conditionally | String | SSL key file path or Amazon S3 path. Amazon S3 path example `s3://<bucketName>/<path>`. Required if `ssl` is set to true and `use_acm_certificate_for_ssl` is set to false.