More updates for journalbeat

journalbeat/docs/config-options.asciidoc

@@ -5,70 +5,68 @@
 <titleabbrev>Configure inputs</titleabbrev>
 ++++
 
-IMPORTANT: This documentation is placeholder content. It has not yet been reviewed.
-
 By default, {beatname_uc} reads log events from the default systemd journals. To
 specify other journal files, set the <<{beatname_lc}-paths,`paths`>> option in
-the +{beatname_lc}.inputs+ section of the +{beatname_lc}.yml+ file. 
-
-The list of paths is a YAML array, so each path begins with a dash (-). Each
-path can be a directory path (to collect events from all journals in a
-directory), or a file path. For example:
+the +{beatname_lc}.inputs+ section of the +{beatname_lc}.yml+ file. Each path
+can be a directory path (to collect events from all journals in a directory), or
+a file path. For example:
 
 ["source","sh",subs="attributes"]
 ----
 {beatname_lc}.inputs:
 - paths:
   - "/dev/log"
-  - "/var/log/messages/my-journal-file"
+  - "/var/log/messages/my-journal-file.journal"
 ----
 
-Within the +{beatname_lc}.inputs+ section, you can also specify options that
-control the position where {beatname_uc} starts reading the journal file, and
-set filters to reduce the fields that {beatname_uc} needs to process. See
-<<{beatname_lc}-options>> for a list of available options.
-
-[float]
-=== Configuration examples
-
-The following example shows how to monitor multiple journals under the
-same directory. {beatname_uc} merges all journals under the directory into a
-single journal and reads them. With `seek` set to `cursor`, {beatname_uc}
-starts reading at the beginning of the journal, but will continue reading where
-it left off after a reload or restart. 
-
+Within the configuration file, you can also specify options that control how
+{beatname_uc} reads the journal files and which fields are sent to the
+configured output. See <<{beatname_lc}-options>> for a list of available
+options.
+
+The following examples show how to configure {beatname_uc} for some common use
+cases.
+
+[[monitor-multiple-journals]]
+.Example 1: Monitor multiple journals under the same directory
+This example configures {beatname_uc} to read from multiple journals that
+are stored under the same directory. {beatname_uc} merges all journals under the
+directory into a single journal and reads them. With `seek` set to `cursor`,
+{beatname_uc} starts reading at the beginning of the journal, but will continue
+reading where it left off after a reload or restart.
 ["source","sh",subs="attributes"]
 ----
 {beatname_lc}.inputs:
 - paths: ["/path/to/journal/directory"]
   seek: cursor
 ----
 
-The following examples show how to get Redis events from a Docker container that
-is tagged as `redis`.
-
-//TODO: Add a better explanation of the options.
-
-This example uses the translated fields by Journald:
-
+[[filter-using-field-names]]
+.Example 2: Fetch log events for Redis running on Docker (uses field names from systemd)
+This example configures {beatname_uc} to fetch log events for Redis running in a
+Docker container. The fields are matched using field names from the systemd
+journal.
 ["source","sh",subs="attributes"]
 ----
 {beatname_lc}.inputs:
 - paths: []
   include_matches:
-    - "container.image.tag=redis"
-    - "process.name=redis"
+    - "CONTAINER_TAG=redis"
+    - "_COMM=redis"
 ----
 
-This example uses the field names from the systemd journal:
-
+[[filter-using-translated-names]]
+.Example 3: Fetch log events for Redis running on Docker (uses translated field names)
+This example also configures {beatname_uc} to fetch log events for Redis running in a
+Docker container. However, in this example the fields are matched using the
+translated field names provided by {beatname_uc}.
 ["source","sh",subs="attributes"]
 ----
 {beatname_lc}.inputs:
 - paths: []
   include_matches:
-    - "CONTAINER_TAG=redis"
-    - "_COMM=redis"
+    - "container.image.tag=redis"
+    - "process.name=redis"
 ----
 
 [id="{beatname_lc}-options"]
@@ -84,9 +82,7 @@ journal files.
 A list of paths that will be crawled and fetched. Each path can be a directory
 path (to collect events from all journals in a directory), or a file path. If
 you specify a directory, {beatname_uc} merges all journals under the directory
-into a single journal and reads them.
-
-//QUESTION: Are globs supported? If so, I need to add more detail here. 
+into a single journal and reads them. 
 
 [float]
 [id="{beatname_lc}-seek"]
@@ -96,7 +92,33 @@ The position to start reading the journal from. Valid settings are:
 
 * `head`: Starts reading at the beginning of the file.
 * `tail`: Starts reading at the end of the file.
-* `cursor`: Initially starts reading at the beginning of the file, but continues
-reading where it left off after a reload or restart. 
+* `cursor`: On first read, starts reading at the beginning of the file. After a
+reload or restart, continues reading where it left off.
+
+When specified under `paths`, the `seek` setting applies to all journals under
+the configured paths. When specified directly under the +{beatname_lc}+
+namespace, the setting applies to all journals read by {beatname_uc}.
+
+[float]
+[id="{beatname_lc}-include-matches"]
+==== `include_matches`
+
+A list of filter expressions used to match fields. The format of the expression
+is `field=value`. {beatname_uc} fetches all events that exactly match the
+expressions. Pattern matching is not supported.
+
+To reference fields, use one of the following:
+
+* The field name used by the systemd journal. For example,
+`CONTAINER_TAG=redis` (<<filter-using-field-names,see a full example>>).
+* The translated field name used by {beatname_uc}. For example,
+`container.image.tag=redis`
+(<<filter-using-translated-names,see a full example>>). {beatname_uc}
+does not translate all fields from the journal. For custom fields, use the name
+specified in the systemd journal.
+
+When specified under `paths`, the `include_matches` filter is applied to all
+journals under the configured paths. When specified directly under the
++{beatname_lc}+ namespace, the setting applies to all journals read by
+{beatname_uc}.
 
-//TODO: ADD OTHER OPTIONS HERE.     

journalbeat/docs/configuring-howto.asciidoc

@@ -4,21 +4,11 @@
 [partintro]
 --
 
-IMPORTANT: This documentation is placeholder content. It has not yet been reviewed.
-
 Before modifying configuration settings, make sure you've completed the
 <<{beatname_lc}-configuration,configuration steps>> in the Getting Started.
 This section describes some common use cases for changing configuration options.
 
-To configure {beatname_uc}, you edit the configuration file. For rpm and deb,
-you’ll find the configuration file at +/etc/{beatname_lc}/{beatname_lc}.yml+.
-There's also a full example configuration file at
-+/etc/{beatname_lc}/{beatname_lc}.reference.yml+ that shows all non-deprecated
-options. For mac and win, look in the archive that you extracted.
-
-The {beatname_uc} configuration file uses http://yaml.org/[YAML] for its syntax.
-See the {libbeat}/config-file-format.html[Config File Format] section of the
-_{libbeat_docs}_ for more about the structure of the config file.
+include::../../libbeat/docs/shared-configuring.asciidoc[]
 
 The following topics describe how to configure {beatname_uc}:
 
@@ -31,7 +21,7 @@ The following topics describe how to configure {beatname_uc}:
 * <<configuring-ingest-node>>
 * <<configuration-path>>
 * <<setup-kibana-endpoint>>
-* <<configuration-dashboards>>
+//* <<configuration-dashboards>>
 * <<configuration-template>>
 * <<configuration-logging>>
 * <<using-environ-vars>>

journalbeat/docs/faq.asciidoc

@@ -1,24 +1,10 @@
 [[faq]]
 == Frequently asked questions
 
-IMPORTANT: This documentation is placeholder content. It has not yet been reviewed.
-
 This section contains frequently asked questions about {beatname_uc}. Also check
 out the https://discuss.elastic.co/c/beats/{beatname_lc}[{beatname_uc}
 discussion forum].
 
-[float]
-[id="{beatname_lc}-sometext"]
-=== Question 1?
-
-ADD DESCRIPTION HERE
-
-[float]
-[id="{beatname_lc}-sometext2"]
-=== Question 2?
-
-ADD DESCRIPTION HERE
-
 include::../../libbeat/docs/faq-limit-bandwidth.asciidoc[]
 
 include::../../libbeat/docs/shared-faq.asciidoc[]

journalbeat/docs/filtering.asciidoc

@@ -1,20 +1,15 @@
 [[filtering-and-enhancing-data]]
 == Filter and enhance the exported data
 
-IMPORTANT: This documentation is placeholder content. It has not yet been reviewed.
-
 Your use case might require only a subset of the data exported by {beatname_uc},
 or you might need to enhance the exported data (for example, by adding
 metadata). {beatname_uc} provides a couple of options for filtering and
 enhancing exported data.
 
-You can configure each input to include or exclude specific lines or files. This
-allows you to specify different filtering criteria for each input. To do this,
-you use the `include_lines`, `exclude_lines`, and `exclude_files` options under
-the +{beatname_lc}.inputs+ section of the config file (see
-<<configuration-{beatname_lc}-options>>). The disadvantage of this approach is
-that you need to implement a configuration option for each filtering criteria
-that you need.
+You can configure {beatname_uc} to include events that match specific filtering
+criteria. To do this, use the <<{beatname_lc}-include-matches,`include_matches`>>
+option. The advantage of this approach is that you can reduce the number of
+fields that {beatname_uc} needs to process.
 
 Another approach (the one described here) is to define processors to configure
 global processing across all data exported by {beatname_uc}.
@@ -26,12 +21,6 @@ global processing across all data exported by {beatname_uc}.
 
 include::../../libbeat/docs/processors.asciidoc[]
 
-[float]
-[[specific-example]]
-==== XYZ example
-
-ADD EXAMPLES SPECIFIC TO THE BEAT, OR DELETE THIS SECTION
-
 // You must set the processor-scope attribute to resolve the attribute reference
 // defined in processors-using.asciidoc. The attribute is used to indicate where
 // processors are valid. If processors are valid in more than two locations

journalbeat/docs/general-options.asciidoc

@@ -1,10 +1,66 @@
 [[configuration-general-options]]
 == Specify general settings
 
-IMPORTANT: This documentation is placeholder content. It has not yet been reviewed.
-
 You can specify settings in the +{beatname_lc}.yml+ config file to control the
-general behavior of {beatname_uc}.
+general behavior of {beatname_uc}. This includes:
+
+* <<configuration-global-options,Global options>> that control things like
+publisher behavior and the location of some files.
+
+* <<configuration-general,General options>> that are supported by all Elastic
+Beats.
+
+[float]
+[[configuration-global-options]]
+=== Global {beatname_uc} configuration options
+
+These options are in the +{beatname_lc}+ namespace.
+
+[float]
+[id="{beatname_lc}-registry-file"]
+==== `registry_file`
+
+The name of the registry file. If a relative path is used, it is considered relative to the
+data path. See the <<directory-layout>> section for details. The default is `${path.data}/registry`.
+
+["source","sh",subs="attributes"]
+----
+{beatname_lc}.registry_file: registry
+----
+
+[float]
+[id="{beatname_lc}-backoff"]
+==== `backoff`
+
+The number of seconds to wait before trying to read again from journals. The
+default is 1s.
+
+[float]
+[id="{beatname_lc}-backoff-factor"]
+==== `backoff_factor`
+
+Multiplier of the backoff value. The default is 1s.
+
+[float]
+[id="{beatname_lc}-max-backoff"]
+==== `max_backoff`
+
+The maximum number of seconds to wait before attempting to read again from
+journals. The default is 60s.
+
+[float]
+==== `seek`
+
+This option is valid as a global setting under the +{beatname_lc}+ namespace
+or under `paths`. For a description of this option, see
+<<{beatname_lc}-seek,`seek`>>. 
+
+[float]
+==== `include_matches`
+
+This option is valid as a global setting under the +{beatname_lc}+ namespace
+or under `paths`. For a description of this option, see
+<<{beatname_lc}-include-matches,`include_matches`>>. 
 
 include::../../libbeat/docs/generalconfig.asciidoc[]