Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[SIEM][Docs][Backport]Clarifies SIEM index and Kibana index pattern requirements #749

Merged
merged 3 commits into from
Dec 11, 2019
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
21 changes: 12 additions & 9 deletions docs/en/siem/installation.asciidoc
Original file line number Diff line number Diff line change
Expand Up @@ -18,11 +18,19 @@ https://www.elastic.co/cloud/elasticsearch-service/signup[Try the {es}
Service for free].
==============

* *{beats}* shippers (version 7.x or later) installed for each system you want to
monitor
* *{beats}* (version 7.x or later) or other data shippers installed for each
system you want to monitor.

You might need to modify UI settings in {kib} to change default behaviors,
such as the index pattern used to query the data. For more information, see {kib}.
[IMPORTANT]
==============
If you use a third-party collector to ship data to the {siem-app}, you must
map its fields to the {ecs-ref}[Elastic Common Schema (ECS)]. Additionally, you
must add its index to the {siem-soln} {es} indices (*{kib}* ->
*Management* -> *Advanced Settings* -> *`siem:defaultIndex`*).

{siem-soln} uses the {ecs-ref}/ecs-host.html[`host.name`] ECS field as the
primary key for identifying hosts.
==============

[float]
[[install-beats]]
Expand All @@ -39,11 +47,6 @@ Windows event logs
* https://www.elastic.co/products/beats/packetbeat[{packetbeat}] for analyzing
network activity

NOTE: The {ecs-ref}[Elastic Common Schema (ECS)] enables {siem-soln} to work
with custom and third-party data sources in addition to
those supported by Beats. It is important to note that {siem-soln} uses the
{ecs-ref}/ecs-host.html[`host.name`] ECS field as the primary key for identifying hosts.

You can install {beats} using a {kib}-based guide or directly from the command line.

[float]
Expand Down
47 changes: 36 additions & 11 deletions docs/en/siem/siem-ui.asciidoc
Original file line number Diff line number Diff line change
Expand Up @@ -16,9 +16,10 @@ collapsible, resizable, moveable, and so forth.
The *{kibana-ref}/kuery-query.html[{kib} Query Language (KQL)]* bar is available
throughout the {siem-app} for searching and filtering.

NOTE: The default index patterns for {siem-soln} events are `endgame-*`,
`auditbeat-*`, `winlogbeat-*`, `filebeat-*`, and `packetbeat-*`. You can change
the default index patterns in {kib} -> Management -> Advanced Settings -> `siem:defaultIndex`.
NOTE: The default index glob patterns defined for {siem-soln} events are
`endgame-*`, `auditbeat-*`, `winlogbeat-*`, `filebeat-*`, and `packetbeat-*`.
You can change the default glob patterns in {kib} -> Management -> Advanced
Settings -> `siem:defaultIndex`.


[float]
Expand Down Expand Up @@ -205,24 +206,48 @@ that aren't present? Let us know. We welcome your input.
Depending on your {kib} setup, to display and interact with data on the map you
might need to:

* <<kibana-index-pattern>>
* <<geoip-data, Add geographical IP data to events>>
* <<private-network>>

NOTE: To see source and destination connections lines on the map, you must
configure `source.geo` and `destination.geo` ECS fields for your indices.

[float]
[[kibana-index-pattern]]
==== Create {kib} index patterns

To display map data, you must define
{kib} {kibana-ref}/tutorial-define-index.html[index patterns] (*Management* ->
*Index Patterns*) with exactly the same names or glob patterns used to define
the {siem-soln} {es} indices.

NOTE: The {siem-soln} {es} indices are defined in the `siem:defaultIndex` field
(*{kib}* -> *Management* -> *Advanced Settings* -> *`siem:defaultIndex`*).

For example, if you define a {siem-soln} {es} `servers-europe-*` glob pattern,
to display map data for the matching indices you must also define a {kib} index
pattern named `servers-europe-*`. If you use a different {kib} index pattern,
such as `servers-*`, map data for the indices is *not* displayed.

// =====================
// Keeping this commented out because it will be true in the future...
// To display data on the map, you must define {kib} index patterns
// (Management -> Index Patterns) that match all the {siem-soln} {es} indices you
// want to visualize. For more information, see
// {kibana-ref}/tutorial-define-index.html[Define your index patterns].
// =====================

[float]
[[geoip-data]]
==== Add geoIP data

If you are not using Beats to ship your data, add the relevant index patterns to
Kibana (Management -> Index patterns) and the SIEM app (Management -> Advanced
settings -> SIEM default index).
When the ECS {ecs-ref}/ecs-geo.html[source.geo.location
and destination.geo.location] fields are mapped, network data is displayed on
When the ECS {ecs-ref}/ecs-geo.html[source.geo.location and
destination.geo.location] fields are mapped, network data is displayed on
the map.

If you use Beats, configure a geoIP processor to add data to the relevant fields:
If you use Beats, configure a geoIP processor to add data to the relevant
fields:

[[geo-pipeleine]]
. Define an ingest node pipeline that uses one or more `geoIP` processors to add
Expand Down Expand Up @@ -283,8 +308,8 @@ that contains the IP address to use for the geographical lookup, and
`"ignore_missing": true` configures the pipeline to continue processing when
it encounters an event that doesn't have the specified field.

. In your Beats configuration files, add the pipeline to the `output.elasticsearch`
tag:
. In your Beats configuration files, add the pipeline to the
`output.elasticsearch`tag:
+
[source,yml]
----------------------------------
Expand Down