Add getting started doc

[ebeahan]

Overview

This PR introduces a Getting Started overview in the ECS documentation. It's intended to give a brief overview of how to adopt and use ECS aimed at newcomers to the schema. The concepts and approach borrow heavily from the original ECS intro blog. The hope is introducing helps ECS newcomers answer the question: "where do I start with ECS?" and helps to address previous user feedback (#516).

Structure

The doc is structured in three sections: parsing, search, and visualization. Again this borrows from the structure of the original blog post. I wanted to retain the value demonstrated in the original blog and bring it into the ECS docs; the value ECS brings to each of these three areas and how ECS improves the user experience across the stack.

Parsing

The first section walks through a simple, common, and well-understood event, an Apache web server log, and provides a detailed field mapping. The intention is to provide newcomers with an event mapping reference example and also priming them to understand ECS adoption into their stack may require some examination of their current data pipeline and data normalization strategy.

Searching

Basic example of how normalizing fields will provide a simplified search experience across diverse data sources.

Visualization

Again a basic example to demonstrate the value of normalized fields. Added a screenshot and images subdirectory to the docs folder. There may be additional changes required for supporting images in the documentations (this is a first for the ECS docs 😄 ), but local testing looks ok.

Placement

Getting Started is added underneath the Using section of the existing docs immediately following the ECS Overview page. This seemed the most useful for a new user (view some example real-world type application before the guidelines and conventions), but I'm open for further discussion on placement.

[ebeahan]

@benskelker would appreciate your feedback here when you have some free cycles.

[benskelker]

Looks good. All my comments are nits and suggestions - do whatever you want with them :).

[webmat]

This is great, thanks @ebeahan!

I've entered a few comments, mostly about the structure of this doc, moving sections around and splitting the table in two. Hopefully my comments cover this clearly. Let me know what you think.

Another thing I haven't covered in comments below is that I think we could end this getting started with pointers to the Logs and Security docs as well. Something along the lines of:

We've covered at a high level how to map your events to ECS. Now if you'd like your events to render well in the Elastic solutions, head over here to learn more about each:

• Logs UI fields reference
• Elastic Security fields reference

Side note on the link for Elastic Security: right now it's published here under SIEM, but the link above is for the future version post merge of "Elastic Security", and it's only available via the /master/ link, until the next release. Not sure when we'll merge this PR and whether we'll want to backport it to the "current" ECS docs (I think we should, this is good stuff). If so, we may have to use the old link first, and make sure to change to the new link after the release.

Finally, I saw some question about whether the image would render well. Note that each PR's doc is rendered via links such as the following, where the subdomain is adjusted with the PR number. https://ecs_888.docs-preview.app.elstc.co/diff. If you scroll to the end of the new page, you'll see it does render well :-) This link can be found at the end of the Jenkins doc build output as well.

[webmat]

I think we should have sections "Simplified Search" and "Unified Visualizations" before the table breakdown.

I think they do a good job of selling people on why they should care. Then when they care, they can dive into the full breakdown of a sample log.

[DanRoscigno]

I agree, a short intro to the reasons helps frame the rest of the doc. Here is how I start an email to encourage use of ECS by new customers:

The Elastic Common Schema (ECS) is an open source specification developed with support from the Elastic user community. ECS defines a common set of fields to be used with data in Elasticsearch, such as logs and metrics.

ECS enables and encourages the normalization of event data, so that you can better analyze, visualize, and correlate the information represented in your events. This drives efficiency in your operations.

[webmat]

Thanks @benskelker for the detailed feedback as well :-)

[ebeahan]

Thanks all for the reviews! I've reworked the doc a bit to incorporate the feedback.

• I split the mapping example into three tables instead of two. However, after splitting the processor-populated fields into its own table, it may be more detail than necessary for a Getting Started doc. I'm interested in other's feedback.

  1. The values extracted verbatim from the original log
  2. ecs.* and event.* metadata fields
  3. Additional fields populated by a processor

• Shifted the search and visualization sections up higher.

• Reworked the intro to incorporate the review feedback and align with the new section ordering.

• Added the links to the field reference guides at the bottom.

• Added link to Getting Started to the top-level ECS index.asciidoc page.

[webmat]

A few final tweaks. The last table I've spotted a few errors that should be corrected.

Some of my other comments are how I would approach it, but not blockers if you feel otherwise.

This is looking great :-)

[webmat]

That's a Chrome version! 😂

Suggested change

	| user_agent.os.version
	| user_agent.version

[ebeahan]

😂 It also was missing the final column and wasn't showing up in the rendered doc either! Cleaned up on the latest commit.

[webmat]

LGTM 👍

[webmat]

I just noticed we hadn't backported this to 1.5. We should do so, in order for it to appear in the "current" docs now, rather than when we release 1.6.

[webmat]

When we do backport, we need to adjust the link from Elastic Security to Elastic SIEM.

[ebeahan]

Thanks for the reminder. I'll work on it 👍