Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

docs mvp #456

Merged
merged 59 commits into from
Jul 10, 2024
Merged
Changes from 1 commit
Commits
Show all changes
59 commits
Select commit Hold shift + click to select a range
051dc91
added patterns page
matryer Jun 13, 2024
2384610
small metadata changes
matryer Jun 14, 2024
693a689
updated Browse log volumes by type
matryer Jun 14, 2024
c0587a5
tweaks
matryer Jun 14, 2024
5069402
more work on docs
matryer Jun 14, 2024
6787f52
updated homepage
matryer Jun 14, 2024
ca33362
removed old file
matryer Jun 14, 2024
37769e9
added labels and detected fields docs
matryer Jun 14, 2024
27cb50a
updated heading
matryer Jun 14, 2024
2e69d74
Merge branch 'main' into docs-mvp
matryer Jun 14, 2024
e86d251
updated description
matryer Jun 14, 2024
7c1066f
better landing page
matryer Jun 14, 2024
7fe29c9
added preview disclaimer
matryer Jun 14, 2024
ca57c1f
added get started
matryer Jun 14, 2024
6c40bb3
simpler
matryer Jun 14, 2024
ef88c95
added Get started to main page
matryer Jun 14, 2024
11bb5c4
tweakz
matryer Jun 14, 2024
8907526
trying this
matryer Jun 14, 2024
93c8284
use correct preview warning
matryer Jun 18, 2024
0a666c7
Merge branch 'main' into docs-mvp
matryer Jun 27, 2024
eba0531
added npm command
matryer Jul 1, 2024
51d9680
Merge branch 'main' into docs-mvp
matryer Jul 2, 2024
9c0a6dd
Merge branch 'main' into docs-mvp
gtk-grafana Jul 2, 2024
7a22252
fixes
matryer Jul 2, 2024
4c3f827
fleshed out the guide
matryer Jul 2, 2024
1c61877
better bullets
matryer Jul 2, 2024
b05a9f0
better formatting
matryer Jul 2, 2024
709bd62
tidied up installation notes
matryer Jul 2, 2024
7c4292b
Merge branch 'main' into docs-mvp
gtk-grafana Jul 2, 2024
f6ed2f3
Update docs/sources/_index.md
gtk-grafana Jul 2, 2024
47401ac
Update docs/sources/_index.md
gtk-grafana Jul 2, 2024
6096527
Update docs/sources/_index.md
gtk-grafana Jul 2, 2024
2bef3fa
Merge branch 'main' into docs-mvp
matryer Jul 3, 2024
7b5f97a
fixed What's Next
matryer Jul 3, 2024
228b567
small docs tweak
matryer Jul 3, 2024
cad8c12
tweaked message
matryer Jul 4, 2024
c0124d4
added feedback prompt to preview notice
matryer Jul 4, 2024
cab2176
added troubleshooting page
matryer Jul 4, 2024
bd46816
Merge branch 'main' into docs-mvp
matryer Jul 4, 2024
d5cdd36
Detected Fields -> Fields
matryer Jul 4, 2024
bdf4442
Merge branch 'docs-mvp' of https://github.com/grafana/explore-logs in…
matryer Jul 4, 2024
13d198a
tweaks
matryer Jul 4, 2024
8a9238f
updated troubleshooting
matryer Jul 4, 2024
da4cb6d
added links to troubleshooting
matryer Jul 4, 2024
352ae0d
Merge branch 'docs-mvp' of https://github.com/grafana/explore-logs in…
matryer Jul 4, 2024
e5126df
minor tweaks
matryer Jul 4, 2024
6a9e704
tweaks
matryer Jul 4, 2024
39b9085
added grafana play
matryer Jul 4, 2024
ef5546b
Apply suggestions from code review
JStickler Jul 8, 2024
ccb401f
added image and updated content
matryer Jul 9, 2024
3766dfe
moved Public Preview banner to the very top
matryer Jul 9, 2024
a5f42b9
better Public Preview banner
matryer Jul 9, 2024
bb87676
small improvements
matryer Jul 9, 2024
0017248
added deciated access page
matryer Jul 10, 2024
7973a24
tweaked title
matryer Jul 10, 2024
5400eff
small tidy up
matryer Jul 10, 2024
edd439c
Merge branch 'main' into docs-mvp
matryer Jul 10, 2024
00255a8
Merge branch 'main' into docs-mvp
matryer Jul 10, 2024
dfc08ad
Merge branch 'main' into docs-mvp
matryer Jul 10, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
96 changes: 96 additions & 0 deletions docs/sources/patterns/_index.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,96 @@
---
aliases:
matryer marked this conversation as resolved.
Show resolved Hide resolved
- /docs/explore-apps/logs/patterns
description:
matryer marked this conversation as resolved.
Show resolved Hide resolved
keywords:
- Logs
- Log Patterns
- Explore
- Patterns
- Drain
- Categorization
- Analysis
menuTitle: Log patterns
title: Log patterns
weight: 800
matryer marked this conversation as resolved.
Show resolved Hide resolved
---

# Log patterns

Log patterns allow you to work with groups of similar log lines. They are automatically extracted when your logs are ingested.

The Explore Logs app shows you the patterns alongside their log volumes. From this view, you can investigate spikes and drill down to view the logs.

## More use cases

Log patterns allow you to:

* Browse the log volume over time of different types of logs.
* Simplify log management by grouping similar log entries.
* Enhance log searches by focusing on relevant patterns.
* Improve troubleshooting efficiency by highlighting critical log lines.
* Reduce storage requirements by minimizing log data.
* Filter out noisy log lines during exploration.
* Identify specific log lines for targeted analysis.

## Quick guides

We've outlined the steps you'll need to take to perform these common use cases.

### Browse log volumes by type



### Targetted analysis

If you know the kind of log line you're looking for, Log patterns are an easy way to get everything else out of the way.

To view only a specific set of patterns, perform the following steps:

1. Go to **Explore > Logs**
matryer marked this conversation as resolved.
Show resolved Hide resolved
2. Select the relevant **Service**
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
2. Select the relevant **Service**
1. Select the relevant **Service**.

3. Open the **Patterns** tab
matryer marked this conversation as resolved.
Show resolved Hide resolved
4. Identify a pattern that matches the type of logs you're interested in
matryer marked this conversation as resolved.
Show resolved Hide resolved
5. Select **Include**
matryer marked this conversation as resolved.
Show resolved Hide resolved
6. Go to the **Logs** tab and notice the filtered view
matryer marked this conversation as resolved.
Show resolved Hide resolved

### Hide noise
matryer marked this conversation as resolved.
Show resolved Hide resolved

To hide noisy log lines, perform the following steps:

1. Go to **Explore > Logs**
matryer marked this conversation as resolved.
Show resolved Hide resolved
2. Select the relevant **Service**
matryer marked this conversation as resolved.
Show resolved Hide resolved
3. Open the **Patterns** tab
gtk-grafana marked this conversation as resolved.
Show resolved Hide resolved
4. Identify a pattern that represents noise in the logs
gtk-grafana marked this conversation as resolved.
Show resolved Hide resolved
5. Select **Exclude**
gtk-grafana marked this conversation as resolved.
Show resolved Hide resolved
6. Go to the **Logs** tab and notice the noise has been removed
gtk-grafana marked this conversation as resolved.
Show resolved Hide resolved

You can repeat steps 4 and 5 to exclude multiple patterns.

## Pattern extraction

Loki uses Drain, an online log template miner, to extract patterns from a stream of log lines.

For example, if your services log lines like this:

```
duration=255ms trace_id=abc001 GET /path/to/endpoint/2
user loaded: 25666
user loaded: 14544
duration=255ms trace_id=abc002 POST /path/to/endpoint/2
user loaded: 25666
duration=355ms trace_id=abc003 GET /path/to/endpoint/1
duration=355ms trace_id=abc004 POST /path/to/endpoint/1
duration=255ms trace_id=abc005 POST /path/to/endpoint/2
user loaded: 89255
duration=4244ms trace_id=abc006 GET /path/to/endpoint/1
user loaded: 25666
duration=255ms trace_id=abc007 GET /path/to/endpoint/2
```

Two patterns would emerge where the static tokens are preserved literally, and dynamic values are turned into placeholders:

Pattern 1: `duration=<_> trace_id=<_> <_> /path/to/endpoint/<_>`

Pattern 2: `user loaded: <_>`