Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Docs: Local install edits #2220

Merged
merged 32 commits into from
Jun 22, 2020
Merged
Show file tree
Hide file tree
Changes from 31 commits
Commits
Show all changes
32 commits
Select commit Hold shift + click to select a range
187808e
Update README.md
oddlittlebird Jun 15, 2020
3698a0c
Update README.md
oddlittlebird Jun 15, 2020
50621f2
Update README.md
oddlittlebird Jun 15, 2020
b5f09a5
Update tanka.md
oddlittlebird Jun 15, 2020
988e8d1
content edits
oddlittlebird Jun 15, 2020
6fdf172
Create install-for-manual-build.md
oddlittlebird Jun 15, 2020
3e2f7ed
Update README.md
oddlittlebird Jun 15, 2020
16c0ade
Update helm.md
oddlittlebird Jun 15, 2020
6d1c37d
Update local.md
oddlittlebird Jun 15, 2020
33c123a
Create get-logs-into-loki.md
oddlittlebird Jun 15, 2020
7b4d916
Update local.md
oddlittlebird Jun 15, 2020
27cecf9
Update get-logs-into-loki.md
oddlittlebird Jun 15, 2020
18adfdf
Update get-logs-into-loki.md
oddlittlebird Jun 15, 2020
94e9a98
Update local.md
oddlittlebird Jun 15, 2020
1734b6d
Update get-logs-into-loki.md
oddlittlebird Jun 15, 2020
4079d4e
Update README.md
oddlittlebird Jun 15, 2020
97b5ed0
Update README.md
oddlittlebird Jun 15, 2020
7c94ee7
Update installation.md
oddlittlebird Jun 15, 2020
bff077e
Update README.md
oddlittlebird Jun 15, 2020
7c47118
Update docs/getting-started/get-logs-into-loki.md
oddlittlebird Jun 16, 2020
eb84176
Update docs/installation/local.md
oddlittlebird Jun 22, 2020
39749bc
Update docs/installation/local.md
oddlittlebird Jun 22, 2020
a0bc598
applied edits
oddlittlebird Jun 22, 2020
c0e2caa
Merge branch 'docs-local-install-edits' of https://github.com/grafana…
oddlittlebird Jun 22, 2020
f0feae6
Update docs/getting-started/get-logs-into-loki.md
oddlittlebird Jun 22, 2020
2651ae7
Update get-logs-into-loki.md
oddlittlebird Jun 22, 2020
84d74b7
Merge branch 'docs-local-install-edits' of https://github.com/grafana…
oddlittlebird Jun 22, 2020
bbee20d
Update docs/getting-started/get-logs-into-loki.md
oddlittlebird Jun 22, 2020
e160bd0
Update docs/getting-started/get-logs-into-loki.md
oddlittlebird Jun 22, 2020
6d67539
Update docs/installation/README.md
oddlittlebird Jun 22, 2020
a9c957d
applied edits
oddlittlebird Jun 22, 2020
e09f375
Update local.md
oddlittlebird Jun 22, 2020
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
16 changes: 8 additions & 8 deletions docs/clients/README.md
Original file line number Diff line number Diff line change
@@ -1,13 +1,13 @@
# Loki Clients
# Loki clients

Loki supports the following official clients for sending logs:

1. [Promtail](./promtail/README.md)
2. [Docker Driver](./docker-driver/README.md)
3. [Fluentd](./fluentd/README.md)
4. [Fluent Bit](../../cmd/fluent-bit/README.md)
- [Promtail](./promtail/README.md)
- [Docker Driver](./docker-driver/README.md)
- [Fluentd](./fluentd/README.md)
- [Fluent Bit](../../cmd/fluent-bit/README.md)

## Picking a Client
## Picking a client

While all clients can be used simultaneously to cover multiple use cases, which
client is initially picked to send logs depends on your use case.
Expand Down Expand Up @@ -41,9 +41,9 @@ and you already have configured `Parser` and `Filter` plugins.
Fluentd also works well for extracting metrics from logs when using its
Prometheus plugin.

# Unofficial Clients
# Unofficial clients

Please note that the Loki API is not stable yet and breaking changes may occur
Please note that the Loki API is not stable yet, so breaking changes might occur
when using or writing a third-party client.

- [promtail-client](https://github.com/afiskon/promtail-client) (Go)
Expand Down
12 changes: 6 additions & 6 deletions docs/clients/promtail/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,14 +6,14 @@ deployed to every machine that has applications needed to be monitored.

It primarily:

1. Discovers targets
2. Attaches labels to log streams
3. Pushes them to the Loki instance.
- Discovers targets
- Attaches labels to log streams
- Pushes them to the Loki instance.

Currently, Promtail can tail logs from two sources: local log files and the
systemd journal (on AMD64 machines only).

## Log File Discovery
## Log file discovery

Before Promtail can ship any data from log files to Loki, it needs to find out
information about its environment. Specifically, this means discovering
Expand All @@ -32,12 +32,12 @@ Just like Prometheus, `promtail` is configured using a `scrape_configs` stanza.
drop, and the final metadata to attach to the log line. Refer to the docs for
[configuring Promtail](configuration.md) for more details.

## Receiving Logs From Syslog
## Receiving logs From Syslog

When the [Syslog Target](./scraping.md#syslog-target) is being used, logs
can be written with the syslog protocol to the configured port.

## Labeling and Parsing
## Labeling and parsing

During service discovery, metadata is determined (pod name, filename, etc.) that
may be attached to the log line as a label for easier identification when
Expand Down
10 changes: 5 additions & 5 deletions docs/clients/promtail/installation.md
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
# Installing Promtail
# Install Promtail

Promtail is distributed as a [binary](#binary), [Docker container](#docker), and
[Helm chart](#helm).
Expand All @@ -12,7 +12,7 @@ Every release includes binaries for Promtail which can be found on the

```bash
# modify tag to most recent version
$ docker pull grafana/promtail:1.5.0
docker pull grafana/promtail:1.5.0
```

## Helm
Expand All @@ -23,13 +23,13 @@ Make sure that Helm is
Then you can add Loki's chart repository to Helm:

```bash
$ helm repo add loki https://grafana.github.io/loki/charts
helm repo add loki https://grafana.github.io/loki/charts
```

And the chart repository can be updated by running:

```bash
$ helm repo update
helm repo update
```

Finally, Promtail can be deployed with:
Expand All @@ -40,7 +40,7 @@ $ helm upgrade --install promtail loki/promtail --set "loki.serviceName=loki"

## Kubernetes

### DaemonSet (Recommended)
### DaemonSet (recommended)

A `DaemonSet` will deploy `promtail` on every node within a Kubernetes cluster.

Expand Down
74 changes: 74 additions & 0 deletions docs/getting-started/get-logs-into-loki.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,74 @@
# Get logs into Loki

After you [install and run Loki](./installation/local.md), you probably want to get logs from other applications into it.

To get application logs into Loki, you need to edit the [Promtail](./clients/promtail/README.md) config file.

Detailed information about configuring Promtail is available in [Promtail configuration](./clients/promtail/configuration.md).

The following instructions should help you get started.

1. If you haven't already, download a Promtail configuration file. Keep track of where it is, because you will need to cite it when you run the binary.

```
wget https://github.com/grafana/loki/blob/master/cmd/promtail/promtail-local-config.yaml
```

2. Open the config file in the text editor of your choice. It should look similar to this:

```
server:
http_listen_port: 9080
grpc_listen_port: 0

positions:
filename: /tmp/positions.yaml

clients:
- url: http://loki:3100/loki/api/v1/push

scrape_configs:
- job_name: system
static_configs:
- targets:
- localhost
labels:
job: varlogs
__path__: /var/log/*log
```

The seven lines under `scrape_configs` are what send the logs that Loki generates to Loki, which then outputs them in the command line and http://localhost:3100/metrics.

3. Copy the seven lines under `scrape_configs`, and then paste them under the original job (you can also just edit the original seven lines).

Below is an example that sends logs from a default Grafana installation to Loki. We updated the following fields:
- job_name - This differentiates the logs collected from other log groups.
- targets - Optional for static_configs, however is often defined because in older versions of Promtail it was not optional. This was an artifact from directly using the Prometheus service discovery code which required this entry.
- labels - Static label to apply to every log line scraped by this definition. Good examples would be environment name, job name, or app name.
- __path__ - The path to where the logs are stored that I want Loki to consume.

```
- job_name: grafana
static_configs:
- targets:
- grafana
labels:
job: grafana
__path__: "C:/Program Files/GrafanaLabs/grafana/data/log/grafana.log"
```

4. Enter the following command to run Promtail. Examples below assume you have put the config file in the same directory as the binary.

**Windows**

```
`.\promtail-windows-amd64.exe --config.file=promtail-local-config.yaml`
```

**Linux**

```
./promtail-linux-amd64 -config.file=promtail-local-config.yaml
```

You should now see your application logs. If you are using Grafana, you might need to refresh your instance in order to see the logs.
25 changes: 20 additions & 5 deletions docs/installation/README.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,21 @@
# Installing Loki
# Install Loki

1. [Installing using Tanka (recommended)](./tanka.md)
2. [Installing through Helm](./helm.md)
3. [Installing through Docker or Docker Compose](./docker.md)
4. [Installing locally](./local.md)
## Installation methods

Instructions for different methods of installing Loki and Promtail.

- [Install using Tanka (recommended)](./tanka.md)
- [Install through Helm](./helm.md)
- [Install through Docker or Docker Compose](./docker.md)
- [Install and run locally](./local.md)
- [Install from source](./install-from-source.md)

## General process

In order to run Loki, you must:

1. Download and install both Loki and Promtail.
1. Download config files for both programs.
1. Start Loki.
1. Update the Promtail config file to get your logs into Loki.
1. Start Promtail.
15 changes: 8 additions & 7 deletions docs/installation/docker.md
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
# Installing Loki with Docker or Docker Compose
# Install Loki with Docker or Docker Compose

You can install Loki with Docker or Docker Compose for evaluating, testing, or developing Loki.
For production, we recommend Tanka or Helm.
You can install Loki and Promtail with Docker or Docker Compose if you are evaluating, testing, or developing Loki.
For production, we recommend installing with Tanka or Helm.

## Prerequisites

Expand All @@ -25,8 +25,7 @@ When finished, `loki-config.yaml` and `promtail-config.yaml` are downloaded in t

Navigate to http://localhost:3100/metrics to view the metrics and http://localhost:3100/ready for readiness.

As of v1.5.0, image is configured to run by default as user loki with UID `10001` and GID `10001`. You can use a different user, specially if you are using bind mounts, by specifying uid with docker run command
by specifying `--user=UID` with numeric UID suited to your needs.
As of v1.5.0, image is configured to run by default as user loki with UID `10001` and GID `10001`. You can use a different user, specially if you are using bind mounts, by specifying the UID with a `docker run` command and using `--user=UID` with numeric UID suited to your needs.

**Windows**

Expand All @@ -46,7 +45,9 @@ Navigate to http://localhost:3100/metrics to view the output.

## Install with Docker Compose

Run the following commands in your command line. They work for Windows or Linux systems.

```bash
$ wget https://raw.githubusercontent.com/grafana/loki/v1.5.0/production/docker-compose.yaml -O docker-compose.yaml
$ docker-compose -f docker-compose.yaml up
wget https://raw.githubusercontent.com/grafana/loki/v1.5.0/production/docker-compose.yaml -O docker-compose.yaml
docker-compose -f docker-compose.yaml up
```
34 changes: 17 additions & 17 deletions docs/installation/helm.md
Original file line number Diff line number Diff line change
@@ -1,72 +1,73 @@
# Installing Loki with Helm
# Install Loki with Helm

## Prerequisites

Make sure you have Helm [installed](https://helm.sh/docs/using_helm/#installing-helm) and
[deployed](https://helm.sh/docs/using_helm/#installing-tiller) to your cluster. Then add
[Loki's chart repository](https://github.com/grafana/loki/tree/master/production/helm/loki) to Helm:
[deployed](https://helm.sh/docs/using_helm/#installing-tiller) to your cluster.

Add [Loki's chart repository](https://github.com/grafana/loki/tree/master/production/helm/loki) to Helm:

```bash
$ helm repo add loki https://grafana.github.io/loki/charts
helm repo add loki https://grafana.github.io/loki/charts
```

You can update the chart repository by running:
To update the chart repository, run:

```bash
$ helm repo update
helm repo update
```

## Deploy Loki to your cluster

### Deploy with default config

```bash
$ helm upgrade --install loki loki/loki-stack
helm upgrade --install loki loki/loki-stack
```

### Deploy in a custom namespace

```bash
$ helm upgrade --install loki --namespace=loki loki/loki
helm upgrade --install loki --namespace=loki loki/loki
```

### Deploy with custom config

```bash
$ helm upgrade --install loki loki/loki --set "key1=val1,key2=val2,..."
helm upgrade --install loki loki/loki --set "key1=val1,key2=val2,..."
```

### Deploy Loki Stack (Loki, Promtail, Grafana, Prometheus)

```bash
$ helm upgrade --install loki loki/loki-stack --set grafana.enabled=true,prometheus.enabled=true,prometheus.alertmanager.persistentVolume.enabled=false,prometheus.server.persistentVolume.enabled=false
helm upgrade --install loki loki/loki-stack --set grafana.enabled=true,prometheus.enabled=true,prometheus.alertmanager.persistentVolume.enabled=false,prometheus.server.persistentVolume.enabled=false
```

### Deploy Loki Stack (Loki, Fluent Bit, Grafana, Prometheus)

```bash
$ helm upgrade --install loki loki/loki-stack \
--set fluent-bit.enabled=true,promtail.enabled=false,grafana.enabled=true,prometheus.enabled=true,prometheus.alertmanager.persistentVolume.enabled=false,prometheus.server.persistentVolume.enabled=false
helm upgrade --install loki loki/loki-stack \
--set fluent-bit.enabled=true,promtail.enabled=false,grafana.enabled=true,prometheus.enabled=true,prometheus.alertmanager.persistentVolume.enabled=false,prometheus.server.persistentVolume.enabled=false
```

## Deploy Grafana to your cluster

To install Grafana on your cluster with Helm, use the following command:

```bash
$ helm install stable/grafana -n loki-grafana
helm install stable/grafana -n loki-grafana
```

To get the admin password for the Grafana pod, run the following command:

```bash
$ kubectl get secret --namespace <YOUR-NAMESPACE> loki-grafana -o jsonpath="{.data.admin-password}" | base64 --decode ; echo
kubectl get secret --namespace <YOUR-NAMESPACE> loki-grafana -o jsonpath="{.data.admin-password}" | base64 --decode ; echo
```

To access the Grafana UI, run the following command:

```bash
$ kubectl port-forward --namespace <YOUR-NAMESPACE> service/loki-grafana 3000:80
kubectl port-forward --namespace <YOUR-NAMESPACE> service/loki-grafana 3000:80
```

Navigate to `http://localhost:3000` and login with `admin` and the password
Expand All @@ -79,8 +80,7 @@ If Loki and Promtail are deployed on different clusters you can add an Ingress
in front of Loki. By adding a certificate you create an HTTPS endpoint. For
extra security you can also enable Basic Authentication on the Ingress.

In Promtail, set the following values to communicate using HTTPS and basic
authentication:
In Promtail, set the following values to communicate using HTTPS and basic authentication:

```yaml
loki:
Expand Down
26 changes: 26 additions & 0 deletions docs/installation/install-from-source.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,26 @@
# Build from source

In order to build Loki manually, you need to clone the GitHub repo and then `make Loki`.

## Prerequisites

- Go 1.13 or later
- Make
- Docker (for updating protobuf files and yacc files)

## Build manually on your local system

Clone Loki to `$GOPATH/src/github.com/grafana/loki`:

```bash
git clone https://github.com/grafana/loki $GOPATH/src/github.com/grafana/loki
```

Then change into that directory and run `make loki`:

```bash
cd $GOPATH/src/github.com/grafana/loki
make loki
```

A file at ./cmd/loki/loki will be created and is the final built binary.
Loading