Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update debug-cluster and crictl files to get them consistent with the… #13183

Merged
merged 1 commit into from
Mar 19, 2019
Merged

Update debug-cluster and crictl files to get them consistent with the… #13183

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion content/en/docs/tasks/debug-application-cluster/_index.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
---
title: "Monitor, Log, and Debug"
title: "Monitoring, Logging, and Debugging"
weight: 80
---

95 changes: 61 additions & 34 deletions content/en/docs/tasks/debug-application-cluster/crictl.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -51,10 +51,9 @@ You can also specify timeout values when connecting to the server and enable or
disable debugging, by specifying `timeout` or `debug` values in the configuration
file or using the `--timeout` and `--debug` command-line flags.

To view or edit the current configuration, view or edit the contents of
`/etc/crictl.yaml`.
To view or edit the current configuration, view or edit the contents of `/etc/crictl.yaml`.

```sh
```shell
cat /etc/crictl.yaml
runtime-endpoint: unix:///var/run/dockershim.sock
image-endpoint: unix:///var/run/dockershim.sock
Expand All @@ -76,10 +75,12 @@ general purpose workflow tool, but a tool that is useful for debugging.

List all pods:

```bash
```shell
crictl pods
```
```none
The output is similar to this:

```
POD ID CREATED STATE NAME NAMESPACE ATTEMPT
926f1b5a1d33a About a minute ago Ready sh-84d7dcf559-4r2gq default 0
4dccb216c4adb About a minute ago Ready nginx-65899c769f-wv2gp default 0
Expand All @@ -89,20 +90,24 @@ a86316e96fa89 17 hours ago Ready kube-proxy-gblk4

List pods by name:

```bash
```shell
crictl pods --name nginx-65899c769f-wv2gp
```
```none
The output is similar to this:

```
POD ID CREATED STATE NAME NAMESPACE ATTEMPT
4dccb216c4adb 2 minutes ago Ready nginx-65899c769f-wv2gp default 0
```

List pods by label:

```bash
```shell
crictl pods --label run=nginx
```
```none
The output is similar to this:

```
POD ID CREATED STATE NAME NAMESPACE ATTEMPT
4dccb216c4adb 2 minutes ago Ready nginx-65899c769f-wv2gp default 0
```
Expand All @@ -111,10 +116,12 @@ POD ID CREATED STATE NAME

List all images:

```bash
```shell
crictl images
```
```none
The output is similar to this:

```
IMAGE TAG IMAGE ID SIZE
busybox latest 8c811b4aec35f 1.15MB
k8s-gcrio.azureedge.net/hyperkube-amd64 v1.10.3 e179bbfe5d238 665MB
Expand All @@ -124,20 +131,24 @@ nginx latest cd5239a0906a6

List images by repository:

```bash
```shell
crictl images nginx
```
```none
The output is similar to this:

```
IMAGE TAG IMAGE ID SIZE
nginx latest cd5239a0906a6 109MB
```

Only list image IDs:

```bash
```shell
crictl images -q
```
```none
The output is similar to this:

```
sha256:8c811b4aec35f259572d0f79207bc0678df4c736eeec50bc9fec37ed936a472a
sha256:e179bbfe5d238de6069f3b03fccbecc3fb4f2019af741bfff1233c4d7b2970c5
sha256:da86e6ba6ca197bf6bc5e9d900febd906b133eaa4750e6bed647b0fbe50ed43e
Expand All @@ -148,10 +159,12 @@ sha256:cd5239a0906a6ccf0562354852fae04bc5b52d72a2aff9a871ddb6bd57553569

List all containers:

```bash
```shell
crictl ps -a
```
```none
The output is similar to this:

```
CONTAINER ID IMAGE CREATED STATE NAME ATTEMPT
1f73f2d81bf98 busybox@sha256:141c253bc4c3fd0a201d32dc1f493bcf3fff003b6df416dea4f41046e0f37d47 7 minutes ago Running sh 1
9c5951df22c78 busybox@sha256:141c253bc4c3fd0a201d32dc1f493bcf3fff003b6df416dea4f41046e0f37d47 8 minutes ago Exited sh 0
Expand All @@ -161,10 +174,12 @@ CONTAINER ID IMAGE

List running containers:

```bash
```
crictl ps
```
```none
The output is similar to this:

```
CONTAINER ID IMAGE CREATED STATE NAME ATTEMPT
1f73f2d81bf98 busybox@sha256:141c253bc4c3fd0a201d32dc1f493bcf3fff003b6df416dea4f41046e0f37d47 6 minutes ago Running sh 1
87d3992f84f74 nginx@sha256:d0a8828cccb73397acb0073bf34f4d7d8aa315263f1e7806bf8c55d8ac139d5f 7 minutes ago Running nginx 0
Expand All @@ -173,32 +188,38 @@ CONTAINER ID IMAGE

### Execute a command in a running container

```bash
```shell
crictl exec -i -t 1f73f2d81bf98 ls
```
```none
The output is similar to this:

```
bin dev etc home proc root sys tmp usr var
```

### Get a container's logs

Get all container logs:

```bash
```shell
crictl logs 87d3992f84f74
```
```none
The output is similar to this:

```
10.240.0.96 - - [06/Jun/2018:02:45:49 +0000] "GET / HTTP/1.1" 200 612 "-" "curl/7.47.0" "-"
10.240.0.96 - - [06/Jun/2018:02:45:50 +0000] "GET / HTTP/1.1" 200 612 "-" "curl/7.47.0" "-"
10.240.0.96 - - [06/Jun/2018:02:45:51 +0000] "GET / HTTP/1.1" 200 612 "-" "curl/7.47.0" "-"
```

Get only the latest `N` lines of logs:

```bash
```shell
crictl logs --tail=1 87d3992f84f74
```
```none
The output is similar to this:

```
10.240.0.96 - - [06/Jun/2018:02:45:51 +0000] "GET / HTTP/1.1" 200 612 "-" "curl/7.47.0" "-"
```

Expand Down Expand Up @@ -226,7 +247,7 @@ deleted by the Kubelet.

2. Use the `crictl runp` command to apply the JSON and run the sandbox.

```bash
```shell
crictl runp pod-config.json
```

Expand All @@ -240,7 +261,7 @@ deleted by the Kubelet.

1. Pull a busybox image

```bash
```shell
crictl pull busybox
Image is up to date for busybox@sha256:141c253bc4c3fd0a201d32dc1f493bcf3fff003b6df416dea4f41046e0f37d47
```
Expand Down Expand Up @@ -284,17 +305,19 @@ deleted by the Kubelet.
container config file, and the pod config file. The ID of the container is
returned.

```bash
```shell
crictl create f84dd361f8dc51518ed291fbadd6db537b0496536c1d2d6c05ff943ce8c9a54f container-config.json pod-config.json
```

4. List all containers and verify that the newly-created container has its
state set to `Created`.

```bash
```shell
crictl ps -a
```
```none
The output is similar to this:

```
CONTAINER ID IMAGE CREATED STATE NAME ATTEMPT
3e025dd50a72d busybox 32 seconds ago Created busybox 0
```
Expand All @@ -303,19 +326,23 @@ deleted by the Kubelet.

To start a container, pass its ID to `crictl start`:

```bash
```shell
crictl start 3e025dd50a72d956c4f14881fbb5b1080c9275674e95fb67f965f6478a957d60
```
```none
The output is similar to this:

```
3e025dd50a72d956c4f14881fbb5b1080c9275674e95fb67f965f6478a957d60
```

Check the container has its state set to `Running`.

```bash
```shell
crictl ps
```
```none
The output is similar to this:

```
CONTAINER ID IMAGE CREATED STATE NAME ATTEMPT
3e025dd50a72d busybox About a minute ago Running busybox 0
```
Expand Down
16 changes: 8 additions & 8 deletions content/en/docs/tasks/debug-application-cluster/debug-cluster.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -36,28 +36,28 @@ of the relevant log files. (note that on systemd-based systems, you may need to

### Master

* /var/log/kube-apiserver.log - API Server, responsible for serving the API
* /var/log/kube-scheduler.log - Scheduler, responsible for making scheduling decisions
* /var/log/kube-controller-manager.log - Controller that manages replication controllers
* `/var/log/kube-apiserver.log` - API Server, responsible for serving the API
* `/var/log/kube-scheduler.log` - Scheduler, responsible for making scheduling decisions
* `/var/log/kube-controller-manager.log` - Controller that manages replication controllers

### Worker Nodes

* /var/log/kubelet.log - Kubelet, responsible for running containers on the node
* /var/log/kube-proxy.log - Kube Proxy, responsible for service load balancing
* `/var/log/kubelet.log` - Kubelet, responsible for running containers on the node
* `/var/log/kube-proxy.log` - Kube Proxy, responsible for service load balancing

## A general overview of cluster failure modes

This is an incomplete list of things that could go wrong, and how to adjust your cluster setup to mitigate the problems.

Root causes:
### Root causes:

- VM(s) shutdown
- Network partition within cluster, or between cluster and users
- Crashes in Kubernetes software
- Data loss or unavailability of persistent storage (e.g. GCE PD or AWS EBS volume)
- Operator error, e.g. misconfigured Kubernetes software or application software

Specific scenarios:
### Specific scenarios:

- Apiserver VM shutdown or apiserver crashing
- Results
Expand Down Expand Up @@ -91,7 +91,7 @@ Specific scenarios:
- users unable to read API
- etc.

Mitigations:
### Mitigations:

- Action: Use IaaS provider's automatic VM restarting feature for IaaS VMs
- Mitigates: Apiserver VM shutdown or apiserver crashing
Expand Down