Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add documentation for Stackdriver event exporter #4155

Merged
merged 1 commit into from
Jun 26, 2017
Merged

Add documentation for Stackdriver event exporter #4155

merged 1 commit into from
Jun 26, 2017

Conversation

crassirostris
Copy link

@crassirostris crassirostris commented Jun 21, 2017

/cc @piosz @fgrzadkowski


This change is Reviewable

@k8s-ci-robot k8s-ci-robot added the cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. label Jun 21, 2017
@crassirostris
Copy link
Author

@chenopis Could you please take a look?

@janetkuo janetkuo added this to the 1.7 milestone Jun 21, 2017
Copy link
Contributor

@chenopis chenopis left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Some suggested edits. Also, the images need to be sized down.

you can find them by selecting an appropriate option from a drop-down menu
of available resources:

![Events location in the Stackdriver Logging interface](/images/docs/stackdriver-event-exporter-resource.png)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You're going to need to adjust the size of the images. You can try using the =WIDTHxHEIGHT syntax. They are showing up very large, see the page preview at: https://deploy-preview-4155--kubernetes-io-master-staging.netlify.com/docs/tasks/debug-application-cluster/events-stackdriver/

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This syntax doesn't work with github pages :(

Replaced with raw html

title: Events Using Stackdriver
---

Kubernetes events are objects that provide an insight into what is happening
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

suggested edit: "Kubernetes events are objects that provide insight into what is happening inside a cluster, such as what decisions were made by scheduler or why some pods were evicted from the node."

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done

inside a cluster, for example, what decisions were made by scheduler or why
some pods were evicted from the node. You can read more about using events
for debugging your application in the [Application Introspection and Debugging
](docs/tasks/debug-application-cluster/debug-application-introspection)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

add leading and trailing slashes in the link URL:
[Application Introspection and Debugging](/docs/tasks/debug-application-cluster/debug-application-introspection/)

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done


Since events are API objects, they are stored in the apiserver on master. To
avoid filling up master's disk, a retention policy is enforced: events are
removed after one hour after the last occurence. To provide longer history
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

unnecessary "after" and spelling typo: "...events are removed one hour after the last occurrence."

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done

Since events are API objects, they are stored in the apiserver on master. To
avoid filling up master's disk, a retention policy is enforced: events are
removed after one hour after the last occurence. To provide longer history
and aggregation capabilities, a third party solution should be installed,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

suggested edit: "To provide longer history and aggregation capabilities, a third party solution should be installed to capture events which may need to be exported in the future."

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The idea was "install 3rd party solution and export events there for future use", not "install 3rd party solution, capture events there and then exported them from there". Changed to a third party solution should be installed to capture events


In Google Container Engine event exporter is deployed by default to the
clusters with master in version 1.7 and higher, if cloud logging is enabled.
Note, that in order to prevent disturbing your workloads, event exporter
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

recommended edit: "To prevent disturbing your workloads, event exporter does not have resources set and is in the best effort QOS class, which means that it will be the first to be killed in the case of resource starvation."

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done

doesn't have resources set and is in the best effort QOS class, which
effectively means that it will be the first to kill in case of resource
starvation. If you want your events to be exported, make sure you have
enough resources to facilitate the event exporter pod (depends on the
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

suggested edit: "If you want your events to be exported, make sure you have enough resources to facilitate the event exporter pod. This may vary depending on the workload, but on average, approximately 100Mb RAM and 100m CPU is needed."

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done

kubectl create -f https://k8s.io/docs/tasks/debug-application-cluster/event-exporter-deploy.yaml
```

Since event exporter accesses Kubernetes API, it requres permissions to
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

add "the" and fix typo: "Since event exporter accesses the Kubernetes API, it requires..."

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done


## User Guide

Events are exported to the `GKE Cluster` resource in Stackdriver Logging,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Break into two sentences: "Events are exported to the GKE Cluster resource in Stackdriver Logging. You can find them by selecting an appropriate option from a drop-down menu of available resources:"

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done


You can filter based on the event object fields using Stackdriver Logging
[filtering mechanism](https://cloud.google.com/logging/docs/view/advanced_filters).
For example, the following query will show events from scheduler
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

add "the": "...show events from the scheduler..."

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done

Copy link
Author

@crassirostris crassirostris left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks a lot for the review! PTAL

Since events are API objects, they are stored in the apiserver on master. To
avoid filling up master's disk, a retention policy is enforced: events are
removed after one hour after the last occurence. To provide longer history
and aggregation capabilities, a third party solution should be installed,
Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The idea was "install 3rd party solution and export events there for future use", not "install 3rd party solution, capture events there and then exported them from there". Changed to a third party solution should be installed to capture events


Since events are API objects, they are stored in the apiserver on master. To
avoid filling up master's disk, a retention policy is enforced: events are
removed after one hour after the last occurence. To provide longer history
Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done

inside a cluster, for example, what decisions were made by scheduler or why
some pods were evicted from the node. You can read more about using events
for debugging your application in the [Application Introspection and Debugging
](docs/tasks/debug-application-cluster/debug-application-introspection)
Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done

title: Events Using Stackdriver
---

Kubernetes events are objects that provide an insight into what is happening
Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done

to where events would be exported for future use.

This article describes a solution that exports Kubernetes events to
Stackdriver Logging, where they can be made useful.
Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done

doesn't have resources set and is in the best effort QOS class, which
effectively means that it will be the first to kill in case of resource
starvation. If you want your events to be exported, make sure you have
enough resources to facilitate the event exporter pod (depends on the
Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done

kubectl create -f https://k8s.io/docs/tasks/debug-application-cluster/event-exporter-deploy.yaml
```

Since event exporter accesses Kubernetes API, it requres permissions to
Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done


## User Guide

Events are exported to the `GKE Cluster` resource in Stackdriver Logging,
Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done


You can filter based on the event object fields using Stackdriver Logging
[filtering mechanism](https://cloud.google.com/logging/docs/view/advanced_filters).
For example, the following query will show events from scheduler
Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done

you can find them by selecting an appropriate option from a drop-down menu
of available resources:

![Events location in the Stackdriver Logging interface](/images/docs/stackdriver-event-exporter-resource.png)
Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This syntax doesn't work with github pages :(

Replaced with raw html

@chenopis
Copy link
Contributor

@piosz Can you do the tech review? Thx

Copy link
Contributor

@fgrzadkowski fgrzadkowski left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Very nice! Only one comment.

assignees:
- crassirostris
- piosz
title: Events Using Stackdriver
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

"Events in Stackdriver"?

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I wanted it to be consistent with the logging page, but I agree, "Events in Stackdriver" sounds better

This article describes a solution that exports Kubernetes events to
Stackdriver Logging, where they can be processed and analyzed.

*Note:* events are considered best-effort, so it's possible that some events
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this comment is rather pessimistic. We estimated we should have pretty high coverage. Maybe we can say that depending on every event exported is not recommended, but alerting based on crashlooping is probably ok.

WDYT?

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agree that it sounds more pessimistic than it actually is, I'll rephrase

@crassirostris
Copy link
Author

@chenopis @fgrzadkowski PTAL

@chenopis
Copy link
Contributor

/docs lgtm
Just waiting on tech lgtm.

Stackdriver Logging, where they can be processed and analyzed.

**Note:** it is not guaranteed that all events happening in a cluster will be
exported to Stackdriver. One example of events that are not exported are
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How about:

One possible scenario when events will not be exported is when event exporter is not running (e.g. during restart or upgrade).

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done

@crassirostris
Copy link
Author

@fgrzadkowski Thanks! PTAL

Also squashed commits

@fgrzadkowski
Copy link
Contributor

/lgtm

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
cncf-cla: yes Indicates the PR's author has signed the CNCF CLA.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

7 participants