Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Proposal] Add a filebeat-like elasticsearch docker logging plugin #12580

Closed
fearful-symmetry opened this issue Jun 17, 2019 · 1 comment
Closed
Assignees
Labels
enhancement new beat Team:Integrations Label for the Integrations team

Comments

@fearful-symmetry
Copy link
Contributor

fearful-symmetry commented Jun 17, 2019

The full proposal can be found here: https://docs.google.com/document/d/1gNH0SnasbcFuhoRf2ra6M8TAycD2shSIP7uKdcy78k0/edit#

The initial discuss issue is here: #10047. This is a longstanding request from the cloud team, and many competing logging solutions have docker log drivers available.

After talking extensively with @urso , I made a second proposal with his help: https://docs.google.com/document/d/1GIMbEuoHSLrl2MMVD_YF2gbZPSc5R72m2daQRTGSjz0

This proposal calls for a front end CLI utility, similar to functionbeat, that's responsible for controlling and configuring the docker plugin.

@fearful-symmetry fearful-symmetry self-assigned this Jun 17, 2019
@fearful-symmetry
Copy link
Contributor Author

Thinking more about the output/config issue. Since we still have global state, we could just spin up a new output/libbeat instance every time we get a different output config, as opposed to per-container or some hacked up global output. Something like:

configHash := hashsum(config["hostname"] + config["index"] + config["auth"])

if _, ok := libbeatHandlers[configHash]; !ok{
libbeatHandlers[configHash] = newHandler(config)
}

It's still a bit of overhead, but at least we can scale for the 90% case where everything is going to one cluster.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement new beat Team:Integrations Label for the Integrations team
Projects
None yet
Development

No branches or pull requests

2 participants