Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add docker_logs datastream for docker container logs collection #4716

Merged
merged 6 commits into from
Nov 28, 2022

Conversation

ChrsMark
Copy link
Member

@ChrsMark ChrsMark commented Nov 24, 2022

What does this PR do?

Add docker_logs data_stream in docker package so as to support containers' logs collection using the docker dynamic provider.

Related issues

Screenshots

Pr1
Pr3
Pr2
PRScreenshot

How to test this PR locally

  1. Make sure that you have access to /var/run/docker.sock and /var/lib/docker/containers. If needed change the permissions for example with sudo chmod -R 777 /var/run/docker.sock && sudo chmod -R 777 /var/lib/docker/containers.
  2. Edit the docker-compose file that is used by elastic-package to bring the local stack up:
    In Elastic Agent section add the socket and the log dirs as volume mounts:
volumes:
    - "../certs/ca-cert.pem:/etc/ssl/certs/elastic-package.pem"
    - "/var/run/docker.sock:/var/run/docker.sock:ro"
    - "/var/lib/docker/containers:/var/lib/docker/containers:ro"
  1. Bring stack up with elastic-package stack up -d -v --version=8.5.0
  2. Add the docker package from fleet UI and make sure that logs from containers are being collected.

@ChrsMark ChrsMark added the Team:Cloudnative-Monitoring Label for the Cloud Native Monitoring team [elastic/obs-cloudnative-monitoring] label Nov 24, 2022
@ChrsMark ChrsMark requested a review from a team as a code owner November 24, 2022 10:44
@ChrsMark ChrsMark self-assigned this Nov 24, 2022
Signed-off-by: ChrsMark <[email protected]>
@elasticmachine
Copy link

elasticmachine commented Nov 24, 2022

💚 Build Succeeded

the below badges are clickable and redirect to their specific view in the CI or DOCS
Pipeline View Test View Changes Artifacts preview preview

Expand to view the summary

Build stats

  • Start Time: 2022-11-28T10:01:55.287+0000

  • Duration: 2 min 6 sec

🤖 GitHub comments

Expand to view the GitHub comments

To re-run your PR in the CI, just comment with:

  • /test : Re-trigger the build.

@elasticmachine
Copy link

elasticmachine commented Nov 24, 2022

🌐 Coverage report

Name Metrics % (covered/total) Diff
Packages 100.0% (0/0) 💚
Files 100.0% (0/0) 💚
Classes 100.0% (0/0) 💚
Methods 96.667% (29/30)
Lines 100.0% (0/0) 💚
Conditionals 100.0% (0/0) 💚

Signed-off-by: ChrsMark <[email protected]>
Signed-off-by: ChrsMark <[email protected]>
packages/docker/changelog.yml Outdated Show resolved Hide resolved
packages/docker/_dev/build/docs/README.md Outdated Show resolved Hide resolved
Signed-off-by: ChrsMark <[email protected]>
@ChrsMark ChrsMark requested review from gizas and gsantoro November 28, 2022 08:21
@ChrsMark
Copy link
Member Author

Changed the default path to /var/lib/docker/containers/${docker.container.id}/*-json.log from /var/lib/docker/containers/${docker.container.id}/${docker.container.id}-json.log to catch also possible rotated files.

@ChrsMark ChrsMark requested review from gsantoro and gizas November 28, 2022 09:40
@ChrsMark ChrsMark merged commit e6fb971 into elastic:main Nov 28, 2022
@elasticmachine
Copy link

Package docker - 2.3.0 containing this change is available at https://epr.elastic.co/search?package=docker

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Team:Cloudnative-Monitoring Label for the Cloud Native Monitoring team [elastic/obs-cloudnative-monitoring]
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants