Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

libbeat monitoring cgroup metrics are all zero under Docker #22844

Closed
axw opened this issue Dec 2, 2020 · 3 comments · Fixed by #22879
Closed

libbeat monitoring cgroup metrics are all zero under Docker #22844

axw opened this issue Dec 2, 2020 · 3 comments · Fixed by #22879
Assignees
@axw
Labels
Team:Services (Deprecated) Label for the former Integrations-Services team v7.11.0

Comments

@axw
Copy link
Member

axw commented Dec 2, 2020

Running a beat (in this case, apm-server) under Docker with monitoring enabled, the beat records cgroup-related metrics but they are all zero.

POST .monitoring-beats-*/_search?filter_path=hits.hits._source.beats_stats.metrics.beat.cgroup
{
  "size": 10,
  "sort": [
    {
      "timestamp": {
        "order": "desc"
      }
    }
  ]
}

{
  "hits" : {
    "hits" : [
      {
        "_source" : {
          "beats_stats" : {
            "metrics" : {
              "beat" : {
                "cgroup" : {
                  "memory" : {
                    "id" : "4aa0ae74b29140844b04752ac5423b1509e10237217be68dd5ca9f11a7e6ed30",
                    "mem" : {
                      "limit" : {
                        "bytes" : 0
                      },
                      "usage" : {
                        "bytes" : 0
                      }
                    }
                  },
                  "cpu" : {
                    "stats" : {
                      "throttled" : {
                        "ns" : 0,
                        "periods" : 0
                      },
                      "periods" : 0
                    },
                    "id" : "4aa0ae74b29140844b04752ac5423b1509e10237217be68dd5ca9f11a7e6ed30",
                    "cfs" : {
                      "period" : {
                        "us" : 0
                      },
                      "quota" : {
                        "us" : 0
                      }
                    }
                  },
                  "cpuacct" : {
                    "id" : "4aa0ae74b29140844b04752ac5423b1509e10237217be68dd5ca9f11a7e6ed30",
                    "total" : {
                      "ns" : 0
                    }
                  }
                }
              }
            }
          }
        }
      },

I think this is the same issue that ES fixed in elastic/elasticsearch#22757
@axw axw added the v7.11.0 label Dec 2, 2020
@axw axw self-assigned this Dec 2, 2020
@botelastic botelastic bot added the needs_team Indicates that the issue/PR needs a Team:* label label Dec 2, 2020
This was referenced Dec 2, 2020
Closed
Merged
@andresrc andresrc added the Team:Services (Deprecated) Label for the former Integrations-Services team label Dec 2, 2020
@elasticmachine
Copy link
Collaborator

Pinging @elastic/integrations-services (Team:Services)

@botelastic botelastic bot removed the needs_team Indicates that the issue/PR needs a Team:* label label Dec 2, 2020
@andresrc
Copy link
Contributor

andresrc commented Dec 2, 2020

ping @fearful-symmetry

@axw axw mentioned this issue Dec 3, 2020
Merged
4 tasks
axw added a commit to axw/beats that referenced this issue Dec 4, 2020
@axw
libbeat: override /sys/fs/cgroup paths in Docker
2ef1aa8 
When running in Docker, make sure metrics are read
from `/sys/fs/cgroup/<subsystem>`, ignoring any paths
in `/proc/self/cgroup`.

Fixes elastic#22844
@axw axw mentioned this issue Dec 7, 2020
Closed
axw added a commit that referenced this issue Dec 10, 2020
@axw
libbeat: override /sys/fs/cgroup paths in Docker (#22879)
7e1520f 
* libbeat: override /sys/fs/cgroup paths in Docker

When running in Docker, make sure metrics are read
from `/sys/fs/cgroup/<subsystem>`, ignoring any paths
in `/proc/self/cgroup`.

Fixes #22844
@axw axw mentioned this issue Dec 10, 2020
Merged
4 tasks
axw added a commit to axw/beats that referenced this issue Dec 10, 2020
@axw
libbeat: override /sys/fs/cgroup paths in Docker (elastic#22879)
38751ce 
* libbeat: override /sys/fs/cgroup paths in Docker

When running in Docker, make sure metrics are read
from `/sys/fs/cgroup/<subsystem>`, ignoring any paths
in `/proc/self/cgroup`.

Fixes elastic#22844

(cherry picked from commit 7e1520f)
axw added a commit that referenced this issue Dec 16, 2020
@axw
libbeat: override /sys/fs/cgroup paths in Docker (#22879) (#23050)
190be26 
* libbeat: override /sys/fs/cgroup paths in Docker

When running in Docker, make sure metrics are read
from `/sys/fs/cgroup/<subsystem>`, ignoring any paths
in `/proc/self/cgroup`.

Fixes #22844

(cherry picked from commit 7e1520f)
@simitt
Copy link
Contributor

simitt commented Jan 21, 2021

Tested with BC 4 on cloud staging and linux outside of docker. Values were populated and reported.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@axw axw

Labels
Team:Services (Deprecated) Label for the former Integrations-Services team v7.11.0
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

libbeat: override /sys/fs/cgroup paths in Docker axw/beats
4 participants
@andresrc @axw @simitt @elasticmachine