Incorrect Errors for Parent Processes That No Longer Exist

[Dylan-M]

Component(s)

receiver/hostmetrics

What happened?

Description

When monitoring processes using the hostmetrics receiver, the log gets filled with errors about parent processes that no longer exist.
Examples:
pid 0 on Linux
winlogon.exe on Windows

Steps to Reproduce

Enable all process/processes monitoring in a host metrics receiver. Check log for these errors.

Expected Result

Receiver understands that processes can fork and exit, thus no longer existing as a parent to the forked process.

Actual Result

Errors in the log about these processes that no longer exist.

Collector version

0.73.0

Environment information

Environment

OS: 5.10.0-21-cloud-amd64 #1 SMP Debian 5.10.162-1 (2023-01-21) x86_64 GNU/Linux
Compiler(if manually compiled): Not manually compiled

OpenTelemetry Collector configuration

receivers:
    hostmetrics:
        collection_interval: 60s
        scrapers:
            cpu:
                metrics:
                    system.cpu.utilization:
                        enabled: true
            disk:
                metrics: null
            filesystem:
                metrics:
                    system.filesystem.utilization:
                        enabled: true
            load:
                metrics: null
            memory:
                metrics:
                    system.memory.utilization:
                        enabled: true
            network:
                metrics:
                    system.network.conntrack.count:
                        enabled: true
                    system.network.conntrack.max:
                        enabled: true
            paging:
                metrics:
                    system.paging.utilization:
                        enabled: true
            process:
                metrics:
                    process.context_switches:
                        enabled: true
                    process.cpu.utilization:
                        enabled: true
                    process.disk.operations:
                        enabled: true
                    process.memory.utilization:
                        enabled: true
                    process.open_file_descriptors:
                        enabled: true
                    process.paging.faults:
                        enabled: true
                    process.signals_pending:
                        enabled: true
                    process.threads:
                        enabled: true
                mute_process_name_error: true
            processes:
processors:
exporters:
    googlecloud:
service:
    pipelines:
        metrics:
            receivers:
                - hostmetrics
            processors:
            exporters:
                - googlecloud

Log output

{"level":"error","ts":"2023-03-16T16:03:25.983Z","caller":"scraperhelper/scrapercontroller.go:212","msg":"Error scraping metrics","kind":"receiver","name":"hostmetrics/source1","data_type":"metrics","error":"failed to read conntrack info: open /proc/sys/net/netfilter/nf_conntrack_count: no such file or directory","scraper":"network","stacktrace":"go.opentelemetry.io/collector/receiver/scraperhelper.(*controller).scrapeMetricsAndReport\n\t/home/runner/go/pkg/mod/go.opentelemetry.io/collector/[email protected]/scraperhelper/scrapercontroller.go:212\ngo.opentelemetry.io/collector/receiver/scraperhelper.(*controller).startScraping.func1\n\t/home/runner/go/pkg/mod/go.opentelemetry.io/collector/[email protected]/scraperhelper/scrapercontroller.go:191"}
{"level":"error","ts":"2023-03-16T16:03:26.011Z","caller":"scraperhelper/scrapercontroller.go:212","msg":"Error scraping metrics","kind":"receiver","name":"hostmetrics/source1","data_type":"metrics","error":"error reading parent pid for process \"systemd\" (pid 1): invalid pid 0","scraper":"process","stacktrace":"go.opentelemetry.io/collector/receiver/scraperhelper.(*controller).scrapeMetricsAndReport\n\t/home/runner/go/pkg/mod/go.opentelemetry.io/collector/[email protected]/scraperhelper/scrapercontroller.go:212\ngo.opentelemetry.io/collector/receiver/scraperhelper.(*controller).startScraping.func1\n\t/home/runner/go/pkg/mod/go.opentelemetry.io/collector/[email protected]/scraperhelper/scrapercontroller.go:191"}

Additional context

No response

[github-actions]

Pinging code owners:

• receiver/hostmetrics: @dmitryax

See Adding Labels via Comments if you do not have permissions to add labels yourself.

[atoulme]

See #18923 for a possible duplicate.

pid 0 and pid 1 are used in linux and Mac OS. PID 0 refers to the kernel: https://www.baeldung.com/linux/process-pid-0#:~:text=What%20Process%20Has%20PID%200,and%20shutting%20down%20the%20system.

[atoulme]

See also #14311 as a possible duplicate.

[F21]

Also seeing this error.

Here's what I have in my logs:

Apr 17 05:27:19 system-1 otelcol[2714]: 2023-04-17T05:27:19.621Z        error        scraperhelper/scrapercontroller.go:212        Error scraping metrics        {"kind": "receiver", "name":
 "hostmetrics", "data_type": "metrics", "error": "error reading parent pid for process \"systemd\" (pid 1): invalid pid 0; error reading parent pid for process \"kthreadd\" (pid 2): invali
d pid 0", "scraper": "process"}
Apr 17 05:27:19 system-1 otelcol[2714]: go.opentelemetry.io/collector/receiver/scraperhelper.(*controller).scrapeMetricsAndReport
Apr 17 05:27:19 system-1 otelcol[2714]:         go.opentelemetry.io/collector/[email protected]/scraperhelper/scrapercontroller.go:212
Apr 17 05:27:19 system-1 otelcol[2714]: go.opentelemetry.io/collector/receiver/scraperhelper.(*controller).startScraping.func1
Apr 17 05:27:19 system-1 otelcol[2714]:         go.opentelemetry.io/collector/[email protected]/scraperhelper/scrapercontroller.go:191

In my config, I am muting all errors, but that doesn't seem to silence the error reading parent pid error:

mute_process_name_error: true
mute_process_exe_error: true
mute_process_io_error: true

[github-actions]

This issue has been inactive for 60 days. It will be closed in 60 days if there is no activity. To ping code owners by adding a component label, see Adding Labels via Comments, or if you are unsure of which component this issue relates to, please ping @open-telemetry/collector-contrib-triagers. If this issue is still relevant, please ping the code owners or leave a comment explaining why it is still relevant. Otherwise, please close it.

Pinging code owners:

• receiver/hostmetrics: @dmitryax

See Adding Labels via Comments if you do not have permissions to add labels yourself.

[github-actions]

This issue has been closed as inactive because it has been stale for 120 days with no activity.

[Dylan-M]

This is still an issue for us. It would be nice to see some action on it, since it is such a simple thing to change for these to be debug messages instead. Or even warnings. However, since this is expected behavior, my belief is that they should be debug.