Unable to restart agent when stopped

[calebhailey]

I have encountered a reproducible error that occurs when I stop and attempt to restart the collector, as follows:

2022-07-27T22:21:27.944Z        info    adapter/receiver.go:54  Starting stanza receiver        {"kind": "receiver", "name": "filelog", "pipeline": "logs"}                                                                          
Error: cannot start pipelines: start stanza: read known files from database: stat: invalid argument
2022/07/27 22:21:27 collector server run finished with error: cannot start pipelines: start stanza: read known files from database: stat: invalid argument

Here's my config.yaml file:

extensions:
  file_storage:
    directory: /var/lib/otelcol-sumo/file_storage
  sumologic:
    install_token: ${SUMO_OTC_INSTALL_TOKEN}
    collector_credentials_directory: /var/lib/otelcol-sumo
    collector_name: ${HOSTNAME}
    collector_category: mission-s3m
    collector_description: Sumo Logic OTC Distro Demo
    collector_fields:
      mission: s3m
    clobber: true
    ephemeral: true

receivers:
  filelog:
    include_file_name: false
    include_file_path_resolved: true
    start_at: end
    include:
      - /tmp/sumologic-otc-example.log

exporters:
  logging:
    loglevel: info
  sumologic:

service:
  extensions: [file_storage, sumologic]
  pipelines:
    logs:
      receivers: [filelog]
      exporters: [sumologic, logging]

For more information about my environment, please see this document, which is a slightly modified from the Sumo Logic OTC Distro installation and configuration guides.

[jspaleta]

More context, by deciphering the log message I've determined that the problem results from
the r.file.Stat() in offsetToEnd() from the stanza/fileconsumer pkg in the opentelemetry-collector-contrib repo.

Ref:

https://github.com/open-telemetry/opentelemetry-collector-contrib/blob/6321d0d76e91a312b409a32df4f4ebe6193cd290/pkg/stanza/fileconsumer/reader.go#L47

This is only called if the filelog start_at configuration is set to end
Ref:

https://github.com/open-telemetry/opentelemetry-collector-contrib/blob/6321d0d76e91a312b409a32df4f4ebe6193cd290/pkg/stanza/fileconsumer/reader_factory.go#L125

If you change the configuration to read as start_at: beginning the problematic function isn't hit and the collector will restart.

I can't get farther than this without actively hacking on the upstream repo.

[aboguszewski-sumo]

Thank you for the details, I've created a pull request in the upstream: open-telemetry/opentelemetry-collector-contrib#12767

[aboguszewski-sumo]

The pull request has been merged into the upstream. The change will be available in the next release.

[aboguszewski-sumo]

This has been fixed in opentelemetry-collector-contrib by open-telemetry/opentelemetry-collector-contrib#12767 and released in https://github.com/open-telemetry/opentelemetry-collector-contrib/releases/tag/v0.57.2
The fix has been included in Sumo Logic distribution by #699 and is available since the following release: https://github.com/SumoLogic/sumologic-otel-collector/releases/tag/v0.57.2-sumo-0