Contribution of Stanza logging agent to OpenTelemetry

[djaglowski]

observIQ proposes to contribute our Stanza logging agent to the OpenTelemetry project, with the intention of merging relevant components into OpenTelemetry's Collector.

Benefits

Stanza is written in Go, so it can be integrated directly into the collector. This would greatly accelerate implementation of the logging specification.

Stanza's operation-oriented architecture would allow for the rapid creation of a variety of useful log receivers which would support log ingestion via File Tailing, Windows Event Log, Journald, Syslog, and Kubernetes. These receivers could also share parsing packages to support various standard formats and values such as JSON, Regex, Timestamps, and Severity.

Logistics

We propose that Stanza is moved into the OpenTelemetry organization as a standalone repository and maintained in such a way that its packages are available to the OpenTelemetry Collector. Eventually, Stanza's capabilities would be more or less redundant with the OpenTelemetry Collector, and Stanza could then be deprecated.

Stanza is currently licensed under Apache 2.0. This could remain the case until the repository is deprecated.

The following people would initially be maintainers on the repository:

• @djaglowski
• @andykellr
• @jsirianni
• @camdencheek (recently left observIQ but is interested in staying on as a maintainer)

Several members of the OpenTelemetry community would be welcomed as maintainers, according to interest and familiarity.

[bhs]

Thank you for raising this, @djaglowski. It also gives us an opportunity to reference the brand new (#594) TC expectations for stuff like this... namely, a 2w turnaround on an initial response to the proposal.

cc @open-telemetry/technical-committee

[tigrannajaryan]

On behalf of TC:

• We are reviewing the proposal.
• TC invites @open-telemetry/specs-approvers and OpenTelemetry community to review and comment on this proposal.

On behalf of myself and @pmm-sumo (Przemek, please feel free to correct me if I missed anything in the summary):
I and @pmm-sumo did a technical due diligence. Summary report here: https://docs.google.com/document/d/1kTMyauz8dDM6mzbFDgU_vDh5Se4svAv-YrVJg1W18ao/edit#

[genebean]

As a current user of OpenTelemetry, this sounds like a good idea to me. The evaluation linked outlines some things that still need to be done, but I agree that this would accelerate things greatly. I'm also excited to see the logging side of things progress generally.

[tedsuo]

This is fabulous @djaglowski, I'm glad this is finally happening. :)

[tigrannajaryan]

The Technical Committee has reviewed the proposal and is glad to approve. We appreciate observeIQ’s contribution of Stanza to OpenTelemetry. It will significantly accelerate the support of logs in the Collector.

The next steps:

1. We will create a new repository under http://github.com/open-telemetry org. @djaglowski @andykellr @jsirianni @camdencheek will become seed maintainers from observeIQ, @tigrannajaryan @bogdandrutu will become seed maintainers from OpenTelemetry.

2. observIQ will commit the existing source code to the new repository. The submission should be a single commit and should carry the regular OpenTelemetry copyright notice. We recommend adding a reference to the original Stanza repository to the README file.

3. Maintainers will work on making the new repository a dependency of OpenTelemetry Collector instead of the current dependency on the Stanza agent that resides in observIQ’s GitHub org.

4. We work towards adding more log collection capabilities in the Collector, stabilize it and make generally available. The TC encourages all interested parties to be involved in this process, contribute individual PRs or become active contributors, approvers or maintainers of the Collector and of the Log specification.

@mtwo @kumoroku @pmm-sumo @zenmoto thank you for helping with reviewing and facilitating the contribution.

[yurishkuro]

🎉 🥳

[gramidt]

This is exciting!!! 🥳

[bhs]

Nice! And thank you, observeIQ folks!! This is a wonderful development for the overall ecosystem (including and beyond Stanza and/or OTel per se).

[djaglowski]

Thank you @tigrannajaryan, @kumoroku, @mtwo, @pmm-sumo, @zenmoto, and everyone else who's been involved.

We look forward to moving through this process as rapidly as possible.

[tigrannajaryan]

Repository created https://github.com/open-telemetry/opentelemetry-log-collection

@lizthegrey, @bogdandrutu Please enable EasyCLA on the repo.

[tigrannajaryan]

@djaglowski @andykellr @jsirianni @camdencheek you are invited as Maintainers to the repo.

[tigrannajaryan]

@djaglowski I temporarily gave you Admin rights on the repo to configure any necessary settings. I already configured the main branch according to https://github.com/open-telemetry/community/blob/master/docs/how-to-configure-new-repository.md#how-we-configure-repositories

Once the initial settings are in place the Admin permissions will be revoked, you will be a Maintainer.

[tigrannajaryan]

@lizthegrey, @bogdandrutu Please enable EasyCLA on https://github.com/open-telemetry/opentelemetry-log-collection repo.

[djaglowski]

Is it possible to add one more maintainer?

@jmwilliams89 is one of the original authors of Stanza and is interested in helping out going forward.

[tigrannajaryan]

Closing, all initial setup is now done.