Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

tracing-syslog crate #1137

Closed
wants to merge 20 commits into from
Closed

tracing-syslog crate #1137

wants to merge 20 commits into from

Conversation

max-heller
Copy link

Adds a tracing-syslog crate to provide support for logging to syslog via tracing without resorting to enabling tracing/log and using a crate like Geal/rust-syslog

Motivation

It would be great to be able to log directly to syslog with tracing. Users have wanted to do so in the past.
Fixes #796

Solution

Adds a tracing-syslog crate implementing a simple logger that writes to syslog using libc::syslog.

Copy link
Member

@davidbarsky davidbarsky left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for opening this PR!

tracing-syslog/src/syslog.rs Outdated Show resolved Hide resolved
tracing-syslog/src/syslog.rs Outdated Show resolved Hide resolved
tracing-syslog/src/syslog.rs Show resolved Hide resolved
Comment on lines +30 to +35
impl std::ops::BitOr for Options {
type Output = Self;
fn bitor(self, rhs: Self) -> Self::Output {
Self(self.0 | rhs.0)
}
}
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hmm, not sure if this carries its weight. Why do you think this is valuable?

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So you can set multiple options at the same time:

use tracing_syslog::Options;
// Log PID with messages and log to stderr as well as `syslog`.
let opts = Options::LOG_PID | Options::LOG_PERROR;

(added this example to Option's docs)

tracing-syslog/src/syslog.rs Outdated Show resolved Hide resolved
Comment on lines +175 to +177
fn drop(&mut self) {
unsafe { libc::closelog() };
}
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why's this safe? What invarients need to be upheld? If you can, can you add comments explaining why?

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I haven't decided yet how to best handle the case where multiple Syslogs are created (and thus openlog() is called multiple times). Recently created loggers will clobber the identity, facility, and options of previously created loggers, but what's worse is that when one is dropped and calls closelog() it will destroy the identity, facility, and options of the other ones. For these reasons, I think it might be best to have an AtomicBool or similar to keep track of whether or not a logger is currently initialized, and panic if a user tries to initialize multiple concurrently.

Comment on lines +1 to +4
#[cfg(unix)]
mod syslog;
#[cfg(unix)]
pub use syslog::*;
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It'd be great to have doc comments with examples and tests here.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've never been sure about this: for a crate that has a primary type (like Syslog in this case), should the majority of the docs go with that type or in lib.rs?

tracing-syslog/src/syslog.rs Outdated Show resolved Hide resolved
Copy link
Member

@hawkw hawkw left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is a great start. I had some thoughts on how we might rework this to allow users to customize the formatting for events written to syslog. Let me know what you think?

tracing-syslog/src/syslog.rs Outdated Show resolved Hide resolved
tracing-syslog/src/syslog.rs Outdated Show resolved Hide resolved
tracing-syslog/src/syslog.rs Outdated Show resolved Hide resolved
tracing-syslog/src/syslog.rs Outdated Show resolved Hide resolved
@max-heller
Copy link
Author

Is there a way to include rustdoc aliases that won't break compilation on pre-1.48 toolchains?

@hawkw
Copy link
Member

hawkw commented Dec 14, 2020

Is there a way to include rustdoc aliases that won't break compilation on pre-1.48 toolchains?

Yes, we currently use several RustDoc features that are only available on nightly. You can do it like this:

#[cfg_attr(docsrs, doc(alias = "..."))]

and then add something like this to Cargo.toml:

[package.metadata.docs.rs]
all-features = true
rustdoc-args = ["--cfg", "docsrs"]

@max-heller
Copy link
Author

Is panicking when the log contents contain an interior nul-terminator the right decision or should we skip logging the message and instead log a warning about it? If we just had static logging messages I'd be okay with panicking, but given that they include dynamic content I wouldn't want to crash someone's program because it was passed a value with a weird debug representation.

@hawkw
Copy link
Member

hawkw commented Dec 14, 2020

Is panicking when the log contents contain an interior nul-terminator the right decision or should we skip logging the message and instead log a warning about it? If we just had static logging messages I'd be okay with panicking, but given that they include dynamic content I wouldn't want to crash someone's program because it was passed a value with a weird debug representation.

We should probably just eprintln! a warning, or something. We could maybe have a debug_assert! so that issues are caught in tests but we don't crash someone's program in production?

davidbarsky pushed a commit that referenced this pull request May 14, 2021
subscriber: add `MakeWriter::make_writer_for`

## Motivation

In some cases, it might be desirable to configure the writer used for
writing out trace data based on the metadata of the span or event being
written. For example, we might want to send different levels to
different outputs, write logs from different targets to separate files,
or wrap formatted output in ANSI color codes based on levels. Currently,
it's not possible for the `MakeWriter` trait to model this kind of
behavior --- it has one method, `make_writer`, which is completely
unaware of *where* the data being written came from.

In particular, this came up in PR #1137, when discussing a proposal that
writing to syslog could be implemented as a `MakeWriter` implementation
rather than as a `Subscribe` implementation, so that all the formatting
logic from `tracing_subscriber::fmt` could be reused. See [here][1] for
details.

## Solution

This branch adds a new `make_writer_for` method to `MakeWriter`, taking
a `Metadata`. Implementations can opt in to metadata-specific behavior
by implementing this method. The method has a default implementation
that just calls `self.make_writer()` and ignores the metadata, so it's
only necessary to implement this when per-metadata behavior is required.
This isn't a breaking change to existing implementations.

There are a couple downsides to this approach: it's possible for callers
to skip the metadata-specific behavior by calling `make_writer` rather
than `make_writer_for`, and the impls for closures can't easily provide
metadata-specific behavior.

Since the upcoming release is going to be a breaking change anyway, we
may want to just make the breaking change of having
`MakeWriter::make_writer` _always_ take a `Metadata`, which solves these
problems. However, that can't be backported to v0.1.x as easily. Additionally,
that would mean that functions like `io::stdout` no longer implement 
`MakeWriter`; they would have to be wrapped in a wrapper type or closure
that ignores metadata.

[1]: #1137 (comment)

Signed-off-by: Eliza Weisman <[email protected]>
hawkw added a commit that referenced this pull request Jun 25, 2021
This backports PR #1141 from `master`.

subscriber: add `MakeWriter::make_writer_for`

## Motivation

In some cases, it might be desirable to configure the writer used for
writing out trace data based on the metadata of the span or event being
written. For example, we might want to send different levels to
different outputs, write logs from different targets to separate files,
or wrap formatted output in ANSI color codes based on levels. Currently,
it's not possible for the `MakeWriter` trait to model this kind of
behavior --- it has one method, `make_writer`, which is completely
unaware of *where* the data being written came from.

In particular, this came up in PR #1137, when discussing a proposal that
writing to syslog could be implemented as a `MakeWriter` implementation
rather than as a `Subscribe` implementation, so that all the formatting
logic from `tracing_subscriber::fmt` could be reused. See [here][1] for
details.

## Solution

This branch adds a new `make_writer_for` method to `MakeWriter`, taking
a `Metadata`. Implementations can opt in to metadata-specific behavior
by implementing this method. The method has a default implementation
that just calls `self.make_writer()` and ignores the metadata, so it's
only necessary to implement this when per-metadata behavior is required.
This isn't a breaking change to existing implementations.

There are a couple downsides to this approach: it's possible for callers
to skip the metadata-specific behavior by calling `make_writer` rather
than `make_writer_for`, and the impls for closures can't easily provide
metadata-specific behavior.

Since the upcoming release is going to be a breaking change anyway, we
may want to just make the breaking change of having
`MakeWriter::make_writer` _always_ take a `Metadata`, which solves these
problems. However, that can't be backported to v0.1.x as easily. Additionally,
that would mean that functions like `io::stdout` no longer implement
`MakeWriter`; they would have to be wrapped in a wrapper type or closure
that ignores metadata.

[1]: #1137 (comment)

Signed-off-by: Eliza Weisman <[email protected]>
hawkw added a commit that referenced this pull request Jun 26, 2021
This backports PR #1141 from `master`.

subscriber: add `MakeWriter::make_writer_for`

## Motivation

In some cases, it might be desirable to configure the writer used for
writing out trace data based on the metadata of the span or event being
written. For example, we might want to send different levels to
different outputs, write logs from different targets to separate files,
or wrap formatted output in ANSI color codes based on levels. Currently,
it's not possible for the `MakeWriter` trait to model this kind of
behavior --- it has one method, `make_writer`, which is completely
unaware of *where* the data being written came from.

In particular, this came up in PR #1137, when discussing a proposal that
writing to syslog could be implemented as a `MakeWriter` implementation
rather than as a `Subscribe` implementation, so that all the formatting
logic from `tracing_subscriber::fmt` could be reused. See [here][1] for
details.

## Solution

This branch adds a new `make_writer_for` method to `MakeWriter`, taking
a `Metadata`. Implementations can opt in to metadata-specific behavior
by implementing this method. The method has a default implementation
that just calls `self.make_writer()` and ignores the metadata, so it's
only necessary to implement this when per-metadata behavior is required.
This isn't a breaking change to existing implementations.

There are a couple downsides to this approach: it's possible for callers
to skip the metadata-specific behavior by calling `make_writer` rather
than `make_writer_for`, and the impls for closures can't easily provide
metadata-specific behavior.

Since the upcoming release is going to be a breaking change anyway, we
may want to just make the breaking change of having
`MakeWriter::make_writer` _always_ take a `Metadata`, which solves these
problems. However, that can't be backported to v0.1.x as easily. Additionally,
that would mean that functions like `io::stdout` no longer implement
`MakeWriter`; they would have to be wrapped in a wrapper type or closure
that ignores metadata.

[1]: #1137 (comment)

Signed-off-by: Eliza Weisman <[email protected]>
@janimo
Copy link

janimo commented Sep 17, 2021

is switching to using make_writer_for the way forward with this PR?

@max-heller
Copy link
Author

is switching to using make_writer_for the way forward with this PR?

Thanks for reminding me @janimo. Yes, make_writer_for is the way forward -- I just pushed a commit updating the crate to use it.

@janimo
Copy link

janimo commented Oct 21, 2021

@max-heller @hawkw does this need more changes?

@nappa85
Copy link

nappa85 commented Dec 19, 2021

Hello there,
are there any news about this PR?
I would really love it to be officially released, are there blocking points I can help with?

@topenkoff
Copy link

Hi everyone! This PR is in draft for a long time. Does this need more changes? Can I help with some blocks, which doesn't allow release crate officially?
@max-heller @hawkw @davidbarsky

@davidbarsky
Copy link
Member

Hi everyone! This PR is in draft for a long time. Does this need more changes? Can I help with some blocks, which doesn't allow release crate officially?

I think I speak for Eliza when I say this:tracing-syslog would be useful, but the tracing monorepo is getting a bit unwieldy and impacting our ability to maintain tracing itself. Therefore, I don't think we should accept this PR adding tracing-syslog, but if @max-heller is interested in publishing it in a standalone repo, we'd be happy link to it as a related crate.

@davidbarsky
Copy link
Member

I will also mention an important detail: we're looking at removing crates from the tracing monorepo. We'd be okay with creating a dedicated repositories for those crates under the Tokio organization if the original author of that crate doesn't want to be sole maintainer.

In the case of tracing-syslog: we're not rejecting the code nor are we saying that it shouldn't exist. We're primarily concerned that changes to tracing-core or tracing-subscriber build lots of unrelated changes and this has resulted in too much work on the CI system itself.

@spacekookie
Copy link

Is there any updates on this? My project could really use tracing-syslog. Currently it looks like we'll have to vendor the code provided by this PR?

spacekookie added a commit to irdest/irdest that referenced this pull request Apr 7, 2022
This is a work-in-progress crate that has not yet been published to
crates.io.  The source was taken from
tokio-rs/tracing#1137

Future updates should remove this crate, once we can pull it from
somewhere else.
@hawkw
Copy link
Member

hawkw commented Apr 8, 2022

@spacekookie

Is there any updates on this? My project could really use tracing-syslog. Currently it looks like we'll have to vendor the code provided by this PR?

I would really like to have a syslog crate in the tracing ecosystem. I'd prefer it to be in a separate repository (which could be within the Tokio organization, to indicate that it's officially maintained) --- the long term plan is to move existing crates such as tracing-journald into their own repositories as well. However, I don't personally have the capacity to be responsible for maintaining these crates, especially when I didn't write any of the original code. If @max-heller (or anyone else interested in this crate) is willing to share responsibility for its continued maintenance with the tracing team, I'm happy to move this code to a new repository in the tokio-rs github organization.

spacekookie added a commit to irdest/irdest that referenced this pull request Apr 10, 2022
This is a work-in-progress crate that has not yet been published to
crates.io.  The source was taken from
tokio-rs/tracing#1137

Future updates should remove this crate, once we can pull it from
somewhere else.
@max-heller
Copy link
Author

Sorry I've been unresponsive, I've been busy with school and other things and didn't get the chance to test any of this. I got the chance to test it a bit and it seems to work at least for simple use cases, so I've moved the code to max-heller/tracing-syslog. If those that were interested could try it out and let me know if it works for your use-cases, I can publish it to crates.io. Going forward, I'd be willing to help share responsibility for maintenance as @hawkw suggests, so once it's in a usable state we could move it to the tokio-rs org.

@spacekookie @topenkoff @nappa85 @janimo @lu-zero

@max-heller
Copy link
Author

Finally published the crate as syslog-tracing. I was trying to get ownership of the name-squat tracing-syslog but couldn't get in contact with the owner.

kaffarell pushed a commit to kaffarell/tracing that referenced this pull request May 22, 2024
This backports PR tokio-rs#1141 from `master`.

subscriber: add `MakeWriter::make_writer_for`

## Motivation

In some cases, it might be desirable to configure the writer used for
writing out trace data based on the metadata of the span or event being
written. For example, we might want to send different levels to
different outputs, write logs from different targets to separate files,
or wrap formatted output in ANSI color codes based on levels. Currently,
it's not possible for the `MakeWriter` trait to model this kind of
behavior --- it has one method, `make_writer`, which is completely
unaware of *where* the data being written came from.

In particular, this came up in PR tokio-rs#1137, when discussing a proposal that
writing to syslog could be implemented as a `MakeWriter` implementation
rather than as a `Subscribe` implementation, so that all the formatting
logic from `tracing_subscriber::fmt` could be reused. See [here][1] for
details.

## Solution

This branch adds a new `make_writer_for` method to `MakeWriter`, taking
a `Metadata`. Implementations can opt in to metadata-specific behavior
by implementing this method. The method has a default implementation
that just calls `self.make_writer()` and ignores the metadata, so it's
only necessary to implement this when per-metadata behavior is required.
This isn't a breaking change to existing implementations.

There are a couple downsides to this approach: it's possible for callers
to skip the metadata-specific behavior by calling `make_writer` rather
than `make_writer_for`, and the impls for closures can't easily provide
metadata-specific behavior.

Since the upcoming release is going to be a breaking change anyway, we
may want to just make the breaking change of having
`MakeWriter::make_writer` _always_ take a `Metadata`, which solves these
problems. However, that can't be backported to v0.1.x as easily. Additionally,
that would mean that functions like `io::stdout` no longer implement
`MakeWriter`; they would have to be wrapped in a wrapper type or closure
that ignores metadata.

[1]: tokio-rs#1137 (comment)

Signed-off-by: Eliza Weisman <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

dedicated tracing-syslog crate
7 participants