Fluentbit as a central log forwarder instead of Fluentd

[Ancitik]

Hi everyone,

Fluentbit having more and more plugins everyday is becoming more connected than Fluentd..

I'm wondering if it would be possible to use Fluentbit instead of Fluentd as a log forwarder?

For instance I'd like to forward some logs to Azure Log Analytics and it seems Fluentd does not have a maintained plugin for that whereas Fluentbit does: https://docs.fluentbit.io/manual/pipeline/outputs/azure

Regards.

[pepov]

Hey!

With the core features provided by fluentd/syslogng, which are mostly around flexible routing of log messages, it wouldn't be possible to implement a fluentbit based log forwarder at the moment.

However if we would consider log forwarders as a dumb forwarder that can do nothing but provide persistent buffering and scaling capabilities with a single output, than it would certainly be doable with a different abstraction that can handle such standalone "outputs", regardless of what is the implementation behind it (fluentbit/fluentd/syslogng/vector/whatever)

Now that is something that is a possible future development direction of the operator, but currently there is no active development on it.

[Ancitik]

I didn't know there was such a difference between both. Reading the Fluentbit documentation it appears there's not.

Both Fluentd and Fluentbit can work as Aggregators or Forwarders, they both can complement each other or use them as standalone solutions.
In the recent years, Cloud Providers switched from Fluentd to Fluent Bit for performance and compatibility reasons. Fluent Bit is now considered the next generation solution.

Source: https://docs.fluentbit.io/manual/about/fluentd-and-fluent-bit

[pepov]

The reason we could implement flexible routing of logs based on metadata (like kubernetes labels) was the introduction of fluentd labels and a special plugin we wrote: https://github.com/kube-logging/fluent-plugin-label-router

In fluentbit I'm not aware of a similar mechanism. You don't have anything but the tag based matching, which provides very little flexibility.