Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

chore: update Fluent Bit status in readme #10510

Merged
merged 1 commit into from
Dec 19, 2021
Merged

chore: update Fluent Bit status in readme #10510

merged 1 commit into from
Dec 19, 2021

Conversation

l2dy
Copy link
Contributor

@l2dy l2dy commented Dec 19, 2021

Fluent Bit 1.7.0 has introduced multi-threading support. See fluent/fluent-bit#1567.

Fluent Bit 1.7.0 has introduced multi-threading support.
@bits-bot
Copy link

bits-bot commented Dec 19, 2021

CLA assistant check
All committers have signed the CLA.

@netlify
Copy link

netlify bot commented Dec 19, 2021

✔️ Deploy Preview for vector-project ready!

🔨 Explore the source changes: ae9fea9

🔍 Inspect the deploy log: https://app.netlify.com/sites/vector-project/deploys/61bf128d249236000858eceb

😎 Browse the preview: https://deploy-preview-10510--vector-project.netlify.app

@binarylogic binarylogic merged commit f1404be into vectordotdev:master Dec 19, 2021
@l2dy l2dy deleted the chore/readme-1 branch December 20, 2021 02:02
@binarylogic
Copy link
Contributor

binarylogic commented Dec 21, 2021

@l2dy I took some time to look into Fluentbit's multi-threading support because I was curious. I had a difficult time finding the actual change that introduced this. I then realized this was an opt-in setting that must be perfectly balanced across all outputs to saturate all cores. Again (#10542), this does not embody the essence of the feature matrix, which is that the tool will vertically scale practically to enable the aggregator use case.

This is a great example of why we built Vector! 😄

@l2dy
Copy link
Contributor Author

l2dy commented Dec 22, 2021

Meanings of each feature is unclear to me though. For example:

⚠ = Not interoperable, metrics are represented as structured logs

What software to interoperate with if only Vector is "interoperable"?

metrics are represented as structured logs

No longer true for Fluent Bit since v1.8, see https://fluentbit.io/announcements/v1.8.1/.

For a long time our community have asked for native metrics support. Despite Fluent Bit has metrics collectors for CPU, Disk I/O, Network, Memory and others the data payload was handled as a simple structured log, that is called ‘logs as metrics’. Now we are introducing ‘native metrics’.

@l2dy
Copy link
Contributor Author

l2dy commented Dec 22, 2021

I then realized this was an [opt-in setting that must be perfectly balanced across all outputs to saturate all cores]

Fluentd requires similar configuration: flush_thread_count and Multi Process Workers. In this case, both are not as good as Vector is.

jdrouet pushed a commit that referenced this pull request Dec 27, 2021
Fluent Bit 1.7.0 has introduced multi-threading support.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants