Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

out_azure: Add support for the x-ms-AzureResourceId Header #2487

Closed
wants to merge 1 commit into from
Closed

out_azure: Add support for the x-ms-AzureResourceId Header #2487

wants to merge 1 commit into from

Conversation

floriankoch
Copy link

@floriankoch floriankoch commented Aug 23, 2020 •
edited
Loading

Signed-off-by: Florian Koch [email protected]

Add support for the x-ms-AzureResourceId Header

#2184

Testing
Before we can approve your change; please submit the following in a comment:

  • Example configuration file for the change
[INPUT]
    Name mem
    Tag  mem

[OUTPUT]
    Name        azure
    Match       *
    Customer_ID xxxxxxxxxxxx 
    Shared_Key xxxxxxxxxxx
    resource_id /subscriptions/xxxxxx/resourceGroups/test
  • Debug log output from testing the change
Fluent Bit v1.6.0
* Copyright (C) 2019-2020 The Fluent Bit Authors
* Copyright (C) 2015-2018 Treasure Data
* Fluent Bit is a CNCF sub-project under the umbrella of Fluentd
* https://fluentbit.io

[2020/08/26 15:36:39] [ info] Configuration:
[2020/08/26 15:36:39] [ info]  flush time     | 1.000000 seconds
[2020/08/26 15:36:39] [ info]  grace          | 5 seconds
[2020/08/26 15:36:39] [ info]  daemon         | 0
[2020/08/26 15:36:39] [ info] ___________
[2020/08/26 15:36:39] [ info]  inputs:
[2020/08/26 15:36:39] [ info]      mem
[2020/08/26 15:36:39] [ info] ___________
[2020/08/26 15:36:39] [ info]  filters:
[2020/08/26 15:36:39] [ info] ___________
[2020/08/26 15:36:39] [ info]  outputs:
[2020/08/26 15:36:39] [ info]      azure.0
[2020/08/26 15:36:39] [ info] ___________
[2020/08/26 15:36:39] [ info]  collectors:
[2020/08/26 15:36:39] [ info] [engine] started (pid=8636)
[2020/08/26 15:36:39] [debug] [engine] coroutine stack size: 24576 bytes (24.0K)
[2020/08/26 15:36:39] [debug] [storage] [cio stream] new stream registered: mem.0
[2020/08/26 15:36:39] [ info] [storage] version=1.0.5, initializing...
[2020/08/26 15:36:39] [ info] [storage] in-memory
[2020/08/26 15:36:39] [ info] [storage] normal synchronization mode, checksum disabled, max_chunks_up=128
[2020/08/26 15:36:39] [ info] [output:azure:azure.0] customer_id='xxxxxx' host='xxxxxx.ods.opinsights.azure.com:443'
[2020/08/26 15:36:39] [debug] [router] match rule mem.0:azure.0
[2020/08/26 15:36:39] [ info] [sp] stream processor started
[2020/08/26 15:36:39] [trace] [input:mem:mem.0 at plugins/in_mem/mem.c:254] memory total=6782084 kb, used=4099800 kb, free=2682284 kb
[2020/08/26 15:36:39] [trace] [input:mem:mem.0 at plugins/in_mem/mem.c:256] swap total=0 kb, used=0 kb, free=0 kb
[2020/08/26 15:36:40] [debug] [task] created task=0x5af46b3820e0 id=0 OK
[2020/08/26 15:36:40] [debug] [output:azure:azure.0] resource_id=/resource/subscriptions/xxxxxx/resourceGroups/test
[2020/08/26 15:36:40] [ info] [output:azure:azure.0] customer_id=xxxxxx, HTTP status=200
[2020/08/26 15:36:40] [debug] [upstream] KA connection #19 to xxxxxx.ods.opinsights.azure.com:443 is now available
  • Attached Valgrind output that shows no leaks or memory corruption was found
sudo valgrind --leak-check=full build/bin/fluent-bit -c ../../azure_test/azure.conf 
==9820== Memcheck, a memory error detector
==9820== Copyright (C) 2002-2017, and GNU GPL'd, by Julian Seward et al.
==9820== Using Valgrind-3.14.0 and LibVEX; rerun with -h for copyright info
==9820== Command: build/bin/fluent-bit -c ../../azure_test/azure.conf
==9820== 
Fluent Bit v1.6.0
* Copyright (C) 2019-2020 The Fluent Bit Authors
* Copyright (C) 2015-2018 Treasure Data
* Fluent Bit is a CNCF sub-project under the umbrella of Fluentd
* https://fluentbit.io

[2020/08/26 19:14:49] [ info] Configuration:
[2020/08/26 19:14:49] [ info]  flush time     | 5.000000 seconds
[2020/08/26 19:14:49] [ info]  grace          | 5 seconds
[2020/08/26 19:14:49] [ info]  daemon         | 0
[2020/08/26 19:14:49] [ info] ___________
[2020/08/26 19:14:49] [ info]  inputs:
[2020/08/26 19:14:49] [ info]      mem
[2020/08/26 19:14:49] [ info] ___________
[2020/08/26 19:14:49] [ info]  filters:
[2020/08/26 19:14:49] [ info] ___________
[2020/08/26 19:14:49] [ info]  outputs:
[2020/08/26 19:14:49] [ info]      azure.0
[2020/08/26 19:14:49] [ info] ___________
[2020/08/26 19:14:49] [ info]  collectors:
[2020/08/26 19:14:49] [ info] [engine] started (pid=9820)
[2020/08/26 19:14:49] [debug] [engine] coroutine stack size: 24576 bytes (24.0K)
[2020/08/26 19:14:49] [debug] [storage] [cio stream] new stream registered: mem.0
[2020/08/26 19:14:49] [ info] [storage] version=1.0.5, initializing...
[2020/08/26 19:14:49] [ info] [storage] in-memory
[2020/08/26 19:14:49] [ info] [storage] normal synchronization mode, checksum disabled, max_chunks_up=128
[2020/08/26 19:14:51] [ info] [output:azure:azure.0] customer_id='xxxxxx' host='xxxxxx.ods.opinsights.azure.com:443'
[2020/08/26 19:14:51] [debug] [router] match rule mem.0:azure.0
[2020/08/26 19:14:51] [ info] [sp] stream processor started
==9820== Warning: client switching stacks?  SP change: 0x1fff000118 --> 0x4fcfd80
==9820==          to suppress, use: --max-stackframe=137338487704 or greater
==9820== Warning: client switching stacks?  SP change: 0x4fcfcf8 --> 0x1fff000118
==9820==          to suppress, use: --max-stackframe=137338487840 or greater
==9820== Warning: client switching stacks?  SP change: 0x1fff000118 --> 0x4fcfcf8
==9820==          to suppress, use: --max-stackframe=137338487840 or greater
==9820==          further instances of this message will not be shown.
[2020/08/26 19:14:55] [debug] [task] created task=0x4fc9960 id=0 OK
[2020/08/26 19:14:57] [debug] [output:azure:azure.0] resource_id=/resource/subscriptions/xxxxxx/resourceGroups/test
[2020/08/26 19:14:58] [ info] [output:azure:azure.0] customer_id=xxxxxx, HTTP status=200
[2020/08/26 19:14:58] [debug] [upstream] KA connection #19 to xxxxxx.ods.opinsights.azure.com:443 is now available
[2020/08/26 19:14:58] [debug] [task] destroy task=0x4fc9960 (task_id=0)
[2020/08/26 19:15:00] [debug] [task] created task=0x5c89030 id=0 OK
[2020/08/26 19:15:00] [debug] [upstream] KA connection #19 is in a failed state to: xxxxxx.ods.opinsights.azure.com:443, cleaning up
[2020/08/26 19:15:01] [debug] [output:azure:azure.0] resource_id=/resource/subscriptions/xxxxxx/resourceGroups/test
[2020/08/26 19:15:02] [ info] [output:azure:azure.0] customer_id=xxxxxx, HTTP status=200
[2020/08/26 19:15:02] [debug] [upstream] KA connection #19 to xxxxxx.ods.opinsights.azure.com:443 is now available
[2020/08/26 19:15:02] [debug] [task] destroy task=0x5c89030 (task_id=0)
^C[engine] caught signal (SIGINT)
==9820== 
==9820== HEAP SUMMARY:
==9820==     in use at exit: 0 bytes in 0 blocks
==9820==   total heap usage: 38,587 allocs, 38,587 frees, 6,667,380 bytes allocated
==9820== 
==9820== All heap blocks were freed -- no leaks are possible
==9820== 
==9820== For counts of detected and suppressed errors, rerun with: -v
==9820== ERROR SUMMARY: 0 errors from 0 contexts (suppressed: 0 from 0)

Documentation

Fluent Bit is licensed under Apache 2.0, by submitting this pull request I understand that this code will be released under the terms of that license.

@floriankoch
Copy link
Author

I have no Idea why the AppVeyor build failed - seems not related to this change

@floriankoch
Copy link
Author

I Close this PR, i submit a better version

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@edsiper edsiper Awaiting requested review from edsiper edsiper is a code owner

@fujimotos fujimotos Awaiting requested review from fujimotos fujimotos is a code owner

@koleini koleini Awaiting requested review from koleini koleini is a code owner

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
1 participant
@floriankoch