Skip to content

Commit

Permalink
out_splunk: add new property "channel" to set channel identifier (#3318)
Browse files Browse the repository at this point in the history 
If we use HEC includes raw events, X-Splunk-Request-Channel header must be needed.
  This new property is to set it.
  https://docs.splunk.com/Documentation/SplunkCloud/latest/Data/FormateventsforHTTPEventCollector#Channel_identifier_header

Signed-off-by: Takahiro Yamashita <[email protected]>
  • Loading branch information
@nokute78
nokute78 authored Apr 20, 2021
1 parent 6e9a5cc commit 23aaf15
Show file tree
Hide file tree
Showing 2 changed files with 19 additions and 0 deletions.
13 changes: 13 additions & 0 deletions plugins/out_splunk/splunk.c
View file
Original file line number Diff line number Diff line change
Expand Up @@ -230,6 +230,13 @@ static void cb_splunk_flush(const void *data, size_t bytes,
ctx->auth_header, flb_sds_len(ctx->auth_header));
}

/* Append Channel identifier header */
if (ctx->channel) {
flb_http_add_header(c, FLB_SPLUNK_CHANNEL_IDENTIFIER_HEADER,
strlen(FLB_SPLUNK_CHANNEL_IDENTIFIER_HEADER),
ctx->channel, ctx->channel_len);
}

/* Content Encoding: gzip */
if (compressed == FLB_TRUE) {
flb_http_set_content_encoding_gzip(c);
Expand Down Expand Up @@ -327,6 +334,12 @@ static struct flb_config_map config_map[] = {
"from the docs for more details to make this option work properly."
},

{
FLB_CONFIG_MAP_STR, "channel", NULL,
0, FLB_TRUE, offsetof(struct flb_splunk, channel),
"Specify X-Splunk-Request-Channel Header for the HTTP Event Collector interface."
},

/* EOF */
{0}
};
Expand Down
6 changes: 6 additions & 0 deletions plugins/out_splunk/splunk.h
View file
Original file line number Diff line number Diff line change
Expand Up @@ -27,6 +27,8 @@
#define FLB_SPLUNK_DEFAULT_TIME "time"
#define FLB_SPLUNK_DEFAULT_EVENT "event"

#define FLB_SPLUNK_CHANNEL_IDENTIFIER_HEADER "X-Splunk-Request-Channel"

#include <fluent-bit/flb_output_plugin.h>
#include <fluent-bit/flb_sds.h>

Expand All @@ -41,6 +43,10 @@ struct flb_splunk {
/* Token Auth */
flb_sds_t auth_header;

/* Channel identifier */
flb_sds_t channel;
size_t channel_len;

/* Send fields directly or pack data into "event" object */
int splunk_send_raw;

Expand Down

4 comments on commit 23aaf15

@sbellan
Copy link

@sbellan sbellan commented on 23aaf15 Jul 11, 2021 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi @nokute78 ,

I'm trying docker build 1.8.1 and I see that channel is one of the valid properties for splunk output,

[2021/07/10 23:31:19] [error] [config] splunk: unknown configuration property 'xxx'. The following properties are allowed: compress, http_user, http_passwd, http_buffer_size, event_key, event_host, event_source, event_sourcetype, event_sourcetype_key, event_index, event_index_key, event_field, splunk_token, splunk_send_raw, and channel.

But when using the splunk output like this,

[OUTPUT]
    Name            splunk
    Match           *
    Host            ${FLUENT_SPLUNK_HOST}
    Port            ${FLUENT_SPLUNK_PORT}
    Splunk_Token    ${FLUENT_SPLUNK_TOKEN}
    splunk_send_raw on
    TLS             On
    TLS.Verify      Off
    channel         FE0ECFAD-13D5-401B-847D-77833BD77131

I still see Splunk returning the error {"text":"Data channel is missing","code":10}

and when I enable trace output I get this,

[2021/07/11 00:37:19] [debug] [task] created task=0x7facc9437a00 id=0 OK
[2021/07/11 00:37:19] [debug] [out coro] cb_destroy coro_id=0
[2021/07/11 00:37:19] [debug] [http_client] not using http_proxy for header
[2021/07/11 00:37:19] [debug] [http_client] header=POST /services/collector/raw HTTP/1.1
Host: vpce-010fde1bf0ee5237e-i5gf1o62.vpce-svc-00398a51b5143f7b0.us-west-2.vpce.amazonaws.com:8088
Content-Length: 2408
User-Agent: Fluent-Bit
Authorization: Splunk XXX-XXX-XXX


[2021/07/11 00:37:19] [ warn] [output:splunk:splunk.1] http_status=400:
{"text":"Data channel is missing","code":10}

I dont see the X-Splunk-Request-Channel channel header being set.

@nokute78
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could you open an issue ?
It is hard to follow an comment for commit.

@nokute78
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@sbellan I found the root cause and I sent a patch #3760
Thank you for reporting issue.

@sbellan
Copy link

@sbellan sbellan commented on 23aaf15 Jul 11, 2021 via email

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please sign in to comment.