-
Notifications
You must be signed in to change notification settings - Fork 1.6k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
How to read multiline json format #2418
Comments
This configuration works for us: custom_parser.conf
Input tail config:
Not sure if this is a best solution 😸 |
Working perfectly. Thanks alot. |
Thanks @simonasr this is where I was looking for, for a long time. Though it looks like a workaround to me, I'm glad the issue didn't end in a show stopper. To get a regular JSON log event (without wrapping all content in the field
|
This issue is stale because it has been open 30 days with no activity. Remove stale label or comment or this will be closed in 5 days. |
This issue was closed because it has been stalled for 5 days with no activity. |
Multiline UpdateAs part of Fluent Bit v1.8, we have released a new Multiline core functionality. This new big feature allows you to configure new For now, you can take at the following documentation resources:
Documentation pages now point to complete config examples that are available on our repository. Thanks everyone for supporting this! |
I'm trying to read below json format but reading line by line .
td-agent-bit version is 1.4.5
Log format :-
{
"thread" : "1",
"level" : "OFF",
"loggerName" : "AuditLogger",
"marker" : {
"name" : "Audit",
"parents" : [ {
"name" : "EVENT"
} ]
},
"message" : "Audit [useCredentials fileName="Credential id=apitoken name=paasjenkins/****** (apitoken-nonprod-jobs)" name="Credential id=apitoken name=paasjenkins/****** (apitoken-nonprod-jobs)" timestamp="2020-06-02T12:34:36.928+04:00" usage="[11,1864)"]",
"endOfBatch" : false,
"loggerFqcn" : "org.apache.logging.log4j.audit.AuditLogger",
"instant" : {
"epochSecond" : 1594986664,
"nanoOfSecond" : 204000000
},
"contextMap" : { },
"threadId" : 129467,
"threadPriority" : 5
}
{
"thread" : "2",
"level" : "OFF",
"loggerName" : "AuditLogger",
"marker" : {
"name" : "Audit",
"parents" : [ {
"name" : "EVENT"
} ]
},
"message" : "Audit [useCredentials fileName="Credential id=apitoken name=paasjenkins/****** (apitoken-nonprod-jobs)" name="Credential id=apitoken name=paasjenkins/****** (apitoken-nonprod-jobs)" timestamp="2020-06-02T12:34:36.928+04:00" usage="[11,1864)"]",
"endOfBatch" : false,
"loggerFqcn" : "org.apache.logging.log4j.audit.AuditLogger",
"instant" : {
"epochSecond" : 1594986664,
"nanoOfSecond" : 204000000
},
"contextMap" : { },
"threadId" : 129467,
"threadPriority" : 5
}
td-agent-bit.conf
[SERVICE]
Parsers_File /etc/td-agent-bit/parsers.conf
Daemon off
Flush 5
Log_Level trace
[INPUT]
Name tail
Path /var/log/jenkins/audit1.log
DB /etc/td-agent-bit/db/enbd-jenkins-audit.db
Path_Key tailed.path
Tag enbd-jenkins-audit
Parser_Firstline firstline2
Multiline On
Parser_1 multiline2
Refresh_Interval 5
Buffer_Chunk_Size 3M
Buffer_Max_Size 200M
Mem_Buf_Limit 300M
Skip_Long_Lines true
[OUTPUT]
Name stdout
Match enbd-jenkins-*
parsers.conf
[PARSER]
Name firstline2
Format regex
Regex ^(?{\n\s{1,}"thread"\s:)
[PARSER]
Name multiline2
Format regex
Regex (?m-ix)^(?{\n\s{1,}"thread".*)
I want to read whole thread-1 is one message and other one is another message.
The text was updated successfully, but these errors were encountered: