Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

How to send tags into multi_value field? #20

Open
ghost opened this issue May 30, 2018 · 0 comments
Open

How to send tags into multi_value field? #20

ghost opened this issue May 30, 2018 · 0 comments
Assignees

Comments

@ghost
Copy link

ghost commented May 30, 2018

Hello,

I'm trying to send the event field [tags] to zabbix using "multi_value" field.

Here's my scenario:

If something wrong happens in my logstash config, like date time parser error, etc, I want to send the tags to zabbix so later on I can check the problem and resend manually the event to be processed.

So in my output, I have a zabbix plugin being called ONLY if tags is found, like this:

input {
    mqtt {
		topic           => "aaaa"
		host            => "${MQTT_HOST}"
		username        => "${MQTT_USERNAME}"
		password        => "${MQTT_PASSWORD}"
		client_id       => "bbbb"
		clean_session   => false
		qos             => 1
  	}
}

filter {
    mutate {        
        copy => {
            "message"                   => "[@metadata][original_message]"
        }
    }

    mutate {
        add_field => { 
            "[@metadata][zabbix_message]"    => "message"
            "[@metadata][zabbix_error]"      => "error"
            "[@metadata][zabbix_type]"       => "type"
            "[@metadata][extra]"       => "extra"
            "[@metadata][zabbix_host]"       => "my_host"
        }
    }

	json {
	    source    => "message"
	}       
    }
}

output {
    if [tags] {
            # call zabbix!
            zabbix {
                id                  => "ping-default-error"
                zabbix_server_host  => "${ZABBIX_SERVER_HOST}"
                zabbix_server_port  => "${ZABBIX_SERVER_PORT}"
                zabbix_host         => "[@metadata][zabbix_host]"
                multi_value         => ["[@metadata][zabbix_message]", "[@metadata][original_message]", "[@metadata][zabbix_error]", "tags", "[@metadata][zabbix_type]", "[@metadata][extra]"]
            }   
    }
}

Here's my environment:

  • Version: logstash 6.2.4
  • Operating System: MacOSX Sierra
  • Sample Data: "test"
  • Steps to Reproduce: send a non-JSON data to the MQTT QUEUE.

If I send a non-JSON data, logstash warns me about the error in the stdout, send data to zabbix but the field that I called "error" is not published, see the output:

[2018-05-30T10:58:27,165][INFO ][logstash.agent           ] Pipelines running {:count=>1, :pipelines=>["main"]}
[2018-05-30T10:58:40,317][WARN ][logstash.filters.json    ] Error parsing json {:source=>"message", :raw=>"batatinha quando nasce esparrama pelo chaoxxxxx", :exception=>#<LogStash::Json::ParserError: Unrecognized token 'batatinha': was expecting ('true', 'false' or 'null')
 at [Source: (byte[])"batatinha quando nasce esparrama pelo chaoxxxxx"; line: 1, column: 11]>}
{
    "@timestamp" => 2018-05-30T13:58:40.177Z,
      "@version" => "1",
          "host" => "casmeiron-macbook.local",
         "topic" => "/teste4",
       "message" => "batatinha quando nasce esparrama pelo chaoxxxxx",
          "tags" => [
        [0] "_jsonparsefailure"
    ]
}
[2018-05-30T10:58:40,739][WARN ][logstash.outputs.zabbix  ] Zabbix server at <server> rejected 1 item(s). {:zabbix_host=>"my_host"}

If the field "[@metadata][zabbix_error]" its removed, no error from zabbix output plugin is printed:


[2018-05-30T11:01:15,018][WARN ][logstash.filters.json    ] Error parsing json {:source=>"message", :raw=>"batatinha quando nasce esparrama pelo chaoxxxxx", :exception=>#<LogStash::Json::ParserError: Unrecognized token 'batatinha': was expecting ('true', 'false' or 'null')
 at [Source: (byte[])"batatinha quando nasce esparrama pelo chaoxxxxx"; line: 1, column: 11]>}
{
    "@timestamp" => 2018-05-30T14:01:14.906Z,
      "@version" => "1",
          "host" => "casmeiron-macbook.local",
         "topic" => "/teste4",
       "message" => "batatinha quando nasce esparrama pelo chaoxxxxx",
          "tags" => [
        [0] "_jsonparsefailure"
    ]
}

And I can see in the zabbix (in both case) the values that are published.

Bottom of line, how can I submit the information that's stored inside [tags] field to zabbix?

Thanks in advance!

@jsvd jsvd self-assigned this Jun 5, 2018
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant