Add timezone to the exported event

[monicasarbu]

There are a few requests to export timezone in order to be able to find the local timestamp. For that, we think the best option would be to export also the timezone (optionally) in the event, and make sure the @timestamp is always sent in UTC.

TODOs:

• Create add_locale processor to add timezone in the event Add processor to set timezone for an event #3902
• Configure the format of the timezone in the add_locale processor. For example America/Curacao or CEST

[martinscholz83]

Hi @monicasarbu. What about a config option for example short_form: false? Or should there more options to format?

[monicasarbu]

There are 3 ways you can display timezone:

1. Using abbreviations, for example PDT
2. North America/Pacific Daylight Time
3. Using offset, for example: -7:00

I think we should have support for all three. In order to decide one or another, ideally would be to configure the timezone format, instead of having a boolean config option for each. I think the best would be to use the format that Logstash supports for timezone, but I didn't have the chance to play with it.

[martinscholz83]

In logstash you have define time as follows America/Curacao. See here. So i think option 1 and 2 are sufficient. We can make America/Curacao as the default output and then maybe add a config option short_form: true.

[martinscholz83]

Hi @monicasarbu. Do you had the chance to look at logstash and decide what format you would use?

[monicasarbu]

@maddin2016 I think the 2nd and 3rd options are more important, not sure about the 1st one. The 3rd option is nice as you can calculate the local time without the need to retrieve timezone information from external systems.

Instead of using a boolean option for the timezone format, I would say to use an option like:

format: "abbreviation" //1st option
format: "canonical_id" //2nd option
format: "offset"           // 3rd option

[ruflin]

+1 on the format config option. The reason I also like option 1 is because if makes filtering for all machines in CEST really simple. Format 2 is not "unique" to a time zone but to an area.

[martinscholz83]

But to work with logstash we need the second option. Why not using all three options?

[ruflin]

+1 on all three option. With the proposed config from monica above, this should be possible.

[martinscholz83]

Ok. Then i open a PR for this issue.

[sarfarazahmad89]

I would like to do all timezone processing on filebeat end itself and avoid using date plugin in logstash.
This leads me to two questions :

1. Can filebeat pick up the timezone directly from the OS ?
   If not I can automate this using a configuration management tool like Puppet, but if there were an auto mode that would be nice.
2. I don't understand how this will fit with date filter in logstash. What is the thought behind adding this ?
   Will this replace date plugin or compliment it ?

[andrewkroh]

Using the add_locale processor FIlebeat can add the timezone information to the event.

Then in Logstash or Ingest Node you can use the value of this field as the timezone argument to the date filter.

[andrewkroh]

Closing because this feature has been implemented.