Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merge master into 6.x #5170

Closed
wants to merge 11 commits into from
Closed

Merge master into 6.x #5170

wants to merge 11 commits into from

Conversation

ruflin
Copy link
Member

@ruflin ruflin commented Sep 13, 2017

Please do merge this branch and not squash the commits.

Steffen Siering and others added 11 commits September 11, 2017 20:39
* [Docs] Add option to log messages in JSON

* Add logging.files.permissions to full reference yml file

Edit description of file permissions
* Reorder processors pipeline order

- reorder processors, such that all client processors run before the
  global processors
- remove support for EventsMetadataKey

* Combine fields and tags if possible

- combine client internal field updates and configured field updates
  into one dictionary
- combine proessors for adding client and global fields+tags if no
  client processors are configured

* Do not copy if no processors are defined

Do not Clone (deep copy) fields being added in the pipeline if no
processors are configured.

As processors might add/remove fields and potentially modify shared
field objects, these must be copied if there is a chance global shared
structured being overwritten by processors. Especially if processors are
guarded by conditions.

* Fix processors order once again

Problem is local fields+tags must be applied after the globaly
configured fields and tests, while client processors must be run before
the global ones.

* review

- use 'disabled' flag
- simplify pipeline fields init
* memqueue cleanup

Move the buffer and message handling from memqueue broker type
to the actual go routines handling the messaging.

* Add specialized buffers to memqueue

Use specialized buffer types and eventloops depending on the memqueue
configuration.

If flushing is disabled, a region based ring buffer is used, as loads of
small batches can be generated and we want to minimize additional
allocations of small objects.

If flushing is enabled a list of active and flushed buffers is managed
by the queue. If a buffer is flushed by timeout, but not yet processed
(and not full), additional events being published are added to the
already flushed buffer.

* ensure flush list does not contain empty buffers
This command can be used in the beats to be run after `make update` is executed. This allows for example to modify the libbeat specific parts of the config file if needed.

The command is optional in a beat, meaning if it does not exist it won't return an error. Also if it exist and it returns an error, make update will still skip the error.
* Add @metadata.version to json events

- pass beat.Info into the codecs
- update json encoder to include @metadata.version (for Console, File, Logstash, Redis, Kafka)

* Update Ls docs to use the version field

* Fix test build

* s/second/third/

* Fix metricbeat unit tests
@ruflin ruflin added review in progress Pull request is currently in progress. labels Sep 13, 2017
@ruflin
Copy link
Member Author

ruflin commented Sep 13, 2017

As soon as #5171 is in will update this PR again.

@tsg
Copy link
Contributor

tsg commented Sep 13, 2017

@ruflin Just FYI, we've got a ./dev-tools/merge_pr script that you can use to create these merging PRs.

@ruflin
Copy link
Member Author

ruflin commented Sep 13, 2017

@tsg Forgot about the script. I think I should also use this in apm-server.

@ruflin
Copy link
Member Author

ruflin commented Sep 13, 2017

Closing this PR and will use the script for a "proper" merge.

@ruflin ruflin closed this Sep 13, 2017
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
in progress Pull request is currently in progress. review
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants