Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Limit number of open files #2236

Closed
ruflin opened this issue Aug 11, 2016 · 0 comments
Closed

Limit number of open files #2236

ruflin opened this issue Aug 11, 2016 · 0 comments
Labels
discuss Issue needs further discussion. Filebeat Filebeat

Comments

@ruflin
Copy link
Member

ruflin commented Aug 11, 2016

Having a large amount of files which need to be harvested by filebeat it can happen that the systems max number of open file handlers is exceeded. In most cases the problem can be solved by closing files faster or not harvesting some files. Still there are edge cases where this can be an issue.

This issue should help to track progress on this topic and discuss potential implementations.
@ruflin ruflin added discuss Issue needs further discussion. Filebeat Filebeat labels Aug 11, 2016
@ruflin ruflin mentioned this issue Aug 12, 2016
Closed
ruflin added a commit to ruflin/beats that referenced this issue Aug 30, 2016
@ruflin
Introduce harvester_limit to limit number of harvesters
9cbbd0b 
This limits the number of harvesters that are started per prospector.

* Changelog entry added
* Docs updated
* Config file updated

Closes elastic#2236
@ruflin ruflin mentioned this issue Aug 30, 2016
Merged
@tsg tsg closed this as completed in #2417 Aug 30, 2016
tsg pushed a commit that referenced this issue Aug 30, 2016
@ruflin @tsg
Introduce harvester_limit to limit number of harvesters (#2417)
45202c5 
This limits the number of harvesters that are started per prospector. Closes #2236.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
discuss Issue needs further discussion. Filebeat Filebeat
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
1 participant
@ruflin