Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Race condition on harvester limit #5458

Closed
tsg opened this issue Oct 27, 2017 · 0 comments
Closed

Race condition on harvester limit #5458

tsg opened this issue Oct 27, 2017 · 0 comments
Assignees
Labels

Comments

@tsg
Copy link
Contributor

tsg commented Oct 27, 2017

The test_prospector.py:Test.test_harvester_limit test is flaky. The fail can be reproduced by running it in a while loop like this:

 while nosetests test_prospector.py:Test.test_harvester_limit; do echo .; done

On my computer, this fails after 10 or 20 iterations.

Looking at the logs and code, it seems to be a race condition between checking p.harvesters.Len() and p.harvesters.Start(). We could serialise the whole startHarvester() method with a new lock, but I'm slightly worried about dead locks, because the harvester registry has it's own lock. So I'm opening a ticket for now to separate this from #5449.

@tsg tsg added bug Filebeat Filebeat labels Oct 27, 2017
@adriansr adriansr self-assigned this Oct 31, 2017
adriansr added a commit to adriansr/beats that referenced this issue Oct 31, 2017
This fixes a race where the size of registry's internal map was
used to count the number of harvesters, but this map is handled
in a different goroutine.

Closes elastic#5458
@urso urso closed this as completed in 09df79c Nov 2, 2017
adriansr added a commit to adriansr/beats that referenced this issue Nov 2, 2017
pull request elastic#5458 was merged with some outstanding changes
kvch pushed a commit that referenced this issue Nov 2, 2017
pull request #5458 was merged with some outstanding changes
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

2 participants