Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Increase sample size on rate sampler test #1076

Merged
merged 1 commit into from
Jun 10, 2020

Conversation

marcotc
Copy link
Member

@marcotc marcotc commented Jun 9, 2020

We have this flaky test in our suite:

  1) Failure:
Datadog::RateByServiceSamplerTest#test_sampling [test/rate_by_service_sampler_test.rb:35]:
Expected |132 - 100.0| (32.0) to be <= 30.0.

This failure happens because our sampling size is too small for the assertions we are trying to make.

This PR changes the sampling sample size from 1.000 to 10.000 runs. I also reduced the allowed deviation as it should go down with the increased sample size.

@marcotc marcotc added the dev/testing Involves testing processes (e.g. RSpec) label Jun 9, 2020
@marcotc marcotc requested a review from a team June 9, 2020 20:35
@marcotc marcotc self-assigned this Jun 9, 2020
@marcotc marcotc changed the title Increse sample size on rate sampler test Increase sample size on rate sampler test Jun 9, 2020
Copy link
Contributor

@delner delner left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍

@marcotc marcotc merged commit df4c39b into master Jun 10, 2020
@marcotc marcotc deleted the test/more-lenient-sampling-rate branch June 10, 2020 17:53
@marcotc marcotc added this to the 0.37.0 milestone Jun 10, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
dev/testing Involves testing processes (e.g. RSpec)
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants