Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

DistributionAggregation fixes #375

Merged

Conversation

c24t
Copy link
Member

@c24t c24t commented Oct 31, 2018

This diff addresses a few bugs in DistributionAggregation and DistributionAggregationData from #373 that needed to be fixed before converting these aggregations to metrics for #335.

This is a noisy diff since I formatted and fixed old lint errors in the files I touched. I'll comment on the important changes.

Fixes #373.

Copy link
Member Author

@c24t c24t left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's possible there are still bugs in this and the other aggregation classes, these are just the ones I found on the way to converting DistributionAggregation into a metric.

@@ -133,7 +133,7 @@ def __init__(
self._boundaries = bucket_boundaries.BucketBoundaries(boundaries)
self._distribution = distribution or {}
self.aggregation_data = aggregation_data.DistributionAggregationData(
0, 0, 0, 0, 0, None, boundaries)
0, 0, float('inf'), float('-inf'), 0, None, boundaries)
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Changed to match the java client's behavior, this previously failed on e.g. negative value points.

self.assertEqual(aggregation_module.Type.DISTRIBUTION,
distribution_aggregation.aggregation_type)

def test_min_max(self):
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Again, all lint and formatting except this one.


dist_agg_data.add_sample(value, timestamp, attachments)
self.assertEqual(5, dist_agg_data.count_data)
self.assertEqual(1.4, dist_agg_data.mean_data)
self.assertEqual(5.2, dist_agg_data.sum_of_sqd_deviations)
self.assertIsNot(0, dist_agg_data.count_data)
self.assertEqual(3, dist_agg_data.exemplars[5].value)
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This was actually asserting the buggy behavior.

else:
last_bucket_index = len(self._bounds)
self._counts_per_bucket[last_bucket_index] += 1
return last_bucket_index
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

One fix here and the other up in __init__, the other changes in this file are lint and formatting.

@songy23
Copy link
Contributor

songy23 commented Oct 31, 2018

Please fix the lint.

opencensus/stats/aggregation_data.py:221:5: E303 too many blank lines (2)
nox > Command flake8 --exclude=opencensus/trace/exporters/gen/opencensus/ opencensus/ failed with exit code 1
nox > Session lint failed. :(

columns=view_columns,
measure=view_measure,
aggregation=view_aggregation)
View(name=view_name,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why is this created but never used?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I didn't start the fire here, I just removed the unused view var to fix the F841 lint error, and same for the changes below.

Many of the tests in this module seem half-baked, but I think this is a problem for another PR.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

SGTM

@@ -343,7 +315,7 @@ def test_record_with_exporter(self):
view_columns = ["testTag1", "testColumn2"]
view_measure = measure
view_aggregation = mock.Mock()
view = View(
View(
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ditto

@@ -378,18 +350,13 @@ def test_export(self):
view_columns = ["testTag1", "testColumn2"]
view_measure = measure
view_aggregation = mock.Mock()
view = View(
View(
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ditto

counts_per_bucket.append(0)
counts_per_bucket = [0 for ii in range(len(bounds) + 1)]
elif len(counts_per_bucket) != len(bounds) + 1:
raise ValueError("counts_per_bucket length does not match bounds "
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we can change to "counts_per_bucket has X fields, which is more than the Y buckets allowed." where, X:len(counts_per_bucket) and Y: len(bounds) + 1. What do you think?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

One benefit of fixed error messages is that you can grep for the exact text, but I'm happy with either here.

Copy link
Contributor

@songy23 songy23 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please re-run CI.

@c24t c24t merged commit d3b2897 into census-instrumentation:master Nov 1, 2018
@c24t c24t deleted the stats-distb-agg-bugfixes-rebase branch November 1, 2018 22:58
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Multiple apparent bugs in DistributionAggregation implementation
4 participants