Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Stackdriver poll-based metrics #593

Merged
merged 24 commits into from
Apr 2, 2019
Merged

Stackdriver poll-based metrics #593

merged 24 commits into from
Apr 2, 2019

Conversation

c24t
Copy link
Member

@c24t c24t commented Apr 2, 2019

This PR changes StackdriverStatsExporter to use metrics data models internally, and switches from a push-based export model (export on each call to record) to a poll-based one (periodically call stats.get_metrics and export the results). It also allows the stackdriver exporter to export gauge metrics. It includes the changes from #560 and #577.

This PR includes multiple breaking behavior and API changes, including starting (and now returning) a background thread from stats_exporter.new_stats_exporter, and exposing Stats as a singleton via stats.stats.

Fixes #583 by exporting less frequently than the stackdriver minimum sampling period.

Fixes #470 by replacing ViewData with metrics objects that are created on the fly.

Addresses #454, which we can close after making the same changes to the prometheus exporter.

@@ -402,21 +363,34 @@ def get_user_agent_slug():
return "opencensus-python/{}".format(__version__)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Curious (although it is not added by this PR) - which version do we want here (opencensus or ext package or both)?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The other clients report the core library version, but I think there's a good argument for replacing this with just the ext version.


def __init__(self, func, interval=None, **kwargs):
super(PeriodicTask, self).__init__(**kwargs)
if interval is None:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Curious, what's the difference with def __init__(self, func, interval=DEFAULT_INTERVAL, **kwargs)?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just that it makes it easy to make the same argument optional in the calling func.

interval comes all the way from stackdriver.stats_exporter.new_stats_exporter, if we used DEFAULT_INTERVAL here we'd have to either have to use it as the default value in the callers (which means importing it in other modules) or make two separate calls, e.g.

def caller_in_another_module(interval=None):
    ...
    if interval is None:
        task = PeriodicTask()
    else:
        task = PeriodicTask(interval=interval)

Using None everywhere is convenient, but there are a few big drawbacks: it's less readable since you can't tell the defaults by looking at the signature, you can't change the default value to something other than None without changing all the callers too, and it doesn't work for arguments that can actually have None as their value.

I'm on the fence here, if you have an opinion on this I'm happy to change it.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't have strong opinion here. Just observed that using None is a common pattern: https://docs.python-guide.org/writing/gotchas/. Feel free to make your call.

Copy link
Contributor

@reyang reyang left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM.

pass


class PeriodicTask(threading.Thread):
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This could be useful for other exporters (e.g. trace exporters), please consider putting it in opencensus.common.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For trace exporters we want a queue that periodically exports batches of spans. It's possible that we'd reuse this, but I think we ought to move it when we do instead of now.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Make sense.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

400 Errors From Stackdriver Stats Exporter Don't expose mutable view data
3 participants