Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[WIP] Views API Prototype #596

Merged
merged 47 commits into from
Aug 3, 2020
Merged
Show file tree
Hide file tree
Changes from 34 commits
Commits
Show all changes
47 commits
Select commit Hold shift + click to select a range
7419bda
view
lzchen Apr 16, 2020
aa7562a
remove label keys
lzchen Apr 17, 2020
4a7f6dd
Merge branch 'master' of https://github.com/open-telemetry/openteleme…
lzchen Apr 20, 2020
e43a984
Remove meter ref, aggregator for
lzchen Apr 21, 2020
4d4351f
view manager as part of meter
lzchen Apr 21, 2020
e667bc8
fix tests
lzchen Apr 21, 2020
5d9ae6f
seperate aggregations from aggregators
lzchen Apr 24, 2020
554dafb
default aggregations
lzchen Apr 24, 2020
f8997ca
Merge branch 'master' of https://github.com/open-telemetry/openteleme…
lzchen May 4, 2020
8331f1b
Add label key logic
lzchen May 4, 2020
d93f159
aggregate config and histogram
lzchen May 7, 2020
53cc0f7
Merge branch 'master' of https://github.com/open-telemetry/openteleme…
lzchen May 7, 2020
e966f7e
Merge branch 'master' of https://github.com/open-telemetry/openteleme…
lzchen Jul 1, 2020
adce43e
views
lzchen Jul 2, 2020
864b644
fix observer
lzchen Jul 2, 2020
fea636f
fix lastvalue
lzchen Jul 2, 2020
af6d003
fix tests
lzchen Jul 7, 2020
176aa55
fix tests
lzchen Jul 7, 2020
12402f7
Merge branch 'master' of https://github.com/open-telemetry/openteleme…
lzchen Jul 7, 2020
8e3a903
changelog
lzchen Jul 7, 2020
9e7332f
fix tests
lzchen Jul 7, 2020
3eb7dcd
Merge branch 'master' of https://github.com/open-telemetry/openteleme…
lzchen Jul 7, 2020
ac3734c
lint
lzchen Jul 7, 2020
82a154c
lint
lzchen Jul 7, 2020
55ad347
lint
lzchen Jul 7, 2020
fbeaf34
lint
lzchen Jul 7, 2020
778223e
lint
lzchen Jul 7, 2020
4875769
lint
lzchen Jul 7, 2020
f793247
Merge branch 'master' of https://github.com/open-telemetry/openteleme…
lzchen Jul 7, 2020
d7fbcef
Merge branch 'master' into views
lzchen Jul 8, 2020
45e96f3
fix state
lzchen Jul 10, 2020
0b22ef7
Merge branch 'views' of https://github.com/lzchen/opentelemetry-pytho…
lzchen Jul 10, 2020
2a55926
fix test
lzchen Jul 10, 2020
b74fb2d
Merge branch 'master' into views
lzchen Jul 10, 2020
a510eb0
Create new aggregator per ViewData, config dict
cnnradams Jul 28, 2020
54594c8
ViewData references, equality updates
cnnradams Jul 30, 2020
8baa90e
Histogram updates
cnnradams Jul 30, 2020
765e8ef
updates
cnnradams Jul 31, 2020
bba29d5
fix batcher
cnnradams Jul 31, 2020
d609602
Merge pull request #8 from cnnradams/views
lzchen Jul 31, 2020
4b49cb7
Fix View hash
c24t Aug 1, 2020
de145ce
Wrap, blacken
c24t Aug 1, 2020
93c3808
Merge branch 'master' of https://github.com/open-telemetry/openteleme…
lzchen Aug 3, 2020
2962561
Merge branch 'views' of https://github.com/lzchen/opentelemetry-pytho…
lzchen Aug 3, 2020
58212ba
fix teests
lzchen Aug 3, 2020
f0e74b7
test
lzchen Aug 3, 2020
a8ab8de
lint
lzchen Aug 3, 2020
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 3 additions & 3 deletions docs/examples/basic_meter/basic_metrics.py
Original file line number Diff line number Diff line change
Expand Up @@ -19,6 +19,7 @@
- How to configure a meter passing a sateful or stateless.
- How to configure an exporter and how to create a controller.
- How to create some metrics intruments and how to capture data with them.
- How to use views to specify aggregation types for each metric instrument.
"""
import sys
import time
Expand Down Expand Up @@ -57,7 +58,6 @@
unit="1",
value_type=int,
metric_type=Counter,
label_keys=("environment",),
)

requests_size = meter.create_metric(
Expand All @@ -66,7 +66,6 @@
unit="1",
value_type=int,
metric_type=ValueRecorder,
label_keys=("environment",),
)

# Labels are used to identify key-values that are associated with a specific
Expand All @@ -86,4 +85,5 @@

requests_counter.add(35, testing_labels)
requests_size.record(2, testing_labels)
time.sleep(5)

input("...\n")
5 changes: 2 additions & 3 deletions docs/examples/basic_meter/calling_conventions.py
Original file line number Diff line number Diff line change
Expand Up @@ -33,7 +33,6 @@
unit="1",
value_type=int,
metric_type=Counter,
label_keys=("environment",),
)

requests_size = meter.create_metric(
Expand All @@ -42,7 +41,6 @@
unit="1",
value_type=int,
metric_type=ValueRecorder,
label_keys=("environment",),
)

clicks_counter = meter.create_metric(
Expand All @@ -51,7 +49,6 @@
unit="1",
value_type=int,
metric_type=Counter,
label_keys=("environment",),
)

labels = {"environment": "staging"}
Expand All @@ -78,3 +75,5 @@
# specified labels for each.
meter.record_batch(labels, ((requests_counter, 50), (clicks_counter, 70)))
time.sleep(5)

input("...\n")
2 changes: 0 additions & 2 deletions docs/examples/basic_meter/observer.py
Original file line number Diff line number Diff line change
Expand Up @@ -41,7 +41,6 @@ def get_cpu_usage_callback(observer):
unit="1",
value_type=float,
observer_type=ValueObserver,
label_keys=("cpu_number",),
)


Expand All @@ -58,7 +57,6 @@ def get_ram_usage_callback(observer):
unit="1",
value_type=float,
observer_type=ValueObserver,
label_keys=(),
)

input("Metrics will be printed soon. Press a key to finish...\n")
111 changes: 111 additions & 0 deletions docs/examples/basic_meter/view.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,111 @@
# Copyright The OpenTelemetry Authors
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
"""
This example shows how to use the different modes to capture metrics.
It shows the usage of the direct, bound and batch calling conventions.
"""
from opentelemetry import metrics
from opentelemetry.sdk.metrics import (
MeterProvider,
UpDownCounter,
ValueRecorder,
)
from opentelemetry.sdk.metrics.export import ConsoleMetricsExporter
from opentelemetry.sdk.metrics.export.aggregate import (
HistogramAggregator,
LastValueAggregator,
MinMaxSumCountAggregator,
SumAggregator,
)
from opentelemetry.sdk.metrics.view import View, ViewConfig

# Use the meter type provided by the SDK package
metrics.set_meter_provider(MeterProvider())
meter = metrics.get_meter(__name__)
metrics.get_meter_provider().start_pipeline(meter, ConsoleMetricsExporter(), 5)

requests_counter = meter.create_metric(
name="requests",
description="number of requests",
unit="1",
value_type=int,
metric_type=UpDownCounter,
)

requests_size = meter.create_metric(
name="requests_size",
description="size of requests",
unit="1",
value_type=int,
metric_type=ValueRecorder,
)

# Views are used to define an aggregation type and label keys to aggregate by
# for a given metric

# Two views with the same metric and aggregation type but different label keys
# With ViewConfig.LABEL_KEYS, all labels but the ones defined in label_keys are
# dropped from the aggregation
counter_view1 = View(
requests_counter,
SumAggregator(),
label_keys=["environment"],
config=ViewConfig.LABEL_KEYS,
)
counter_view2 = View(
requests_counter,
MinMaxSumCountAggregator(),
label_keys=["os_type"],
config=ViewConfig.LABEL_KEYS,
)
# This view has ViewConfig set to UNGROUPED, meaning all recorded metrics take
# the labels directly without and consideration for label_keys
counter_view3 = View(
requests_counter,
LastValueAggregator(),
label_keys=["environment"], # is not used due to ViewConfig.UNGROUPED
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If this isn't used, maybe this parameter should just be the signal that you want to use the LABEL_KEYS configuration, rather than having it only being used in that one case?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Interesting thought. I don't know what is more confusing, the ignoring of the ViewConfig if label_keys is defined or the ignoring of the keys if ViewConfig.Ungrouped is defined.

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why not combine them into one struct that contains both the config and the data that it needs?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Even if they are combined to one struct, i'm not sure if it eases the confusion? There's still implicit behavior that is occurring if both values are supplied, even as arguments to a struct.

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I guess I was thinking more of a class that could indicate both the type and the attributes needed to implement it, and constructors that made it clear what was required for what. Or something along those lines.

config=ViewConfig.UNGROUPED,
)
# This view uses the HistogramAggregator which accepts an option config
# parameter to specify the bucket ranges
size_view = View(
requests_size,
HistogramAggregator(config=[20, 40, 60, 80, 100]),
label_keys=["environment"], # is not used due to ViewConfig.UNGROUPED
config=ViewConfig.UNGROUPED,
)

# Register the views to the view manager to use the views. Views MUST be
# registered before recording metrics. Metrics that are recorded without
# views defined for them will use a default for that type of metric
meter.register_view(counter_view1)
meter.register_view(counter_view2)
meter.register_view(counter_view3)
meter.register_view(size_view)

# The views will evaluate the labels passed into the record and aggregate upon
# the unique labels that are generated
# view1 labels will evaluate to {"environment": "staging"}
# view2 labels will evaluate to {"os_type": linux}
# view3 labels will evaluate to {"environment": "staging", "os_type": "linux"}
requests_counter.add(100, {"environment": "staging", "os_type": "linux"})

# Since this is using the HistogramAggregator, the bucket counts will be reflected
# with each record
requests_size.record(25, {"test": "value"})
requests_size.record(-3, {"test": "value"})
requests_size.record(200, {"test": "value"})

input("...\n")
Original file line number Diff line number Diff line change
Expand Up @@ -25,7 +25,7 @@
Counter,
MeterProvider,
ValueRecorder,
get_labels_as_key,
get_dict_as_key,
)
from opentelemetry.sdk.metrics.export import (
MetricRecord,
Expand All @@ -42,7 +42,7 @@ def setUpClass(cls):
metrics.set_meter_provider(MeterProvider())
cls._meter = metrics.get_meter(__name__)
cls._labels = {"environment": "staging"}
cls._key_labels = get_labels_as_key(cls._labels)
cls._key_labels = get_dict_as_key(cls._labels)

def test_constructor(self):
mock_get_node = mock.Mock()
Expand Down Expand Up @@ -119,7 +119,7 @@ def test_export(self):
client=mock_client, host_name=host_name
)
test_metric = self._meter.create_metric(
"testname", "testdesc", "unit", int, Counter, ["environment"]
"testname", "testdesc", "unit", int, Counter,
)
record = MetricRecord(
test_metric, self._key_labels, aggregate.SumAggregator(),
Expand All @@ -142,7 +142,7 @@ def test_export(self):

def test_translate_to_collector(self):
test_metric = self._meter.create_metric(
"testname", "testdesc", "unit", int, Counter, ["environment"]
"testname", "testdesc", "unit", int, Counter,
)
aggregator = aggregate.SumAggregator()
aggregator.update(123)
Expand Down
29 changes: 6 additions & 23 deletions ext/opentelemetry-ext-prometheus/tests/test_prometheus_exporter.py
Original file line number Diff line number Diff line change
Expand Up @@ -32,15 +32,10 @@ def setUp(self):
set_meter_provider(metrics.MeterProvider())
self._meter = get_meter_provider().get_meter(__name__)
self._test_metric = self._meter.create_metric(
"testname",
"testdesc",
"unit",
int,
metrics.Counter,
["environment"],
"testname", "testdesc", "unit", int, metrics.Counter,
)
labels = {"environment": "staging"}
self._labels_key = metrics.get_labels_as_key(labels)
self._labels_key = metrics.get_dict_as_key(labels)

self._mock_registry_register = mock.Mock()
self._registry_register_patch = mock.patch(
Expand Down Expand Up @@ -78,15 +73,10 @@ def test_export(self):
def test_counter_to_prometheus(self):
meter = get_meter_provider().get_meter(__name__)
metric = meter.create_metric(
"test@name",
"testdesc",
"unit",
int,
metrics.Counter,
["environment@", "os"],
"test@name", "testdesc", "unit", int, metrics.Counter,
)
labels = {"environment@": "staging", "os": "Windows"}
key_labels = metrics.get_labels_as_key(labels)
key_labels = metrics.get_dict_as_key(labels)
aggregator = SumAggregator()
aggregator.update(123)
aggregator.take_checkpoint()
Expand Down Expand Up @@ -117,7 +107,7 @@ def test_invalid_metric(self):
"tesname", "testdesc", "unit", int, StubMetric
)
labels = {"environment": "staging"}
key_labels = metrics.get_labels_as_key(labels)
key_labels = metrics.get_dict_as_key(labels)
record = MetricRecord(metric, key_labels, None)
collector = CustomCollector("testprefix")
collector.add_metrics_data([record])
Expand All @@ -143,15 +133,8 @@ def __init__(
unit: str,
value_type,
meter,
label_keys,
enabled: bool = True,
):
super().__init__(
name,
description,
unit,
value_type,
meter,
label_keys=label_keys,
enabled=enabled,
name, description, unit, value_type, meter, enabled=enabled,
)
3 changes: 0 additions & 3 deletions opentelemetry-api/src/opentelemetry/metrics/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -342,7 +342,6 @@ def create_metric(
unit: str,
value_type: Type[ValueT],
metric_type: Type[MetricT],
label_keys: Sequence[str] = (),
enabled: bool = True,
) -> "Metric":
"""Creates a ``metric_kind`` metric with type ``value_type``.
Expand All @@ -354,7 +353,6 @@ def create_metric(
(https://unitsofmeasure.org/ucum.html).
value_type: The type of values being recorded by the metric.
metric_type: The type of metric being created.
label_keys: The keys for the labels with dynamic values.
enabled: Whether to report the metric by default.
Returns: A new ``metric_type`` metric with values of ``value_type``.
"""
Expand Down Expand Up @@ -413,7 +411,6 @@ def create_metric(
unit: str,
value_type: Type[ValueT],
metric_type: Type[MetricT],
label_keys: Sequence[str] = (),
enabled: bool = True,
) -> "Metric":
# pylint: disable=no-self-use
Expand Down
2 changes: 2 additions & 0 deletions opentelemetry-sdk/CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,8 @@

- Add support for resources and resource detector
([#853](https://github.com/open-telemetry/opentelemetry-python/pull/853))
- Implement Views in metrics SDK
([#596](https://github.com/open-telemetry/opentelemetry-python/pull/596))

## Version 0.10b0

Expand Down
Loading