Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add support for System.Diagnostics.Metrics #3479

Merged
merged 22 commits into from
Feb 1, 2023
Merged
Show file tree
Hide file tree
Changes from 13 commits
Commits
Show all changes
22 commits
Select commit Hold shift + click to select a range
4f12598
Add prototype support for system diagnostics metrics
kkeirstead Nov 15, 2022
87f934b
[feature branch, do not review] Update counter api, fixup metadata pa…
wiktork Dec 16, 2022
f09f1e6
[System Diagnostics Metrics] Quantile->Percentile Changes & Testing I…
kkeirstead Dec 21, 2022
e0c2573
Rename counterpipeline
wiktork Jan 13, 2023
0fbbfb0
Fix invariant
wiktork Jan 13, 2023
9a43a8b
Fixup regex
wiktork Jan 13, 2023
37c1edc
Added tests for histogram and timeseries; addressed some other feedba…
kkeirstead Jan 19, 2023
07987dd
Fix counter name
kkeirstead Jan 19, 2023
a794a28
Use one payload for Histogram
wiktork Jan 25, 2023
e45d950
Added MetricType option for configuration; added testing to go along …
kkeirstead Jan 25, 2023
b4cf9af
Move documentation to correct file
wiktork Jan 26, 2023
567bc4c
Initial PR feedback
wiktork Jan 27, 2023
92925b5
Do not allow CounterEnded events to displace existing metrics
wiktork Jan 30, 2023
2418cb1
Deduplicate errors
wiktork Jan 31, 2023
5949109
Update dependencies from https://github.com/dotnet/diagnostics build …
dotnet-maestro[bot] Jan 31, 2023
e094792
Merge branch 'main' into shared/systemDiagnosticsMetricsMain
wiktork Jan 31, 2023
67a2d7d
Fix minor formatting issues
wiktork Jan 31, 2023
5bb6ed6
Fix spell check issues
wiktork Jan 31, 2023
33e093c
Update documentation/configuration/metrics-configuration.md
kkeirstead Jan 31, 2023
e2eef4d
Update documentation/configuration/metrics-configuration.md
kkeirstead Jan 31, 2023
e06b6c1
PR Feedback
kkeirstead Jan 31, 2023
ea6db9a
Add comment from PR
wiktork Feb 1, 2023
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
50 changes: 48 additions & 2 deletions documentation/api/livemetrics-custom.md
Original file line number Diff line number Diff line change
Expand Up @@ -60,7 +60,9 @@ The expected content type is `application/json`.

## Examples

### Sample Request
### EventCounter

#### Sample Request

```http
POST /livemetrics?pid=21632&durationSeconds=60 HTTP/1.1
Expand All @@ -81,7 +83,7 @@ Authorization: Bearer fffffffffffffffffffffffffffffffffffffffffff=
}
```

### Sample Response
#### Sample Response

```http
HTTP/1.1 200 OK
Expand Down Expand Up @@ -113,6 +115,50 @@ Location: localhost:52323/operations/67f07e40-5cca-4709-9062-26302c484f18
}
```

### System.Diagnostics.Metrics

#### Sample Request

```http
GET /livemetrics?pid=21632&durationSeconds=60 HTTP/1.1
wiktork marked this conversation as resolved.
Show resolved Hide resolved
Host: localhost:52323
Authorization: Bearer fffffffffffffffffffffffffffffffffffffffffff=

{
"includeDefaultProviders": false,
"providers": [
{
"providerName": "CustomProvider",
"counterNames": [
"myHistogram"
]
}
]
}
```

#### Sample Histogram Response

```http
HTTP/1.1 200 OK
Content-Type: application/json-seq
Location: localhost:52323/operations/67f07e40-5cca-4709-9062-26302c484f18

{
"timestamp": "2021-08-31T16:58:39.7514031+00:00",
"provider": "CustomProvider",
"name": "myHistogram",
"displayName": "myHistogram",
"unit": null,
"counterType": "Metric",
"value": {
"0.5": 2892,
"0.95": 4848,
"0.99": 4984
}
}
```

## Supported Runtimes

| Operating System | Runtime Version |
Expand Down
66 changes: 66 additions & 0 deletions documentation/configuration/metrics-configuration.md
Original file line number Diff line number Diff line change
Expand Up @@ -168,6 +168,72 @@ When `CounterNames` are not specified, all the counters associated with the `Pro

[7.1+] Custom metrics support labels for metadata. Metadata cannot include commas (`,`); the inclusion of a comma in metadata will result in all metadata being removed from the custom metric.

## Limit How Many Histograms To Track (7.1+)
kkeirstead marked this conversation as resolved.
Show resolved Hide resolved

For System.Diagnostics.Metrics, `dotnet monitor` allows you to set the maximum number of histograms that can be tracked. Each unique combination of provider name, histogram name, and dimension values counts as one histogram. Tracking more histograms uses more memory in the target process so this bound guards against unintentional high memory use. `MaxHistograms` has a default value of `10`.
kkeirstead marked this conversation as resolved.
Show resolved Hide resolved

<details>
<summary>JSON</summary>

```json
{
"Metrics": {
"MaxHistograms": 5
}
}
```
</details>

<details>
<summary>Kubernetes ConfigMap</summary>

```yaml
Metrics__MaxHistograms: "5"
```
</details>

<details>
<summary>Kubernetes Environment Variables</summary>

```yaml
- name: DotnetMonitor_Metrics__MaxHistograms
value: "5"
```
</details>

## Limit How Many Time Series To Track (7.1+)

For System.Diagnostics.Metrics, `dotnet monitor` allows you to set the maximum number of time series that can be tracked. Each unique combination of provider name, metric name, and dimension values counts as one time series. Tracking more time series uses more memory in the target process so this bound guards against unintentional high memory use. `MaxTimeSeries` has a default value of `1000`.

<details>
<summary>JSON</summary>

```json
{
"Metrics": {
"MaxTimeSeries": 500
}
}
```
</details>

<details>
<summary>Kubernetes ConfigMap</summary>

```yaml
Metrics__MaxTimeSeries: "500"
```
</details>

<details>
<summary>Kubernetes Environment Variables</summary>

```yaml
- name: DotnetMonitor_Metrics__MaxTimeSeries
value: "500"
```
</details>

## Disable default providers

In addition to enabling custom providers, `dotnet monitor` also allows you to disable collection of the default providers. You can do so via the following configuration:
Expand Down
11 changes: 11 additions & 0 deletions documentation/openapi.json
Original file line number Diff line number Diff line change
Expand Up @@ -1461,6 +1461,9 @@
"type": "string"
},
"nullable": true
},
"metricType": {
"$ref": "#/components/schemas/MetricProviderType"
}
},
"additionalProperties": false
Expand Down Expand Up @@ -1553,6 +1556,14 @@
},
"additionalProperties": false
},
"MetricProviderType": {
"enum": [
"EventCounter",
"Meter",
"All"
],
"type": "string"
},
"OperationError": {
"type": "object",
"properties": {
Expand Down
59 changes: 59 additions & 0 deletions documentation/schema.json
Original file line number Diff line number Diff line change
Expand Up @@ -877,6 +877,28 @@
"default": 5.0,
"maximum": 86400.0,
"minimum": 1.0
},
"MaxHistograms": {
"type": [
"integer",
"null"
],
"description": "The maximum number of histograms that can be tracked. Each unique combination of provider name, histogram name, and dimension values counts as one histogram. Tracking more histograms uses more memory in the target process so this bound guards against unintentional high memory use.",
"format": "int32",
"default": 20,
"maximum": 2147483647.0,
"minimum": 1.0
},
"MaxTimeSeries": {
"type": [
"integer",
"null"
],
"description": "The maximum number of time series that can be tracked. Each unique combination of provider name, metric name, and dimension values counts as one time series. Tracking more time series uses more memory in the target process so this bound guards against unintentional high memory use.",
"format": "int32",
"default": 1000,
"maximum": 2147483647.0,
"minimum": 1.0
}
}
},
Expand Down Expand Up @@ -1304,9 +1326,36 @@
"items": {
"type": "string"
}
},
"MetricType": {
"description": "The type of metrics this provider consumes",
"default": "All",
"oneOf": [
{
"type": "null"
},
{
"$ref": "#/definitions/MetricProviderType"
}
]
}
}
},
"MetricProviderType": {
"type": "string",
"description": "",
"x-enumFlags": true,
"x-enumNames": [
"EventCounter",
"Meter",
"All"
],
"enum": [
"EventCounter",
"Meter",
"All"
]
},
"StorageOptions": {
"type": "object",
"additionalProperties": false,
Expand Down Expand Up @@ -1636,6 +1685,16 @@
"items": {
"type": "string"
}
},
"MetricType": {
"oneOf": [
{
"type": "null"
},
{
"$ref": "#/definitions/MetricProviderType"
}
]
}
}
},
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -18,11 +18,31 @@ public class GlobalCounterOptions
[Range(IntervalMinSeconds, IntervalMaxSeconds)]
[DefaultValue(GlobalCounterOptionsDefaults.IntervalSeconds)]
public float? IntervalSeconds { get; set; }

[Display(
ResourceType = typeof(OptionsDisplayStrings),
Description = nameof(OptionsDisplayStrings.DisplayAttributeDescription_MetricsOptions_MaxHistograms))]
[DefaultValue(GlobalCounterOptionsDefaults.MaxHistograms)]
[Range(1, int.MaxValue)]
public int? MaxHistograms { get; set; }

[Display(
ResourceType = typeof(OptionsDisplayStrings),
Description = nameof(OptionsDisplayStrings.DisplayAttributeDescription_MetricsOptions_MaxTimeSeries))]
[DefaultValue(GlobalCounterOptionsDefaults.MaxTimeSeries)]
[Range(1, int.MaxValue)]
public int? MaxTimeSeries { get; set; }
}

internal static class GlobalCounterOptionsExtensions
{
public static float GetIntervalSeconds(this GlobalCounterOptions options) =>
options.IntervalSeconds.GetValueOrDefault(GlobalCounterOptionsDefaults.IntervalSeconds);

public static int GetMaxHistograms(this GlobalCounterOptions options) =>
options.MaxHistograms.GetValueOrDefault(GlobalCounterOptionsDefaults.MaxHistograms);

public static int GetMaxTimeSeries(this GlobalCounterOptions options) =>
options.MaxTimeSeries.GetValueOrDefault(GlobalCounterOptionsDefaults.MaxTimeSeries);
}
}
Original file line number Diff line number Diff line change
Expand Up @@ -6,5 +6,9 @@ namespace Microsoft.Diagnostics.Monitoring.WebApi
internal static class GlobalCounterOptionsDefaults
{
public const float IntervalSeconds = 5.0f;

public const int MaxHistograms = 20;

public const int MaxTimeSeries = 1000;
}
}
15 changes: 15 additions & 0 deletions src/Microsoft.Diagnostics.Monitoring.Options/MetricsOptions.cs
Original file line number Diff line number Diff line change
@@ -1,6 +1,7 @@
// Licensed to the .NET Foundation under one or more agreements.
// The .NET Foundation licenses this file to you under the MIT license.

using System;
using System.Collections.Generic;
using System.ComponentModel;
using System.ComponentModel.DataAnnotations;
Expand Down Expand Up @@ -57,5 +58,19 @@ public class MetricProvider
ResourceType = typeof(OptionsDisplayStrings),
Description = nameof(OptionsDisplayStrings.DisplayAttributeDescription_MetricProvider_CounterNames))]
public List<string> CounterNames { get; set; } = new List<string>(0);

[Display(
ResourceType = typeof(OptionsDisplayStrings),
Description = nameof(OptionsDisplayStrings.DisplayAttributeDescription_MetricProvider_MetricType))]
[DefaultValue(MetricsOptionsDefaults.MetricType)]
public MetricProviderType? MetricType { get; set; }
}

[Flags]
public enum MetricProviderType
{
EventCounter = 0x1,
Meter = 0x2,
All = 0xFF
}
}
Original file line number Diff line number Diff line change
Expand Up @@ -10,5 +10,7 @@ internal static class MetricsOptionsDefaults
public const int MetricCount = 3;

public const bool IncludeDefaultProviders = true;

public const MetricProviderType MetricType = MetricProviderType.All;
}
}

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

Original file line number Diff line number Diff line change
Expand Up @@ -799,4 +799,16 @@
<value>A mapping of event payload field names to their expected value. A subset of the payload fields may be specified.</value>
<comment>The description provided for the PayloadFilter parameter on TraceEventFilter.</comment>
</data>
<data name="DisplayAttributeDescription_MetricsOptions_MaxHistograms" xml:space="preserve">
<value>The maximum number of histograms that can be tracked. Each unique combination of provider name, histogram name, and dimension values counts as one histogram. Tracking more histograms uses more memory in the target process so this bound guards against unintentional high memory use.</value>
<comment>The description provided for the MaxHistograms parameter on MetricsOptions.</comment>
</data>
<data name="DisplayAttributeDescription_MetricsOptions_MaxTimeSeries" xml:space="preserve">
<value>The maximum number of time series that can be tracked. Each unique combination of provider name, metric name, and dimension values counts as one time series. Tracking more time series uses more memory in the target process so this bound guards against unintentional high memory use.</value>
<comment>The description provided for the MaxTimeSeries parameter on MetricsOptions.</comment>
</data>
<data name="DisplayAttributeDescription_MetricProvider_MetricType" xml:space="preserve">
<value>The type of metrics this provider consumes</value>
<comment>The description provided for the MetricType parameter on MetricProvider.</comment>
</data>
</root>
Loading