Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

add helper.go and tests #5

Closed
wants to merge 70 commits into from
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
70 commits
Select commit Hold shift + click to select a range
06753d6
set up project structure
huyan0 Jul 30, 2020
287a4e5
placeholder text for README
huyan0 Jul 30, 2020
c7e2a42
finish Test_validateMetrics
huyan0 Jul 30, 2020
0f6033f
finish Test_validateMetrics
huyan0 Jul 30, 2020
0686608
finish Test_validateMetrics
huyan0 Jul 30, 2020
a896535
remove extra import
huyan0 Jul 30, 2020
6335403
fix format error so that make runs
huyan0 Jul 30, 2020
f76b22a
update unit tests
huyan0 Jul 30, 2020
91dddb9
Testbed for the config and factory constructions:
Jul 30, 2020
66224aa
Testbed for the config and factory constructions, possible README.md …
Jul 30, 2020
2e4b1e1
Testbed oop
Jul 30, 2020
01441e7
add metrictest util code
huyan0 Jul 31, 2020
a9eaaec
tempeorary code
huyan0 Jul 31, 2020
7cf119b
finish testutil and Test_addSample
huyan0 Jul 31, 2020
5625330
add more unit tests
huyan0 Aug 1, 2020
cbd7f6e
add unit tests
huyan0 Aug 2, 2020
a4d7eeb
use assert.Exactly
huyan0 Aug 2, 2020
ae1758c
finish Test_handleHistogramMetric
huyan0 Aug 2, 2020
93df3e9
fix Test_handleScalarMetric to include metric name in label set
huyan0 Aug 2, 2020
346fce0
finish Test_handleSummaryMetric
huyan0 Aug 2, 2020
d215b33
add Test_shutdown
huyan0 Aug 2, 2020
b001c18
add unit test for functions
huyan0 Aug 2, 2020
64aad65
add unit tests for cortex.go
huyan0 Aug 3, 2020
b55576e
associated methods with cortexExporter
huyan0 Aug 3, 2020
ad4f1b1
some more config and factory stuff:
Aug 3, 2020
5806a51
M branch 'unit-tests' of https://github.com/open-o11y/opentelemetry-c…
Aug 3, 2020
d670578
config and some factory, export skeleton
Aug 3, 2020
d125a28
config and some factory, export skeleton
Aug 3, 2020
34b40fe
add nil test cases for cortex.go
huyan0 Aug 3, 2020
32a9db9
implement and test validateMetrics and addSample
huyan0 Aug 3, 2020
c466bd9
implement and test timeSeriesSignature
huyan0 Aug 3, 2020
c7a0dc1
implement and test createLabelSet
huyan0 Aug 3, 2020
b978fd4
finish createMetricsExporter method
huyan0 Aug 3, 2020
3992ea7
work
Aug 3, 2020
01ad87a
factory
Aug 3, 2020
da4dbcc
no more const labels for config
Aug 3, 2020
0312ced
implement and test handleScalarMetric
huyan0 Aug 3, 2020
d868fd9
implement and test handleScalarMetric
huyan0 Aug 3, 2020
cadd3bf
Update factory.go
huyan0 Aug 3, 2020
184e16a
implement and test handleHistogramMetric
huyan0 Aug 3, 2020
d0d700b
Merge branch 'unit-tests' of github.com:open-o11y/opentelemetry-colle…
huyan0 Aug 3, 2020
a133149
implement and test handleHistogramMetric
huyan0 Aug 3, 2020
222ea87
implement and test handleSummaryMetric
huyan0 Aug 3, 2020
c7ce3c6
implement shutdown
huyan0 Aug 3, 2020
a4f09ab
opt headers for config
Aug 3, 2020
100d482
Mergeae branch 'unit-tests' of https://github.com/open-o11y/opentelem…
Aug 3, 2020
ca1295e
undupped create Metrics Exporter
Aug 3, 2020
4bc9a4f
test shutdown
huyan0 Aug 3, 2020
db605f2
Merge branch 'unit-tests' of github.com:open-o11y/opentelemetry-colle…
huyan0 Aug 3, 2020
4a29b98
yes
Aug 3, 2020
c4d8d47
okay, worksMerge branch 'unit-tests' of https://github.com/open-o11y/…
Aug 3, 2020
6073ce3
remove usage of getSample in cortex.go
huyan0 Aug 3, 2020
3c12ab7
Merge branch 'unit-tests' of github.com:open-o11y/opentelemetry-colle…
huyan0 Aug 3, 2020
37db943
implement and test newMetricsExporter
huyan0 Aug 4, 2020
e9ac299
Test_pushMetrics not working
huyan0 Aug 4, 2020
53fbe2b
add comments to and format cortex.go
huyan0 Aug 4, 2020
96807eb
make cortex_test readable
huyan0 Aug 4, 2020
4182996
modify things in factory.go
huyan0 Aug 4, 2020
8ac6b25
update TODOs in cortex_test
huyan0 Aug 4, 2020
9a69f18
updated go.mod
huyan0 Aug 4, 2020
cf8fcaf
add headers field to exporter struct
huyan0 Aug 4, 2020
f223b37
fix config_test
huyan0 Aug 4, 2020
3371a59
add todos
huyan0 Aug 4, 2020
2cb879a
create helper.go and helper_test.go
huyan0 Aug 4, 2020
071b82d
add cortex.go and helper.go with tests
huyan0 Aug 4, 2020
fda582b
fix cortex_test.go
huyan0 Aug 4, 2020
05bcd9d
add comments to tests and address style issues
huyan0 Aug 5, 2020
4042156
add DO NOT REVIEW message in certain files
huyan0 Aug 5, 2020
72289fd
new branch with only helper.go and tests
huyan0 Aug 5, 2020
a0e505b
update comments
huyan0 Aug 5, 2020
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions exporter/cortexexporter/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
To be added.
176 changes: 176 additions & 0 deletions exporter/cortexexporter/helper.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,176 @@
package cortexexporter

import (
"fmt"
"github.com/prometheus/prometheus/prompb"
common "go.opentelemetry.io/collector/internal/data/opentelemetry-proto-gen/common/v1"
otlp "go.opentelemetry.io/collector/internal/data/opentelemetry-proto-gen/metrics/v1"
"sort"
"strings"
"log"
"unicode"
)


const (
nameStr = "name"
sumStr = "_sum"
countStr = "_count"
bucketStr = "_bucket"
leStr = "le"
quantileStr = "quantile"
pInfStr = "+Inf"
totalStr = "total"
delimeter = "_"
keyStr = "key"
)

// ByLabelName enables the usage of sort.Sort() with a slice of labels
type ByLabelName []prompb.Label
func (a ByLabelName) Len() int { return len(a) }
func (a ByLabelName) Less(i, j int) bool { return a[i].Name < a[j].Name }
func (a ByLabelName) Swap(i, j int) { a[i], a[j] = a[j], a[i] }

// validateMetrics returns a bool representing whether the metric has a valid type and temporality combination.
func validateMetrics(desc *otlp.MetricDescriptor) bool {
if desc == nil {
return false
}
switch desc.GetType() {
case otlp.MetricDescriptor_MONOTONIC_DOUBLE, otlp.MetricDescriptor_MONOTONIC_INT64,
otlp.MetricDescriptor_HISTOGRAM, otlp.MetricDescriptor_SUMMARY:
return desc.GetTemporality() == otlp.MetricDescriptor_CUMULATIVE
case otlp.MetricDescriptor_INT64, otlp.MetricDescriptor_DOUBLE:
return true
}
return false
}

// addSample finds a TimeSeries in tsMap that corresponds to the label set lbs, and add sample to the TimeSeries; it
// creates a new TimeSeries in the map if not found. tsMap is unmodified if either of its parameters is nil.
func addSample(tsMap map[string]*prompb.TimeSeries, sample *prompb.Sample, lbs []prompb.Label,
ty otlp.MetricDescriptor_Type) {
if sample == nil || lbs == nil || tsMap == nil {
return
}
sig := timeSeriesSignature(ty, &lbs)
ts, ok := tsMap[sig]
if ok {
ts.Samples = append(ts.Samples, *sample)
} else {
newTs := &prompb.TimeSeries{
Labels: lbs,
Samples: []prompb.Sample{*sample},
}
tsMap[sig] = newTs
}
}

// timeSeries return a string signature of a OTLP metric in the form of:
// TYPE-label1-value1- ... -labelN-valueN
// the label slice should not contain duplicate label names; this method sorts the slice by label name before creating
// the signature.
func timeSeriesSignature(t otlp.MetricDescriptor_Type, lbs *[]prompb.Label) string {
b := strings.Builder{}
fmt.Fprintf(&b, t.String())

sort.Sort(ByLabelName(*lbs))

for _, lb := range *lbs {
fmt.Fprintf(&b, "-%s-%s", lb.GetName(),lb.GetValue())
}

return b.String()
}

// createLabelSet c
// Unpaired string value is ignored. String pairs overwrites OTLP labels if collision happens, and the overwrite is
// logged. Resultant label names are sanitized.
func createLabelSet(labels []*common.StringKeyValue, extras ...string) []prompb.Label {
l := map[string]prompb.Label{}
for _, lb := range labels {
l[lb.Key] = prompb.Label{
Name: sanitize(lb.Key),
Value: lb.Value,
}
}
for i := 0; i < len(extras); i+=2 {
if i+1 >= len(extras){
break
}
_,found:= l[extras[i]]
if found {
log.Println("label " + extras[i] + " is overwritten. Check if Prometheus reserved labels are used.")
}
l[extras[i]] = prompb.Label{
Name: sanitize(extras[i]),
Value: extras[i+1],
}
}
s := make([]prompb.Label,0,len(l))
for _, lb := range l {
s = append(s,lb)
}
return s
}

// copied from prometheus-go-metric-exporter
// sanitize replaces non-alphanumeric characters with underscores in s.
func sanitize(s string) string {
if len(s) == 0 {
return s
}

// Note: No length limit for label keys because Prometheus doesn't
// define a length limit, thus we should NOT be truncating label keys.
// See https://github.com/orijtech/prometheus-go-metrics-exporter/issues/4.

s = strings.Map(sanitizeRune, s)
if unicode.IsDigit(rune(s[0])) {
s = keyStr + delimeter + s
}
if s[0] == '_' {
s = keyStr + s
}
return s
}

// copied from prometheus-go-metric-exporter
// sanitizeRune converts anything that is not a letter or digit to an underscore
func sanitizeRune(r rune) rune {
if unicode.IsLetter(r) || unicode.IsDigit(r) {
return r
}
// Everything else turns into an underscore
return '_'
}

// getPromMetricName creates a Prometheus metric name by attaching namespace prefix, and _total suffix for Monotonic
// metrics.
func getPromMetricName(desc *otlp.MetricDescriptor, ns string) string {
if desc == nil {
return ""
}
isCounter := desc.Type == otlp.MetricDescriptor_MONOTONIC_INT64 ||
desc.Type == otlp.MetricDescriptor_MONOTONIC_DOUBLE
b := strings.Builder{}
fmt.Fprintf(&b, ns)
if b.Len() > 0 {
fmt.Fprintf(&b, delimeter)
}
fmt.Fprintf(&b, sanitize(desc.GetName()))

// Including units makes two metrics with the same name and label set belong to two different TimeSeries if the
// units are different.
/*
if b.Len() > 0 && len(desc.GetUnit()) > 0{
fmt.Fprintf(&b, delimeter)
fmt.Fprintf(&b, desc.GetUnit())
}*/

if b.Len()>0 && isCounter {
fmt.Fprintf(&b, delimeter)
fmt.Fprintf(&b, totalStr)
}
return b.String()
}
Loading