Refactor the complete code #49

rustycl0ck · 2020-08-04T05:58:48Z

Make the working of this exporter similar to that of the blackbox_exporter to allow probing multiple targets.
Add functionality to add headers to the request
Update the example config to use headers as well as the metrics keys in alignment with the new code
Add default header 'Accept: application/json'

* Make the working of this exporter similar to that of the blackbox_exporter to allow probing multiple targets. * Add functionality to add headers to the request * Update the example config to use `headers` as well as the `metrics` keys in alignment with the new code * Add default header 'Accept: application/json' Signed-off-by: rustyclock <[email protected]>

SuperQ · 2020-08-04T06:22:12Z

It would be useful to add a Prometheus config example with the required relabeling to the README.

Signed-off-by: rustyclock <[email protected]>

rustycl0ck · 2020-08-05T09:42:28Z

Migrated to promlog, but don't know if I should have used a global logger variable in the package, instead of passing it around to each function. Will wait for review/feedback.

SuperQ · 2020-08-05T14:49:23Z

Good question, I'm not very opinionated on how the logger is supposed to be used.

Maybe @roidelapluie has some opinions?

roidelapluie · 2020-08-05T14:53:24Z

Global - no, but you could create a struct that implements Collector and attach the functions to it - and use it from there

Signed-off-by: rustyclock <[email protected]>

rustycl0ck · 2020-08-05T22:36:16Z

Switched to kingpin.

Closes #50 #28

cmd/main.go

Signed-off-by: rustyclock <[email protected]>

rustycl0ck · 2020-08-06T11:47:45Z

Closes #33

Signed-off-by: rustyclock <[email protected]>

rustycl0ck · 2020-08-20T02:56:33Z

Addresses #44 also, since I added some unit tests now.
I don't think I'll be making anymore changes now (unless I find any bugs or you guys request any modifications) except creating a helm chart for deployment (which can/will probably be a separate PR).

SuperQ

LGTM

SuperQ · 2020-09-23T21:13:36Z

Ping @roidelapluie, would you mind reviewing this?

roidelapluie · 2020-09-23T21:20:21Z

Can we use another package name that internal as it prevents reuse of the code elsewhere?

roidelapluie

Some comments :) Awesome work

internal/util.go

roidelapluie · 2020-09-23T21:28:30Z

examples/prometheus.yml

+  relabel_configs:
+    - source_labels: [__param_target]
+      target_label: endpoint
+      action: replace


An exemples à la blackbox_exporter might be better? With the json as target

Maybe I didn't understand this fully I think.
Current example set up:
prometheus -> scrapes json_exporter -> fetches localhost:8000/examples/data.json
Proposed example set up:
prometheus -> scrapes blackbox_exporter -> probes json_exporter? -> fetches localhost:8000/examples/data.json?
🤔

No, it's more about how you use Prometheus configs.

... static_configs: - targets: - http://host-1:8000/examples/data.json - http://host-2:8000/other-examples/data.json relabel_configs: - source_labels: [__address__] target_label: __param_target - source_labels: [__param_target] target_label: instance - target_label: __address__ replacement: 127.0.0.1:7979 # The json exporter's real hostname:port.

Thanks for the clarification, I have updated the example prometheus config. Let me know if the updated one looks good.

config/config.go

cmd/main_test.go

cmd/main.go

Signed-off-by: rustyclock <[email protected]>

rustycl0ck · 2020-10-03T00:32:36Z

Thanks for the awesome review @roidelapluie. I have made the requested changes and addressed the remaining.

examples/prometheus.yml

Signed-off-by: rustyclock <[email protected]>

rustycl0ck · 2020-10-17T06:42:14Z

Ping @SuperQ @roidelapluie Can this be merged if all looks good now?

roidelapluie · 2020-10-28T16:45:12Z

Thanks!!

SuperQ · 2020-10-28T17:48:41Z

@rustycl0ck Anything else we should do before we cut a new release?

rustycl0ck · 2020-10-29T03:14:03Z

@SuperQ Nothing more yet. Thanks 👍
Although, I have a use case where I need to provide different bearerToken for authentication to different servers.But I'll create a new issue for discussing the right way to handle that.

After prometheus-community#49 was merged to master the build failed due to a linting error: GO111MODULE=on /go/bin/golangci-lint run ./... exporter/util.go:162:10: Error return value of `io.Copy` is not checked (errcheck) io.Copy(ioutil.Discard, resp.Body) ^ make: *** [Makefile.common:192: common-lint] Error 1 build failure here: https://app.circleci.com/pipelines/github/prometheus-community/json_exporter/28/workflows/e2c53db3-d3a3-4faa-a5a6-3b1fbb5754a0/jobs/71 Signed-off-by: Konstantinos Koukopoulos <[email protected]>

This was referenced Aug 4, 2020

Add functionality to add headers to the request #43

Closed

Multiple end points? #14

Closed

rustycl0ck force-pushed the feature/scrape-multiple-endpoints branch from 915489f to 4ea7ef2 Compare August 4, 2020 06:19

rustycl0ck force-pushed the feature/scrape-multiple-endpoints branch from 4ea7ef2 to c869516 Compare August 4, 2020 06:21

rustycl0ck added 3 commits August 4, 2020 15:37

Add example prometheus config

0b34d68

Signed-off-by: rustyclock <[email protected]>

Migrate logging to promlog

f57d3e9

Signed-off-by: rustyclock <[email protected]>

Fix lint errors

1e1ba27

Signed-off-by: rustyclock <[email protected]>

SuperQ requested review from roidelapluie and SuperQ August 5, 2020 14:49

Switch to kingpin

fa085b2

Signed-off-by: rustyclock <[email protected]>

SuperQ reviewed Aug 6, 2020

View reviewed changes

cmd/main.go Outdated Show resolved Hide resolved

SuperQ reviewed Aug 6, 2020

View reviewed changes

cmd/main.go Outdated Show resolved Hide resolved

rustycl0ck added 2 commits August 6, 2020 20:03

Add HTTP config support

bf7590a

Signed-off-by: rustyclock <[email protected]>

Remove probe_duration metric

4479a45

Signed-off-by: rustyclock <[email protected]>

rustycl0ck added 2 commits August 18, 2020 12:28

Return 503 on connection errors

a9d3155

Signed-off-by: rustyclock <[email protected]>

Use NewConstMetric instead of regular Gauge

a6b9654

Signed-off-by: rustyclock <[email protected]>

rustycl0ck force-pushed the feature/scrape-multiple-endpoints branch from cfb0e53 to a6b9654 Compare August 19, 2020 02:59

Add unit tests

014e2df

Signed-off-by: rustyclock <[email protected]>

SuperQ approved these changes Sep 4, 2020

View reviewed changes

roidelapluie reviewed Sep 23, 2020

View reviewed changes

rustycl0ck added 2 commits October 3, 2020 08:58

Fix review comments

9a50f9d

Signed-off-by: rustyclock <[email protected]>

Change package name from 'internal' to 'exporter'

9f94906

Signed-off-by: rustyclock <[email protected]>

SuperQ reviewed Oct 3, 2020

View reviewed changes

examples/prometheus.yml Outdated Show resolved Hide resolved

SuperQ reviewed Oct 3, 2020

View reviewed changes

examples/prometheus.yml Show resolved Hide resolved

SuperQ reviewed Oct 3, 2020

View reviewed changes

examples/prometheus.yml Outdated Show resolved Hide resolved

SuperQ reviewed Oct 3, 2020

View reviewed changes

examples/prometheus.yml Outdated Show resolved Hide resolved

Update prometheus sample config

a46be32

Signed-off-by: rustyclock <[email protected]>

roidelapluie approved these changes Oct 28, 2020

View reviewed changes

roidelapluie merged commit f76b12d into prometheus-community:master Oct 28, 2020

rustycl0ck deleted the feature/scrape-multiple-endpoints branch October 29, 2020 03:14

kouk mentioned this pull request Nov 2, 2020

Fix unchecked call to io.Copy #57

Merged

This was referenced Nov 2, 2020

Add support for turning off TLS verification for scraped sites #55

Closed

Add support for endpoint with self signed certificate #12

Closed

Add functionality to add headers to the request #30

Closed

This was referenced Nov 3, 2020

Update logging #28

Closed

Switch the cli flags to kingpin #50

Closed

Support http config #33

Closed

SuperQ mentioned this pull request Nov 3, 2020

Allow specifying bind address #34

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Refactor the complete code #49

Refactor the complete code #49

rustycl0ck commented Aug 4, 2020 •

edited

Loading

SuperQ commented Aug 4, 2020

rustycl0ck commented Aug 5, 2020

SuperQ commented Aug 5, 2020

roidelapluie commented Aug 5, 2020

rustycl0ck commented Aug 5, 2020

rustycl0ck commented Aug 6, 2020

rustycl0ck commented Aug 20, 2020 •

edited

Loading

SuperQ left a comment

SuperQ commented Sep 23, 2020

roidelapluie commented Sep 23, 2020

roidelapluie left a comment

roidelapluie Sep 23, 2020

rustycl0ck Oct 3, 2020

SuperQ Oct 3, 2020

rustycl0ck Oct 5, 2020

rustycl0ck commented Oct 3, 2020

rustycl0ck commented Oct 17, 2020

roidelapluie commented Oct 28, 2020

SuperQ commented Oct 28, 2020

rustycl0ck commented Oct 29, 2020

Refactor the complete code #49

Refactor the complete code #49

Conversation

rustycl0ck commented Aug 4, 2020 • edited Loading

SuperQ commented Aug 4, 2020

rustycl0ck commented Aug 5, 2020

SuperQ commented Aug 5, 2020

roidelapluie commented Aug 5, 2020

rustycl0ck commented Aug 5, 2020

rustycl0ck commented Aug 6, 2020

rustycl0ck commented Aug 20, 2020 • edited Loading

SuperQ left a comment

Choose a reason for hiding this comment

SuperQ commented Sep 23, 2020

roidelapluie commented Sep 23, 2020

roidelapluie left a comment

Choose a reason for hiding this comment

roidelapluie Sep 23, 2020

Choose a reason for hiding this comment

rustycl0ck Oct 3, 2020

Choose a reason for hiding this comment

SuperQ Oct 3, 2020

Choose a reason for hiding this comment

rustycl0ck Oct 5, 2020

Choose a reason for hiding this comment

rustycl0ck commented Oct 3, 2020

rustycl0ck commented Oct 17, 2020

roidelapluie commented Oct 28, 2020

SuperQ commented Oct 28, 2020

rustycl0ck commented Oct 29, 2020

rustycl0ck commented Aug 4, 2020 •

edited

Loading

rustycl0ck commented Aug 20, 2020 •

edited

Loading