Input plugin 2673 traefik #3798

mkboudreau · 2018-02-16T22:17:34Z

Required for all PRs:

Signed CLA.
Associated README.md updated.
Has appropriate unit tests.

This submission is a continuance of #2673

* traefik/traefik: Update all.go clean icinga2 files add missing comma fix icinga2 test change icinga2 readme title update title Update README.md add traefik to README inputs section traefik readme traefik input plugin traefik input plugin fix test update readme icinga2 plugin unit test filter icinga2 objects to query add icinga2 to readme implement telegraf interface functions init icinga2 input plugin

jkowalleck · 2018-04-08T11:22:13Z

plugins/inputs/traefik/traefik.go

+	json.NewDecoder(resp.Body).Decode(&healthData)
+	t.lastRequestTiming = time.Since(start).Seconds()
+
+	tags := map[string]string{"server": t.Address}


since there is no option for credentials in the plugin config, i guess they need to be entered in the address field like "http://user:pass@host:port" (if so: missing config comment).
does this mean the credentials are exposed in the "server" tag ?

I think we'll get this merged and it can be extended later to support basic authentication.

jkowalleck · 2018-04-08T11:22:44Z

plugins/inputs/traefik/traefik.go

+type HttpCodes map[string]int
+
+var sampleConfig = `
+# Required Traefik server address, host and port (default: "127.0.0.1")


how to add credentials?

Great question. I took over this pull request to help drive it through completion. Adding credentials was not part of the original PR and not really needed to get through the initial completion of this PR. Obviously there are people out there who may want to secure this endpoint, so that functionality should probably be added. However, since the most common use case for this kind of metric data is probably unauthenticated, it makes sense to push this through as is and submit another PR with the added functionality.

Would you want to work on adding the ability to add credentials?

Traefik supports several methods for authentication: https://docs.traefik.io/configuration/entrypoints/. If we add support for this (and I agree it does not need to be in the initial PR) we should do it in a way that we could configure any of the methods even if we only do HTTP Basic Auth initially.

So, if you do work on this don't use the userinfo section of the URL, instead add options like https://github.com/influxdata/telegraf/blob/master/plugins/inputs/http/http.go#L58-L60

@mkboudreau One thing I would like to add initially though is support for TLS. Should be pretty easy we just need to mirror the code in the http input: https://github.com/influxdata/telegraf/blob/master/plugins/inputs/apache/apache.go#L100-L115

@mkboudreau wrote

Would you want to work on adding the ability to add credentials?

sorry, i dont know GO, so i think i would not be helpfull.
i tried to read the mongodb input plugin code to see how they strip credentials from the server... was not able to understand it at all ;-)

@jkowalleck Are you interested in HTTP Basic Auth then? I could add this in once it's merged pretty easily.

@danielnelson Since traefik supports BASIC and DIGEST i suggest having both available.
see https://docs.traefik.io/configuration/entrypoints/

my current setup uses BASIC auth.
currently using httpjson input plugin as following:

[[inputs.httpjson]] ## traefik # ## NOTE This plugin only reads numerical measurements, strings and booleans # ## will be ignored. # ## Name for the service being polled. Will be appended to the name of the # ## measurement e.g. httpjson_webserver_stats # ## # ## Deprecated (1.3.0): Use name_override, name_suffix, name_prefix instead. # name = "webserver_stats" name_override = "traefik" # overwrite target - default target is "httpjson" # ## URL of each server in the service's cluster # ## !!!! DO NOT ADD auth here - it might(will) be written to the (influx) # ## output plugin as a tag servers = ["http://traefik:57475/health"] # ## Set response_timeout (default 5 seconds) # response_timeout = "5s" # ## HTTP method to use: GET or POST (case-sensitive) # method = "GET" # ## HTTP Headers (all values must be strings) [inputs.httpjson.headers] authorization = "Basic c3RhdHM6c3RhdHM="

@jkowalleck I'll add support for Basic Auth after we merge, might take a bit longer to get digest. It will probably look more like the http input (successor to httpjson in 1.6).

@mkboudreau One thing I would like to add initially though is support for TLS. Should be pretty easy we just need to mirror the code in the http input: https://github.com/influxdata/telegraf/blob/master/plugins/inputs/apache/apache.go#L100-L115

@danielnelson I should be able to update and add TLS support. Might have to wait until at least next week.

deric · 2018-05-29T14:05:10Z

Any update on this PR?

mkboudreau · 2018-05-30T21:01:42Z

I was planning on doing it weeks ago, but just have not had a moment to spare. Still hoping to add in the requested changes very soon unless someone beats me to it.

* master: (329 commits) Fix panic with unicode counter names in win_perf_counters (influxdata#4255) Update go-syslog version Update changelog Update tengine docs Restore tengine input plugin (influxdata#4160) Fix TLS and SSL config option parsing (influxdata#4247) Update changelog Use same flags for all bsd family ping varients (influxdata#4241) Ignore more boring filesystems from disk plugin (influxdata#4244) Update changelog Add SSL/TLS support to Redis input (influxdata#4236) Don't skip metrics during startup in aggregate phase (influxdata#4230) Set 1.6.4 release date Update master version to 1.8 Update sample config Add go-syslog to dependencies licenses list Update changelog Revert "Update aerospike-client-go version to latest release (influxdata#4128)" Update changelog Fix misnamed option in varnish sample config ...

glinton · 2018-09-10T23:39:23Z

@mkboudreau do you think you'll have time soon to address the latest feedback?

mkboudreau · 2018-09-11T14:22:19Z

@glinton sorry greg, for whatever reason I didn't realize there was anything being waited on. I'll take care of those changes right away.

glinton

I just remembered we're trying to standardize that tag. Thanks

plugins/inputs/traefik/README.md

glinton · 2018-09-26T00:10:42Z

plugins/inputs/traefik/traefik.go

+	json.NewDecoder(resp.Body).Decode(&healthData)
+	t.lastRequestTiming = time.Since(start).Seconds()
+
+	tags := map[string]string{"server": t.Address}


Lets call the tag source (rather than server (4413)) and only use the parsed url's .Hostname() as the value. Also, save this as a variable on the struct so it doesn't have to be computed each call to Gather.

…luxdata#4413)

kerams · 2019-03-16T17:57:43Z

Are stats split by frontend/backend something that can be added or does Traefik not expose those at all?

mkboudreau · 2019-07-29T22:48:57Z

@danielnelson I haven't checked up on this in a while. I assumed it'd already be merged. Any idea what is preventing this PR from being merged?

danielnelson · 2019-08-06T05:58:33Z

plugins/inputs/traefik/traefik.go

+	json.NewDecoder(resp.Body).Decode(&healthData)
+	t.lastRequestTiming = time.Since(start).Seconds()
+
+	tags := map[string]string{"source": t.Address}


The source tag should only contain the hostname.

danielnelson · 2019-08-06T06:07:21Z

plugins/inputs/traefik/traefik.go

+
+	healthData := &HealthCheck{}
+	json.NewDecoder(resp.Body).Decode(&healthData)
+	t.lastRequestTiming = time.Since(start).Seconds()


Normally we add timing information like this as a selfstat (enabled by turning on the internal plugin). You can see some examples of these in the influxdb_listener plugin. It may not be worth adding though since I imagine it will be quite close to the built in gather_time_ns value.

danielnelson · 2019-08-06T06:12:24Z

plugins/inputs/traefik/README.md

+- `traefik` measurement have the following fields:
+    - **total_count** (int) - total number of responses since the application was last restarted
+    - **average_response_time_sec** (float64) - average time (in seconds) of all responses
+    - **total_response_time_sec** (float64) - total time (in seconds) of all responses


For the duration types, we are trying to standardize on reporting as an integer in nanoseconds with the suffix _ns.

danielnelson · 2019-08-06T06:15:26Z

plugins/inputs/traefik/README.md

+    - **total_count** (int) - total number of responses since the application was last restarted
+    - **average_response_time_sec** (float64) - average time (in seconds) of all responses
+    - **total_response_time_sec** (float64) - total time (in seconds) of all responses
+    - **unixtime** (int) - unix timestamp of when this service was last restarted


Is this description accurate? Traefik's docs say // current server date in seconds.

At any rate, for times we are standardizing on saving them as integers in nanoseconds since the epoch. I don't think we need to call it unixtime, something descriptive of the correct definition should work: server_time or ``startup_time`

danielnelson · 2019-08-06T06:15:45Z

plugins/inputs/traefik/README.md

+    - **average_response_time_sec** (float64) - average time (in seconds) of all responses
+    - **total_response_time_sec** (float64) - total time (in seconds) of all responses
+    - **unixtime** (int) - unix timestamp of when this service was last restarted
+    - **uptime_sec** (int) - elapsed time (in seconds) since this service was restarted


uptime_ns

danielnelson · 2019-08-06T06:18:37Z

plugins/inputs/traefik/README.md

+  # address = "http://127.0.0.1:8080"
+  #
+  # [inputs.traefik.tags]
+  #  instance = "prod"


Remove the tags usage example, we don't usually show it even though it can, of course, be set on all plugins. Just showing the default config will be enough.

danielnelson · 2019-08-06T06:26:46Z

plugins/inputs/traefik/README.md

+	- **status_code_500** (int)
+	- *status_code_*...
+
+- `traefik_status_codes` measurement have the following fields:


The cardinality of this measurement isn't going to be much of a problem, in almost all cases we are talking several dozen codes. Even 1000 series wouldn't really be a problem, so I wouldn't make it configurable, keep it simple and always send the status code metrics.

So with that in mind, and given the shared data between the two measurements, I think a single measurement would work okay if we did it similar to:

traefik,source=localhost total_count=42i,average_response_time_ns=42i traefik,source=localhost,status_code=400 status_code_count=42i traefik,source=localhost,status_code=500 status_code_count=42i

If we do it like this, I don't mind having them all under the same measurement because there is no overlap between the field names. However if you prefer, two measurements are still okay but we should keep the shared fields on the primary measurement only:

traefik,source=localhost total_count=42i,average_response_time_ns=42i traefik_status_code,source=localhost,status_code=400 count=42i traefik_status_code,source=localhost,status_code=500 count=42i

danielnelson · 2019-08-06T06:28:06Z

plugins/inputs/traefik/traefik.go

+func init() {
+	inputs.Add("traefik", func() telegraf.Input {
+		return &Traefik{
+			Address: "127.0.0.1:8080",


I believe this address needs to be a URL: http://127.0.0.1:8080

ssoroka · 2020-02-27T16:47:49Z

I took a look at what it would take to get this up to date and merged in to the project. Digging into this I discovered there's a better option. Traefik now supports exporting data to InfluxDB directly, with many more stats, and we can even pipe that through Telegraf if we want, using the influxdb_listener plugin.

For Example:

Telegraf config:

[[inputs.influxdb_listener]]
  service_address = ":8186"
  read_timeout = "10s"
  write_timeout = "10s"
  max_body_size = "500MiB"
  max_line_size = "64KiB"

Traefik config:

traefik --metrics.influxdb=true \
  --metrics.influxdb.address=telegraf.server.location:8186 \
  --metrics.influxdb.protocol=http

Going to close this now as there's nothing else to do.

mlabouardy and others added 21 commits April 12, 2017 16:22

init icinga2 input plugin

5311efa

implement telegraf interface functions

c139f41

add icinga2 to readme

f04cd31

filter icinga2 objects to query

fd19010

icinga2 plugin unit test

0c0bd2f

update readme

b064c99

fix test

6cee6cc

traefik input plugin

033fd8c

traefik input plugin

1f91758

traefik readme

41c1626

add traefik to README inputs section

cfca856

Update README.md

a27fb1f

update title

72a6eb2

change icinga2 readme title

0b07143

fix icinga2 test

b43e0f4

add missing comma

912ff96

Merge branch 'master' of github.com:mlabouardy/telegraf

da158c3

clean icinga2 files

b2739c9

Update all.go

63b96d4

Complete traefik input plugin along with enhancements

97d015c

danielnelson added the new plugin label Mar 5, 2018

danielnelson added this to the 1.7.0 milestone Mar 5, 2018

jkowalleck reviewed Apr 8, 2018

View reviewed changes

danielnelson modified the milestones: 1.7.0, 1.8.0 Jun 3, 2018

danielnelson modified the milestones: 1.8.0, 1.9.0 Sep 6, 2018

Add newline between functions and adjust documentation

5647b12

glinton approved these changes Sep 11, 2018

View reviewed changes

glinton requested a review from danielnelson September 11, 2018 16:28

glinton suggested changes Sep 26, 2018

View reviewed changes

mkboudreau added 2 commits October 3, 2018 08:00

Change response tag 'server' to 'source' in traefik input plugin (inf…

0615ea8

…luxdata#4413)

Reformat with go 1.9 to pass automated testsuite

112a3a7

glinton approved these changes Oct 4, 2018

View reviewed changes

danielnelson modified the milestones: 1.9.0, 1.10 Oct 29, 2018

russorat modified the milestones: 1.10.0, 1.11.0 Jan 14, 2019

danielnelson modified the milestones: 1.11.0, 1.12.0 May 24, 2019

danielnelson suggested changes Aug 6, 2019

View reviewed changes

glinton assigned mkboudreau Aug 6, 2019

danielnelson modified the milestones: 1.12.0, 1.13.0 Aug 20, 2019

sjwang90 modified the milestones: 1.13.0, 1.14.0 Nov 15, 2019

sjwang90 added this to the 1.14.0 milestone Jan 13, 2020

sjwang90 assigned ssoroka Feb 24, 2020

ssoroka closed this Feb 27, 2020

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Input plugin 2673 traefik #3798

Input plugin 2673 traefik #3798

mkboudreau commented Feb 16, 2018

jkowalleck Apr 8, 2018

glinton Aug 14, 2018

jkowalleck Apr 8, 2018

mkboudreau Apr 9, 2018

danielnelson Apr 9, 2018

jkowalleck Apr 9, 2018 •

edited

Loading

danielnelson Apr 9, 2018

jkowalleck Apr 10, 2018 •

edited

Loading

danielnelson Apr 11, 2018

mkboudreau Apr 11, 2018

deric commented May 29, 2018

mkboudreau commented May 30, 2018

glinton commented Sep 10, 2018

mkboudreau commented Sep 11, 2018

glinton left a comment

glinton Sep 26, 2018

kerams commented Mar 16, 2019

mkboudreau commented Jul 29, 2019

danielnelson Aug 6, 2019

danielnelson Aug 6, 2019

danielnelson Aug 6, 2019

danielnelson Aug 6, 2019

danielnelson Aug 6, 2019

danielnelson Aug 6, 2019

danielnelson Aug 6, 2019

danielnelson Aug 6, 2019

ssoroka commented Feb 27, 2020

Input plugin 2673 traefik #3798

Input plugin 2673 traefik #3798

Conversation

mkboudreau commented Feb 16, 2018

Required for all PRs:

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

jkowalleck Apr 9, 2018 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

jkowalleck Apr 10, 2018 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

deric commented May 29, 2018

mkboudreau commented May 30, 2018

glinton commented Sep 10, 2018

mkboudreau commented Sep 11, 2018

glinton left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

kerams commented Mar 16, 2019

mkboudreau commented Jul 29, 2019

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

ssoroka commented Feb 27, 2020

jkowalleck Apr 9, 2018 •

edited

Loading

jkowalleck Apr 10, 2018 •

edited

Loading