Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add per-plugin execution time to debug output #479

Closed
yuankui opened this issue Jan 5, 2016 · 8 comments
Closed

Add per-plugin execution time to debug output #479

yuankui opened this issue Jan 5, 2016 · 8 comments

Comments

@yuankui
Copy link

yuankui commented Jan 5, 2016 

2016/01/05 11:03:43 Gathered metrics, ({2s} interval), from 6 plugins in 3.229187695s
2016/01/05 11:03:43 Gathered metrics, ({2s} interval), from 6 plugins in 546.383772ms
2016/01/05 11:03:44 Gathered metrics, ({2s} interval), from 6 plugins in 451.010904ms
2016/01/05 11:03:46 Gathered metrics, ({2s} interval), from 6 plugins in 751.141487ms
2016/01/05 11:03:48 Gathered metrics, ({2s} interval), from 6 plugins in 692.393806ms
2016/01/05 11:03:50 Gathered metrics, ({2s} interval), from 6 plugins in 662.366727ms
2016/01/05 11:03:52 Gathered metrics, ({2s} interval), from 6 plugins in 973.684338ms

here is my config

[tags]
dc = "all"

[agent]
interval = "2s"

# OUTPUTS
[outputs]
[outputs.influxdb]
url = "http://192.168.48.164:8086" # required.
database = "telegraf" # required.
precision = "s"

# PLUGINS
[cpu]
percpu = false
totalcpu = true
pass = ["cpu_usage"]

[mem]
[system]

[net]
pass = ["net_bytes"]
interfaces = ["eth0","em1"]

[swap]

[netstat]
@sparrc
Copy link
Contributor

sparrc commented Jan 5, 2016

It would help if you could be more specific. Try to narrow down which plugin is taking too long, and also specify how long you would expect it to take.

@zstyblik
Copy link
Contributor

zstyblik commented Jan 5, 2016

@sparrc out of curiosity, is there a way to get per-plugin execution times?

@sparrc
Copy link
Contributor

sparrc commented Jan 5, 2016

@zstyblik there isn't, you would need to run the plugins one-by-one

@zstyblik
Copy link
Contributor

zstyblik commented Jan 5, 2016

@sparrc I see :\ Thanks.

@yuankui
Copy link
Author

yuankui commented Jan 6, 2016

Maybe it's good for debug if add some info like per-plugin execution time in the schedule framework?

@sparrc sparrc changed the title Gathered metrics too slow... Add per-plugin execution time to debug output Jan 20, 2016
@sparrc
Copy link
Contributor

sparrc commented Jan 20, 2016

changing this case to just be for adding per-plugin execution time to the debug output

sparrc added a commit that referenced this issue May 21, 2016
@sparrc
Set a timeout for calls to input.Gather
ae54ec2 
Changing the internal behavior around running plugins. Each plugin
will now have it's own goroutine with it's own ticker. This means that a
hung plugin will not block any other plugins. When a plugin is hung, we
will log an error message every interval, letting users know which
plugin is hung.

Currently the input interface does not have any methods for killing a
running Gather call, so there is nothing we can do but log an "ERROR"
and move on. This will give some visibility into the plugin that is
acting up.

closes #1230
fixes #479
sparrc added a commit that referenced this issue May 21, 2016
@sparrc
Set a timeout for calls to input.Gather
df0ddd0 
Changing the internal behavior around running plugins. Each plugin
will now have it's own goroutine with it's own ticker. This means that a
hung plugin will not block any other plugins. When a plugin is hung, we
will log an error message every interval, letting users know which
plugin is hung.

Currently the input interface does not have any methods for killing a
running Gather call, so there is nothing we can do but log an "ERROR"
and move on. This will give some visibility into the plugin that is
acting up.

closes #1230
fixes #479
sparrc added a commit that referenced this issue May 21, 2016
@sparrc
Set a timeout for calls to input.Gather
8dc9fb9 
Changing the internal behavior around running plugins. Each plugin
will now have it's own goroutine with it's own ticker. This means that a
hung plugin will not block any other plugins. When a plugin is hung, we
will log an error message every interval, letting users know which
plugin is hung.

Currently the input interface does not have any methods for killing a
running Gather call, so there is nothing we can do but log an "ERROR"
and move on. This will give some visibility into the plugin that is
acting up.

closes #1230
fixes #479
@sparrc sparrc closed this as completed in 4f5d592 May 23, 2016
@Mikanoshi
Copy link

It also would be useful to have a per-plugin exec time sent to outputs even when debug mode is off to build graphs and monitor it. Maybe also a per-command time in exec plugin.

@sparrc
Copy link
Contributor

sparrc commented Jun 12, 2016

agreed, see #1348

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
4 participants
@yuankui @Mikanoshi @zstyblik @sparrc