Add Redis Sentinel Input Plugin #5978

adamflott · 2019-06-11T22:47:44Z

Fixed PR #5487

Required for all PRs:

Signed CLA.
Associated README.md updated.
Has appropriate unit tests.

adamflott · 2019-06-11T22:54:18Z

If you'd like me to add more tests for the other measurements I can do that, wanted to get an initial review before I spent more time on it.

adamflott · 2019-08-04T13:57:21Z

@danielnelson Any update? We'd like to start pulling my PR changes into mainline

glinton

Thanks

etc/telegraf.conf

plugins/inputs/redis_sentinel/README.md

plugins/inputs/redis_sentinel/redis_sentinel.go

plugins/inputs/redis_sentinel/redis_sentinel_test.go

adamflott · 2019-08-15T00:53:29Z

@glinton will address tomorrow, thanks for the review!

joeymiller · 2019-09-04T17:37:03Z

Any update on merge? Excited to see this work.

adamflott · 2019-09-05T14:47:13Z

Any update on merge? Excited to see this work.

Soon. Personal and work life have taken a priority to this.

Address code review comments

adamflott · 2019-09-13T02:07:15Z

Still trying to figure out why this the tests are failing, the output isn't terribly useful.

	accumulator.go:341: 
			Error Trace:	accumulator.go:341
			            				redis_sentinel_test.go:87
			Error:      	unknown measurement redis_sentinel with tags map[host:redis.net]
			Test:       	TestRedisSentinelParseInfo

danielnelson · 2019-09-13T02:14:20Z

I recommend using testutil.MustMetric and testutil.RequireMetricsEqual for testing metrics instead of the acc.AssertContainsTaggedFields method. The output is a bit more helpful.

Here is an example of use, you can ignore the metric time using the testutil.IgnoreTime:

https://github.com/influxdata/telegraf/blob/master/plugins/processors/pivot/pivot_test.go

Use `uptime_ns` per upstream request and get tests passing

Record all errors with accumulator AddError()

- Add tests against `sentinel {masters,sentinels,replicas}` - Convert `info all` test to use testutil.RequireMetricsEqual - Change has-quorum to int (from bool)

adamflott · 2019-10-04T17:05:44Z

Everything (code, tests, documentation) should be in now, please re-review. I'd love to get this in for 1.13 time

plugins/inputs/redis_sentinel/redis_sentinel_test.go

glinton · 2019-10-04T17:26:19Z

plugins/inputs/redis_sentinel/README.md

+
+  - fields:
+    - config-epoch (int)
+    - down-after-milliseconds (int)


Would down_after_ns be feasable (convert ms to ns as well)?

glinton · 2019-10-04T17:26:19Z

plugins/inputs/redis_sentinel/README.md

+
+  - fields:
+    - config-epoch (int)
+    - down-after-milliseconds (int)


Would down_after_ns be feasable (convert ms to ns as well)?

plugins/inputs/redis_sentinel/README.md

plugins/inputs/redis_sentinel/redis_sentinel.go

Correct README uptime -> uptime_ns

Address a PR suggestion

Per project standards change all metric names to snake case

plugins/inputs/redis_sentinel/redis_sentinel.go

Ensure compatibility with Go < 1.12

adamflott · 2019-10-14T16:34:16Z

Any updates?

The keys coming out did not have their dashes properly replaced with underscores

glinton

Sorry for the delays in re-reviewing the requested changes. It looks like you've addressed them all. Thanks.

adamflott · 2019-12-03T22:03:53Z

Sorry for the delays in re-reviewing the requested changes. It looks like you've addressed them all. Thanks.

Can I assume this plugin will not be a part of 1.13?

danielnelson · 2019-12-03T22:20:26Z

Yes, sorry. Let's put it down for 1.14 though for sure.

adamflott · 2020-01-25T13:45:42Z

@danielnelson ready to merge? Don't wanna miss 1.14 :)

danielnelson · 2020-01-28T07:30:36Z

The Influxdata team is team building out in Phoenix this week, but I'll try to review on Friday once I'm back home.

danielnelson · 2020-02-03T05:13:38Z

plugins/inputs/redis_sentinel/redis_sentinel.go

+
+const measurementMasters = "redis_sentinel_masters"
+const measurementSentinel = "redis_sentinel"
+const measurementSentinels = "redis_sentinels"


redis_sentinel_sentinels

danielnelson · 2020-02-03T05:13:50Z

plugins/inputs/redis_sentinel/redis_sentinel.go

+const measurementMasters = "redis_sentinel_masters"
+const measurementSentinel = "redis_sentinel"
+const measurementSentinels = "redis_sentinels"
+const measurementReplicas = "redis_replicas"


redis_sentinel_replicas

danielnelson · 2020-02-03T05:17:15Z

plugins/inputs/redis_sentinel/redis_sentinel.go

+func (r *RedisSentinel) init(acc telegraf.Accumulator) error {
+	if r.initialized {
+		return nil
+	}


Rename this function func (r *RedisSentinel) Init() error {, you can remove the initialized boolean.

danielnelson · 2020-02-03T05:19:40Z

plugins/inputs/redis_sentinel/redis_sentinel.go

+
+	for i, serv := range r.Servers {
+		if !strings.HasPrefix(serv, "tcp://") && !strings.HasPrefix(serv, "unix://") {
+			log.Printf("W! [inputs.redis_sentinel]: server URL found without scheme; please update your configuration file")


Don't add in this support for addresses with a scheme, this is in the normal redis plugin only for backwards compatibility. If the address doesn't begin with tcp or unix return an error instead.

danielnelson · 2020-02-03T05:21:51Z

plugins/inputs/redis_sentinel/redis_sentinel.go

+
+		u, err := url.Parse(serv)
+		if err != nil {
+			acc.AddError(fmt.Errorf("Unable to parse to address %q: %v", serv, err))


Return an error here, which will cause the failure to be fatal. Since we are removing the warning on line 106, do this first and it will ease checking the scheme.

danielnelson · 2020-02-03T06:28:51Z

plugins/inputs/redis_sentinel/README.md

+  - fields:
+    - down_after_milliseconds (int)
+    - flags (string)
+    - ip (string)


Does ip differ from sentinel_ip? I think we should skip it as a field.

danielnelson · 2020-02-03T06:31:12Z

plugins/inputs/redis_sentinel/README.md

+    - link_pending_commands (int)
+    - link_refcount (int)
+    - name (string)
+    - port (int)


Is port the same as sentinel_port? If so let's skip this as well.

danielnelson · 2020-02-03T06:31:45Z

plugins/inputs/redis_sentinel/README.md

+    - total_connections_received (int)
+    - total_net_input_bytes (int)
+    - total_net_output_bytes (int)
+    - uptime_ns (int, seconds)


List units as nanoseconds.

danielnelson · 2020-02-03T06:32:38Z

plugins/inputs/redis_sentinel/README.md

+    - replica_port
+    - source
+
+  - fields:


Like with the sentinels, I think we should remove the port, ip, and runid fields.

danielnelson · 2020-02-03T06:38:43Z

plugins/inputs/redis_sentinel/redis_sentinel.go

+				// ------------------------------------------------------------
+
+				// Check other Replicas
+				replicasCmd := redis.NewSliceCmd("sentinel", "replicas", m["name"])


I found that in versions of Redis < 5, this command is called sentinel slaves <master-group-name>. If we make this call with the older name, the plugin will work with at least back to Redis 3.0, and it continues to work current version.

sjwang90 · 2020-02-25T00:58:07Z

Hi @adamflott, if you have a chance to take a look at @danielnelson's comments we are seeing if we can get this in to 1.14. Thanks!

sjwang90 · 2020-03-09T18:53:36Z

Hey @adamflott! Just a heads up we're a week out from finalizing our 1.14 release candidate if you have time to finish this plugin. We can help wrap things up as well if we have time. Just let us know, thanks!

debu99 · 2020-06-14T14:49:45Z

any progress?

sjwang90 · 2021-01-12T18:13:06Z

There's a last few fixes for this plugin @adamflott. If you'd like anyone to take this over just let us know.

This plugin can also be used faster as an external plugin that can be used with execd to run seamlessly with Telegraf. Telegraf users would be able to use the plugin sooner by having it as an external plugin since it wouldn't have to go through the typical review process.

adamflott · 2021-01-13T15:18:10Z

@sjwang90 I have dropped the ball on this, but would like to finish it out since my project that uses this will finally be going live. Give me a few weeks to get the code changes in.

srebhan · 2021-04-28T12:51:58Z

Hey @adamflott any news on this?

adamflott · 2021-05-06T14:32:16Z

Hey @adamflott any news on this?

Due to my $work no longer taking interest in Redis, I too have lost interest.

sjwang90 · 2021-05-07T17:49:54Z

Thanks for letting us know @adamflott. We'll see if anyone else in the community is willing to take over your PR

spideyfusion · 2021-11-02T09:37:26Z

Continues in #10042

sjwang90 · 2021-11-02T15:37:39Z

Thanks @spideyfusion! Going to close this PR.

Add Redis Sentinel Input Plugin

08ed64f

Fixed PR influxdata#5487

danielnelson added area/redis new plugin labels Jun 12, 2019

adamflott mentioned this pull request Aug 13, 2019

Capture and parse the cmdstat timings from INFO ALL command #5926

Merged

3 tasks

glinton suggested changes Aug 14, 2019

View reviewed changes

glinton assigned adamflott Aug 28, 2019

adamflott added 2 commits September 12, 2019 21:52

Add Redis Sentinel Input Plugin influxdata#5978

367c2f8

Address code review comments

Merge remote-tracking branch 'origin/master' into redis-sentinel

ef6cfa6

Adam Flott and others added 4 commits September 24, 2019 11:28

Add Redis Sentinel Input Plugin influxdata#5978

89ade4d

Use `uptime_ns` per upstream request and get tests passing

Add Redis Sentinel Input Plugin influxdata#5978

57c811e

Record all errors with accumulator AddError()

Merge remote-tracking branch 'origin/master' into redis-sentinel

052467f

Add Redis Sentinel Input Plugin influxdata#5978

9a68a35

- Add tests against `sentinel {masters,sentinels,replicas}` - Convert `info all` test to use testutil.RequireMetricsEqual - Change has-quorum to int (from bool)

glinton reviewed Oct 4, 2019

View reviewed changes

adamflott added 3 commits October 4, 2019 14:52

Add Redis Sentinel Input Plugin influxdata#5978

b45f324

Correct README uptime -> uptime_ns

Add Redis Sentinel Input Plugin influxdata#5978

9f23e16

Address a PR suggestion

Add Redis Sentinel Input Plugin influxdata#5978

7b31e2a

Per project standards change all metric names to snake case

adamflott requested a review from glinton October 8, 2019 19:12

glinton reviewed Oct 8, 2019

View reviewed changes

plugins/inputs/redis_sentinel/redis_sentinel.go Outdated Show resolved Hide resolved

Add Redis Sentinel Input Plugin influxdata#5978

ec816a8

Ensure compatibility with Go < 1.12

Add Redis Sentinel Input Plugin influxdata#5978

dbc023c

The keys coming out did not have their dashes properly replaced with underscores

glinton approved these changes Nov 5, 2019

View reviewed changes

danielnelson added this to the 1.14.0 milestone Dec 3, 2019

danielnelson suggested changes Feb 3, 2020

View reviewed changes

sjwang90 removed this from the 1.14.0 milestone Mar 13, 2020

sjwang90 added external plugin Plugins that would be ideal external plugin and expedite being able to use plugin w/ Telegraf plugin/input 1. Request for new input plugins 2. Issues/PRs that are related to input plugins labels Jan 12, 2021

srebhan self-assigned this Feb 19, 2021

srebhan added the waiting for response waiting for response from contributor label Apr 28, 2021

sjwang90 mentioned this pull request May 7, 2021

redis plugin sentinel support #5487

Closed

srebhan added help wanted Request for community participation, code, contribution and removed waiting for response waiting for response from contributor labels May 18, 2021

spideyfusion mentioned this pull request Nov 2, 2021

feat: Add Redis Sentinel input plugin #10042

Merged

3 tasks

sjwang90 closed this Nov 2, 2021

Add Redis Sentinel Input Plugin #5978

Add Redis Sentinel Input Plugin #5978

Conversation

adamflott commented Jun 11, 2019

Required for all PRs:

adamflott commented Jun 11, 2019

adamflott commented Aug 4, 2019

glinton left a comment

Choose a reason for hiding this comment

adamflott commented Aug 15, 2019

joeymiller commented Sep 4, 2019

adamflott commented Sep 5, 2019

adamflott commented Sep 13, 2019 • edited Loading

danielnelson commented Sep 13, 2019

adamflott commented Oct 4, 2019

Choose a reason for hiding this comment

Choose a reason for hiding this comment

adamflott commented Oct 14, 2019

glinton left a comment

Choose a reason for hiding this comment

adamflott commented Dec 3, 2019

danielnelson commented Dec 3, 2019

adamflott commented Jan 25, 2020

danielnelson commented Jan 28, 2020

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

sjwang90 commented Feb 25, 2020

sjwang90 commented Mar 9, 2020

debu99 commented Jun 14, 2020

sjwang90 commented Jan 12, 2021

adamflott commented Jan 13, 2021

srebhan commented Apr 28, 2021

adamflott commented May 6, 2021

sjwang90 commented May 7, 2021

spideyfusion commented Nov 2, 2021

sjwang90 commented Nov 2, 2021

adamflott commented Sep 13, 2019 •

edited

Loading