Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

redis-rb 5.0 #1120

Merged
merged 25 commits into from
Aug 17, 2022
Merged

redis-rb 5.0 #1120

merged 25 commits into from
Aug 17, 2022

Conversation

byroot
Copy link
Collaborator

@byroot byroot commented Aug 12, 2022

Ref: #1070

Ok, let's start the work to see what kind of roadblocks we'll run into.

One idea I'm toying with right now, is that rather than delegate everything to redis-client which would require Redis 6+, we could use redis-client with theRESP2 protocol and keep all our type casting. The downside is that it doesn't allow to delete as much code, the upside is that we don't break people who are still on Redis 5, and we get the much better drivers of redis-client.

@mperham
Copy link
Contributor

mperham commented Aug 12, 2022

I'd suggest being more aggressive. Why not drop Redis 5? redis-rb 4.x supports Redis 5 just fine, people can continue using it if they don't want to upgrade for some reason. Redis 6.0 is from April 2020 and every major Redis provider supports 6.x.

@byroot
Copy link
Collaborator Author

byroot commented Aug 12, 2022

Why not drop Redis 5?

The problem isn't so much Redis 5, but RESP2. Some people migrated to Redis 6 or 7, but used proxies such as envoyproxy/envoy#17571 which don't yet support the newer protocol. That pains me but...

So if I can support RESP2 in redis-rb 5.0, I can remove all the deprecated stuff and allow these people to upgrade, and in a shorter time I can drop RESP2 support in 6.0.

That's the idea so far, we'll see how it goes. In casperisfine#1 a surprising amount of tests pass with extremely little effort.

@casperisfine casperisfine force-pushed the redis-rb-5.0 branch 2 times, most recently from 1d4c0a3 to 71c2deb Compare August 12, 2022 14:48
byroot added 4 commits August 13, 2022 17:14
Even 5.0 is EOL, and it's the oldest version we test against,
so no point keeping all these version checks.
`main` and `distributed` test suites pass.
`cluster` and `sentinel` still need some updating
Now that we no longer swap the client during pipelining and such,
we can get rid of that mess.
@casperisfine
Copy link

casperisfine commented Aug 16, 2022

Ok, so another status update:

Outstanding issues with cluster:

  • redis-cluster-client requires ruby 2.7, not sure if necessary or not, but that makes testing harder.
  • There's a bunch of properties normally access on the config object that are broken (e.g. client.id). I need to iterate on all the node, that may require some small change in cluster-client.
  • A handful of broken tests around transactions and pipelining, I'll have to dig into them.

Overall I'm quite tempted to extra Redis::Cluster to its own gem (or inside redis-cluster-client?). But not sure how much it would complicate testing. @supercaracal what do you think?

@casperisfine
Copy link

After sleeping on it, I think short term the best is to indeed split it in another gem, but keep it in the same repo so that it's easy to share code.

One issue though is that the gem name is already in use -_- https://rubygems.org/gems/redis-cluster

@casperisfine
Copy link

Ok so:

  • The cluster suite now pass:
    • We have a regression on transactions (multi). Unless I missed something redis-cluster-client outright don't support them, we used to support some of them before. Not sure how big a deal it is.
  • sentinel is still flaky I still need to debug, but I confirmed at least a part of the failures are happening consistently with specific seeds, so I think some tests put the sentinel group in a bad state and make later tests fail.
  • I tested this branch against sidekiq and activesupport test suites, aside from some test only code they have to select the driver, they all pass.

I think I'll bump the version number and merge this this afternoon, further changes can happen on master. I've pushed a 4.x branch in case we want to backport some stuff to 4.x.

@casperisfine
Copy link

Oh, and jruby is flaky too now...

byroot added 3 commits August 17, 2022 13:45
Now that synchrony is gone and hiredis support SSL
we don't need to single out some drivers.
byroot added 2 commits August 17, 2022 15:19
Some threads would stay subscribed to various channel
causing state leak across threads.

It's still not perfect but should be much more stable.
@casperisfine casperisfine force-pushed the redis-rb-5.0 branch 2 times, most recently from f705f90 to 578f752 Compare August 17, 2022 13:22
@byroot byroot marked this pull request as ready for review August 17, 2022 13:37
@byroot
Copy link
Collaborator Author

byroot commented Aug 17, 2022

Aaaand it's green!

Further work will happen on master.

@byroot byroot merged commit 93e5dc8 into redis:master Aug 17, 2022
@supercaracal
Copy link
Contributor

I’m sorry for the late reply. I agree with the decision.

@ZackMattor
Copy link

@byroot I might create a new issue... and let me know if I should, but wanted to see if I was missing anything. We use clustering for our Rails Cache (using the official redis_cache_store adapter in rails), and it seems like this change makes it so there is no easy path forward for us to upgrade to redis > 5... Before you could just pass the cluster parameter to the redis_cache_store adapter and it would work as intended and now it doesn't seem like it can support using Redis::Cluster.

https://github.com/rails/rails/blob/6-0-stable/activesupport/lib/active_support/cache/redis_cache_store.rb#L116-L128

Maybe I should work with the rails project to get clustering support working over there.

@byroot
Copy link
Collaborator Author

byroot commented Aug 2, 2023

Rails RedisCacheStore accept a Redis instance as argument:

ActiveSupport::Cache::RedisCacheStore.new(redis: Redis::Cluster.new(....))

or

config.cache_store = :redis_store, { redis: Redis::Cluster.new(....) }

NB: I didn't test, I may have got some of the syntax partially wrong, but the general idea works.

@ZackMattor
Copy link

ZackMattor commented Aug 2, 2023

Oh I definitely missed that... I'll play around with it. Thanks a lot!

...it was even right there in the method I highlighted 😅

kbrock added a commit to kbrock/discourse that referenced this pull request Apr 18, 2024
Ok.

There was an initializer introduced in discourse#11088

```
Rack::MiniProfiler.counter_method(Redis::Client, :call) { 'redis' }
```

This now throws an error

```
gems/rack-mini-profiler-3.3.1/lib/mini_profiler/profiling_methods.rb:87:in alias_method': undefined method call' for class `Redis::Client'
```

The error states that the method `def call` is no longer defined for Redis::Client`. Which is accurate since that method was dropped in 5.0 via redis/redis-rb@349ddb1 ( redis/redis-rb#1120 )

This change ensures a known redis api
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

6 participants