[TT-10520] Migrating from go-redis to storage library #765

mativm02 · 2024-01-09T13:52:39Z

Description

This PR migrates all the Redis usage to storage/temporal.

Related Issue

https://tyktech.atlassian.net/browse/TT-10520?atlOrigin=eyJpIjoiMmY4YzJkMDA1YTRkNDg4MDlmODA5Y2VhYmJmYjM5ZjYiLCJwIjoiaiJ9

Motivation and Context

How This Has Been Tested

Screenshots (if appropriate)

Types of changes

Bug fix (non-breaking change which fixes an issue)
New feature (non-breaking change which adds functionality)
Breaking change (fix or feature that would cause existing functionality to change)

Checklist

sweep-ai · 2024-01-09T13:53:18Z

Apply Sweep Rules to your PR?

Apply: All new business logic should have corresponding unit tests.
Apply: Refactor large functions to be more modular.
Apply: Add docstrings to all functions and file headers.

buger · 2024-01-09T13:59:29Z

API tests result - postgres15-murmur64 env: success ✅
Branch used: refs/heads/master
Commit: 407c373 TT-10520 Migrating from go-redis to storage library (#765)

migrating from go-redis to storage library
fixing issues
linting
fixing compilation error
linting
linting part 2
adding constructor unit test
Updating storage to v1.1.0
Fixing GetAndDeleteSet method
Modifying errors to 'fatal' type
Running go mod tidy
improving tests
improving tests
linting
removing fatal error
linting
fixing test
adding TLS Options
linting
simplifying code
Improving code readingness
Making model.Connector global variable a singleton
Update dependencies
removing commented code
linting
Triggered by: push (@mativm02)
Execution page

buger · 2024-01-09T13:59:30Z

API tests result - postgres15-sha256 env: success ✅
Branch used: refs/heads/master
Commit: 407c373 TT-10520 Migrating from go-redis to storage library (#765)

migrating from go-redis to storage library
fixing issues
linting
fixing compilation error
linting
linting part 2
adding constructor unit test
Updating storage to v1.1.0
Fixing GetAndDeleteSet method
Modifying errors to 'fatal' type
Running go mod tidy
improving tests
improving tests
linting
removing fatal error
linting
fixing test
adding TLS Options
linting
simplifying code
Improving code readingness
Making model.Connector global variable a singleton
Update dependencies
removing commented code
linting
Triggered by: push (@mativm02)
Execution page

buger · 2024-01-09T13:59:32Z

API tests result - mongo44-sha256 env: success ✅
Branch used: refs/heads/master
Commit: 407c373 TT-10520 Migrating from go-redis to storage library (#765)

migrating from go-redis to storage library
fixing issues
linting
fixing compilation error
linting
linting part 2
adding constructor unit test
Updating storage to v1.1.0
Fixing GetAndDeleteSet method
Modifying errors to 'fatal' type
Running go mod tidy
improving tests
improving tests
linting
removing fatal error
linting
fixing test
adding TLS Options
linting
simplifying code
Improving code readingness
Making model.Connector global variable a singleton
Update dependencies
removing commented code
linting
Triggered by: push (@mativm02)
Execution page

buger · 2024-01-09T13:59:35Z

API tests result - mongo44-murmur64 env: success ✅
Branch used: refs/heads/master
Commit: 407c373 TT-10520 Migrating from go-redis to storage library (#765)

migrating from go-redis to storage library
fixing issues
linting
fixing compilation error
linting
linting part 2
adding constructor unit test
Updating storage to v1.1.0
Fixing GetAndDeleteSet method
Modifying errors to 'fatal' type
Running go mod tidy
improving tests
improving tests
linting
removing fatal error
linting
fixing test
adding TLS Options
linting
simplifying code
Improving code readingness
Making model.Connector global variable a singleton
Update dependencies
removing commented code
linting
Triggered by: push (@mativm02)
Execution page

…10520

sweep-ai · 2024-01-24T14:23:37Z

Resolving merge conflicts: track the progress here.

I'm currently resolving the merge conflicts in this PR. I will stack a new PR once I'm done.

✨ Created Pull Request: #775

mativm02 · 2024-01-24T15:13:17Z

storage/redis_test.go

-	t.Run("Default addresses", func(t *testing.T) {
-		opts := &redis.UniversalOptions{}
-		simpleOpts := opts.Simple()
-
-		if simpleOpts.Addr != "127.0.0.1:6379" {
-			t.Fatal("Wrong default single node address")
-		}
-
-		opts.Addrs = []string{}
-		clusterOpts := opts.Cluster()
-
-		if clusterOpts.Addrs[0] != "127.0.0.1:6379" || len(clusterOpts.Addrs) != 1 {
-			t.Fatal("Wrong default cluster mode address")
-		}
-
-		opts.Addrs = []string{}
-		failoverOpts := opts.Failover()
-
-		if failoverOpts.SentinelAddrs[0] != "127.0.0.1:26379" || len(failoverOpts.SentinelAddrs) != 1 {
-			t.Fatal("Wrong default sentinel mode address")
-		}
-	})


This test has been removed since it was testing Redis library-specific stuff - not related to the storage library

tbuchaillot

@mativm02 let's remove all the "Redis" references from structures and file names. Also double check if Redis disconnect - pump is able to handle the reconnection and monitor the number of Redis connection created ( I don't think it should be a problem with the singleton but just in case)

storage/redis.go

tbuchaillot

@mativm02 please add redis TLS opts and configurations

storage/temporal_storage.go

tbuchaillot

LGTM

mativm02 · 2024-01-26T11:37:20Z

/release to release-1.9

tykbot · 2024-01-26T11:37:36Z

Working on it! Note that it can take a few minutes.

* migrating from go-redis to storage library * fixing issues * linting * fixing compilation error * linting * linting part 2 * adding constructor unit test * Updating storage to v1.1.0 * Fixing GetAndDeleteSet method * Modifying errors to 'fatal' type * Running go mod tidy * improving tests * improving tests * linting * removing fatal error * linting * fixing test * adding TLS Options * linting * simplifying code * Improving code readingness * Making model.Connector global variable a singleton * Update dependencies * removing commented code * linting (cherry picked from commit 407c373)

tykbot · 2024-01-26T11:37:48Z

@mativm02 Succesfully merged PR

… library (#765) (#777) * [TT-10520] Migrating from go-redis to storage library (#765) * migrating from go-redis to storage library * fixing issues * linting * fixing compilation error * linting * linting part 2 * adding constructor unit test * Updating storage to v1.1.0 * Fixing GetAndDeleteSet method * Modifying errors to 'fatal' type * Running go mod tidy * improving tests * improving tests * linting * removing fatal error * linting * fixing test * adding TLS Options * linting * simplifying code * Improving code readingness * Making model.Connector global variable a singleton * Update dependencies * removing commented code * linting (cherry picked from commit 407c373) * Fixing conflicts --------- Co-authored-by: Matias <[email protected]>

migrating from go-redis to storage library

547aaa8

mativm02 added 9 commits January 9, 2024 13:36

fixing issues

192adf2

linting

bc3c969

fixing compilation error

97601c9

linting

524fb01

Merge branch 'master' of github.com:TykTechnologies/tyk-pump into TT-…

4352f70

…10520

linting part 2

c393441

adding constructor unit test

4c4963d

Updating storage to v1.1.0

3714ad9

Fixing GetAndDeleteSet method

feca5d2

sweep-ai bot mentioned this pull request Jan 24, 2024

Sweep: Resolve merge conflicts for PR #765: [TT-10520] Migrating from go-redis to storage library #775

Open

mativm02 and others added 3 commits January 24, 2024 11:26

Modifying errors to 'fatal' type

476061c

Merge branch 'master' into TT-10520

0a51a76

Running go mod tidy

f3db86b

mativm02 commented Jan 24, 2024

View reviewed changes

mativm02 added 3 commits January 24, 2024 12:25

improving tests

920fed3

improving tests

048684c

linting

1cef0ff

tbuchaillot requested changes Jan 25, 2024

View reviewed changes

storage/redis.go Outdated Show resolved Hide resolved

storage/redis.go Outdated Show resolved Hide resolved

storage/redis.go Outdated Show resolved Hide resolved

mativm02 added 3 commits January 25, 2024 09:46

removing fatal error

7d412e0

linting

1c6b11e

fixing test

d0ba49a

mativm02 requested review from tbuchaillot and sredxny January 25, 2024 13:45

tbuchaillot requested changes Jan 25, 2024

View reviewed changes

tbuchaillot reviewed Jan 25, 2024

View reviewed changes

storage/temporal_storage.go Outdated Show resolved Hide resolved

tbuchaillot reviewed Jan 25, 2024

View reviewed changes

storage/temporal_storage.go Outdated Show resolved Hide resolved

mativm02 added 3 commits January 25, 2024 11:01

adding TLS Options

be20d1c

linting

d6c3bbf

simplifying code

c4d6763

mativm02 requested a review from tbuchaillot January 25, 2024 14:15

mativm02 added 5 commits January 25, 2024 13:50

Improving code readingness

8b240d2

Making model.Connector global variable a singleton

b993821

Update dependencies

bb51e9f

removing commented code

f27c763

linting

52d7dfc

tbuchaillot approved these changes Jan 26, 2024

View reviewed changes

mativm02 merged commit 407c373 into master Jan 26, 2024
22 checks passed

mativm02 deleted the TT-10520 branch January 26, 2024 11:37

buger mentioned this pull request Jan 26, 2024

Merging to release-1.9: [TT-10520] Migrating from go-redis to storage library (#765) #777

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[TT-10520] Migrating from go-redis to storage library #765

[TT-10520] Migrating from go-redis to storage library #765

mativm02 commented Jan 9, 2024 •

edited

Loading

sweep-ai bot commented Jan 9, 2024

buger commented Jan 9, 2024 •

edited by jira bot

Loading

buger commented Jan 9, 2024 •

edited by jira bot

Loading

buger commented Jan 9, 2024 •

edited by jira bot

Loading

buger commented Jan 9, 2024 •

edited by jira bot

Loading

sweep-ai bot commented Jan 24, 2024 •

edited

Loading

Resolving merge conflicts: track the progress here.

mativm02 Jan 24, 2024

tbuchaillot left a comment •

edited

Loading

tbuchaillot left a comment

tbuchaillot left a comment

mativm02 commented Jan 26, 2024

tykbot bot commented Jan 26, 2024

tykbot bot commented Jan 26, 2024

[TT-10520] Migrating from go-redis to storage library #765

[TT-10520] Migrating from go-redis to storage library #765

Conversation

mativm02 commented Jan 9, 2024 • edited Loading

Description

Related Issue

Motivation and Context

How This Has Been Tested

Screenshots (if appropriate)

Types of changes

Checklist

sweep-ai bot commented Jan 9, 2024

Apply Sweep Rules to your PR?

buger commented Jan 9, 2024 • edited by jira bot Loading

buger commented Jan 9, 2024 • edited by jira bot Loading

buger commented Jan 9, 2024 • edited by jira bot Loading

buger commented Jan 9, 2024 • edited by jira bot Loading

sweep-ai bot commented Jan 24, 2024 • edited Loading

Resolving merge conflicts: track the progress here.

mativm02 Jan 24, 2024

Choose a reason for hiding this comment

tbuchaillot left a comment • edited Loading

Choose a reason for hiding this comment

tbuchaillot left a comment

Choose a reason for hiding this comment

tbuchaillot left a comment

Choose a reason for hiding this comment

mativm02 commented Jan 26, 2024

tykbot bot commented Jan 26, 2024

tykbot bot commented Jan 26, 2024

mativm02 commented Jan 9, 2024 •

edited

Loading

buger commented Jan 9, 2024 •

edited by jira bot

Loading

buger commented Jan 9, 2024 •

edited by jira bot

Loading

buger commented Jan 9, 2024 •

edited by jira bot

Loading

buger commented Jan 9, 2024 •

edited by jira bot

Loading

sweep-ai bot commented Jan 24, 2024 •

edited

Loading

tbuchaillot left a comment •

edited

Loading