Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Automatic master discovery #113

Closed
Totktonada opened this issue Nov 18, 2021 · 1 comment · Fixed by #148
Closed

Automatic master discovery #113

Totktonada opened this issue Nov 18, 2021 · 1 comment · Fixed by #148
Assignees
@AnaNek
Labels
feature A new functionality

Comments

@Totktonada
Copy link
Member

Totktonada commented Nov 18, 2021 •
edited
Loading

I propose to implement quite simple master discovery algorithm on top of existing connection pool (multi). Key points:

  • Poll box.info.ro on each instance every X seconds.
  • Add mode = 'any' / 'ro' / 'prefer_ro' / 'prefer_rw' / 'rw' parameter for Call, Eval, Execute, Select, Ping, Insert, Replace, Delete, Update, Upsert.
  • Fail fast: if we have no rw instance ATM -> report the fail to a caller.
  • Fail fast: got ER_READ_ONLY -> report it to a caller.

Later we'll reimplement it for fresh tarantool versions using watchers instead of polling. Not in the scope of this issue.

Later we'll design a good retry strategy, but it is okay for now to leave it for a user.

mode parameter may be discussed: proposals backed by arguments (not just opinion) are appreciated.

RFC is here.
@Totktonada Totktonada added feature A new functionality teamE labels Nov 18, 2021
@Totktonada
Copy link
Member Author

Updated the description with a short proposal what to implement.

@Totktonada Totktonada added the 8sp label Dec 13, 2021
@Totktonada Totktonada mentioned this issue Dec 14, 2021
Closed
@Totktonada Totktonada mentioned this issue Dec 24, 2021
Merged
@AnaNek AnaNek mentioned this issue Apr 19, 2022
Merged
AnaNek added a commit that referenced this issue Apr 19, 2022
@AnaNek
connection_pool: implement connection pool with automatic master disc…
976f1f5 
…overy

Main features:

- Check active connection with configurable time interval and on connection fail switch to next in pool.
- Automatic master discovery by `mode` parameter.

Additional options (configurable via `ConnectWithOpts`):

* `CheckTimeout` - time interval to check for connection timeout and try to switch connection

`Mode` parameter:

* `RW` (writteable instance (master)) - the request can only be executed on master.
* `PREFER_RO` (read only instance (replica)) - if there is one, otherwise fallback to a writteable one (master).
* `PREFER_RW` (write only instance (master)) - if there is one, otherwise fallback to a read only one (replica).

Closes #113
AnaNek added a commit that referenced this issue Apr 20, 2022
@AnaNek
connection_pool: implement connection pool with automatic master disc…
d44850b 
…overy

Main features:

- Return available connection from pool according to round-robin strategy.
- Automatic master discovery by `mode` parameter.

Additional options (configurable via `ConnectWithOpts`):

* `CheckTimeout` - time interval to check for connection timeout and try to switch connection

`Mode` parameter:

* `ANY` (use any instance) - the request can be executed on any instance (master or replica).
* `RW` (writeable instance (master)) - the request can only be executed on master.
* `RO` (read only instance (replica)) - the request can only be executed on replica.
* `PREFER_RO` (prefer read only instance (replica)) - if there is one, otherwise fallback to a writeable one (master).
* `PREFER_RW` (prefer write only instance (master)) - if there is one, otherwise fallback to a read only one (replica).

Closes #113
AnaNek added a commit that referenced this issue Apr 21, 2022
@AnaNek
connection_pool: implement connection pool with automatic master disc…
d31ffce 
…overy

Main features:

- Return available connection from pool according to round-robin strategy.
- Automatic master discovery by `mode` parameter.

Additional options (configurable via `ConnectWithOpts`):

* `CheckTimeout` - time interval to check for connection timeout and try to switch connection

`Mode` parameter:

* `ANY` (use any instance) - the request can be executed on any instance (master or replica).
* `RW` (writeable instance (master)) - the request can only be executed on master.
* `RO` (read only instance (replica)) - the request can only be executed on replica.
* `PREFER_RO` (prefer read only instance (replica)) - if there is one, otherwise fallback to a writeable one (master).
* `PREFER_RW` (prefer write only instance (master)) - if there is one, otherwise fallback to a read only one (replica).

Closes #113
AnaNek added a commit that referenced this issue Apr 21, 2022
@AnaNek
connection_pool: implement connection pool with automatic master disc…
f17cf39 
…overy

Main features:

- Return available connection from pool according to round-robin strategy.
- Automatic master discovery by `mode` parameter.

Additional options (configurable via `ConnectWithOpts`):

* `CheckTimeout` - time interval to check for connection timeout and try to switch connection

`Mode` parameter:

* `ANY` (use any instance) - the request can be executed on any instance (master or replica).
* `RW` (writeable instance (master)) - the request can only be executed on master.
* `RO` (read only instance (replica)) - the request can only be executed on replica.
* `PREFER_RO` (prefer read only instance (replica)) - if there is one, otherwise fallback to a writeable one (master).
* `PREFER_RW` (prefer write only instance (master)) - if there is one, otherwise fallback to a read only one (replica).

Closes #113
AnaNek added a commit that referenced this issue Apr 21, 2022
@AnaNek
Implement connection pool with master discovery
d6358d4 
Main features:

- Return available connection from pool according to
  round-robin strategy.
- Automatic master discovery by `mode` parameter.

Additional options (configurable via `ConnectWithOpts`):

* `CheckTimeout` - time interval to check for connection
  timeout and try to switch connection

`Mode` parameter:

* `ANY` (use any instance) - the request can be executed on any
  instance (master or replica).
* `RW` (writeable instance (master)) - the request can only
  be executed on master.
* `RO` (read only instance (replica)) - the request can only
  be executed on replica.
* `PREFER_RO` (prefer read only instance (replica)) - if there is one,
  otherwise fallback to a writeable one (master).
* `PREFER_RW` (prefer write only instance (master)) - if there is one,
  otherwise fallback to a read only one (replica).

Closes #113
AnaNek added a commit that referenced this issue Apr 25, 2022
@AnaNek
connection-pool: implement connection pool with master discovery
e1f2309 
Main features:

- Return available connection from pool according to
  round-robin strategy.
- Automatic master discovery by `mode` parameter.

Additional options (configurable via `ConnectWithOpts`):

* `CheckTimeout` - time interval to check for connection
  timeout and try to switch connection

`Mode` parameter:

* `ANY` (use any instance) - the request can be executed on any
  instance (master or replica).
* `RW` (writeable instance (master)) - the request can only
  be executed on master.
* `RO` (read only instance (replica)) - the request can only
  be executed on replica.
* `PREFER_RO` (prefer read only instance (replica)) - if there is one,
  otherwise fallback to a writeable one (master).
* `PREFER_RW` (prefer write only instance (master)) - if there is one,
  otherwise fallback to a read only one (replica).

Closes #113
AnaNek added a commit that referenced this issue Apr 25, 2022
@AnaNek
connection-pool: implement connection pool with master discovery
7d8e478 
Main features:

- Return available connection from pool according to
  round-robin strategy.
- Automatic master discovery by `mode` parameter.

Additional options (configurable via `ConnectWithOpts`):

* `CheckTimeout` - time interval to check for connection
  timeout and try to switch connection

`Mode` parameter:

* `ANY` (use any instance) - the request can be executed on any
  instance (master or replica).
* `RW` (writeable instance (master)) - the request can only
  be executed on master.
* `RO` (read only instance (replica)) - the request can only
  be executed on replica.
* `PREFER_RO` (prefer read only instance (replica)) - if there is one,
  otherwise fallback to a writeable one (master).
* `PREFER_RW` (prefer write only instance (master)) - if there is one,
  otherwise fallback to a read only one (replica).

Closes #113
AnaNek added a commit that referenced this issue Apr 25, 2022
@AnaNek
connection-pool: implement connection pool with master discovery
571f326 
Main features:

- Return available connection from pool according to
  round-robin strategy.
- Automatic master discovery by `mode` parameter.

Additional options (configurable via `ConnectWithOpts`):

* `CheckTimeout` - time interval to check for connection
  timeout and try to switch connection

`Mode` parameter:

* `ANY` (use any instance) - the request can be executed on any
  instance (master or replica).
* `RW` (writeable instance (master)) - the request can only
  be executed on master.
* `RO` (read only instance (replica)) - the request can only
  be executed on replica.
* `PREFER_RO` (prefer read only instance (replica)) - if there is one,
  otherwise fallback to a writeable one (master).
* `PREFER_RW` (prefer write only instance (master)) - if there is one,
  otherwise fallback to a read only one (replica).

Closes #113
AnaNek added a commit that referenced this issue Apr 25, 2022
@AnaNek
connection-pool: implement connection pool with master discovery
9379fb7 
Main features:

- Return available connection from pool according to
  round-robin strategy.
- Automatic master discovery by `mode` parameter.

Additional options (configurable via `ConnectWithOpts`):

* `CheckTimeout` - time interval to check for connection
  timeout and try to switch connection

`Mode` parameter:

* `ANY` (use any instance) - the request can be executed on any
  instance (master or replica).
* `RW` (writeable instance (master)) - the request can only
  be executed on master.
* `RO` (read only instance (replica)) - the request can only
  be executed on replica.
* `PREFER_RO` (prefer read only instance (replica)) - if there is one,
  otherwise fallback to a writeable one (master).
* `PREFER_RW` (prefer write only instance (master)) - if there is one,
  otherwise fallback to a read only one (replica).

Closes #113
AnaNek added a commit that referenced this issue Apr 25, 2022
@AnaNek
connection-pool: implement connection pool with master discovery
e9b9ba1 
Main features:

- Return available connection from pool according to
  round-robin strategy.
- Automatic master discovery by `mode` parameter.

Additional options (configurable via `ConnectWithOpts`):

* `CheckTimeout` - time interval to check for connection
  timeout and try to switch connection

`Mode` parameter:

* `ANY` (use any instance) - the request can be executed on any
  instance (master or replica).
* `RW` (writeable instance (master)) - the request can only
  be executed on master.
* `RO` (read only instance (replica)) - the request can only
  be executed on replica.
* `PREFER_RO` (prefer read only instance (replica)) - if there is one,
  otherwise fallback to a writeable one (master).
* `PREFER_RW` (prefer write only instance (master)) - if there is one,
  otherwise fallback to a read only one (replica).

Closes #113
oleg-jukovec added a commit that referenced this issue Jun 1, 2022
@oleg-jukovec
Release 1.6.0
c7b0468 
Overview

This release adds a number of features and updates documentation.

Breaking changes

	There are no breaking changes in the release.

New features

	Coveralls support (#149).

	Reusable testing workflow (integration testing with latest Tarantool) (#123).

	Simple CI based on GitHub actions (#114).

	Support UUID type in msgpack (#90).

	Go modules support (#91).

	queue-utube handling (#85).

	Master discovery (#113).

	SQL support (#62).

    Handle everything with `go test` (#115).

	Update API documentation: comments and examples (#123).

Bugfixes

	Reset buffer if its average use size smaller than quater of capacity (#95).

	Fix queue tests (#107).

	Make test case consistent with comments (#105).
oleg-jukovec added a commit that referenced this issue Jun 1, 2022
@oleg-jukovec
Release 1.6.0
7dcaa9a 
Overview

This release adds a number of features and updates documentation.

Breaking changes

	There are no breaking changes in the release.

New features

	Coveralls support (#149).

	Reusable testing workflow (integration testing with latest
	Tarantool) (#123).

	Simple CI based on GitHub actions (#114).

	Support UUID type in msgpack (#90).

	Go modules support (#91).

	queue-utube handling (#85).

	Master discovery (#113).

	SQL support (#62).

    Handle everything with `go test` (#115).

	Update API documentation: comments and examples (#123).

Bugfixes

	Reset buffer if its average use size smaller than quater of
	capacity (#95).

	Fix queue tests (#107).

	Make test case consistent with comments (#105).
oleg-jukovec added a commit that referenced this issue Jun 1, 2022
@oleg-jukovec
Release 1.6.0
b4475b4 
Overview

This release adds a number of features and updates documentation.

Breaking changes

	There are no breaking changes in the release.

New features

	Coveralls support (#149).

	Reusable testing workflow (integration testing with latest
	Tarantool) (#123).

	Simple CI based on GitHub actions (#114).

	Support UUID type in msgpack (#90).

	Go modules support (#91).

	queue-utube handling (#85).

	Master discovery (#113).

	SQL support (#62).

	Handle everything with `go test` (#115).

	Update API documentation: comments and examples (#123).

Bugfixes

	Reset buffer if its average use size smaller than quater of
	capacity (#95).

	Fix queue tests (#107).

	Make test case consistent with comments (#105).
@oleg-jukovec oleg-jukovec mentioned this issue Jun 1, 2022
Merged
oleg-jukovec added a commit that referenced this issue Jun 1, 2022
@oleg-jukovec
Release 1.6.0
4b9a5c3 
Overview

This release adds a number of features and updates documentation.

Breaking changes

	There are no breaking changes in the release.

New features

	Coveralls support (#149).

	Reusable testing workflow (integration testing with latest
	Tarantool) (#112).

	Simple CI based on GitHub actions (#114).

	Support UUID type in msgpack (#90).

	Go modules support (#91).

	queue-utube handling (#85).

	Master discovery (#113).

	SQL support (#62).

	Handle everything with `go test` (#115).

	Update API documentation: comments and examples (#123).

Bugfixes

	Reset buffer if its average use size smaller than quater of
	capacity (#95).

	Fix queue tests (#107).

	Make test case consistent with comments (#105).
oleg-jukovec added a commit that referenced this issue Jun 2, 2022
@oleg-jukovec
Release 1.6.0
6cf656c 
Overview

This release adds a number of features. Also it significantly improves
testing, CI and documentation.

Breaking changes

    There are no breaking changes in the release.

New features

    Support UUID type in msgpack (#90).

    queue-utube handling (#85).

    Master discovery (#113).

    SQL support (#62).

Bugfixes

    Reset buffer if its average use size smaller than quater of
    capacity (#95).

Testing

    Coveralls support (#149).

    Reusable testing workflow (integration testing with latest
    Tarantool) (#112).

    Simple CI based on GitHub actions (#114).

    Handle everything with `go test` (#115).

    Fix queue tests (#107).

    Make test case consistent with comments (#105).

Other

    Go modules support (#91).

    Update API documentation: comments and examples (#123).
oleg-jukovec added a commit that referenced this issue Jun 2, 2022
@oleg-jukovec
Release 1.6.0
a0685c4 
Overview

This release adds a number of features. Also it significantly improves
testing, CI and documentation.

Breaking changes

    There are no breaking changes in the release.

New features

    Support UUID type in msgpack (#90).

    queue-utube handling (#85).

    Master discovery (#113).

    SQL support (#62).

Bugfixes

    Reset buffer if its average use size smaller than quater of
    capacity (#95).

Testing

    Coveralls support (#149).

    Reusable testing workflow (integration testing with latest
    Tarantool) (#112).

    Simple CI based on GitHub actions (#114).

    Handle everything with `go test` (#115).

    Fix queue tests (#107).

    Make test case consistent with comments (#105).

Other

    Go modules support (#91).

    Update API documentation: comments and examples (#123).
@TarantoolBot TarantoolBot mentioned this issue Aug 15, 2022
Closed
oleg-jukovec added a commit to tarantool/doc that referenced this issue Aug 15, 2022
@oleg-jukovec
Update go-tarantool in comparison table (#3058)
01a77a5 
The patch adds context, streams, SQL, push messages and master
discovery support for connection pool to go-tarantool. It also updates
GitHub starts for Go connectors.

Follows up tarantool/go-tarantool#48
Follows up tarantool/go-tarantool#62
Follows up tarantool/go-tarantool#67
Follows up tarantool/go-tarantool#101
Follows up tarantool/go-tarantool#113
@oleg-jukovec oleg-jukovec mentioned this issue Aug 15, 2022
Merged
patiencedaur pushed a commit to tarantool/doc that referenced this issue Aug 15, 2022
@oleg-jukovec
Update go-tarantool in comparison table
83ba625 
Resolves #3094 

The patch adds context, streams, SQL, push messages and master
discovery support for connection pool to go-tarantool. It also updates
GitHub starts for Go connectors.

Follows up tarantool/go-tarantool#48
Follows up tarantool/go-tarantool#62
Follows up tarantool/go-tarantool#67
Follows up tarantool/go-tarantool#101
Follows up tarantool/go-tarantool#113
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@AnaNek AnaNek

Labels
feature A new functionality
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Master discovery tarantool/go-tarantool
2 participants
@Totktonada @AnaNek