Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

crud: introduce tarantool 3 roles #427

Merged

Conversation

DifferentialOrange
Copy link
Member

@DifferentialOrange DifferentialOrange commented Mar 25, 2024

This patch introduce Tarantool 3 roles for crud routers and storages, similar to Cartridge ones. It also introduces testing with Tarantool 3 cluster started from configuration. See commits for more info.

I didn't forget about

  • Tests
  • Changelog
  • Documentation

Closes #415

@DifferentialOrange DifferentialOrange force-pushed the DifferentialOrange/gh-412-3.0-grants branch 2 times, most recently from 0a352fe to 106f2aa Compare March 26, 2024 08:43
@DifferentialOrange DifferentialOrange force-pushed the DifferentialOrange/gh-415-role-for-tarantool3 branch from fc50243 to d080a31 Compare March 26, 2024 08:44
@DifferentialOrange DifferentialOrange force-pushed the DifferentialOrange/gh-412-3.0-grants branch from 106f2aa to ab1a690 Compare March 26, 2024 08:53
@DifferentialOrange DifferentialOrange force-pushed the DifferentialOrange/gh-415-role-for-tarantool3 branch from d080a31 to 97b008d Compare March 26, 2024 09:34
@DifferentialOrange DifferentialOrange force-pushed the DifferentialOrange/gh-412-3.0-grants branch from ab1a690 to e2b8486 Compare March 26, 2024 09:44
@DifferentialOrange DifferentialOrange force-pushed the DifferentialOrange/gh-415-role-for-tarantool3 branch from 97b008d to 483d7e0 Compare March 26, 2024 12:23
@DifferentialOrange DifferentialOrange force-pushed the DifferentialOrange/gh-412-3.0-grants branch from e2b8486 to b21560e Compare March 26, 2024 13:09
@DifferentialOrange DifferentialOrange force-pushed the DifferentialOrange/gh-415-role-for-tarantool3 branch 7 times, most recently from 6b51642 to 91168b4 Compare March 27, 2024 07:22
@DifferentialOrange DifferentialOrange force-pushed the DifferentialOrange/gh-412-3.0-grants branch 2 times, most recently from cfbc6b3 to 195881f Compare March 27, 2024 13:23
@DifferentialOrange DifferentialOrange force-pushed the DifferentialOrange/gh-415-role-for-tarantool3 branch 5 times, most recently from e32dcc9 to 15011d6 Compare March 27, 2024 17:29
@DifferentialOrange
Copy link
Member Author

DifferentialOrange commented Mar 27, 2024

A single time cluster had started with broken vshard and Error getting storage info for <alias>: Connection closed issues. It doesn't seem to be related to crud, our bootstrap scripts or insufficient test retries. I'll explore it later, but it seems that for now we will restart them.

@DifferentialOrange DifferentialOrange force-pushed the DifferentialOrange/gh-415-role-for-tarantool3 branch from 15011d6 to 3f15368 Compare March 27, 2024 17:39
@DifferentialOrange DifferentialOrange marked this pull request as ready for review March 27, 2024 18:04
@DifferentialOrange DifferentialOrange changed the title [WIP] Role for Tarantool 3 crud: introduce tarantool 3 roles Mar 28, 2024
@DifferentialOrange DifferentialOrange force-pushed the DifferentialOrange/gh-415-role-for-tarantool3 branch 4 times, most recently from 8ec877e to 3c6749f Compare March 28, 2024 10:00
@DifferentialOrange DifferentialOrange force-pushed the DifferentialOrange/gh-415-role-for-tarantool3 branch from 3c6749f to cf2dc02 Compare March 29, 2024 11:14
@DifferentialOrange DifferentialOrange force-pushed the DifferentialOrange/gh-412-3.0-grants branch from 195881f to 3a7328b Compare March 29, 2024 14:29
@DifferentialOrange DifferentialOrange force-pushed the DifferentialOrange/gh-415-role-for-tarantool3 branch from cf2dc02 to eb14f0e Compare March 29, 2024 14:29
@DifferentialOrange DifferentialOrange force-pushed the DifferentialOrange/gh-412-3.0-grants branch from 3a7328b to ce65f86 Compare April 1, 2024 11:47
@DifferentialOrange DifferentialOrange force-pushed the DifferentialOrange/gh-415-role-for-tarantool3 branch 2 times, most recently from f37b0c3 to df1ede6 Compare April 1, 2024 13:48
@DifferentialOrange DifferentialOrange force-pushed the DifferentialOrange/gh-412-3.0-grants branch from ce65f86 to 14864f2 Compare April 1, 2024 16:18
Base automatically changed from DifferentialOrange/gh-412-3.0-grants to master April 2, 2024 07:09
@DifferentialOrange DifferentialOrange force-pushed the DifferentialOrange/gh-415-role-for-tarantool3 branch from df1ede6 to ba62a52 Compare April 2, 2024 07:26
After this patch, it is possible to start a Tarantool 3.0 cluster
from the config for tests. It doesn't yet integrated with common test
matrix yet. The tests are basic and shallow since it would be tested
against the whole crud test suite in the next commits of the patchset.

Part of #415
After this patch, all unit and integration tests run also on cluster
started from configuration file.

Part of #415
Sometimes it is possible to get a vshard router with broken connection
to a storage. It seems that existing 5 seconds timeout does not allow
to reconnect to a storage, so we try to increase it.

Part of #415
Logging to file is not captured by luatest, so debugging in case cluster
start had failed is nearly impossible.
Before this patch, it was possible to get a successful storage info
response, while some replicas still could be treated as erroneous
(backoff) by vshard ro calls.
This patch introduce basic crud-router and crud-storage roles for
Tarantool 3. The roles are similar to Cartridge ones.
Roles support Tarantool 3.0.2, Tarantool 3.1.0 and newer due to [1, 2].

This commit makes all existing config tests run with roles enabled
instead of manual bootstrap.

This commit does not yet introduce metrics configuration through roles
config.

[1] tarantool/tarantool#9643
[2] tarantool/tarantool#9649

Part of #415
This patch introduces `roles.crud-router` role configuration through
`roles_cfg`, similar to existing Cartridge clusterwide configuration
support. For now, storages don't have any configuration, so they remain
unchanged.

After this patch, Tarantool 3 roles have all features supported in
Cartridge roles.

Closes #415
@DifferentialOrange DifferentialOrange force-pushed the DifferentialOrange/gh-415-role-for-tarantool3 branch from ba62a52 to c106a7a Compare April 2, 2024 07:55
@DifferentialOrange DifferentialOrange merged commit 8d7cae0 into master Apr 3, 2024
25 checks passed
@DifferentialOrange DifferentialOrange deleted the DifferentialOrange/gh-415-role-for-tarantool3 branch April 3, 2024 11:01
DifferentialOrange added a commit that referenced this pull request Apr 3, 2024
Overview

  This release introduces roles for Tarantool 3 configuration (supported
  for versions 3.0.2, 3.1.0 and newer).

Added
* Asynchronous bootstrap support for storages (#412).
* Tarantool 3 roles for setting up crud routers and storages (#415).
* Ability to configure crud through Tarantool 3 roles
  configuration (#415).

Changed
* Explicitly forbid datetime interval conditions (#373).
* Storage initialization is now asynchronous by default for
  Tarantool 3.0+ (#412).
* Additionally check backoff error on storage info fetch (#427).

Fixed
* Working with datetime conditions in case of non-indexed fields or
  non-iterating indexes (#373).
* Precision loss for decimal conditions in case of non-indexed fields or
  non-iterating indexes (#373).
* Passing errors from storages for merger operations (`crud.select`,
  `crud.pairs`, `readview:select`, `readview:pairs`) (#423).
* Working with `nil` operand conditions in case of non-indexed fields or
  non-iterating indexes (#422).
@DifferentialOrange DifferentialOrange mentioned this pull request Apr 3, 2024
DifferentialOrange added a commit that referenced this pull request Apr 4, 2024
Overview

  This release introduces roles for Tarantool 3 configuration (supported
  for versions 3.0.2, 3.1.0 and newer).

Added
* Asynchronous bootstrap support for storages (#412).
* Tarantool 3 roles for setting up crud routers and storages (#415).
* Ability to configure crud through Tarantool 3 roles
  configuration (#415).

Changed
* Explicitly forbid datetime interval conditions (#373).
* Storage initialization is now asynchronous by default for
  Tarantool 3.0+ (#412).
* Additionally check backoff error on storage info fetch (#427).

Fixed
* Working with datetime conditions in case of non-indexed fields or
  non-iterating indexes (#373).
* Precision loss for decimal conditions in case of non-indexed fields or
  non-iterating indexes (#373).
* Passing errors from storages for merger operations (`crud.select`,
  `crud.pairs`, `readview:select`, `readview:pairs`) (#423).
* Working with `nil` operand conditions in case of non-indexed fields or
  non-iterating indexes (#422).
DifferentialOrange added a commit that referenced this pull request May 20, 2024
Overview

  This release introduces roles for Tarantool 3 configuration (supported
  for versions 3.0.2, 3.1.0 and newer).

Added
* Asynchronous bootstrap support for storages (#412).
* Tarantool 3 roles for setting up crud routers and storages (#415).
* Ability to configure crud through Tarantool 3 roles
  configuration (#415).

Changed
* Explicitly forbid datetime interval conditions (#373).
* Storage initialization is now asynchronous by default for
  Tarantool 3.0+ (#412).
* Additionally check backoff error on storage info fetch (#427).

Fixed
* Working with datetime conditions in case of non-indexed fields or
  non-iterating indexes (#373).
* Precision loss for decimal conditions in case of non-indexed fields or
  non-iterating indexes (#373).
* Passing errors from storages for merger operations (`crud.select`,
  `crud.pairs`, `readview:select`, `readview:pairs`) (#423).
* Working with `nil` operand conditions in case of non-indexed fields or
  non-iterating indexes (#422).
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Add a role for tarantool 3.0
2 participants