-
Notifications
You must be signed in to change notification settings - Fork 14
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Fix plan in case of select operation with custom sharding key #213
Labels
bug
Something isn't working
Comments
3 tasks
ligurio
added a commit
that referenced
this issue
Sep 16, 2021
Previously there were two different ways to obtain bucket id in CRUD: - calculate bucket id automatically using primary key (default) - pass it from outside explicitly in options on CRUD operation call Users who uses DDL module [1] may specify sharding key (that are actually names of tuple fields), but it was not possible to use DDL sharding key for bucket id calculation. Now CRUD allows to use that custom sharding key to calculate bucket id, it will be done automatically when used DDL schema with non-empty sharding_key [1] or when space _ddl_sharding_key contains a tuple with space name and it's sharding key. Table below describe what operations supports custom sharding key: | CRUD method | Added sharding key support | | ---------------------------- | -------------------------- | | get() | Yes | | insert() / insert_object() | Yes | | delete() | Yes | | replace() / replace_object() | Yes | | upsert() / upsert_object() | Yes | | select() / pairs() | Yes | | update() | Yes | | upsert() / upsert_object() | Yes | | replace() / replace_object() | Yes | | min() / max() | No (not required) | | cut_rows() / cut_objects() | No (not required) | | truncate() | No (not required) | | len() | No (not required) | Limitations: - It's not possible to update sharding keys automatically when schema is updated on storages, see [2]. However it is possible to do it manually with sharding_key.update_sharding_keys_cache(). - CRUD select may lead map reduce in some cases, see [3]. 1. https://github.com/tarantool/ddl 2. #212 3. #213 Closes #166
ligurio
added a commit
that referenced
this issue
Sep 16, 2021
Previously there were two different ways to obtain bucket id in CRUD: - calculate bucket id automatically using primary key (default) - pass it from outside explicitly in options on CRUD operation call Users who uses DDL module [1] may specify sharding key (that are actually names of tuple fields), but it was not possible to use DDL sharding key for bucket id calculation. Now CRUD allows to use that custom sharding key to calculate bucket id, it will be done automatically when used DDL schema with non-empty sharding_key [1] or when space _ddl_sharding_key contains a tuple with space name and it's sharding key. Table below describe what operations supports custom sharding key: | CRUD method | Added sharding key support | | ---------------------------- | -------------------------- | | get() | Yes | | insert() / insert_object() | Yes | | delete() | Yes | | replace() / replace_object() | Yes | | upsert() / upsert_object() | Yes | | select() / pairs() | Yes | | update() | Yes | | upsert() / upsert_object() | Yes | | replace() / replace_object() | Yes | | min() / max() | No (not required) | | cut_rows() / cut_objects() | No (not required) | | truncate() | No (not required) | | len() | No (not required) | Limitations: - It's not possible to update sharding keys automatically when schema is updated on storages, see [2]. However it is possible to do it manually with sharding_key.update_sharding_keys_cache(). - CRUD select may lead map reduce in some cases, see [3]. 1. https://github.com/tarantool/ddl 2. #212 3. #213 Closes #166
ligurio
added a commit
that referenced
this issue
Sep 17, 2021
Previously there were two different ways to obtain bucket id in CRUD: - calculate bucket id automatically using primary key (default) - pass it from outside explicitly in options on CRUD operation call Users who uses DDL module [1] may specify sharding key (that are actually names of tuple fields), but it was not possible to use DDL sharding key for bucket id calculation. Now CRUD allows to use that custom sharding key to calculate bucket id, it will be done automatically when used DDL schema with non-empty sharding_key [1] or when space _ddl_sharding_key contains a tuple with space name and it's sharding key. Table below describe what operations supports custom sharding key: | CRUD method | Added sharding key support | | ---------------------------- | -------------------------- | | get() | Yes | | insert() / insert_object() | Yes | | delete() | Yes | | replace() / replace_object() | Yes | | upsert() / upsert_object() | Yes | | select() / pairs() | Yes | | update() | Yes | | upsert() / upsert_object() | Yes | | replace() / replace_object() | Yes | | min() / max() | No (not required) | | cut_rows() / cut_objects() | No (not required) | | truncate() | No (not required) | | len() | No (not required) | Limitations: - It's not possible to update sharding keys automatically when schema is updated on storages, see [2]. However it is possible to do it manually with sharding_key.update_sharding_keys_cache(). - CRUD select may lead map reduce in some cases, see [3]. 1. https://github.com/tarantool/ddl 2. #212 3. #213 Closes #166
ligurio
added a commit
that referenced
this issue
Sep 20, 2021
Previously there were two different ways to obtain bucket id in CRUD: - calculate bucket id automatically using primary key (default) - pass it from outside explicitly in options on CRUD operation call Users who uses DDL module [1] may specify sharding key (that are actually names of tuple fields), but it was not possible to use DDL sharding key for bucket id calculation. Now CRUD allows to use that custom sharding key to calculate bucket id, it will be done automatically when used DDL schema with non-empty sharding_key [1] or when space _ddl_sharding_key contains a tuple with space name and it's sharding key. Table below describe what operations supports custom sharding key: | CRUD method | Added sharding key support | | ---------------------------- | -------------------------- | | get() | Yes | | insert() / insert_object() | Yes | | delete() | Yes | | replace() / replace_object() | Yes | | upsert() / upsert_object() | Yes | | select() / pairs() | Yes | | update() | Yes | | upsert() / upsert_object() | Yes | | replace() / replace_object() | Yes | | min() / max() | No (not required) | | cut_rows() / cut_objects() | No (not required) | | truncate() | No (not required) | | len() | No (not required) | Limitations: - It's not possible to update sharding keys automatically when schema is updated on storages, see [2]. However it is possible to do it manually with sharding_key.update_sharding_keys_cache(). - CRUD select may lead map reduce in some cases, see [3]. 1. https://github.com/tarantool/ddl 2. #212 3. #213 Closes #166
ligurio
added a commit
that referenced
this issue
Sep 21, 2021
Previously there were two different ways to obtain bucket id in CRUD: - calculate bucket id automatically using primary key (default) - pass it from outside explicitly in options on CRUD operation call Users who uses DDL module [1] may specify sharding key (that are actually names of tuple fields), but it was not possible to use DDL sharding key for bucket id calculation. Now CRUD allows to use that custom sharding key to calculate bucket id, it will be done automatically when used DDL schema with non-empty sharding_key [1] or when space _ddl_sharding_key contains a tuple with space name and it's sharding key. Table below describe what operations supports custom sharding key: | CRUD method | Added sharding key support | | ---------------------------- | -------------------------- | | get() | Yes | | insert() / insert_object() | Yes | | delete() | Yes | | replace() / replace_object() | Yes | | upsert() / upsert_object() | Yes | | select() / pairs() | Yes | | update() | Yes | | upsert() / upsert_object() | Yes | | replace() / replace_object() | Yes | | min() / max() | No (not required) | | cut_rows() / cut_objects() | No (not required) | | truncate() | No (not required) | | len() | No (not required) | Limitations: - It's not possible to update sharding keys automatically when schema is updated on storages, see [2]. However it is possible to do it manually with sharding_key.update_sharding_keys_cache(). - CRUD select may lead map reduce in some cases, see [3]. 1. https://github.com/tarantool/ddl 2. #212 3. #213 Closes #166
ligurio
added a commit
that referenced
this issue
Sep 29, 2021
ligurio
added a commit
that referenced
this issue
Sep 29, 2021
ligurio
added a commit
that referenced
this issue
Sep 29, 2021
ligurio
added a commit
that referenced
this issue
Sep 30, 2021
ligurio
added a commit
that referenced
this issue
Sep 30, 2021
ligurio
added a commit
that referenced
this issue
Sep 30, 2021
ligurio
added a commit
that referenced
this issue
Sep 30, 2021
ligurio
added a commit
that referenced
this issue
Sep 30, 2021
ligurio
added a commit
that referenced
this issue
Oct 1, 2021
ligurio
added a commit
that referenced
this issue
Nov 18, 2021
ligurio
added a commit
that referenced
this issue
Nov 18, 2021
ligurio
added a commit
that referenced
this issue
Nov 18, 2021
ligurio
added a commit
that referenced
this issue
Nov 18, 2021
ligurio
added a commit
that referenced
this issue
Nov 18, 2021
ligurio
added a commit
that referenced
this issue
Nov 19, 2021
Describe functionality and current limitations (#212, #213 and #219) with custom sharding key in CHANGELOG and README. Closes #166 Reviewed-by: Oleg Babin <[email protected]> Reviewed-by: Alexander Turenko <[email protected]>
ligurio
added a commit
that referenced
this issue
Nov 19, 2021
Describe functionality and current limitations (#212, #213 and #219) with custom sharding key in CHANGELOG and README. Closes #166 Reviewed-by: Oleg Babin <[email protected]> Reviewed-by: Alexander Turenko <[email protected]>
ligurio
added a commit
that referenced
this issue
Nov 19, 2021
Describe functionality and current limitations (#212, #213 and #219) with custom sharding key in CHANGELOG and README. Closes #166 Reviewed-by: Oleg Babin <[email protected]> Reviewed-by: Alexander Turenko <[email protected]>
ligurio
added a commit
that referenced
this issue
Nov 19, 2021
Describe functionality and current limitations (#212, #213 and #219) with custom sharding key in CHANGELOG and README. Closes #166 Reviewed-by: Oleg Babin <[email protected]> Reviewed-by: Alexander Turenko <[email protected]>
AnaNek
pushed a commit
that referenced
this issue
Nov 24, 2021
DifferentialOrange
added a commit
that referenced
this issue
Nov 24, 2021
PR #181 introduced support of DDL sharding keys. But if sharding key hasn't got a separate index in schema, select with equal conditions for all required sharding key fields still led to map-reduce instead of a single storage call. This patch introduces impoved support of sharding keys extraction and fixes the issue. Closes #213
DifferentialOrange
added a commit
that referenced
this issue
Nov 24, 2021
PR #181 introduced support of DDL sharding keys. But if sharding key hasn't got a separate index in schema, select with equal conditions for all required sharding key fields still led to map-reduce instead of a single storage call. This patch introduces impoved support of sharding keys extraction and fixes the issue. Closes #213
3 tasks
DifferentialOrange
added a commit
that referenced
this issue
Nov 25, 2021
PR #181 introduced support of DDL sharding keys. But if sharding key hasn't got a separate index in schema, select with equal conditions for all required sharding key fields still led to map-reduce instead of a single storage call. This patch introduces impoved support of sharding keys extraction and fixes the issue. Closes #213
ligurio
added a commit
that referenced
this issue
Nov 25, 2021
Describe functionality and current limitations (#212, #213 and #219) with custom sharding key in CHANGELOG and README. Closes #166 Reviewed-by: Oleg Babin <[email protected]> Reviewed-by: Alexander Turenko <[email protected]> Co-authored-by: Georgy Moiseev <[email protected]>
ligurio
added a commit
that referenced
this issue
Nov 26, 2021
Describe functionality and current limitations (#212, #213 and #219) with custom sharding key in CHANGELOG and README. Closes #166 Reviewed-by: Oleg Babin <[email protected]> Reviewed-by: Alexander Turenko <[email protected]> Co-authored-by: Georgy Moiseev <[email protected]>
ligurio
added a commit
that referenced
this issue
Nov 26, 2021
Describe functionality and current limitations (#212, #213 and #219) with custom sharding key in CHANGELOG and README. Closes #166 Reviewed-by: Oleg Babin <[email protected]> Reviewed-by: Alexander Turenko <[email protected]> Co-authored-by: Georgy Moiseev <[email protected]>
ligurio
added a commit
that referenced
this issue
Nov 26, 2021
Describe functionality and current limitations (#212, #213, #219, #243) with custom sharding key in CHANGELOG and README. Closes #166 Reviewed-by: Oleg Babin <[email protected]> Reviewed-by: Alexander Turenko <[email protected]> Co-authored-by: Georgy Moiseev <[email protected]>
ligurio
added a commit
that referenced
this issue
Nov 26, 2021
Describe functionality and current limitations (#212, #213, #219, #243) with custom sharding key in CHANGELOG and README. Closes #166 Reviewed-by: Oleg Babin <[email protected]> Reviewed-by: Alexander Turenko <[email protected]> Co-authored-by: Georgy Moiseev <[email protected]>
ligurio
added a commit
that referenced
this issue
Nov 27, 2021
Describe functionality and current limitations (#212, #213, #219, #243) with custom sharding key in CHANGELOG and README. Thanks to Oleg Babin (@olegrok) and Alexander Turenko (@Totktonada) for help with feature implementation. Closes #166 Reviewed-by: Oleg Babin <[email protected]> Reviewed-by: Alexander Turenko <[email protected]> Co-authored-by: Georgy Moiseev <[email protected]>
ligurio
added a commit
that referenced
this issue
Nov 27, 2021
Describe functionality and current limitations (#212, #213, #219, #243) with custom sharding key in CHANGELOG and README. Thanks to Oleg Babin (@olegrok) and Alexander Turenko (@Totktonada) for help with feature implementation. Closes #166 Reviewed-by: Oleg Babin <[email protected]> Reviewed-by: Alexander Turenko <[email protected]> Co-authored-by: Georgy Moiseev <[email protected]>
DifferentialOrange
added a commit
that referenced
this issue
Nov 29, 2021
PR #181 introduced support of DDL sharding keys. But if sharding key hasn't got a separate index in schema, select with equal conditions for all required sharding key fields still led to map-reduce instead of a single storage call. This patch introduces impoved support of sharding keys extraction and fixes the issue. Closes #213
DifferentialOrange
added a commit
that referenced
this issue
Nov 29, 2021
PR #181 introduced support of DDL sharding keys. But if sharding key hasn't got a separate index in schema, select with equal conditions for all required sharding key fields still led to map-reduce instead of a single storage call. This patch introduces impoved support of sharding keys extraction and fixes the issue. Closes #213
DifferentialOrange
added a commit
that referenced
this issue
Nov 30, 2021
PR #181 introduced support of DDL sharding keys. But if sharding key hasn't got a separate index in schema, select with equal conditions for all required sharding key fields still led to map-reduce instead of a single storage call. This patch introduces impoved support of sharding keys extraction and fixes the issue. Closes #213
DifferentialOrange
added a commit
that referenced
this issue
Nov 30, 2021
PR #181 introduced support of DDL sharding keys. But if sharding key hasn't got a separate index in schema, select with equal conditions for all required sharding key fields still led to map-reduce instead of a single storage call. This patch introduces impoved support of sharding keys extraction and fixes the issue. Closes #213
DifferentialOrange
added a commit
that referenced
this issue
Nov 30, 2021
PR #181 introduced support of DDL sharding keys. But if sharding key hasn't got a separate index in schema, select with equal conditions for all required sharding key fields still led to map-reduce instead of a single storage call. This patch introduces impoved support of sharding keys extraction and fixes the issue. Closes #213
DifferentialOrange
added a commit
that referenced
this issue
Nov 30, 2021
PR #181 introduced support of DDL sharding keys. But if sharding key hasn't got a separate index in schema, select with equal conditions for all required sharding key fields still led to map-reduce instead of a single storage call. This patch introduces impoved support of sharding keys extraction and fixes the issue. Closes #213
DifferentialOrange
added a commit
that referenced
this issue
Dec 1, 2021
PR #181 introduced support of DDL sharding keys. But if sharding key hasn't got a separate index in schema, select with equal conditions for all required sharding key fields still led to map-reduce instead of a single storage call. This patch introduces impoved support of sharding keys extraction and fixes the issue. Closes #213
DifferentialOrange
added a commit
that referenced
this issue
Dec 1, 2021
PR #181 introduced support of DDL sharding keys. But if sharding key hasn't got a separate index in schema, select with equal conditions for all required sharding key fields still led to map-reduce instead of a single storage call. This patch introduces impoved support of sharding keys extraction and fixes the issue. Closes #213
DifferentialOrange
added a commit
that referenced
this issue
Dec 1, 2021
PR #181 introduced support of DDL sharding keys. But if sharding key hasn't got a separate index in schema, select with equal conditions for all required sharding key fields still led to map-reduce instead of a single storage call. This patch introduces impoved support of sharding keys extraction and fixes the issue. Closes #213
Totktonada
pushed a commit
that referenced
this issue
Dec 1, 2021
PR #181 introduced support of DDL sharding keys. But if sharding key hasn't got a separate index in schema, select with equal conditions for all required sharding key fields still led to map-reduce instead of a single storage call. This patch introduces impoved support of sharding keys extraction and fixes the issue. Closes #213
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Cases:
==
triggered map-reduce if there is no a separate index by filed 'name' and name 'name'rough patch:
Part of #166
The text was updated successfully, but these errors were encountered: