Support instance/replicaset/cluster names instead of UUIDs in the configuration #426

Totktonada · 2023-07-14T21:17:45Z

The key difference from the configuration using UUIDs is that instance/replicaset/cluster names are known at the configuration stage.

It is especially important in case of a cluster configuration introduced in tarantool/tarantool#8724.

There is a workaround: use a hash function to generate UUIDs from names. However, it complicates upgrading from old snapshots, where there are no persistent instance/replicaset/cluster names.

I would appreciate hints about possible pitfalls on the road. What else we need to do except just passing the parameters down to box.cfg()?

The text was updated successfully, but these errors were encountered:

Gerold103 · 2023-07-14T21:20:09Z

There are no pitfalls. It just needs to be done and shouldn't be even complicated.

This commit allows vtest to create named config, wich will be introduced in the following commits. If identification_mode is 'name_as_key', then names are used instead of UUIDs as keys, replica/replicaset.name is not specified, as it's forbidden, UUIDs are set as replica/replicaset.uuid. The commit also introduces new server API: instance/replicaset_name(), which is meant to be used in tests for Tarantool >= 3.0.0. Part of tarantool#426 NO_DOC=internal

In the following commits it'll be allowed to use names instead of UUIDs. This patch refactors error module, changing the name of arguments, accepted by errors: from now on it may be either UUID or persistent name, so we cannot say, that this is always UUID, as it's done now. The alternative would be to rename arguments according to the configuration used on the fly. It's not selected, as it complicates expluatation. Note, that this is breaking change, if user's code relies on UUID arguments of returned errors! Part of tarantool#426 NO_DOC=refactoring

This commit replaces all appearances of UUIDs in replicaset's code, where it's possible to do without any functional changes. This commit has only code-readability purpose, as in the future named identification will be introduced. Part of tarantool#426 NO_DOC=refactoring NO_TEST=refactoring

From now on keys inside replicaset and replicaset.replicas may be either replicaset/instance_name or UUID, depending on cfg.identification_mode. We cannot use UUIDs always now, as they might not be specified at all. Part of tarantool#426 NO_DOC=internal

@TarantoolBot

Part of tarantool#426 @TarantoolBot document Title: vshard: config identification mode The option `identification_mode` should be specified in the root of the config. It can have one of those values: * `'uuid_as_key'` - default. Means, that default uuid config identification is used. replica.name is allowed and should not be interpreted as `box.cfg.instance_name`. replica/replicaset.uuid is forbidden. The config should have the following format: { ['cbf06940-0790-498b-948d-042b62cf3d29'] = { -- replicaset tarantool#1 replicas = { ['8a274925-a26d-47fc-9e1b-af88ce939412'] = { name = 'storage_1_a', ... }, ... }, }, ... } * `'name_as_key'`. Name identification is used, supported only by Tarantool >= 3.0.0. It's forbidden to specify replica.name in such format. UUIDs are optional and can be specified via replicaset/replica.uuid: { replicaset_1 = { uuid = 'cbf06940-0790-498b-948d-042b62cf3d29', replicas = { replica_1_a = { uuid = '8a274925-a26d-47fc-9e1b-af88ce939412' ... }, ... } }, ... } Note, that names, used as keys in config are passed to box.cfg.replicaset/instance_name for storage. In case of reconfiguration it's strictly validated, that both replicaset and instance name corresponds to the passed config. Vshard doesn't deal with changing or setting names, it must be done externally (using Tarantool's config module, for example).

@TarantoolBot

This commit allows storage to use named config identification. Part of tarantool#426 @TarantoolBot document Title: vshard: vshard.storage.cfg argument change vshard.storage.cfg(cfg, instance_id) Configure the database and start sharding for the specified storage instance. `istance_id` depends on `identification_mode` used in config. If it's 'uuid_as_key', which is default, then UUID must be used. Otherwise - instance_name. Parameters: cfg – a storage configuration instance_id – UUID or name of the instance

@TarantoolBot

It's not good to show only uuid in info, when name identification is used. Let's also show a name of a master and replicaset in such case. We don't show master name for UUID identification, as it's not a persisted name. We also don't replace uuid with names for name identification in order to preserve backward compatibility of vshard.storage.info(). Part of tarantool#426 @TarantoolBot document Title: vshard: vshard.storage.info shows name When `identification_mode` is `name_as_key`, persisted master and replicaset name is shown in vshard.storage.info: ``` tarantool> vshard.storage.info() --- - replicasets: replicaset_1: name: replicaset_1 master: uri: admin@localhost:3302 name: replica_1_a ... ```

@TarantoolBot

This commit allows router to use named config identification. Part of tarantool#426 @TarantoolBot document Title: vshard: vshard.router.buckets_info shows name When `identification_mode` is `name_as_key`, buckets_info() also returns name of the replicas, where buckets are stored: ``` tarantool> vshard.router.buckets_info() --- - - uuid: aaaaaaaa-0000-4000-a000-000000000000 status: available_rw name: replica_1_a ```

@TarantoolBot

It's not good to show only uuid in info, when name identification is used. Let's also show a name of a master and replicaset in such case. We don't show master name for UUID identification, as it's not a persisted name. We also don't replace uuid with names for name identification in order to preserve backward compatibility of vshard.storage.info(). Part of tarantool#426 @TarantoolBot document Title: vshard: vshard.router.info shows name When `identification_mode` is `name_as_key`, persisted master and replicaset name is shown in vshard.router.info: ``` tarantool> vshard.router.info() --- - replicasets: replicaset_1: replica: &0 network_timeout: 0.5 status: available uri: [email protected]:3303 name: replica_1_a name: replicaset_1 master: *0 ```

@TarantoolBot

It's not good to show only uuid in info, when name identification is used. Let's also show a name of a master and replicaset in such case. We don't show master name for UUID identification, as it's not a persisted name. We also don't replace uuid with names for name identification in order to preserve backward compatibility of vshard.storage.info(). Part of tarantool#426 @TarantoolBot document Title: vshard: vshard.router.info shows name When `identification_mode` is `name_as_key`, persisted master and replicaset name is shown in vshard.router.info: ``` tarantool> vshard.router.info() --- - replicasets: replicaset_1: replica: &0 network_timeout: 0.5 status: available uri: [email protected]:3303 name: replica_1_a name: replicaset_1 master: *0 ```

@TarantoolBot

This commit allows router to use named config identification. Part of tarantool#426 @TarantoolBot document Title: vshard: vshard.router.buckets_info shows name When `identification_mode` is `name_as_key`, buckets_info() also returns name of the replicas, where buckets are stored: ``` tarantool> vshard.router.buckets_info() --- - - uuid: aaaaaaaa-0000-4000-a000-000000000000 status: available_rw name: replica_1_a ```

@TarantoolBot

It's not good to show only uuid in info, when name identification is used. Let's also show a name of a master and replicaset in such case. We don't show master name for UUID identification, as it's not a persisted name. We also don't replace uuid with names for name identification in order to preserve backward compatibility of vshard.storage.info(). Part of tarantool#426 @TarantoolBot document Title: vshard: vshard.router.info shows name When `identification_mode` is `name_as_key`, persisted master and replicaset name is shown in vshard.router.info: ``` tarantool> vshard.router.info() --- - replicasets: replicaset_1: replica: &0 network_timeout: 0.5 status: available uri: [email protected]:3303 name: replica_1_a name: replicaset_1 master: *0 ```

In case of name identification, no UUID may be passed at all, so we cannot verify only UUID, when connecting to storage. It seems impossible to extend the current net.box greeting by exposing net_box.conn.name to it, as iproto greeting doesn't have enough free space to save 64 bit instance name. So, we should deal with name validation on vshard side. Let's validate name only during connection establishing, not on every reconnect, as it's done for UUID now. The motivation for that, is the fact, that it's not cheap to validate name. Closes tarantool#426 NO_DOC=internal

This commit allows vtest to create named config, wich will be introduced in the following commits. If identification_mode is 'name_as_key', then names are used instead of UUIDs as keys, replica/replicaset.name is not specified, as it's forbidden, UUIDs are set as replica/replicaset.uuid. The commit also introduces new server API: instance/replicaset_name(), which is meant to be used in tests for Tarantool >= 3.0.0. Part of tarantool#426 NO_DOC=internal

In the following commits it'll be allowed to use names instead of UUIDs. This patch refactors error module, changing the name of arguments, accepted by errors: from now on it may be either UUID or persistent name, so we cannot say, that this is always UUID, as it's done now. The alternative would be to rename arguments according to the configuration used on the fly. It's not selected, as it complicates expluatation. Note, that this is breaking change, if user's code relies on UUID arguments of returned errors! Part of tarantool#426 NO_DOC=refactoring

This commit replaces all appearances of UUIDs in replicaset's code, where it's possible to do without any functional changes. This commit has only code-readability purpose, as in the future named identification will be introduced. Part of tarantool#426 NO_DOC=refactoring NO_TEST=refactoring

From now on keys inside replicaset and replicaset.replicas may be either replicaset/instance_name or UUID, depending on cfg.identification_mode. We cannot use UUIDs always now, as they might not be specified at all. Part of tarantool#426 NO_DOC=internal

@TarantoolBot

Part of tarantool#426 @TarantoolBot document Title: vshard: config identification mode The option `identification_mode` should be specified in the root of the config. It can have one of those values: * `'uuid_as_key'` - default. Means, that default uuid config identification is used. replica.name is allowed and should not be interpreted as `box.cfg.instance_name`. replica/replicaset.uuid is forbidden. The config should have the following format: ``` { ['cbf06940-0790-498b-948d-042b62cf3d29'] = { -- replicaset tarantool#1 replicas = { ['8a274925-a26d-47fc-9e1b-af88ce939412'] = { name = 'storage_1_a', ... }, ... }, }, ... } ``` * `'name_as_key'`. Name identification is used, supported only by Tarantool >= 3.0.0. It's forbidden to specify replica.name in such format. UUIDs are optional and can be specified via replicaset/replica.uuid: ``` { replicaset_1 = { uuid = 'cbf06940-0790-498b-948d-042b62cf3d29', replicas = { replica_1_a = { uuid = '8a274925-a26d-47fc-9e1b-af88ce939412' ... }, ... } }, ... } ``` Note, that names, used as keys in config are passed to box.cfg.replicaset/instance_name for storage. In case of reconfiguration it's strictly validated, that both replicaset and instance name corresponds to the passed config. Vshard doesn't deal with changing or setting names, it must be done externally (using Tarantool's config module, for example).

This commit replaces all appearances of UUIDs in storage's code, where it's possible to do without any functional changes. This commit has only code-readability purpose, as in the future named identification will be introduced. Part of tarantool#426 NO_DOC=refactoring NO_TEST=refactoring

@TarantoolBot

This commit allows storage to use named config identification. Part of tarantool#426 @TarantoolBot document Title: vshard: vshard.storage.cfg argument change vshard.storage.cfg(cfg, instance_id) Configure the database and start sharding for the specified storage instance. `istance_id` depends on `identification_mode` used in config. If it's 'uuid_as_key', which is default, then UUID must be used. Otherwise - instance_name. Parameters: cfg – a storage configuration instance_id – UUID or name of the instance

@TarantoolBot

When named config is used, there may be no UUIDs specified for replicas and replicasets. Consequently, in such case we must also show the names in addition to UUIDs in storage.info(). Part of tarantool#426 @TarantoolBot document Title: vshard: vshard.storage.info shows name When `identification_mode` is `name_as_key`, persisted master and replicaset name is shown in vshard.storage.info: ``` tarantool> vshard.storage.info() --- - replicasets: replicaset_1: name: replicaset_1 master: uri: admin@localhost:3302 name: replica_1_a ... ```

This commit replaces all appearances of UUIDs in router's code, where it's possible to do without any functional changes. This commit has only code-readability purpose, as in the future named identification will be introduced. Part of #426 NO_DOC=refactoring NO_TEST=refactoring

This commit allows router to use named config identification. Part of #426 NO_DOC=<no API changed>

@TarantoolBot

When named config is used, there may be no UUIDs specified for replicas. Consequently, in such case we must also show the name of replica, where bucket is located, not only UUID. Part of #426 @TarantoolBot document Title: vshard: vshard.router.buckets_info shows name When `identification_mode` is `name_as_key`, buckets_info() also returns name of the replicas, where buckets are stored: ``` tarantool> vshard.router.buckets_info() --- - - uuid: aaaaaaaa-0000-4000-a000-000000000000 status: available_rw name: replica_1_a ```

@TarantoolBot

When named config is used, there may be no UUIDs specified for replicas and replicasets. Consequently, in such case we must also show the names in addition to UUIDs in router.info(). Part of #426 @TarantoolBot document Title: vshard: vshard.router.info shows name When `identification_mode` is `name_as_key`, persisted master and replicaset name is shown in vshard.router.info: ``` tarantool> vshard.router.info() --- - replicasets: replicaset_1: replica: &0 network_timeout: 0.5 status: available uri: [email protected]:3303 name: replica_1_a name: replicaset_1 master: *0 ```

It's needed to check, whether the error was a timeout or not in the following commit. Let's move this function from vtest to error module. Needed for #426 NO_DOC=internal

In case of name identification, no UUID may be passed at all, so we cannot verify only UUID, when connecting to storage. It seems impossible to extend the current net.box greeting by exposing net_box.conn.name to it, as iproto greeting doesn't have enough free space to save 64 bit instance name. So, we should deal with name validation on vshard side. For this, conn.vconnect is introduced. It's asynchronous vshard greeting, saved inside netbox connection. It stores future object and additional info, needed for its work. Future is initialized, when the connection is established (inside netbox_on_connect). The connection cannot be considered "connected" until vconnect is properly validated. Currently only instance_name is validated inside conn.vconnect. Closes #426 NO_DOC=internal

During upgrade to Tarantool 3.0.0 there's a time, when instance name have not been set yet. If vshard strictly validates names on connection, it leads to the cluster unavailability by the time, when all names are configured. This commit softens the check and allows name to be nil, when instance UUID is specified in configuration, only UUID is validated in such case. In conclusion we have the following checks on connection: 1. UUID is validated in the following cases: a. When `identification_mode` is `uuid_as_key` b. When `identification_mode` is `name_as_key` and replica.uuid is specified 2. Instance name is validated only when `identification_mode` is `name_as_key` and can work in the following modes: a. Strict validation (nil instance name is not allowed), when replica.uuid is not specified. b. Soft validation (nil instance name is allowed), when replica.uuid is specified. Follow-up tarantool#426 NO_DOC=internal

During upgrade to Tarantool 3.0.0 there's a time, when instance name have not been set yet. If vshard strictly validates names on connection, it leads to the cluster unavailability by the time, when all names are configured. This commit softens the check and allows name to be nil, when instance UUID is specified in configuration, only UUID is validated in such case. In conclusion we have the following checks on connection: 1. UUID is validated in the following cases: a. When `identification_mode` is `uuid_as_key` b. When `identification_mode` is `name_as_key` and replica.uuid is specified 2. Instance name is validated only when `identification_mode` is `name_as_key` and can work in the following modes: a. Strict validation (nil instance name is not allowed), when replica.uuid is not specified. b. Soft validation (nil instance name is allowed), when replica.uuid is specified. Follow-up #426 NO_DOC=internal

In vshard 0.1.25, new feature related to vshard configuration and storage info was introduced [1]. If the new mode is used, crud module fails to bootstrap and work in several places. This feature is enabled by Tarantool 3.0 if vshard cluster was configured with 3.0 config. After this patch, it is possible to bootstrap a vshard cluster with new configuration mode. We run all existing vshard tests with new modes after this patch. Module code was updated to support both possible mods. 1. tarantool/vshard#426 Closes #403

In vshard 0.1.25, new feature related to vshard configuration and storage info was introduced [1]. If the new mode is used, crud module fails to bootstrap and work in several places. This feature is enabled by Tarantool 3.0 if vshard cluster was configured with 3.0 config. After this patch, it is possible to bootstrap a vshard cluster with new configuration mode. We run all existing vshard tests with new modes after this patch. Module code was updated to support both possible modes. 1. tarantool/vshard#426 Closes #403

Currently, the name validation is not used, when locate_master() is called. It makes an explicit call via the connection to obtain a future object in order to figure out, whether the node is a master. We should not make any calls to a replica until the time, we definitely know, that its name and uuid was validated. Let's use replica_call instead of conn.call. Follow-up tarantool#426 NO_DOC=bugfix

Currently, the name validation is not used, when locate_master() is called. It makes an explicit call via the connection to obtain a future object in order to figure out, whether the node is a master. We should not make any calls to a replica until the time, we definitely know, that its name and uuid was validated. Let's use replica_call instead of conn.call. Follow-up #426 NO_DOC=bugfix

Totktonada added feature A new functionality configuration labels Jul 14, 2023

Totktonada assigned ImeevMA Jul 26, 2023

sergepetrenko unassigned ImeevMA Aug 25, 2023

Serpentian self-assigned this Oct 17, 2023

Serpentian mentioned this issue Dec 4, 2023

Introduce named config #451

Merged

Gerold103 pushed a commit that referenced this issue Dec 19, 2023

router: introduce name identification

515506f

This commit allows router to use named config identification. Part of #426 NO_DOC=<no API changed>

Gerold103 pushed a commit that referenced this issue Dec 19, 2023

error: introduce is_timeout

9d52e1e

It's needed to check, whether the error was a timeout or not in the following commit. Let's move this function from vtest to error module. Needed for #426 NO_DOC=internal

Serpentian mentioned this issue Dec 20, 2023

replicaset: soften name validation #458

Merged

DifferentialOrange mentioned this issue Dec 22, 2023

Expose current identification_mode #460

Closed

Serpentian mentioned this issue Dec 26, 2023

replicaset: validate name in replicaset_locate_master #462

Merged

This was referenced Dec 27, 2023

vshard: config identification mode tarantool/doc#3940

Open

vshard: vshard.storage.cfg argument change tarantool/doc#3941

Open

This was referenced Jan 11, 2024

vshard: vshard.storage.info shows name tarantool/doc#3942

Open

vshard: vshard.router.buckets_info shows name tarantool/doc#3943

Open

vshard: vshard.router.info shows name tarantool/doc#3944

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Support instance/replicaset/cluster names instead of UUIDs in the configuration #426

Support instance/replicaset/cluster names instead of UUIDs in the configuration #426

Totktonada commented Jul 14, 2023 •

edited

Loading

Gerold103 commented Jul 14, 2023

Support instance/replicaset/cluster names instead of UUIDs in the configuration #426

Support instance/replicaset/cluster names instead of UUIDs in the configuration #426

Comments

Totktonada commented Jul 14, 2023 • edited Loading

Gerold103 commented Jul 14, 2023

Totktonada commented Jul 14, 2023 •

edited

Loading