Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update Mirror Gateway document #2207

Merged
Merged
Show file tree
Hide file tree
Changes from 1 commit
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
78 changes: 1 addition & 77 deletions docs/api/mirror-gateway.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,13 +2,11 @@

## Overview

Mirror Service is responsible for providing the `Register` and `Advertise` interface for the Vald Mirror Gateway.
Mirror Service is responsible for providing the `Register` interface for the Vald Mirror Gateway.

```rpc
service Mirror {
rpc Register(payload.v1.Mirror.Targets) returns (payload.v1.Mirror.Targets) {}

rpc Advertise(payload.v1.Mirror.Targets) returns (payload.v1.Mirror.Targets) {}
}
```

Expand Down Expand Up @@ -85,77 +83,3 @@ Register RPC is the method to register other Vald Mirror Gateway targets.
| 3 | INVALID_ARGUMENT |
| 4 | DEADLINE_EXCEEDED |
| 13 | INTERNAL |

## Advertise RPC

Advertise RPC is the method to advertise Vald Mirror Gateway targets.

### Input

- the scheme of `payload.v1.Mirror.Targets`.

```rpc
message Mirror {
message Target {
string host = 1;
uint32 port = 2;
}

message Targets {
repeated Target targets = 1;
}
}
```

- Mirror.Targets

| field | type | label | required | desc. |
| :-----: | :------------ | :----------------------------- | :------: | :------------------------------- |
| targets | Mirror.Target | repeated(Array[Mirror.Target]) | \* | The multiple target information. |

- Mirror.Target

| field | type | label | required | desc. |
| :---: | :----- | :---- | :------: | :------------------- |
| host | string | | \* | The target hostname. |
| port | uint32 | | \* | The target port. |

### Output

- the scheme of `payload.v1.Mirror.Targets`.

```rpc
message Mirror {
message Target {
string host = 1;
uint32 port = 2;
}

message Targets {
repeated Target targets = 1;
}
}
```

- Mirror.Targets

| field | type | label | required | desc. |
| :-----: | :------------ | :----------------------------- | :------: | :------------------------------- |
| targets | Mirror.Target | repeated(Array[Mirror.Target]) | | The multiple target information. |

- Mirror.Target

| field | type | label | required | desc. |
| :---: | :----- | :---- | :------: | :------------------- |
| host | string | | \* | The target hostname. |
| port | uint32 | | \* | The target port. |

### Status Code

| code | desc. |
| :--: | :---------------- |
| 0 | OK |
| 1 | CANCELLED |
| 3 | INVALID_ARGUMENT |
| 4 | DEADLINE_EXCEEDED |
| 13 | INTERNAL |
46 changes: 22 additions & 24 deletions docs/overview/component/mirror-gateway.md
Original file line number Diff line number Diff line change
Expand Up @@ -44,35 +44,33 @@ Then, while forwarding the user request, the Vald Mirror Gateway bypasses the in

On the other hand, if the incoming user request is an [Object API](https://vald.vdaas.org/docs/api/object/) or [Search API](https://vald.vdaas.org/docs/api/search/), it is bypassed to only a Vald LB Gateway in its own cluster without forwarding it to other Vald Mirror Gateways.

### Automatic rollback on failure
### Continuous processing on failure

The request may fail at the forwarding destination or the bypass destination.

If some requests fail, the vector data will not be consistent across Vald clusters.
If some of the requests fails, the processing continues based on their status code.

To keep index state consistency, the Vald Mirror Gateway will send the rollback request for the failed request. After the rollback request succeeds, the index state will be the same as before requesting.

The following is the list of rollback types.
The following is an overview of the process for each request.

- Insert Request
- Rollback Condition/Trigger
- Status code other than `ALREADY_EXISTS` exists
- Rollback request to the successful request
- REMOVE request
- Remove Request
- Rollback Condition/Trigger
- Status code other than `NOT_FOUND` exists
- Rollback request to the successful request
- UPSERT Request with old vector

- If the target host returns a status of `ALREADY_EXISTS`, the Update request is sent to this host.
- If all target hosts returns a status of `ALREADY_EXISTS`, returns `ALREADY_EXISTS` to the user without continuous processing.
hlts2 marked this conversation as resolved.
Show resolved Hide resolved
- If all target hosts returns a status of `OK` or `ALREADY_EXISTS`, returns `OK` to the user without continuous processing.
hlts2 marked this conversation as resolved.
Show resolved Hide resolved

- Update Request
- Rollback Condition/Trigger
- Status code other than `ALREADY_EXISTS` exists
- Rollback request to the successful request
- REMOVE Request if there is no old vector data
- UPDATE Request if there is old vector data

- If the target host returns a status of `NOT_FOUND`, the Insert request is sent to this host.
- If all target hosts returns a status of `ALREADY_EXISTS`, returns `ALREADY_EXISTS` to the user without continuous processing.
hlts2 marked this conversation as resolved.
Show resolved Hide resolved
- If all target hosts returns a status of `OK` or `ALREADY_EXISTS`, returns `OK` to the user without continuous processing.
hlts2 marked this conversation as resolved.
Show resolved Hide resolved

- Upsert Request
- Rollback Condition/Trigger
- Status code other than `ALREADY_EXISTS` exists
- Rollback request to the successful request
- REMOVE Request if there is no old vector data
- UPDATE Request if there is old vector data

- If all target hosts returns a status of `OK` or `ALREADY_EXISTS`, returns `OK` to the user without continuous processing.
hlts2 marked this conversation as resolved.
Show resolved Hide resolved
- If all target hosts returns a status of `ALREADY_EXISTS`, returns `ALREADY_EXISTS` to the user without continuous processing.
hlts2 marked this conversation as resolved.
Show resolved Hide resolved

- Remove Request
- If all target hosts returns a status of `OK` or `NOT_FOUND`, returns `OK` to the user without continuous processing.
hlts2 marked this conversation as resolved.
Show resolved Hide resolved
- If all target hosts returns a status of `NOT_FOUND`, returns `NOT_FOUND` to the user without continuous processing.
hlts2 marked this conversation as resolved.
Show resolved Hide resolved

For more information, please refer to [Mirror Gateway Troubleshooting](https://vald.vdaas.org/docs/troubleshooting/mirror-gateway/).
4 changes: 2 additions & 2 deletions docs/user-guides/mirroring-configuration.md
Original file line number Diff line number Diff line change
Expand Up @@ -41,8 +41,8 @@ gateway:
...
# gRPC client configuration (overrides defaults.grpc.client)
client: {}
# The interval to advertise addresses of Mirror Gateway to other Mirror Gateway.
advertise_interval: "1s"
# The duration to register other Mirror Gateways.
register_duration: "1s"
hlts2 marked this conversation as resolved.
Show resolved Hide resolved
# The target namespace to discover ValdMirrorTarget (CR) resource.
# The default value is its own namespace.
namespace: "vald"
Expand Down