Skip to content

Latest commit

 

History

History
309 lines (231 loc) · 11.5 KB

remove.md

File metadata and controls

309 lines (231 loc) · 11.5 KB
Raw

Vald Remove APIs

Overview

Remove Service is responsible for removing vectors that are indexed in the vald-agent.

service Remove {

  rpc Remove(payload.v1.Remove.Request) returns (payload.v1.Object.Location) {}

  rpc StreamRemove(stream payload.v1.Remove.Request)
      returns (stream payload.v1.Object.StreamLocation) {}

  rpc MultiRemove(payload.v1.Remove.MultiRequest)
      returns (payload.v1.Object.Locations) {}
}

Remove RPC

Remove RPC is the method to remove a single vector.

Input

  • the scheme of payload.v1.Remove.Request

    message Remove {
    message Request {
        Object.ID id = 1;
        Config config = 2;
    }

    message Config {
        bool skip_strict_exist_check = 1;
        int64 timestamp = 3;
    }
}

message Object {
    message ID {
        string id = 1 [ (validate.rules).string.min_len = 1 ];
    }
}

    • Remove.Request

      field type label required desc.
      id Object.ID * the id of vector
      config Config * the configuration of the remove request

    • Remove.Config

      field type label required desc.
      skip_strict_exist_check bool check the same vector is already inserted or not.
      the ID should be unique if the value is true.
      timestamp int64 the timestamp of the vector removed.
      if it is N/A, the current time will be used.

    • Object.ID

      field type label required desc.
      id string * the ID of a vector. ID should consist of 1 or more characters.

Output

  • the scheme of payload.v1.Object.Location

    message Object {
    message Location {
      string name = 1;
      string uuid = 2;
      repeated string ips = 3;
    }
}

    • Object.Location

      field type label desc.
      name string the name of vald agent pod where the request vector is removed.
      uuid string the ID of the removed vector. It is the same as an Object.ID.
      ips string repeated(Array[string]) the IP list of vald-agent pods where the request vector is removed.

Status Code

code desc.
0 OK
1 CANCELLED
3 INVALID_ARGUMENT
4 DEADLINE_EXCEEDED
5 NOT_FOUND
13 INTERNAL

StreamRemove RPC

StreamRemove RPC is the method to remove multiple vectors using the bidirectional streaming RPC.
By using the bidirectional streaming RPC, the remove request can be communicated in any order between client and server. Each Remove request and response are independent. It's the recommended method to remove a large number of vectors.

Input

  • the scheme of payload.v1.Remove.Request stream

    message Remove {
    message Request {
        Object.ID id = 1;
        Config config = 2;
    }

    message Config {
        bool skip_strict_exist_check = 1;
        int64 timestamp = 3;
    }
}

message Object {
    message ID {
        string id = 1 [ (validate.rules).string.min_len = 1 ];
    }
}

    • Remove.Request

      field type label required desc.
      id Object.ID * the id of vector
      config Config * the configuration of the insert request

    • Remove.Config

      field type label required desc.
      skip_strict_exist_check bool check the same vector is already inserted or not.
      the ID should be unique if the value is true.
      timestamp int64 the timestamp of the vector removed.
      if it is N/A, the current time will be used.

    • Object.ID

      field type label required desc.
      id string * the ID of a vector. ID should consist of 1 or more characters.

Output

  • the scheme of payload.v1.Object.StreamLocation

    message Object {
    message StreamLocation {
      oneof payload {
          Location location = 1;
          google.rpc.Status status = 2;
      }
    }

    message Location {
      string name = 1;
      string uuid = 2;
      repeated string ips = 3;
    }
}

    • Object.StreamLocation

      field type label desc.
      location Object.Location the information of Object.Location data.
      status google.rpc.Status the status of google RPC

    • Object.Location

      field type label desc.
      name string the name of vald agent pod where the request vector is removed.
      uuid string the ID of the removed vector. It is the same as an Object.ID.
      ips string repeated(Array[string]) the IP list of vald-agent pods where the request vector is removed.

    • google.rpc.Status

      field type label desc.
      code int32 status code (code list is next section)
      message string error message
      details google.protobuf.Any repeated(Array[any]) the details error message list

Status Code

code desc.
0 OK
1 CANCELLED
3 INVALID_ARGUMENT
4 DEADLINE_EXCEEDED
5 NOT_FOUND
13 INTERNAL

MultiRemove RPC

MultiRemove is the method to remove multiple vectors in 1 request.

gRPC has a message size limitation.
Please be careful that the size of the request exceeds the limit.

Input

  • the scheme of payload.v1.Remove.MultiRequest

    message Remove {
    message MultiRequest {
      repeated Request requests = 1;
    }

    message Request {
        Object.ID id = 1;
        Config config = 2;
    }

    message Config {
        bool skip_strict_exist_check = 1;
        int64 timestamp = 3;
    }
}

message Object {
    message ID {
        string id = 1 [ (validate.rules).string.min_len = 1 ];
    }
}

    • Remove.MultiRequest

      field type label required desc.
      requests Remove.Request repeated(Array[Insert.Request]) * the request list

    • Remove.Request

      field type label required desc.
      id Object.ID * the id of vector
      config Config * the configuration of the remove request

    • Remove.Config

      field type label required desc.
      skip_strict_exist_check bool check the same vector is already inserted or not.
      the ID should be unique if the value is true.
      timestamp int64 the timestamp of the vector removed.
      if it is N/A, the current time will be used.

    • Object.ID

      field type label required desc.
      id string * the ID of a vector. ID should consist of 1 or more characters.

Output

  • the scheme of payload.v1.Object.Locations.

    message Object {
    message Locations { repeated Location locations = 1; }

    message Location {
      string name = 1;
      string uuid = 2;
      repeated string ips = 3;
    }
}

    • Object.Locations

      field type label desc.
      location Object.Location repeated(Array[Object.Location]) the list of Object.Location.

    • Object.Location

      field type label desc.
      name string the name of vald agent pod where the request vector is removed.
      uuid string the ID of the removed vector. It is the same as an Object.Vector.
      ips string repeated(Array[string]) the IP list of vald-agent pods where the request vector is removed.

Status Code

code desc.
0 OK
1 CANCELLED
3 INVALID_ARGUMENT
4 DEADLINE_EXCEEDED
5 NOT_FOUND
13 INTERNAL