Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: add support for Row Affinity app profiles #2341

Open
wants to merge 10 commits into
base: main
Choose a base branch
from

Conversation

djyau
Copy link
Contributor

@djyau djyau commented Sep 16, 2024

Thank you for opening a Pull Request! Before submitting your PR, there are a few things you can do to make sure it goes smoothly:

  • Make sure to open an issue as a bug/issue before writing your code! That way we can discuss the change, evaluate designs, and agree on the general idea
  • Ensure the tests and linter pass
  • Code coverage does not decrease (if any source code was changed)
  • Appropriate docs were updated (if necessary)
  • Rollback plan is reviewed and LGTMed
  • All new data plane features have a completed end to end testing plan

Fixes #<issue_number_goes_here> ☕️

If you write sample code, please follow the samples format.

Change-Id: I7cdd6c9ce85f9132c0fa3db3aa8c70abac26f12b
Change-Id: Id2cedefd90c52c249b2b15734eb1fbbcc5db2bf4
Change-Id: I01ef72617d391bdb956ae9f469f1fd5af208a2e1
Change-Id: I9f18991cb61d99f30b811cea0988963d9e2577cd
@product-auto-label product-auto-label bot added size: l Pull request size is large. api: bigtable Issues related to the googleapis/java-bigtable API. labels Sep 16, 2024
@djyau djyau marked this pull request as ready for review September 17, 2024 14:29
@djyau djyau requested review from a team as code owners September 17, 2024 14:29
@@ -267,6 +271,32 @@ public static MultiClusterRoutingPolicy of(Set<String> clusterIds) {
MultiClusterRoutingUseAny.newBuilder().addAllClusterIds(clusterIds).build());
}

/** Creates a new instance of {@link MultiClusterRoutingPolicy}. */
public static MultiClusterRoutingPolicy ofWithRowAffinity() {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think withRowAffinity is a bit less akward

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also can you describe what the difference between this variant and withRowAffinity(String... clusterIds)

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also is it possible to update an app profile to change cluster ids? or to change the routing away from row affinity?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also do you need to update the mask?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also can you describe what the difference between this variant and withRowAffinity(String... clusterIds)

Yes - Row affinity, when enabled, routes single row key requests deterministically to the same cluster each time to improve read-your-writes consistency.

When row affinity is enabled with a cluster group (with cluster IDs specified), then when the user adds a new cluster to their instance, it does not affect which clusters each row key routes to. It is a way to protect read-your-writes consistency from being impacted negatively from new clusters as well as from attempts to delete clusters (since you cannot delete a cluster if its in the cluster group). Without the cluster group, when a user adds a new cluster, row keys will undergo a shuffling of which cluster they route to, which temporarily breaks read-your-writes consistency.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also do you need to update the mask?

I don't think so, but correct me if I am wrong: row affinity is a subfield/submessage of MultiClusterRoutingUseAny (routing policy), and we already have existing methods for updating the routing policy with the appropriate mask: https://github.com/googleapis/java-bigtable/blob/main/google-cloud-bigtable/src/main/java/com/google/cloud/bigtable/admin/v2/models/UpdateAppProfileRequest.java#L106-L125

Also is it possible to update an app profile to change cluster ids? or to change the routing away from row affinity?

It should be possible to do both of these. The user should call UpdateAppProfileRequest::SetRoutingPolicy and supply a routing policy with new cluster ids or with row affinity absent

Change-Id: I9f88209e04cde4f628c878fbdb659c7344aa1c3c
…to feature-affinity-proto

Change-Id: I3fae97fa5283129ae449d0f6bbc8f2ee6fc2601d
adminClient.createAppProfile(
CreateAppProfileRequest.of(INSTANCE_ID, APP_PROFILE_ID)
.setDescription("my description")
.setRoutingPolicy(MultiClusterRoutingPolicy.withRowAffinity("cluster-id-1", "cluster-id-2")));
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this the same as the test above?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It is similar. The one above adds the cluster_ids one at a time via "addClusterIds". This test adds all the cluster_ids in one list via "addAllClusterIds"

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think that's testing the generated proto instead of testing our wrapper class. We can probably just keep one of the 2 test.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Okie - removed this one.

Change-Id: Ieda39f0d9825fae649755350b4f228fe0e8985e5
Change-Id: I6d8e7bd644ce2cd8f098c8ee37b4eec1a0d642e2
@mutianf
Copy link
Contributor

mutianf commented Oct 11, 2024

/gcbrun

@mutianf mutianf added the kokoro:run Add this label to force Kokoro to re-run the tests. label Oct 11, 2024
@yoshi-kokoro yoshi-kokoro removed the kokoro:run Add this label to force Kokoro to re-run the tests. label Oct 11, 2024
Change-Id: I7b072214c8b38646058d0ec668bbb469bf16f23e
@mutianf mutianf added the kokoro:run Add this label to force Kokoro to re-run the tests. label Oct 14, 2024
@yoshi-kokoro yoshi-kokoro removed the kokoro:run Add this label to force Kokoro to re-run the tests. label Oct 14, 2024
Change-Id: Ieb04b2141c8843ff59cc4d756d736b4c00609ee2
@mutianf mutianf added the kokoro:run Add this label to force Kokoro to re-run the tests. label Oct 14, 2024
@yoshi-kokoro yoshi-kokoro removed the kokoro:run Add this label to force Kokoro to re-run the tests. label Oct 14, 2024
@mutianf
Copy link
Contributor

mutianf commented Oct 14, 2024

/gcbrun

@mutianf mutianf added the do not merge Indicates a pull request not ready for merge, due to either quality or timing. label Oct 14, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
api: bigtable Issues related to the googleapis/java-bigtable API. do not merge Indicates a pull request not ready for merge, due to either quality or timing. size: l Pull request size is large.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants