feat: add support for transaction-level exclusion from change streams

[dengwe1]

Add support for excluding transactions from being recorded in the change streams by passing Options.excludeTxnFromChangeStreams() in the various write APIs:

• writeWithOptions
• writeAtleastOnceWithOptions
• batchWriteAtLeastOnce
• executePartitionedUpdate
• readWriteTransaction
• transactionManager
• transactionManagerAsync
• runAsync

Note: Samples and integration tests will be added in separate PRs.

[dengwe1]

@ShuranZhang @arpan14 Please review, thanks!

[ShuranZhang]

Overall looks good, I'll take a another deeper look in details tomorrow.

One major comment:
What will happen if users try to pass Options.excludeTxnFromChangeStreams() to request-level DML APIs which also take UpdateOptions as a parameter(ie:executeUpdate)? Seems like it will simply be a no-op since the new UpdateTransactionOption you added also extends UpdateOptions. I think it's better to prevent users from even trying to set Options.excludeTxnFromChangeStreams() for these request level DML APIs from the client library side, to avoid any potential confusions(ie: they were able to pass the option per-request and run the code, but records are still not excluded).

For example, for above scenarios we might want to throw an error like “Option.excludeTxnFromChangeStreams() is not supported at request level”:

databaseClient
      .readWriteTransaction()
      .run(transaction -> {
        transaction.executeUpdate(
            Statement.of("UPDATE T1 SET Name='fool' WHERE id = 123"),Options.excludeTxnFromChangeStreams());
        return null;
      });

This should also apply for executeUpdateAsync, batchUpdate and batchUpdateAsync.

Thanks!

[dengwe1]

Overall looks good, I'll take a another deeper look in details tomorrow.

One major comment: What will happen if users try to pass Options.excludeTxnFromChangeStreams() to request-level DML APIs which also take UpdateOptions as a parameter(ie:executeUpdate)? Seems like it will simply be a no-op since the new UpdateTransactionOption you added also extends UpdateOptions. I think it's better to prevent users from even trying to set Options.excludeTxnFromChangeStreams() for these request level DML APIs from the client library side, to avoid any potential confusions(ie: they were able to pass the option per-request and run the code, but records are still not excluded).

For example, for above scenarios we might want to throw an error like “Option.excludeTxnFromChangeStreams() is not supported at request level”:

databaseClient
      .readWriteTransaction()
      .run(transaction -> {
        transaction.executeUpdate(
            Statement.of("UPDATE T1 SET Name='fool' WHERE id = 123"),Options.excludeTxnFromChangeStreams());
        return null;
      });

This should also apply for executeUpdateAsync, batchUpdate and batchUpdateAsync.

Thanks!

Great point! Yes, we only want to allow Options.excludeTxnFromChangeStreams() in the executePartitionedUpdate method, but not the other request-level Update methods.

I'm thinking of the following approach:

• Introduce a new interface PartitionedUpdateOption that is only applicable to the executePartitionedUpdate method.
• Modify the executePartitionedUpdate method to take PartitionedUpdateOption instead of UpdateOption.
• Let UpdateOption extend PartitionedUpdateOption, so that existing code that passes UpdateOption to executePartitionedUpdate would still work.
• Let excludeTxnFromChangeStreams() return a sub-type of both PartitionedUpdateOption and TransactionOption. Thus, Options.excludeTxnFromChangeStreams() can be passed to either executePartitionedUpdate or the other write APIs that accept TransactionOption. But it cannot be passed to the other Update methods that expect UpdateOption.

public interface PartitionedUpdateOption {}

public interface UpdateOption extends PartitionedUpdateOption {}

public interface PartitionedUpdateAndTransactionOption extends PartitionedUpdateOption, TransactionOption {}

public static PartitionedUpdateAndTransactionOption excludeTxnFromChangeStreams()

What do you think?

[ShuranZhang]

Overall looks good, I'll take a another deeper look in details tomorrow.
One major comment: What will happen if users try to pass Options.excludeTxnFromChangeStreams() to request-level DML APIs which also take UpdateOptions as a parameter(ie:executeUpdate)? Seems like it will simply be a no-op since the new UpdateTransactionOption you added also extends UpdateOptions. I think it's better to prevent users from even trying to set Options.excludeTxnFromChangeStreams() for these request level DML APIs from the client library side, to avoid any potential confusions(ie: they were able to pass the option per-request and run the code, but records are still not excluded).
For example, for above scenarios we might want to throw an error like “Option.excludeTxnFromChangeStreams() is not supported at request level”:

databaseClient
      .readWriteTransaction()
      .run(transaction -> {
        transaction.executeUpdate(
            Statement.of("UPDATE T1 SET Name='fool' WHERE id = 123"),Options.excludeTxnFromChangeStreams());
        return null;
      });

This should also apply for executeUpdateAsync, batchUpdate and batchUpdateAsync.
Thanks!

Great point! Yes, we only want to allow Options.excludeTxnFromChangeStreams() in the executePartitionedUpdate method, but not the other request-level Update methods.

I'm thinking of the following approach:

• Introduce a new interface PartitionedUpdateOption that is only applicable to the executePartitionedUpdate method.
• Modify the executePartitionedUpdate method to take PartitionedUpdateOption instead of UpdateOption.
• Let UpdateOption extend PartitionedUpdateOption, so that existing code that passes UpdateOption to executePartitionedUpdate would still work.
• Let excludeTxnFromChangeStreams() return a sub-type of both PartitionedUpdateOption and TransactionOption. Thus, Options.excludeTxnFromChangeStreams() can be passed to either executePartitionedUpdate or the other write APIs that accept TransactionOption. But it cannot be passed to the other Update methods that expect UpdateOption.

public interface PartitionedUpdateOption {}

public interface UpdateOption extends PartitionedUpdateOption {}

public interface PartitionedUpdateAndTransactionOption extends PartitionedUpdateOption, TransactionOption {}

public static PartitionedUpdateAndTransactionOption excludeTxnFromChangeStreams()

What do you think?

Sounds good! Thanks for the explanation.
Yeah it would be better in your approach, if the code itself does not compile rather than throwing an error afterwords.
@arpan14 Could you please also take a look and confirm this is an acceptable approach? Thanks!

[dengwe1]

Overall looks good, I'll take a another deeper look in details tomorrow.
One major comment: What will happen if users try to pass Options.excludeTxnFromChangeStreams() to request-level DML APIs which also take UpdateOptions as a parameter(ie:executeUpdate)? Seems like it will simply be a no-op since the new UpdateTransactionOption you added also extends UpdateOptions. I think it's better to prevent users from even trying to set Options.excludeTxnFromChangeStreams() for these request level DML APIs from the client library side, to avoid any potential confusions(ie: they were able to pass the option per-request and run the code, but records are still not excluded).
For example, for above scenarios we might want to throw an error like “Option.excludeTxnFromChangeStreams() is not supported at request level”:

databaseClient
      .readWriteTransaction()
      .run(transaction -> {
        transaction.executeUpdate(
            Statement.of("UPDATE T1 SET Name='fool' WHERE id = 123"),Options.excludeTxnFromChangeStreams());
        return null;
      });

This should also apply for executeUpdateAsync, batchUpdate and batchUpdateAsync.
Thanks!

Great point! Yes, we only want to allow Options.excludeTxnFromChangeStreams() in the executePartitionedUpdate method, but not the other request-level Update methods.
I'm thinking of the following approach:

• Introduce a new interface PartitionedUpdateOption that is only applicable to the executePartitionedUpdate method.
• Modify the executePartitionedUpdate method to take PartitionedUpdateOption instead of UpdateOption.
• Let UpdateOption extend PartitionedUpdateOption, so that existing code that passes UpdateOption to executePartitionedUpdate would still work.
• Let excludeTxnFromChangeStreams() return a sub-type of both PartitionedUpdateOption and TransactionOption. Thus, Options.excludeTxnFromChangeStreams() can be passed to either executePartitionedUpdate or the other write APIs that accept TransactionOption. But it cannot be passed to the other Update methods that expect UpdateOption.

public interface PartitionedUpdateOption {}

public interface UpdateOption extends PartitionedUpdateOption {}

public interface PartitionedUpdateAndTransactionOption extends PartitionedUpdateOption, TransactionOption {}

public static PartitionedUpdateAndTransactionOption excludeTxnFromChangeStreams()

What do you think?

Sounds good! Thanks for the explanation. Yeah it would be better in your approach, if the code itself does not compile rather than throwing an error afterwords. @arpan14 Could you please also take a look and confirm this is an acceptable approach? Thanks!

Thanks @olavloite for pointing out that this will break binary compatibility. I have reverted this change and added error handling in those Update methods.