Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fixes an issue with splitReadStream #6112

Closed
wants to merge 2 commits into from
Closed

Fixes an issue with splitReadStream #6112

Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
be995b6
Remove auto-generated comments
aryann Aug 20, 2019
15bfb9e
Add a split fraction
aryann Aug 20, 2019
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
23 changes: 8 additions & 15 deletions ...torage/src/main/java/com/google/cloud/bigquery/storage/v1beta1/BigQueryStorageClient.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -203,7 +203,6 @@ public final ReadSession createReadSession(
return createReadSession(request);
}

// AUTO-GENERATED DOCUMENTATION AND METHOD
/**
* Creates a new read session. A read session divides the contents of a BigQuery table into one or
* more streams, which can then be used to read data from the table. The read session also
Expand Down Expand Up @@ -237,7 +236,6 @@ public final ReadSession createReadSession(CreateReadSessionRequest request) {
return createReadSessionCallable().call(request);
}

// AUTO-GENERATED DOCUMENTATION AND METHOD
/**
* Creates a new read session. A read session divides the contents of a BigQuery table into one or
* more streams, which can then be used to read data from the table. The read session also
Expand Down Expand Up @@ -270,7 +268,6 @@ public final UnaryCallable<CreateReadSessionRequest, ReadSession> createReadSess
return stub.createReadSessionCallable();
}

// AUTO-GENERATED DOCUMENTATION AND METHOD
/**
* Reads rows from the table in the format prescribed by the read session. Each response contains
* one or more table rows, up to a maximum of 10 MiB per response; read requests which attempt to
Expand Down Expand Up @@ -301,7 +298,6 @@ public final ServerStreamingCallable<ReadRowsRequest, ReadRowsResponse> readRows
return stub.readRowsCallable();
}

// AUTO-GENERATED DOCUMENTATION AND METHOD
/**
* Creates additional streams for a ReadSession. This API can be used to dynamically adjust the
* parallelism of a batch processing task upwards by adding additional workers.
Expand Down Expand Up @@ -333,7 +329,6 @@ public final BatchCreateReadSessionStreamsResponse batchCreateReadSessionStreams
return batchCreateReadSessionStreams(request);
}

// AUTO-GENERATED DOCUMENTATION AND METHOD
/**
* Creates additional streams for a ReadSession. This API can be used to dynamically adjust the
* parallelism of a batch processing task upwards by adding additional workers.
Expand All @@ -360,7 +355,6 @@ public final BatchCreateReadSessionStreamsResponse batchCreateReadSessionStreams
return batchCreateReadSessionStreamsCallable().call(request);
}

// AUTO-GENERATED DOCUMENTATION AND METHOD
/**
* Creates additional streams for a ReadSession. This API can be used to dynamically adjust the
* parallelism of a batch processing task upwards by adding additional workers.
Expand All @@ -387,7 +381,6 @@ public final BatchCreateReadSessionStreamsResponse batchCreateReadSessionStreams
return stub.batchCreateReadSessionStreamsCallable();
}

// AUTO-GENERATED DOCUMENTATION AND METHOD
/**
* Triggers the graceful termination of a single stream in a ReadSession. This API can be used to
* dynamically adjust the parallelism of a batch processing task downwards without losing data.
Expand Down Expand Up @@ -418,7 +411,6 @@ public final void finalizeStream(Stream stream) {
finalizeStream(request);
}

// AUTO-GENERATED DOCUMENTATION AND METHOD
/**
* Triggers the graceful termination of a single stream in a ReadSession. This API can be used to
* dynamically adjust the parallelism of a batch processing task downwards without losing data.
Expand Down Expand Up @@ -451,7 +443,6 @@ public final void finalizeStream(FinalizeStreamRequest request) {
finalizeStreamCallable().call(request);
}

// AUTO-GENERATED DOCUMENTATION AND METHOD
/**
* Triggers the graceful termination of a single stream in a ReadSession. This API can be used to
* dynamically adjust the parallelism of a batch processing task downwards without losing data.
Expand Down Expand Up @@ -483,7 +474,6 @@ public final UnaryCallable<FinalizeStreamRequest, Empty> finalizeStreamCallable(
return stub.finalizeStreamCallable();
}

// AUTO-GENERATED DOCUMENTATION AND METHOD
/**
* Splits a given read stream into two Streams. These streams are referred to as the primary and
* the residual of the split. The original stream can still be read from in the same manner as
Expand All @@ -501,21 +491,24 @@ public final UnaryCallable<FinalizeStreamRequest, Empty> finalizeStreamCallable(
*
* <pre><code>
* try (BigQueryStorageClient bigQueryStorageClient = BigQueryStorageClient.create()) {
* Stream originalStream = Stream.newBuilder().build();
* Stream originalStream = getStream();
* SplitReadStreamResponse response = bigQueryStorageClient.splitReadStream(originalStream);
* }
* </code></pre>
*
* @param originalStream Stream to split.
* @throws com.google.api.gax.rpc.ApiException if the remote call fails
* @deprecated Use {@link BigQueryStorageClient#splitReadStream(SplitReadStreamRequest)}.
*/
public final SplitReadStreamResponse splitReadStream(Stream originalStream) {
SplitReadStreamRequest request =
SplitReadStreamRequest.newBuilder().setOriginalStream(originalStream).build();
SplitReadStreamRequest.newBuilder()
.setOriginalStream(originalStream)
.setFraction(0.5f)
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would suggest to move the 0.5f value to a constant STREAM_SPLIT_FRACTION and use it in all places!

.build();
return splitReadStream(request);
}

// AUTO-GENERATED DOCUMENTATION AND METHOD
/**
* Splits a given read stream into two Streams. These streams are referred to as the primary and
* the residual of the split. The original stream can still be read from in the same manner as
Expand All @@ -533,9 +526,10 @@ public final SplitReadStreamResponse splitReadStream(Stream originalStream) {
*
* <pre><code>
* try (BigQueryStorageClient bigQueryStorageClient = BigQueryStorageClient.create()) {
* Stream originalStream = Stream.newBuilder().build();
* Stream originalStream = getStream()
* SplitReadStreamRequest request = SplitReadStreamRequest.newBuilder()
* .setOriginalStream(originalStream)
* .setFraction(0.5f)
* .build();
* SplitReadStreamResponse response = bigQueryStorageClient.splitReadStream(request);
* }
Expand All @@ -548,7 +542,6 @@ public final SplitReadStreamResponse splitReadStream(SplitReadStreamRequest requ
return splitReadStreamCallable().call(request);
}

// AUTO-GENERATED DOCUMENTATION AND METHOD
/**
* Splits a given read stream into two Streams. These streams are referred to as the primary and
* the residual of the split. The original stream can still be read from in the same manner as
Expand Down