Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: support append() without offset in StreamWriter #1452

Merged
Merged
Show file tree
Hide file tree
Changes from 1 commit
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
Expand Up @@ -80,7 +80,8 @@ private JsonStreamWriter(Builder builder)

/**
* Writes a JSONArray that contains JSONObjects to the BigQuery table by first converting the JSON
* data to protobuf messages, then using StreamWriter's append() to write the data.
* data to protobuf messages, then using StreamWriter's append() to write the data at current
* end of stream.
*
* @param jsonArr The JSON array that contains JSONObjects to be written
* @return ApiFuture<AppendRowsResponse> returns an AppendRowsResponse message wrapped in an
Expand All @@ -92,7 +93,8 @@ public ApiFuture<AppendRowsResponse> append(JSONArray jsonArr) {

/**
* Writes a JSONArray that contains JSONObjects to the BigQuery table by first converting the JSON
* data to protobuf messages, then using StreamWriter's append() to write the data.
* data to protobuf messages, then using StreamWriter's append() to write the data at the
* specified offset.
*
* @param jsonArr The JSON array that contains JSONObjects to be written
* @param offset Offset for deduplication
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -224,14 +224,23 @@ public void run(Throwable finalStatus) {
}

/**
* Schedules the writing of a message.
* Schedules the writing of rows at the end of current stream.
*
* <p>Example of writing a message.
* @param rows the rows in serialized format to write to BigQuery.
* @return the append response wrapped in a future.
*/
public ApiFuture<AppendRowsResponse> append(ProtoRows rows) {
return append(rows, -1);
}

/**
* Schedules the writing of rows at given offset.
*
* <p>Example of writing rows with specific offset.
*
* <pre>{@code
* AppendRowsRequest message;
* ApiFuture<AppendRowsResponse> messageIdFuture = writer.append(message);
* ApiFutures.addCallback(messageIdFuture, new ApiFutureCallback<AppendRowsResponse>() {
* ApiFuture<AppendRowsResponse> future = writer.append(rows, 0);
* ApiFutures.addCallback(future, new ApiFutureCallback<AppendRowsResponse>() {
* public void onSuccess(AppendRowsResponse response) {
* if (!response.hasError()) {
* System.out.println("written with offset: " + response.getAppendResult().getOffset());
Expand All @@ -247,7 +256,7 @@ public void run(Throwable finalStatus) {
* }</pre>
*
* @param rows the rows in serialized format to write to BigQuery.
* @param offset the offset of the first row.
* @param offset the offset of the first row. Provide -1 to write at the current end of stream.
* @return the append response wrapped in a future.
*/
public ApiFuture<AppendRowsResponse> append(ProtoRows rows, long offset) {
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -131,7 +131,7 @@ private AppendRowsResponse createAppendResponseWithError(Status.Code code, Strin
}

private ApiFuture<AppendRowsResponse> sendTestMessage(StreamWriter writer, String[] messages) {
return writer.append(createProtoRows(messages), -1);
return writer.append(createProtoRows(messages));
}

private ApiFuture<AppendRowsResponse> sendTestMessage(
Expand Down