From 6f2370bef3bc4363cb7b3ff995bf75c79371ede0 Mon Sep 17 00:00:00 2001 From: Karim Wadie Date: Tue, 14 Feb 2023 16:06:47 +0100 Subject: [PATCH] updated readme with new diagram --- README.md | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/README.md b/README.md index 6579007..017985d 100644 --- a/README.md +++ b/README.md @@ -116,10 +116,11 @@ A cloud scheduler is used to send a BigQuery “Scan Scope” to the dispatcher #### BigQuery Snapshoter * Calls the BigQuery API to execute a snapshot operation given the snapshot config passed from the previous service * Waits for the operation to report results, if the return status is success it sends a tagging request to the Tagger service +* Snapshot jobs finishes almost instantaneously regardless of table size, so there are no benefits in async calls. #### GCS Snapshoter * Calls the BigQuery API to execute a full table export to GCS given the snapshot config passed from the previous service -* Submits the export job asynchronously +* Export jobs could take few minutes depending on the table size. Thus, the export job is submitted asynchronously * Stage a tagging request to a persistent storage (i.e. Cloud Storage) and terminate the HTTP call * When the export job completes, BigQuery will log an event to Cloud Logging that is captured by a Log Sink and sent to the Tagger service