-
Notifications
You must be signed in to change notification settings - Fork 239
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
- Loading branch information
1 parent
b6c7578
commit 158bf2c
Showing
7 changed files
with
67 additions
and
0 deletions.
There are no files selected for viewing
20 changes: 20 additions & 0 deletions
20
resources/dataflowjob/batch-dataflow-job/dataflow_v1beta1_dataflowjob.yaml
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,20 @@ | ||
apiVersion: dataflow.cnrm.cloud.google.com/v1beta1 | ||
kind: DataflowJob | ||
metadata: | ||
annotations: | ||
cnrm.cloud.google.com/on-delete: "cancel" | ||
labels: | ||
label-one: "value-one" | ||
name: dataflowjob-sample-batch | ||
spec: | ||
tempGcsLocation: gs://dataflowjob-dep-batch/tmp | ||
# This is a public, Google-maintained Dataflow Job template of a batch job | ||
templateGcsPath: gs://dataflow-templates/2020-02-03-01_RC00/Word_Count | ||
parameters: | ||
# This is a public, Google-maintained text file | ||
inputFile: gs://dataflow-samples/shakespeare/various.txt | ||
output: gs://dataflowjob-dep-batch/output | ||
zone: us-central1-a | ||
machineType: "n1-standard-1" | ||
maxWorkers: 3 | ||
ipConfiguration: "WORKER_IP_PUBLIC" |
6 changes: 6 additions & 0 deletions
6
resources/dataflowjob/batch-dataflow-job/storage_v1beta1_storagebucket.yaml
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,6 @@ | ||
apiVersion: storage.cnrm.cloud.google.com/v1beta1 | ||
kind: StorageBucket | ||
metadata: | ||
annotations: | ||
cnrm.cloud.google.com/force-destroy: "true" | ||
name: dataflowjob-dep-batch |
4 changes: 4 additions & 0 deletions
4
resources/dataflowjob/streaming-dataflow-job/bigquery_v1beta1_bigquerydataset.yaml
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,4 @@ | ||
apiVersion: bigquery.cnrm.cloud.google.com/v1beta1 | ||
kind: BigQueryDataset | ||
metadata: | ||
name: dataflowjobdepstreaming |
7 changes: 7 additions & 0 deletions
7
resources/dataflowjob/streaming-dataflow-job/bigquery_v1beta1_bigquerytable.yaml
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,7 @@ | ||
apiVersion: bigquery.cnrm.cloud.google.com/v1beta1 | ||
kind: BigQueryTable | ||
metadata: | ||
name: dataflowjobdepstreaming | ||
spec: | ||
datasetRef: | ||
name: dataflowjobdepstreaming |
20 changes: 20 additions & 0 deletions
20
resources/dataflowjob/streaming-dataflow-job/dataflow_v1beta1_dataflowjob.yaml
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,20 @@ | ||
apiVersion: dataflow.cnrm.cloud.google.com/v1beta1 | ||
kind: DataflowJob | ||
metadata: | ||
annotations: | ||
cnrm.cloud.google.com/on-delete: "cancel" | ||
labels: | ||
label-one: "value-one" | ||
name: dataflowjob-sample-streaming | ||
spec: | ||
tempGcsLocation: gs://dataflowjob-dep-streaming/tmp | ||
# This is a public, Google-maintained Dataflow Job template of a streaming job | ||
templateGcsPath: gs://dataflow-templates/2020-02-03-01_RC00/PubSub_to_BigQuery | ||
parameters: | ||
# replace ${PROJECT_ID?} with your project name | ||
inputTopic: projects/${PROJECT_ID?}/topics/dataflowjob-dep-streaming | ||
outputTableSpec: ${PROJECT_ID?}:dataflowjobdepstreaming.dataflowjobdepstreaming | ||
zone: us-central1-a | ||
machineType: "n1-standard-1" | ||
maxWorkers: 3 | ||
ipConfiguration: "WORKER_IP_PUBLIC" |
4 changes: 4 additions & 0 deletions
4
resources/dataflowjob/streaming-dataflow-job/pubsub_v1beta1_pubsubtopic.yaml
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,4 @@ | ||
apiVersion: pubsub.cnrm.cloud.google.com/v1beta1 | ||
kind: PubSubTopic | ||
metadata: | ||
name: dataflowjob-dep-streaming |
6 changes: 6 additions & 0 deletions
6
resources/dataflowjob/streaming-dataflow-job/storage_v1beta1_storagebucket.yaml
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,6 @@ | ||
apiVersion: storage.cnrm.cloud.google.com/v1beta1 | ||
kind: StorageBucket | ||
metadata: | ||
annotations: | ||
cnrm.cloud.google.com/force-destroy: "true" | ||
name: dataflowjob-dep-streaming |