chore: move samples from python-docs-sample (#66)

* Add XMPP Sample

* Add Dataproc Sample

* Add more region tags

* Minor dataproc fixes

* Fix Dataproc e2e for Python 3

* Update reqs

* updating requirements [(#358)](GoogleCloudPlatform/python-docs-samples#358)

Change-Id: I6177a17fad021e26ed76679d9db34848c17b62a8

* Update Reqs

* Wrong arg description

* Auto-update dependencies. [(#456)](GoogleCloudPlatform/python-docs-samples#456)

* Auto-update dependencies. [(#459)](GoogleCloudPlatform/python-docs-samples#459)

* Fix import order lint errors

Change-Id: Ieaf7237fc6f925daec46a07d2e81a452b841198a

* bump

Change-Id: I02e7767d13ba267ee9fc72c5b68a57013bb8b8d3

* Auto-update dependencies. [(#486)](GoogleCloudPlatform/python-docs-samples#486)

* Auto-update dependencies. [(#540)](GoogleCloudPlatform/python-docs-samples#540)

* Auto-update dependencies. [(#542)](GoogleCloudPlatform/python-docs-samples#542)

* Move to google-cloud [(#544)](GoogleCloudPlatform/python-docs-samples#544)

* Auto-update dependencies. [(#584)](GoogleCloudPlatform/python-docs-samples#584)

* Auto-update dependencies. [(#629)](GoogleCloudPlatform/python-docs-samples#629)

* Update samples to support latest Google Cloud Python [(#656)](GoogleCloudPlatform/python-docs-samples#656)

* Update README.md [(#691)](GoogleCloudPlatform/python-docs-samples#691)

* Auto-update dependencies. [(#715)](GoogleCloudPlatform/python-docs-samples#715)

* Auto-update dependencies. [(#735)](GoogleCloudPlatform/python-docs-samples#735)

* Auto-update dependencies.
* Fix language OCR sample
* Remove unused import

* Auto-update dependencies. [(#790)](GoogleCloudPlatform/python-docs-samples#790)

* Remove usage of GoogleCredentials [(#810)](GoogleCloudPlatform/python-docs-samples#810)

* Fix a typo [(#813)](GoogleCloudPlatform/python-docs-samples#813)

* Remove cloud config fixture [(#887)](GoogleCloudPlatform/python-docs-samples#887)

* Remove cloud config fixture

* Fix client secrets

* Fix bigtable instance

* Fix reference to our testing tools

* Auto-update dependencies. [(#914)](GoogleCloudPlatform/python-docs-samples#914)

* Auto-update dependencies.

* xfail the error reporting test

* Fix lint

* Auto-update dependencies. [(#922)](GoogleCloudPlatform/python-docs-samples#922)

* Auto-update dependencies.

* Fix pubsub iam samples

* Auto-update dependencies. [(#1005)](GoogleCloudPlatform/python-docs-samples#1005)

* Auto-update dependencies.

* Fix bigtable lint

* Fix IOT iam interaction

* Auto-update dependencies. [(#1011)](GoogleCloudPlatform/python-docs-samples#1011)

* Properly forwarding the "region" parameter provided as an input argument. [(#1029)](GoogleCloudPlatform/python-docs-samples#1029)

* Auto-update dependencies. [(#1055)](GoogleCloudPlatform/python-docs-samples#1055)

* Auto-update dependencies.

* Explicitly use latest bigtable client

Change-Id: Id71e9e768f020730e4ca9514a0d7ebaa794e7d9e

* Revert language update for now

Change-Id: I8867f154e9a5aae00d0047c9caf880e5e8f50c53

* Remove pdb. smh

Change-Id: I5ff905fadc026eebbcd45512d4e76e003e3b2b43

* Fix region handling and allow to use an existing cluster. [(#1053)](GoogleCloudPlatform/python-docs-samples#1053)

* Auto-update dependencies. [(#1094)](GoogleCloudPlatform/python-docs-samples#1094)

* Auto-update dependencies.

* Relax assertions in the ocr_nl sample

Change-Id: I6d37e5846a8d6dd52429cb30d501f448c52cbba1

* Drop unused logging apiary samples

Change-Id: I545718283773cb729a5e0def8a76ebfa40829d51

* Auto-update dependencies. [(#1133)](GoogleCloudPlatform/python-docs-samples#1133)

* Auto-update dependencies.

* Fix missing http library

Change-Id: I99faa600f2f3f1f50f57694fc9835d7f35bda250

* Auto-update dependencies. [(#1186)](GoogleCloudPlatform/python-docs-samples#1186)

* Auto-update dependencies. [(#1199)](GoogleCloudPlatform/python-docs-samples#1199)

* Auto-update dependencies.

* Fix iot lint

Change-Id: I6289e093bdb35e38f9e9bfc3fbc3df3660f9a67e

* Fixed Failed Kokoro Test (Dataproc) [(#1203)](GoogleCloudPlatform/python-docs-samples#1203)

* Fixed Failed Kokoro Test (Dataproc)

* Fixed Lint Error

* Update dataproc_e2e_test.py

* Update dataproc_e2e_test.py

* Fixing More Lint Errors

* Fixed b/65407087

* Revert "Merge branch 'master' of https://github.com/michaelawyu/python-docs-samples"

This reverts commit 1614c7d3ef33630a8ab095792b27fc25fd91f0ad, reversing
changes made to cd1dbfd25997a154a8a85cc754cc2a85b18a63c4.

* Revert "Fixed b/65407087"

This reverts commit cd1dbfd25997a154a8a85cc754cc2a85b18a63c4.

* Fixed Lint Error

* Fixed Lint Error

* Auto-update dependencies. [(#1208)](GoogleCloudPlatform/python-docs-samples#1208)

* Dataproc GCS sample plus doc touchups [(#1151)](GoogleCloudPlatform/python-docs-samples#1151)

* Auto-update dependencies. [(#1217)](GoogleCloudPlatform/python-docs-samples#1217)

* Auto-update dependencies. [(#1239)](GoogleCloudPlatform/python-docs-samples#1239)

* Added "Open in Cloud Shell" buttons to README files [(#1254)](GoogleCloudPlatform/python-docs-samples#1254)

* Auto-update dependencies. [(#1282)](GoogleCloudPlatform/python-docs-samples#1282)

* Auto-update dependencies.

* Fix storage acl sample

Change-Id: I413bea899fdde4c4859e4070a9da25845b81f7cf

* Auto-update dependencies. [(#1309)](GoogleCloudPlatform/python-docs-samples#1309)

* Auto-update dependencies. [(#1320)](GoogleCloudPlatform/python-docs-samples#1320)

* Auto-update dependencies. [(#1355)](GoogleCloudPlatform/python-docs-samples#1355)

* Auto-update dependencies. [(#1359)](GoogleCloudPlatform/python-docs-samples#1359)

* Auto-update dependencies.

* update Dataproc region tags to standard format [(#1826)](GoogleCloudPlatform/python-docs-samples#1826)

* Update submit_job_to_cluster.py [(#1708)](GoogleCloudPlatform/python-docs-samples#1708)

switch region to new 'global' region and remove unnecessary function.

* Auto-update dependencies. [(#1846)](GoogleCloudPlatform/python-docs-samples#1846)

ACK, merging.

* Need separate install for google-cloud-storage [(#1863)](GoogleCloudPlatform/python-docs-samples#1863)

* Revert "Update dataproc/submit_job_to_cluster.py" [(#1864)](GoogleCloudPlatform/python-docs-samples#1864)

* Revert "Remove test configs for non-testing directories [(#1855)](GoogleCloudPlatform/python-docs-samples#1855)"

This reverts commit 73a73321579337312e8ba85c34fe9c37b42b7f6e.

* Revert "Auto-update dependencies. [(#1846)](GoogleCloudPlatform/python-docs-samples#1846)"

This reverts commit 3adc94f.

* Revert "Tweak slack sample [(#1847)](GoogleCloudPlatform/python-docs-samples#1847)"

This reverts commit a48c010481c166968d9f1bd58106054c5d1c58f9.

* Revert "Non-client library example of constructing a Signed URL [(#1837)](GoogleCloudPlatform/python-docs-samples#1837)"

This reverts commit fc3284d995a8a35c473a207e80490fad265782af.

* Revert "GCF samples: handle {empty JSON, GET} requests + remove commas [(#1832)](GoogleCloudPlatform/python-docs-samples#1832)"

This reverts commit 6928491ed3d52b0bec694e6b30257f08caac5f2b.

* Revert "Correct the maintenance event types [(#1830)](GoogleCloudPlatform/python-docs-samples#1830)"

This reverts commit c22840fd23586349b7b665d851dea046a94ba7c7.

* Revert "Fix GCF region tags [(#1827)](GoogleCloudPlatform/python-docs-samples#1827)"

This reverts commit 0fbfef27d35cea23ad0e20fd2c9df3e8a4a046cb.

* Revert "Updated to Flask 1.0 [(#1819)](GoogleCloudPlatform/python-docs-samples#1819)"

This reverts commit d52ccf99503311bba2cec2881e8cb0f9b5a6f2bf.

* Revert "Fix deprecation warning [(#1801)](GoogleCloudPlatform/python-docs-samples#1801)"

This reverts commit 981737e85f60eca5cc337f172249deddca9b291b.

* Revert "Update submit_job_to_cluster.py [(#1708)](GoogleCloudPlatform/python-docs-samples#1708)"

This reverts commit df1f2b2.

* Create python-api-walkthrough.md [(#1966)](GoogleCloudPlatform/python-docs-samples#1966)

* Create python-api-walkthrough.md

This Google Cloud Shell walkthrough is linked to Cloud Dataproc documentation to be published at: https://cloud.google.com/dataproc/docs/tutorials/python-library-example

* Update python-api-walkthrough.md

* Update list_clusters.py [(#1887)](GoogleCloudPlatform/python-docs-samples#1887)

* Auto-update dependencies. [(#1980)](GoogleCloudPlatform/python-docs-samples#1980)

* Auto-update dependencies.

* Update requirements.txt

* Update requirements.txt

* Update Dataproc samples. [(#2158)](GoogleCloudPlatform/python-docs-samples#2158)

* Update requirements.txt

* Update python-api-walkthrough.md

* Update submit_job_to_cluster.py

* Update list_clusters.py

* Update python-api-walkthrough.md [(#2172)](GoogleCloudPlatform/python-docs-samples#2172)

* Adds updates including compute [(#2436)](GoogleCloudPlatform/python-docs-samples#2436)

* Adds updates including compute

* Python 2 compat pytest

* Fixing weird \r\n issue from GH merge

* Put asset tests back in

* Re-add pod operator test

* Hack parameter for k8s pod operator

* feat: adding samples for dataproc - create cluster [(#2536)](GoogleCloudPlatform/python-docs-samples#2536)

* adding sample for cluster create

* small fix

* Add create cluster samples

* Fixed copyright, added 'dataproc' to region tag and changed imports from 'dataproc' to 'dataproc_v1'

* Fix copyright in create_cluster.py

* Auto-update dependencies. [(#2005)](GoogleCloudPlatform/python-docs-samples#2005)

* Auto-update dependencies.

* Revert update of appengine/flexible/datastore.

* revert update of appengine/flexible/scipy

* revert update of bigquery/bqml

* revert update of bigquery/cloud-client

* revert update of bigquery/datalab-migration

* revert update of bigtable/quickstart

* revert update of compute/api

* revert update of container_registry/container_analysis

* revert update of dataflow/run_template

* revert update of datastore/cloud-ndb

* revert update of dialogflow/cloud-client

* revert update of dlp

* revert update of functions/imagemagick

* revert update of functions/ocr/app

* revert update of healthcare/api-client/fhir

* revert update of iam/api-client

* revert update of iot/api-client/gcs_file_to_device

* revert update of iot/api-client/mqtt_example

* revert update of language/automl

* revert update of run/image-processing

* revert update of vision/automl

* revert update testing/requirements.txt

* revert update of vision/cloud-client/detect

* revert update of vision/cloud-client/product_search

* revert update of jobs/v2/api_client

* revert update of jobs/v3/api_client

* revert update of opencensus

* revert update of translate/cloud-client

* revert update to speech/cloud-client

Co-authored-by: Kurtis Van Gent <[email protected]>
Co-authored-by: Doug Mahugh <[email protected]>

* feat: dataproc quickstart sample added and create_cluster updated [(#2629)](GoogleCloudPlatform/python-docs-samples#2629)

* Adding quickstart sample

* Added new quickstart sample and updated create_cluster sample

* Fix to create_cluster.py

* deleted dataproc quickstart files not under dataproc/quickstart/

* Added quickstart test

* Linting and formatting fixes

* Revert "Linting and formatting fixes"

This reverts commit c5afcbcdf9deccbb7a21ddd82ae0fc305e79c008.

* Added bucket cleanup to quickstart test

* Changes to samples and tests

* Linting fixes

* Removed todos in favor of clearer docstring

* Fixed lint error

Co-authored-by: Leah E. Cole <[email protected]>

* Update Python Cloud Shell walkthrough script [(#2733)](GoogleCloudPlatform/python-docs-samples#2733)

Cloud Shell walkthrough scripts no longer support enabling APIs. APIs must be enabled by linking to the console.
Updated product name: "Cloud Dataproc" -> "Dataproc".

* fix: added cli functionality to dataproc quickstart example [(#2734)](GoogleCloudPlatform/python-docs-samples#2734)

* Added CLI functionality to quickstart

* Fixed Dataproc quickstart test to properly clean up GCS bucket [(#3001)](GoogleCloudPlatform/python-docs-samples#3001)

* splitting up #2651 part 1/3 - dataproc + endpoints [(#3025)](GoogleCloudPlatform/python-docs-samples#3025)

* splitting up #2651

* fix typos

* chore(deps): update dependency google-auth to v1.11.2 [(#2724)](GoogleCloudPlatform/python-docs-samples#2724)

Co-authored-by: Leah E. Cole <[email protected]>

* chore(deps): update dependency google-cloud-storage to v1.26.0 [(#3046)](GoogleCloudPlatform/python-docs-samples#3046)

* chore(deps): update dependency google-cloud-storage to v1.26.0

* chore(deps): specify dependencies by python version

* chore: up other deps to try to remove errors

Co-authored-by: Leah E. Cole <[email protected]>
Co-authored-by: Leah Cole <[email protected]>

* chore(deps): update dependency google-cloud-dataproc to v0.7.0 [(#3083)](GoogleCloudPlatform/python-docs-samples#3083)

* feat: added dataproc workflows samples [(#3056)](GoogleCloudPlatform/python-docs-samples#3056)

* Added workflows sample

* chore(deps): update dependency grpcio to v1.27.2 [(#3173)](GoogleCloudPlatform/python-docs-samples#3173)

This PR contains the following updates:

| Package | Update | Change |
|---|---|---|
| [grpcio](https://grpc.io) | minor | `==1.25.0` -> `==1.27.2` |
| [grpcio](https://grpc.io) | minor | `==1.23.0` -> `==1.27.2` |
| [grpcio](https://grpc.io) | minor | `==1.26.0` -> `==1.27.2` |
| [grpcio](https://grpc.io) | patch | `==1.27.1` -> `==1.27.2` |

---

### Renovate configuration

:date: **Schedule**: At any time (no schedule defined).

:vertical_traffic_light: **Automerge**: Disabled by config. Please merge this manually once you are satisfied.

:recycle: **Rebasing**: Never, or you tick the rebase/retry checkbox.

:no_bell: **Ignore**: Close this PR and you won't be reminded about these updates again.

---

 - [ ] <!-- rebase-check -->If you want to rebase/retry this PR, check this box

---

This PR has been generated by [WhiteSource Renovate](https://renovate.whitesourcesoftware.com). View repository job log [here](https://app.renovatebot.com/dashboard#GoogleCloudPlatform/python-docs-samples).

* Simplify noxfile setup. [(#2806)](GoogleCloudPlatform/python-docs-samples#2806)

* chore(deps): update dependency requests to v2.23.0

* Simplify noxfile and add version control.

* Configure appengine/standard to only test Python 2.7.

* Update Kokokro configs to match noxfile.

* Add requirements-test to each folder.

* Remove Py2 versions from everything execept appengine/standard.

* Remove conftest.py.

* Remove appengine/standard/conftest.py

* Remove 'no-sucess-flaky-report' from pytest.ini.

* Add GAE SDK back to appengine/standard tests.

* Fix typo.

* Roll pytest to python 2 version.

* Add a bunch of testing requirements.

* Remove typo.

* Add appengine lib directory back in.

* Add some additional requirements.

* Fix issue with flake8 args.

* Even more requirements.

* Readd appengine conftest.py.

* Add a few more requirements.

* Even more Appengine requirements.

* Add webtest for appengine/standard/mailgun.

* Add some additional requirements.

* Add workaround for issue with mailjet-rest.

* Add responses for appengine/standard/mailjet.

Co-authored-by: Renovate Bot <[email protected]>

* fix: add mains to samples [(#3284)](GoogleCloudPlatform/python-docs-samples#3284)

Added mains to two samples: create_cluster and instantiate_inline_workflow_templates.

Fixed their associated tests to accommodate this.

Removed subprocess from quickstart/quickstart_test.py to fix [2873](GoogleCloudPlatform/python-docs-samples#2873)

fixes #2873

* Update dependency grpcio to v1.28.1 [(#3276)](GoogleCloudPlatform/python-docs-samples#3276)

Co-authored-by: Leah E. Cole <[email protected]>

* Update dependency google-auth to v1.14.0 [(#3148)](GoogleCloudPlatform/python-docs-samples#3148)

Co-authored-by: Leah E. Cole <[email protected]>

* chore(deps): update dependency google-auth to v1.14.1 [(#3464)](GoogleCloudPlatform/python-docs-samples#3464)

This PR contains the following updates:

| Package | Update | Change |
|---|---|---|
| [google-auth](https://togithub.com/googleapis/google-auth-library-python) | patch | `==1.14.0` -> `==1.14.1` |
| [google-auth](https://togithub.com/googleapis/google-auth-library-python) | minor | `==1.11.2` -> `==1.14.1` |

---

### Release Notes

<details>
<summary>googleapis/google-auth-library-python</summary>

### [`v1.14.1`](https://togithub.com/googleapis/google-auth-library-python/blob/master/CHANGELOG.md#&#8203;1141-httpswwwgithubcomgoogleapisgoogle-auth-library-pythoncomparev1140v1141-2020-04-21)

[Compare Source](https://togithub.com/googleapis/google-auth-library-python/compare/v1.14.0...v1.14.1)

</details>

---

### Renovate configuration

:date: **Schedule**: At any time (no schedule defined).

:vertical_traffic_light: **Automerge**: Disabled by config. Please merge this manually once you are satisfied.

:recycle: **Rebasing**: Never, or you tick the rebase/retry checkbox.

:no_bell: **Ignore**: Close this PR and you won't be reminded about these updates again.

---

 - [ ] <!-- rebase-check -->If you want to rebase/retry this PR, check this box

---

This PR has been generated by [WhiteSource Renovate](https://renovate.whitesourcesoftware.com). View repository job log [here](https://app.renovatebot.com/dashboard#GoogleCloudPlatform/python-docs-samples).

* chore(deps): update dependency google-cloud-storage to v1.28.0 [(#3260)](GoogleCloudPlatform/python-docs-samples#3260)

Co-authored-by: Takashi Matsuo <[email protected]>

* chore(deps): update dependency google-auth to v1.14.2 [(#3724)](GoogleCloudPlatform/python-docs-samples#3724)

This PR contains the following updates:

| Package | Update | Change |
|---|---|---|
| [google-auth](https://togithub.com/googleapis/google-auth-library-python) | patch | `==1.14.1` -> `==1.14.2` |

---

### Release Notes

<details>
<summary>googleapis/google-auth-library-python</summary>

### [`v1.14.2`](https://togithub.com/googleapis/google-auth-library-python/blob/master/CHANGELOG.md#&#8203;1142-httpswwwgithubcomgoogleapisgoogle-auth-library-pythoncomparev1141v1142-2020-05-07)

[Compare Source](https://togithub.com/googleapis/google-auth-library-python/compare/v1.14.1...v1.14.2)

</details>

---

### Renovate configuration

:date: **Schedule**: At any time (no schedule defined).

:vertical_traffic_light: **Automerge**: Disabled by config. Please merge this manually once you are satisfied.

:recycle: **Rebasing**: Never, or you tick the rebase/retry checkbox.

:no_bell: **Ignore**: Close this PR and you won't be reminded about this update again.

---

 - [ ] <!-- rebase-check -->If you want to rebase/retry this PR, check this box

---

This PR has been generated by [WhiteSource Renovate](https://renovate.whitesourcesoftware.com). View repository job log [here](https://app.renovatebot.com/dashboard#GoogleCloudPlatform/python-docs-samples).

* chore: some lint fixes [(#3743)](GoogleCloudPlatform/python-docs-samples#3743)

* chore(deps): update dependency google-auth to v1.14.3 [(#3728)](GoogleCloudPlatform/python-docs-samples#3728)

This PR contains the following updates:

| Package | Update | Change |
|---|---|---|
| [google-auth](https://togithub.com/googleapis/google-auth-library-python) | patch | `==1.14.2` -> `==1.14.3` |

---

### Release Notes

<details>
<summary>googleapis/google-auth-library-python</summary>

### [`v1.14.3`](https://togithub.com/googleapis/google-auth-library-python/blob/master/CHANGELOG.md#&#8203;1143-httpswwwgithubcomgoogleapisgoogle-auth-library-pythoncomparev1142v1143-2020-05-11)

[Compare Source](https://togithub.com/googleapis/google-auth-library-python/compare/v1.14.2...v1.14.3)

</details>

---

### Renovate configuration

:date: **Schedule**: At any time (no schedule defined).

:vertical_traffic_light: **Automerge**: Disabled by config. Please merge this manually once you are satisfied.

:recycle: **Rebasing**: Never, or you tick the rebase/retry checkbox.

:no_bell: **Ignore**: Close this PR and you won't be reminded about this update again.

---

 - [x] <!-- rebase-check -->If you want to rebase/retry this PR, check this box

---

This PR has been generated by [WhiteSource Renovate](https://renovate.whitesourcesoftware.com). View repository job log [here](https://app.renovatebot.com/dashboard#GoogleCloudPlatform/python-docs-samples).

* chore(deps): update dependency grpcio to v1.29.0 [(#3786)](GoogleCloudPlatform/python-docs-samples#3786)

* chore(deps): update dependency google-cloud-storage to v1.28.1 [(#3785)](GoogleCloudPlatform/python-docs-samples#3785)

* chore(deps): update dependency google-cloud-storage to v1.28.1

* [asset] testing: use uuid instead of time

Co-authored-by: Takashi Matsuo <[email protected]>

* update google-auth to 1.15.0 part 3 [(#3816)](GoogleCloudPlatform/python-docs-samples#3816)

* Update dependency google-cloud-dataproc to v0.8.0 [(#3837)](GoogleCloudPlatform/python-docs-samples#3837)

* chore(deps): update dependency google-auth to v1.16.0 [(#3903)](GoogleCloudPlatform/python-docs-samples#3903)

* update google-auth part 3 [(#3963)](GoogleCloudPlatform/python-docs-samples#3963)

* chore(deps): update dependency google-cloud-dataproc to v0.8.1 [(#4015)](GoogleCloudPlatform/python-docs-samples#4015)

This PR contains the following updates:

| Package | Update | Change |
|---|---|---|
| [google-cloud-dataproc](https://togithub.com/googleapis/python-dataproc) | patch | `==0.8.0` -> `==0.8.1` |

---

### Release Notes

<details>
<summary>googleapis/python-dataproc</summary>

### [`v0.8.1`](https://togithub.com/googleapis/python-dataproc/blob/master/CHANGELOG.md#&#8203;081-httpswwwgithubcomgoogleapispython-dataproccomparev080v081-2020-06-05)

[Compare Source](https://togithub.com/googleapis/python-dataproc/compare/v0.8.0...v0.8.1)

</details>

---

### Renovate configuration

:date: **Schedule**: At any time (no schedule defined).

:vertical_traffic_light: **Automerge**: Disabled by config. Please merge this manually once you are satisfied.

:recycle: **Rebasing**: Never, or you tick the rebase/retry checkbox.

:no_bell: **Ignore**: Close this PR and you won't be reminded about this update again.

---

 - [ ] <!-- rebase-check -->If you want to rebase/retry this PR, check this box

---

This PR has been generated by [WhiteSource Renovate](https://renovate.whitesourcesoftware.com). View repository job log [here](https://app.renovatebot.com/dashboard#GoogleCloudPlatform/python-docs-samples).

* Replace GCLOUD_PROJECT with GOOGLE_CLOUD_PROJECT. [(#4022)](GoogleCloudPlatform/python-docs-samples#4022)

* Update dependency google-auth to v1.17.0 [(#4058)](GoogleCloudPlatform/python-docs-samples#4058)

* chore(deps): update dependency google-auth to v1.17.1 [(#4073)](GoogleCloudPlatform/python-docs-samples#4073)

* Update dependency google-auth to v1.17.2 [(#4083)](GoogleCloudPlatform/python-docs-samples#4083)

* Update dependency google-auth to v1.18.0 [(#4125)](GoogleCloudPlatform/python-docs-samples#4125)

* Update dependency google-cloud-dataproc to v1 [(#4109)](GoogleCloudPlatform/python-docs-samples#4109)

Co-authored-by: Takashi Matsuo <[email protected]>

* chore(deps): update dependency google-cloud-storage to v1.29.0 [(#4040)](GoogleCloudPlatform/python-docs-samples#4040)

* chore(deps): update dependency grpcio to v1.30.0 [(#4143)](GoogleCloudPlatform/python-docs-samples#4143)

Co-authored-by: Takashi Matsuo <[email protected]>

* Update dependency google-auth-httplib2 to v0.0.4 [(#4255)](GoogleCloudPlatform/python-docs-samples#4255)

Co-authored-by: Takashi Matsuo <[email protected]>

* chore(deps): update dependency pytest to v5.4.3 [(#4279)](GoogleCloudPlatform/python-docs-samples#4279)

* chore(deps): update dependency pytest to v5.4.3

* specify pytest for python 2 in appengine

Co-authored-by: Leah Cole <[email protected]>

* chore(deps): update dependency google-auth to v1.19.0 [(#4293)](GoogleCloudPlatform/python-docs-samples#4293)

* chore(deps): update dependency google-cloud-dataproc to v1.0.1 [(#4309)](GoogleCloudPlatform/python-docs-samples#4309)

This PR contains the following updates:

| Package | Update | Change |
|---|---|---|
| [google-cloud-dataproc](https://togithub.com/googleapis/python-dataproc) | patch | `==1.0.0` -> `==1.0.1` |

---

### Release Notes

<details>
<summary>googleapis/python-dataproc</summary>

### [`v1.0.1`](https://togithub.com/googleapis/python-dataproc/blob/master/CHANGELOG.md#&#8203;101-httpswwwgithubcomgoogleapispython-dataproccomparev100v101-2020-07-16)

[Compare Source](https://togithub.com/googleapis/python-dataproc/compare/v1.0.0...v1.0.1)

</details>

---

### Renovate configuration

:date: **Schedule**: At any time (no schedule defined).

:vertical_traffic_light: **Automerge**: Disabled by config. Please merge this manually once you are satisfied.

:recycle: **Rebasing**: Never, or you tick the rebase/retry checkbox.

:no_bell: **Ignore**: Close this PR and you won't be reminded about this update again.

---

 - [ ] <!-- rebase-check -->If you want to rebase/retry this PR, check this box

---

This PR has been generated by [WhiteSource Renovate](https://renovate.whitesourcesoftware.com). View repository job log [here](https://app.renovatebot.com/dashboard#GoogleCloudPlatform/python-docs-samples).

* chore(deps): update dependency google-auth to v1.19.1 [(#4304)](GoogleCloudPlatform/python-docs-samples#4304)

* chore(deps): update dependency google-auth to v1.19.2 [(#4321)](GoogleCloudPlatform/python-docs-samples#4321)

This PR contains the following updates:

| Package | Update | Change |
|---|---|---|
| [google-auth](https://togithub.com/googleapis/google-auth-library-python) | patch | `==1.19.1` -> `==1.19.2` |

---

### Release Notes

<details>
<summary>googleapis/google-auth-library-python</summary>

### [`v1.19.2`](https://togithub.com/googleapis/google-auth-library-python/blob/master/CHANGELOG.md#&#8203;1192-httpswwwgithubcomgoogleapisgoogle-auth-library-pythoncomparev1191v1192-2020-07-17)

[Compare Source](https://togithub.com/googleapis/google-auth-library-python/compare/v1.19.1...v1.19.2)

</details>

---

### Renovate configuration

:date: **Schedule**: At any time (no schedule defined).

:vertical_traffic_light: **Automerge**: Disabled by config. Please merge this manually once you are satisfied.

:recycle: **Rebasing**: Never, or you tick the rebase/retry checkbox.

:no_bell: **Ignore**: Close this PR and you won't be reminded about this update again.

---

 - [ ] <!-- rebase-check -->If you want to rebase/retry this PR, check this box

---

This PR has been generated by [WhiteSource Renovate](https://renovate.whitesourcesoftware.com). View repository job log [here](https://app.renovatebot.com/dashboard#GoogleCloudPlatform/python-docs-samples).

* Update dependency google-auth to v1.20.0 [(#4387)](GoogleCloudPlatform/python-docs-samples#4387)

* Update dependency pytest to v6 [(#4390)](GoogleCloudPlatform/python-docs-samples#4390)

* Update dependency grpcio to v1.31.0 [(#4438)](GoogleCloudPlatform/python-docs-samples#4438)

* chore(deps): update dependency google-auth to v1.20.1 [(#4452)](GoogleCloudPlatform/python-docs-samples#4452)

* chore: update templates

Co-authored-by: Bill Prin <[email protected]>
Co-authored-by: Bill Prin <[email protected]>
Co-authored-by: Jon Wayne Parrott <[email protected]>
Co-authored-by: Eran Kampf <[email protected]>
Co-authored-by: DPE bot <[email protected]>
Co-authored-by: aman-ebay <[email protected]>
Co-authored-by: Martial Hue <[email protected]>
Co-authored-by: Gioia Ballin <[email protected]>
Co-authored-by: michaelawyu <[email protected]>
Co-authored-by: michaelawyu <[email protected]>
Co-authored-by: Alix Hamilton <[email protected]>
Co-authored-by: James Winegar <[email protected]>
Co-authored-by: Charles Engelke <[email protected]>
Co-authored-by: Gus Class <[email protected]>
Co-authored-by: Brad Miro <[email protected]>
Co-authored-by: Kurtis Van Gent <[email protected]>
Co-authored-by: Doug Mahugh <[email protected]>
Co-authored-by: Leah E. Cole <[email protected]>
Co-authored-by: WhiteSource Renovate <[email protected]>
Co-authored-by: Leah Cole <[email protected]>
Co-authored-by: Takashi Matsuo <[email protected]>

.github/CODEOWNERS

@@ -0,0 +1,8 @@
+# Code owners file.
+# This file controls who is tagged for review for any given pull request.
+#
+# For syntax help see:
+# https://help.github.com/en/github/creating-cloning-and-archiving-repositories/about-code-owners#codeowners-syntax
+
+
+/samples/**/*.py                           @bradmiro @googleapis/python-samples-owners

.gitignore

@@ -46,6 +46,7 @@ pip-log.txt
 # Built documentation
 docs/_build
 bigquery/docs/generated
+docs.metadata
 
 # Virtual environment
 env/
@@ -57,4 +58,4 @@ system_tests/local_test_setup
 
 # Make sure a generated file isn't accidentally committed.
 pylintrc
-pylintrc.test
+pylintrc.test

.kokoro/build.sh

@@ -36,4 +36,10 @@ python3.6 -m pip uninstall --yes --quiet nox-automation
 python3.6 -m pip install --upgrade --quiet nox
 python3.6 -m nox --version
 
-python3.6 -m nox
+# If NOX_SESSION is set, it only runs the specified session,
+# otherwise run all the sessions.
+if [[ -n "${NOX_SESSION:-}" ]]; then
+    python3.6 -m nox -s "${NOX_SESSION:-}"
+else
+    python3.6 -m nox
+fi

.kokoro/docker/docs/Dockerfile

@@ -0,0 +1,98 @@
+# Copyright 2020 Google LLC
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+from ubuntu:20.04
+
+ENV DEBIAN_FRONTEND noninteractive
+
+# Ensure local Python is preferred over distribution Python.
+ENV PATH /usr/local/bin:$PATH
+
+# Install dependencies.
+RUN apt-get update \
+  && apt-get install -y --no-install-recommends \
+    apt-transport-https \
+    build-essential \
+    ca-certificates \
+    curl \
+    dirmngr \
+    git \
+    gpg-agent \
+    graphviz \
+    libbz2-dev \
+    libdb5.3-dev \
+    libexpat1-dev \
+    libffi-dev \
+    liblzma-dev \
+    libreadline-dev \
+    libsnappy-dev \
+    libssl-dev \
+    libsqlite3-dev \
+    portaudio19-dev \
+    redis-server \
+    software-properties-common \
+    ssh \
+    sudo \
+    tcl \
+    tcl-dev \
+    tk \
+    tk-dev \
+    uuid-dev \
+    wget \
+    zlib1g-dev \
+  && add-apt-repository universe \
+  && apt-get update \
+  && apt-get -y install jq \
+  && apt-get clean autoclean \
+  && apt-get autoremove -y \
+  && rm -rf /var/lib/apt/lists/* \
+  && rm -f /var/cache/apt/archives/*.deb
+
+
+COPY fetch_gpg_keys.sh /tmp
+# Install the desired versions of Python.
+RUN set -ex \
+    && export GNUPGHOME="$(mktemp -d)" \
+    && echo "disable-ipv6" >> "${GNUPGHOME}/dirmngr.conf" \
+    && /tmp/fetch_gpg_keys.sh \
+    && for PYTHON_VERSION in 3.7.8 3.8.5; do \
+        wget --no-check-certificate -O python-${PYTHON_VERSION}.tar.xz "https://www.python.org/ftp/python/${PYTHON_VERSION%%[a-z]*}/Python-$PYTHON_VERSION.tar.xz" \
+        && wget --no-check-certificate -O python-${PYTHON_VERSION}.tar.xz.asc "https://www.python.org/ftp/python/${PYTHON_VERSION%%[a-z]*}/Python-$PYTHON_VERSION.tar.xz.asc" \
+        && gpg --batch --verify python-${PYTHON_VERSION}.tar.xz.asc python-${PYTHON_VERSION}.tar.xz \
+        && rm -r python-${PYTHON_VERSION}.tar.xz.asc \
+        && mkdir -p /usr/src/python-${PYTHON_VERSION} \
+        && tar -xJC /usr/src/python-${PYTHON_VERSION} --strip-components=1 -f python-${PYTHON_VERSION}.tar.xz \
+        && rm python-${PYTHON_VERSION}.tar.xz \
+        && cd /usr/src/python-${PYTHON_VERSION} \
+        && ./configure \
+            --enable-shared \
+            # This works only on Python 2.7 and throws a warning on every other
+            # version, but seems otherwise harmless.
+            --enable-unicode=ucs4 \
+            --with-system-ffi \
+            --without-ensurepip \
+        && make -j$(nproc) \
+        && make install \
+        && ldconfig \
+  ; done \
+  && rm -rf "${GNUPGHOME}" \
+  && rm -rf /usr/src/python* \
+  && rm -rf ~/.cache/
+
+RUN wget -O /tmp/get-pip.py 'https://bootstrap.pypa.io/get-pip.py' \
+  && python3.7 /tmp/get-pip.py \
+  && python3.8 /tmp/get-pip.py \
+  && rm /tmp/get-pip.py
+
+CMD ["python3.7"]

.kokoro/docker/docs/fetch_gpg_keys.sh

@@ -0,0 +1,45 @@
+#!/bin/bash
+# Copyright 2020 Google LLC
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#      http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+# A script to fetch gpg keys with retry.
+# Avoid jinja parsing the file.
+# 
+
+function retry {
+    if [[ "${#}" -le 1 ]]; then
+	echo "Usage: ${0} retry_count commands.."
+	exit 1
+    fi
+    local retries=${1}
+    local command="${@:2}"
+    until [[ "${retries}" -le 0 ]]; do
+	$command && return 0
+	if [[ $? -ne 0 ]]; then
+	    echo "command failed, retrying"
+	    ((retries--))
+	fi
+    done
+    return 1
+}
+
+# 3.6.9, 3.7.5 (Ned Deily)
+retry 3 gpg --keyserver ha.pool.sks-keyservers.net --recv-keys \
+      0D96DF4D4110E5C43FBFB17F2D347EA6AA65421D
+
+# 3.8.0 (Łukasz Langa)
+retry 3 gpg --keyserver ha.pool.sks-keyservers.net --recv-keys \
+      E3FF2839C048B25C084DEBE9B26995E310250568
+
+# 

.kokoro/docs/common.cfg

@@ -11,12 +11,12 @@ action {
 gfile_resources: "/bigstore/cloud-devrel-kokoro-resources/trampoline"
 
 # Use the trampoline script to run in docker.
-build_file: "python-dataproc/.kokoro/trampoline.sh"
+build_file: "python-dataproc/.kokoro/trampoline_v2.sh"
 
 # Configure the docker image for kokoro-trampoline.
 env_vars: {
     key: "TRAMPOLINE_IMAGE"
-    value: "gcr.io/cloud-devrel-kokoro-resources/python-multi"
+    value: "gcr.io/cloud-devrel-kokoro-resources/python-lib-docs"
 }
 env_vars: {
     key: "TRAMPOLINE_BUILD_FILE"
@@ -28,6 +28,23 @@ env_vars: {
     value: "docs-staging"
 }
 
+env_vars: {
+    key: "V2_STAGING_BUCKET"
+    value: "docs-staging-v2-staging"
+}
+
+# It will upload the docker image after successful builds.
+env_vars: {
+    key: "TRAMPOLINE_IMAGE_UPLOAD"
+    value: "true"
+}
+
+# It will always build the docker image.
+env_vars: {
+    key: "TRAMPOLINE_DOCKERFILE"
+    value: ".kokoro/docker/docs/Dockerfile"
+}
+
 # Fetch the token needed for reporting release status to GitHub
 before_action {
   fetch_keystore {

.kokoro/docs/docs-presubmit.cfg

@@ -0,0 +1,17 @@
+# Format: //devtools/kokoro/config/proto/build.proto
+
+env_vars: {
+    key: "STAGING_BUCKET"
+    value: "gcloud-python-test"
+}
+
+env_vars: {
+    key: "V2_STAGING_BUCKET"
+    value: "gcloud-python-test"
+}
+
+# We only upload the image in the main `docs` build.
+env_vars: {
+    key: "TRAMPOLINE_IMAGE_UPLOAD"
+    value: "false"
+}

.kokoro/publish-docs.sh

@@ -18,26 +18,16 @@ set -eo pipefail
 # Disable buffering, so that the logs stream through.
 export PYTHONUNBUFFERED=1
 
-cd github/python-dataproc
-
-# Remove old nox
-python3.6 -m pip uninstall --yes --quiet nox-automation
+export PATH="${HOME}/.local/bin:${PATH}"
 
 # Install nox
-python3.6 -m pip install --upgrade --quiet nox
-python3.6 -m nox --version
+python3 -m pip install --user --upgrade --quiet nox
+python3 -m nox --version
 
 # build docs
 nox -s docs
 
-python3 -m pip install gcp-docuploader
-
-# install a json parser
-sudo apt-get update
-sudo apt-get -y install software-properties-common
-sudo add-apt-repository universe
-sudo apt-get update
-sudo apt-get -y install jq
+python3 -m pip install --user gcp-docuploader
 
 # create metadata
 python3 -m docuploader create-metadata \
@@ -52,4 +42,23 @@ python3 -m docuploader create-metadata \
 cat docs.metadata
 
 # upload docs
-python3 -m docuploader upload docs/_build/html --metadata-file docs.metadata --staging-bucket docs-staging
+python3 -m docuploader upload docs/_build/html --metadata-file docs.metadata --staging-bucket "${STAGING_BUCKET}"
+
+
+# docfx yaml files
+nox -s docfx
+
+# create metadata.
+python3 -m docuploader create-metadata \
+  --name=$(jq --raw-output '.name // empty' .repo-metadata.json) \
+  --version=$(python3 setup.py --version) \
+  --language=$(jq --raw-output '.language // empty' .repo-metadata.json) \
+  --distribution-name=$(python3 setup.py --name) \
+  --product-page=$(jq --raw-output '.product_documentation // empty' .repo-metadata.json) \
+  --github-repository=$(jq --raw-output '.repo // empty' .repo-metadata.json) \
+  --issue-tracker=$(jq --raw-output '.issue_tracker // empty' .repo-metadata.json)
+
+cat docs.metadata
+
+# upload docs
+python3 -m docuploader upload docs/_build/html/docfx_yaml --metadata-file docs.metadata --destination-prefix docfx --staging-bucket "${V2_STAGING_BUCKET}"