Add job for extracting tables to CloudStorage #1068

tseaver · 2015-08-17T23:24:26Z

~~Uses #1064 as a base.~~

Inverse of 'jobs.LoadTableFromStorageJob': extracts a table to one or more files in CloudStorage.

tseaver · 2015-08-18T03:20:27Z

Rebased after landing #1064.

gcloud/bigquery/client.py

@@ -133,3 +134,22 @@ def copy_table(self, name, destination, *sources):
        :returns: a new ``CopyJob`` instance
        """
        return CopyJob(name, destination, sources, client=self)
+
+    def extract_table_to_storage(self, name, source, *destination_uris):
+        """Construct a job for copying one or more tables into another table.


Addresses: #1068 (comment)

gcloud/bigquery/client.py

@@ -95,7 +96,6 @@ def dataset(self, name):
        :param name: Name of the dataset.

        :rtype: :class:`gcloud.bigquery.dataset.Dataset`
-        :returns: a new ``Dataset`` instance


Addresses: #1068 (comment) #1068 (comment).

Addresses: #1068 (comment)

dhermes · 2015-08-19T04:44:45Z

Are we all good here? AFAICT we got the :returns: issue sorted out. If yes, LGTM and feel free to merge. Ping me on the dependent PRs once rebased.

Add job for extracting tables to CloudStorage

* Add samples for Cloud Tasks [(#1068)](GoogleCloudPlatform/python-docs-samples#1068) * Add samples for Cloud Tasks * Respond to tasks sample review * Update app engine queues samples * Address review feedback * Address review issues and convert pull queue sample to not use API key auth * Reform pull queues to match appengine queues changes to auth, command line input, readme * flake8 and fix comment * Fix Tasks sample test issues. * Remove queue.yaml, now unused * Add required flag on certain arguments to command-line tools * Auto-update dependencies. [(#1116)](GoogleCloudPlatform/python-docs-samples#1116) * Auto-update dependencies. [(#1133)](GoogleCloudPlatform/python-docs-samples#1133) * Auto-update dependencies. * Fix missing http library Change-Id: I99faa600f2f3f1f50f57694fc9835d7f35bda250 * Update Task Queue samples for beta * Use full import to get client * Auto-update dependencies. [(#1186)](GoogleCloudPlatform/python-docs-samples#1186) * Auto-update dependencies. [(#1217)](GoogleCloudPlatform/python-docs-samples#1217) * Added "Open in Cloud Shell" buttons to README files [(#1254)](GoogleCloudPlatform/python-docs-samples#1254) * Add comments and region tags to Cloud Tasks samples [(#1271)](GoogleCloudPlatform/python-docs-samples#1271) * Standardize on CamelCase, reword confusing endpoint name [(#1288)](GoogleCloudPlatform/python-docs-samples#1288) * Auto-update dependencies. [(#1309)](GoogleCloudPlatform/python-docs-samples#1309) * Rename pull to lease and fix name/parent confusion [(#1311)](GoogleCloudPlatform/python-docs-samples#1311) * Updated region tags for pull queue samples [(#1329)](GoogleCloudPlatform/python-docs-samples#1329) * Auto-update dependencies. [(#1320)](GoogleCloudPlatform/python-docs-samples#1320) * Auto-update dependencies. [(#1355)](GoogleCloudPlatform/python-docs-samples#1355) * Auto-update dependencies. [(#1359)](GoogleCloudPlatform/python-docs-samples#1359) * Auto-update dependencies. * Update Cloud Tasks Samples [(#1529)](GoogleCloudPlatform/python-docs-samples#1529) * passing create task * Passing tests * updates to region tags * update region tags [(#1532)](GoogleCloudPlatform/python-docs-samples#1532) * update Tasks Sample for App Engine [(#1541)](GoogleCloudPlatform/python-docs-samples#1541) * update gcloud command for creating queues * deploys and runs * update license * passing tests * Fix run command [(#1563)](GoogleCloudPlatform/python-docs-samples#1563) * Updated library for TTS GA [(#1552)](GoogleCloudPlatform/python-docs-samples#1552) * update gcloud command for Cloud Tasks [(#1566)](GoogleCloudPlatform/python-docs-samples#1566) * update gcloud command * update pull queue command * update pull queue command * Update Cloud Tasks Push Queue Sample [(#1698)](GoogleCloudPlatform/python-docs-samples#1698) * deleted pull queues * updated samples * fix dependency versions * [Cloud Tasks] Move samples to new folder [(#2114)](GoogleCloudPlatform/python-docs-samples#2114) * Move samples to keep consistent with other langauges * Ad system tests as well * [Cloud Tasks] Add task with authentication sample [(#2113)](GoogleCloudPlatform/python-docs-samples#2113) * Add task with authentication sample * Fix linting * Fix linting * Fix spacing * Update tests with service account * Move samples and update READMEs * Update version and linting * Update task sample comments [(#2156)](GoogleCloudPlatform/python-docs-samples#2156) * Update task comments * Update readme * Update queue name * update gcloud [(#2208)](GoogleCloudPlatform/python-docs-samples#2208) * Add protobuf dep and install instructions [(#2250)](GoogleCloudPlatform/python-docs-samples#2250) * Add Migration Guide Snippets for Cloud Tasks [(#2316)](GoogleCloudPlatform/python-docs-samples#2316) * Migration guide * remove app and update migraitonn * snippets for migration guide - tests added * lint * remove print statements * Styling changes * Travis trigger * Update create_http_task.py [(#2187)](GoogleCloudPlatform/python-docs-samples#2187) Updates `create_http_task.py` to have missing `in_seconds` variable * Adds updates for samples profiler ... vision [(#2439)](GoogleCloudPlatform/python-docs-samples#2439) * Update Cloud Tasks library version [(#2516)](GoogleCloudPlatform/python-docs-samples#2516) * Update to new library * update library version * Adds Task name while creating Task [(#2543)](GoogleCloudPlatform/python-docs-samples#2543) * Task Name in Creating HTTP Task with Token [(#2700)](GoogleCloudPlatform/python-docs-samples#2700) * chore(deps): update dependency google-cloud-tasks to v1.5.0 [(#3168)](GoogleCloudPlatform/python-docs-samples#3168) * chore(deps): update dependency googleapis-common-protos to v1.51.0 [(#3171)](GoogleCloudPlatform/python-docs-samples#3171) This PR contains the following updates: | Package | Update | Change | |---|---|---| | [googleapis-common-protos](https://togithub.com/googleapis/googleapis) | minor | `==1.6.0` -> `==1.51.0` | --- ### Renovate configuration :date: **Schedule**: At any time (no schedule defined). :vertical_traffic_light: **Automerge**: Disabled by config. Please merge this manually once you are satisfied. :recycle: **Rebasing**: Never, or you tick the rebase/retry checkbox. :no_bell: **Ignore**: Close this PR and you won't be reminded about this update again. --- - [ ] If you want to rebase/retry this PR, check this box --- This PR has been generated by [WhiteSource Renovate](https://renovate.whitesourcesoftware.com). View repository job log [here](https://app.renovatebot.com/dashboard#GoogleCloudPlatform/python-docs-samples). * Simplify noxfile setup. [(#2806)](GoogleCloudPlatform/python-docs-samples#2806) * chore(deps): update dependency requests to v2.23.0 * Simplify noxfile and add version control. * Configure appengine/standard to only test Python 2.7. * Update Kokokro configs to match noxfile. * Add requirements-test to each folder. * Remove Py2 versions from everything execept appengine/standard. * Remove conftest.py. * Remove appengine/standard/conftest.py * Remove 'no-sucess-flaky-report' from pytest.ini. * Add GAE SDK back to appengine/standard tests. * Fix typo. * Roll pytest to python 2 version. * Add a bunch of testing requirements. * Remove typo. * Add appengine lib directory back in. * Add some additional requirements. * Fix issue with flake8 args. * Even more requirements. * Readd appengine conftest.py. * Add a few more requirements. * Even more Appengine requirements. * Add webtest for appengine/standard/mailgun. * Add some additional requirements. * Add workaround for issue with mailjet-rest. * Add responses for appengine/standard/mailjet. Co-authored-by: Renovate Bot <[email protected]> * Remove dependency googleapis-common-protos [(#3955)](GoogleCloudPlatform/python-docs-samples#3955) * Update dependency googleapis-common-protos to v1.52.0 * Update requirements.txt * Update requirements.txt * Update requirements.txt Co-authored-by: Bu Sun Kim <[email protected]> * Replace GCLOUD_PROJECT with GOOGLE_CLOUD_PROJECT. [(#4022)](GoogleCloudPlatform/python-docs-samples#4022) * [tasks] testing: use fixtures for the queue [(#4049)](GoogleCloudPlatform/python-docs-samples#4049) fixes #4045 fixes #4044 I don't know why these tests started to fail, but anyways we'd better use fixtures and temporary queues. * add python snippets and tests for creating, listing, and deleting queues [(#4012)](GoogleCloudPlatform/python-docs-samples#4012) * add python snippets and tests for creating, listing, and deleting queues * fix grammar * update licenses * apply suggested fixes and format with black * refine delete_queue_test with fixture for setup * utilize fixtures and match format of create_http_task_test * utilize fixtures in list_queues_test and create_queue_test * make create_queue_test call the right function * still attempt to delete queue after test runs in case of failure * attempt to delete queue in case of failure, using try/except approach * add print when NotFound is caught * fix import Co-authored-by: Averi Kitsch <[email protected]> Co-authored-by: Takashi Matsuo <[email protected]> * docs(tasks): service_account_email parameter example [(#4183)](GoogleCloudPlatform/python-docs-samples#4183) * docs(tasks): serviceAccountEmail parameter example * docs(tasks): parameter name camel => snake case * docs(tasks): comment order = function param order * chore(deps): update dependency pytest to v5.4.3 [(#4279)](GoogleCloudPlatform/python-docs-samples#4279) * chore(deps): update dependency pytest to v5.4.3 * specify pytest for python 2 in appengine Co-authored-by: Leah Cole <[email protected]> * Update dependency pytest to v6 [(#4390)](GoogleCloudPlatform/python-docs-samples#4390) * tasks: added json content-type request [(#4473)](GoogleCloudPlatform/python-docs-samples#4473) - added json payload compatibility - fix imports and code block used on https://cloud.google.com/tasks/docs/creating-http-target-tasks#python ## Description Fixes #<ISSUE-NUMBER> Note: It's a good idea to open an issue first for discussion. ## Checklist - [ ] I have followed [Sample Guidelines from AUTHORING_GUIDE.MD](https://github.com/GoogleCloudPlatform/python-docs-samples/blob/master/AUTHORING_GUIDE.md) - [ ] README is updated to include [all relevant information](https://github.com/GoogleCloudPlatform/python-docs-samples/blob/master/AUTHORING_GUIDE.md#readme-file) - [ ] **Tests** pass: `nox -s py-3.6` (see [Test Environment Setup](https://github.com/GoogleCloudPlatform/python-docs-samples/blob/master/AUTHORING_GUIDE.md#test-environment-setup)) - [ ] **Lint** pass: `nox -s lint` (see [Test Environment Setup](https://github.com/GoogleCloudPlatform/python-docs-samples/blob/master/AUTHORING_GUIDE.md#test-environment-setup)) - [ ] These samples need a new **API enabled** in testing projects to pass (let us know which ones) - [ ] These samples need a new/updated **env vars** in testing projects set to pass (let us know which ones) - [ ] Please **merge** this PR for me once it is approved. - [ ] This sample adds a new sample directory, and I updated the [CODEOWNERS file](https://github.com/GoogleCloudPlatform/python-docs-samples/blob/master/.github/CODEOWNERS) with the codeowners for this sample * docs: add samples from python-docs-samples/tasks * chore: update templates Co-authored-by: Andrew Gorcester <[email protected]> Co-authored-by: Andrew Gorcester <[email protected]> Co-authored-by: DPE bot <[email protected]> Co-authored-by: michaelawyu <[email protected]> Co-authored-by: ellenevans <[email protected]> Co-authored-by: Averi Kitsch <[email protected]> Co-authored-by: michaelawyu <[email protected]> Co-authored-by: Noah Negrey <[email protected]> Co-authored-by: Alex Voorhees <[email protected]> Co-authored-by: Gus Class <[email protected]> Co-authored-by: Sarath Kaul <[email protected]> Co-authored-by: Sarath Kaul <[email protected]> Co-authored-by: WhiteSource Renovate <[email protected]> Co-authored-by: Kurtis Van Gent <[email protected]> Co-authored-by: Bu Sun Kim <[email protected]> Co-authored-by: Takashi Matsuo <[email protected]> Co-authored-by: Aaron Johnson <[email protected]> Co-authored-by: Adam Ross <[email protected]> Co-authored-by: Leah Cole <[email protected]> Co-authored-by: Joab Leite S. Neto <[email protected]>

tseaver added the api: bigquery Issues related to the BigQuery API. label Aug 17, 2015

googlebot added the cla: yes This human has signed the Contributor License Agreement. label Aug 17, 2015

tseaver changed the title ~~Add job for managing extract ot tables to CloudStorage~~ Add job for extracting tables to CloudStorage Aug 17, 2015

tseaver added 3 commits August 17, 2015 23:19

Add 'jobs.ExtractTableToStorageJob'.

1b6320d

Inverse of 'jobs.LoadTableFromStorageJob': extracts a table to one or more files in CloudStorage.

Link spec URLs for job configuration properties.

0feb8f3

Add job factory: 'Client.extract_table_to_storage'.

4bcaaf5

dhermes reviewed Aug 18, 2015
View reviewed changes

Fix docstring for 'Client.extract_table_to_storage'.

a94c14e

Addresses: #1068 (comment)

dhermes reviewed Aug 18, 2015
View reviewed changes

gcloud/bigquery/client.py

@@ -95,7 +96,6 @@ def dataset(self, name):

:param name: Name of the dataset.

:rtype: :class:`gcloud.bigquery.dataset.Dataset`

:returns: a new ``Dataset`` instance

This comment was marked as spam.

Sign in to view

This comment was marked as spam.

Sign in to view

tseaver added 3 commits August 18, 2015 16:29

Fix docstring copy-pasta.

01deb80

Addresses: #1068 (comment) #1068 (comment).

Docstring for 'ExtractTableToStorageJob._populate_config_resource'.

56bf5d5

Addresses: #1068 (comment)

Improve Sphinx method-links.

d865a2c

Addresses: #1068 (comment)

tseaver mentioned this pull request Aug 19, 2015

Add job for running queries #1072

Merged

tseaver added a commit that referenced this pull request Aug 19, 2015

Merge pull request #1068 from tseaver/bigquery-jobs_extract

6f1debf

Add job for extracting tables to CloudStorage

tseaver merged commit 6f1debf into googleapis:master Aug 19, 2015

tseaver deleted the bigquery-jobs_extract branch August 19, 2015 13:04

dhermes mentioned this pull request Sep 30, 2015

Upgrading version to 0.8.0. #1158

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add job for extracting tables to CloudStorage #1068

Add job for extracting tables to CloudStorage #1068

tseaver commented Aug 17, 2015

tseaver commented Aug 18, 2015

This comment was marked as spam.

This comment was marked as spam.

This comment was marked as spam.

This comment was marked as spam.

This comment was marked as spam.

This comment was marked as spam.

dhermes commented Aug 19, 2015

Add job for extracting tables to CloudStorage #1068

Add job for extracting tables to CloudStorage #1068

Conversation

tseaver commented Aug 17, 2015

tseaver commented Aug 18, 2015

This comment was marked as spam.

This comment was marked as spam.

This comment was marked as spam.

This comment was marked as spam.

This comment was marked as spam.

This comment was marked as spam.

dhermes commented Aug 19, 2015