New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

BigTable: Add truncate table and drop by prefix on top of GAPIC integration #5360

Merged

tseaver merged 12 commits into googleapis:master from zakons:feature/add_truncate_table_and_drop_by_prefix

Jun 11, 2018

Contributor

aneepct commented May 22, 2018

Add methods on Table to delete rows by prefix and truncate all rows.

truncate
drop_by_prefix


          Add truncate and drop by prefix

1f824c5

aneepct requested a review from lukesneeringer as a code owner

May 22, 2018 11:42

googlebot added the cla: yes label

aneepct mentioned this pull request

BigTable: Add data app profile id #5275

Closed

sduskis suggested changes

View reviewed changes

bigtable/google/cloud/bigtable/table.py Outdated

@@ @@ -386,6 +386,47 @@ def sample_row_keys(self): @@
                           self.name)
                       return response_iterator
+                  def truncate(self, timeout=60):

This comment was marked as spam.

Sign in to view

bigtable/google/cloud/bigtable/table.py Outdated

+                                                        delete_all_data_from_table=True,
+                                                        timeout=timeout)
+                  def drop_by_prefix(self, row_key_prefix, timeout=60):

This comment was marked as spam.

Sign in to view

tseaver added the api: bigtable label

aneepct added 4 commits

May 23, 2018 11:27


          Add default timeout value

b7efabb


          Update timestamp logic

4da1be6


          Added unit test for timestamp logic

02dc657


          Merge branch 'master' of https://github.com/zakons/google-cloud-python …

4dfde6c

…into feature/add_truncate_table_and_drop_by_prefix

Contributor Author

aneepct commented May 29, 2018

@sduskis Fixed please review

aneepct mentioned this pull request

BigTable: Fixing client for system tests #5398

Merged

sduskis reviewed

View reviewed changes

bigtable/tests/unit/test_table.py Outdated


		result = table.truncate()

		self.assertEqual(result, expected_result)

This comment was marked as spam.

Sign in to view

This comment was marked as spam.

Sign in to view

tseaver reviewed

View reviewed changes

bigtable/tests/unit/test_table.py Outdated


		self.assertEqual(result, expected_result)

		def test_drop_by_prefix_w_timestamp(self):

This comment was marked as spam.

Sign in to view


          Done required changes

2c6d8a7

tseaver suggested changes

View reviewed changes

bigtable/tests/unit/test_table.py Outdated

+                      data_api = bigtable_client.BigtableClient(mock.Mock())
+                      table_api = bigtable_table_admin_client.BigtableTableAdminClient(
+                          mock.Mock())

This comment was marked as spam.

Sign in to view

bigtable/tests/unit/test_table.py Outdated

+                      data_api = bigtable_client.BigtableClient(mock.Mock())
+                      table_api = bigtable_table_admin_client.BigtableTableAdminClient(
+                          mock.Mock())

This comment was marked as spam.

Sign in to view

bigtable/tests/unit/test_table.py Outdated

+                      data_api = bigtable_client.BigtableClient(mock.Mock())
+                      table_api = bigtable_table_admin_client.BigtableTableAdminClient(
+                          mock.Mock())

This comment was marked as spam.

Sign in to view

aneepct added 3 commits

June 1, 2018 08:08


          Fix required changes on unit test cases

c63b0e6


          Merge branch 'master' of https://github.com/zakons/google-cloud-python …

444db17

…into feature/add_truncate_table_and_drop_by_prefix


          Create system tests and update truncate and drop by prefix code

6609be2

sduskis reviewed

View reviewed changes

bigtable/tests/system.py Outdated

@@ @@ -84,7 +84,7 @@ def setUpModule(): @@
                       credentials = EmulatorCreds()
                       Config.CLIENT = Client(admin=True, credentials=credentials)
                   else:
-                      Config.CLIENT = Client(admin=True)
+                      Config.CLIENT = Client(project='grass-clump-479', admin=True)

This comment was marked as spam.

Sign in to view

This comment was marked as spam.

Sign in to view


          Remove static project defined

bcbfc75

sduskis approved these changes

View reviewed changes

Contributor

sduskis left a comment

I'm good to go when @tseaver is

tseaver suggested changes

View reviewed changes

bigtable/tests/system.py Outdated

+                      row_keys = [
+                          'row_key_1', 'row_key_2', 'row_key_3', 'row_key_4', 'row_key_5',
+                          'row_key_pr_1', 'row_key_pr_2', 'row_key_pr_3', 'row_key_pr_4',
+                          'row_key_pr_5']

This comment was marked as spam.

Sign in to view

bigtable/tests/system.py Outdated

+                      row_keys = [
+                          'row_key_1', 'row_key_2', 'row_key_3', 'row_key_4', 'row_key_5',
+                          'row_key_pr_1', 'row_key_pr_2', 'row_key_pr_3', 'row_key_pr_4',
+                          'row_key_pr_5']

This comment was marked as spam.

Sign in to view

aneepct added 2 commits

June 8, 2018 17:11


          Fix requried changes


          Remove static project id

bb7d47e

tseaver approved these changes

View reviewed changes

Contributor

zakons commented Jun 11, 2018

@tseaver Ready to merge? Thanks.

tseaver merged commit 5f39feb into googleapis:master

Contributor

tseaver commented Jun 12, 2018

@aneepct One of the system tests added by this PR is failing on CI this morning 1 2 3 4:

____________________ TestDataAPI.test_drop_by_prefix_table _____________________

self = &lt;tests.system.TestDataAPI testMethod=test_drop_by_prefix_table&gt;

    def test_drop_by_prefix_table(self):
        row_keys = [
            b'row_key_1', b'row_key_2', b'row_key_3', b'row_key_4',
            b'row_key_5', b'row_key_pr_1', b'row_key_pr_2', b'row_key_pr_3',
            b'row_key_pr_4', b'row_key_pr_5']
    
        for row_key in row_keys:
            row = self._table.row(row_key)
            row.set_cell(COLUMN_FAMILY_ID1, COL_NAME1, CELL_VAL1)
            row.commit()
            self.rows_to_delete.append(row)
    
        self._table.drop_by_prefix(row_key_prefix='row_key_pr', timeout=200)
    
        read_rows = self._table.yield_rows()
        expected_rows_count = 5
        read_rows_count = 0
    
        for row in read_rows:
            if row.row_key.decode('utf-8') in row_keys:
                read_rows_count += 1
    
&gt;       self.assertEqual(expected_rows_count, read_rows_count)
E       AssertionError: 5 != 0

tseaver added a commit that referenced this pull request


          Fix Py3 breakage in new system test.

63c61f4

Added in PR #5360.

tseaver mentioned this pull request

Fix Py3 breakage in new system test. #5474

Merged

tseaver added a commit that referenced this pull request


          Fix Py3 breakage in new system test. (#5474)

7a0a97f

Added in PR #5360.

tseaver mentioned this pull request

Fix Failing BigTable System tests #5362

Closed

tseaver mentioned this pull request

tests: refactor systests into separate modules w/ pytest fixtures googleapis/python-bigtable#393

Merged

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

api: bigtable cla: yes