Deprecate mock_dynamodb2 #4919
Merged
Deprecate mock_dynamodb2 #4919
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
bblommers
force-pushed
the
deprecate-dynamodb2
branch
from
d02e0a1 to
d7012a4
Compare
bblommers
force-pushed
the
deprecate-dynamodb2
branch
from
d7012a4 to
e03095a
Compare
Codecov Report
@@ Coverage Diff @@
## master #4919 +/- ##
==========================================
- Coverage 95.54% 95.53% -0.01%
==========================================
Files 618 618
Lines 65481 65484 +3
==========================================
Hits 62562 62562
- Misses 2919 2922 +3
Flags with carried forward coverage won't be shown. Click here to find out more.
Continue to review full report at Codecov.
|
|
This is now part of moto >= 3.1.0.dev4 |
sahilshah6196
pushed a commit
to EBSCOIS/moto
that referenced
this pull request
Jul 1, 2022
chhabrakadabra
added a commit
to chhabrakadabra/feast
that referenced
this pull request
Sep 30, 2022
The latest rebuilding of requirements has upgraded the `moto` library past the `4.0.0` release, which has a couple of breaking changes. Specifically, the `mock_dynamodb2` decorator has been deprecated. See https://github.com/spulec/moto/blob/master/CHANGELOG.md#400 for more details. The actual PR (getmoto/moto#4919) mentions that it's because the `mock_dynamodb` decorator is now equivalent to the `mock_dynamodb2` decorator. Signed-off-by: Abhin Chhabra <[email protected]>
feast-ci-bot
pushed a commit
to feast-dev/feast
that referenced
this pull request
Oct 5, 2022
* Initial implementation of BigTable online store. Signed-off-by: Abhin Chhabra <[email protected]> * Attempt to run bigtable integration tests. Currently focusing on just getting the tests running locally. I've only build python3.8 requirements. Signed-off-by: Abhin Chhabra <[email protected]> * Got the BigTable tests running in local containers Signed-off-by: Abhin Chhabra <[email protected]> * Set serialization version when computing entity ID Signed-off-by: Abhin Chhabra <[email protected]> * Switch to the recommended layout in bigtable. This was recommended by the BigTable dev team. Details of this layout will be added to the documentation in a future commit. Signed-off-by: Abhin Chhabra <[email protected]> * Minor bugfixes. - If a row is empty when fetching data, don't process it more. - If a task in the threadpool fails, bubble up that failure. - If a `created_ts` is not available, use an empty string. `None` does not automatically serialize to bytes. Signed-off-by: Abhin Chhabra <[email protected]> * Move BigTable online store out of contrib As per feedback on the PR. Signed-off-by: Abhin Chhabra <[email protected]> * Attempt to run integration tests in CI. Provide the GCP project and the bigtable instance ID for the tests to connect to. Signed-off-by: Abhin Chhabra <[email protected]> * Delete tables for entity-less feature views. Signed-off-by: Abhin Chhabra <[email protected]> * Table names should be smaller than 50 characters This is BigTable's table length limit and it's causing test failures. Signed-off-by: Abhin Chhabra <[email protected]> * Optimize bigtable reads. - Fetch all the rows in one bigtable fetch. - Get only the columns that are necessary (using a column regex filter). Signed-off-by: Abhin Chhabra <[email protected]> * dynamodb: switch to `mock_dynamodb` The latest rebuilding of requirements has upgraded the `moto` library past the `4.0.0` release, which has a couple of breaking changes. Specifically, the `mock_dynamodb2` decorator has been deprecated. See https://github.com/spulec/moto/blob/master/CHANGELOG.md#400 for more details. The actual PR (getmoto/moto#4919) mentions that it's because the `mock_dynamodb` decorator is now equivalent to the `mock_dynamodb2` decorator. Signed-off-by: Abhin Chhabra <[email protected]> * minor: rename `BigTable` to `Bigtable` This matches the GCP docs. Signed-off-by: Abhin Chhabra <[email protected]> * Wrote some Bigtable documentation. Closely mirrors the docs for the other online stores. Signed-off-by: Abhin Chhabra <[email protected]> * Bugfix: Deal with missing row keys. It looks like the bigtable client will just skip over non-existent row keys. Signed-off-by: Abhin Chhabra <[email protected]> * Fix linting issues. Signed-off-by: Abhin Chhabra <[email protected]> * Generate requirements files. - As of version `1.49`, the various python packages in the [grpc repo](https://github.com/grpc/grpc/tree/master/src/python) require `protobuf>=4.21.3`. Unfortunately, this is incompatible with all versions of `tensorflow-metadata` (see [this issue](tensorflow/metadata#37)). And since `piptools` doesn't backtrack during dependency resolution, the requirement files cannot be regenerated without adding an upper limit on these grpc libraries directly in `setup.py`. - The previous attempt to upgrade usages of the `mock_dynamodb2` decorator to the newest version failed. Since I'm not an expert in dynamodb, it made sense to just cap the test tool to the version already being used in CI. Signed-off-by: Abhin Chhabra <[email protected]> * Don't bother materializing created timestamp. Had a discussion with Danny about whether it's useful to copy this column. He agreed that there's no value to storing this in the online store. Signed-off-by: Abhin Chhabra <[email protected]> * Remove `tensorflow-metadata`. Turns out that this dependency is not required. We removed all references to it in [this PR](#2063), but did not remove it from `setup.py`. Removing it has caused many of the restrictions imposed in previous commits to be unnecessary. Signed-off-by: Abhin Chhabra <[email protected]> * Minor fix to Bigtable documentation. Feedback from Danny mentioned that Bigtable should be able to store multiple versions of the same key and fetch the latest at read time. This makes sense and means that concurrent writes should work just fine. Signed-off-by: Abhin Chhabra <[email protected]> * update roadmap docs Signed-off-by: Danny Chiao <[email protected]> * Fix roadmap doc Signed-off-by: Danny Chiao <[email protected]> * Change link to point to roadmap page Signed-off-by: Danny Chiao <[email protected]> * change order in roadmap Signed-off-by: Danny Chiao <[email protected]> Signed-off-by: Abhin Chhabra <[email protected]> Signed-off-by: Abhin Chhabra <[email protected]> Signed-off-by: Danny Chiao <[email protected]> Co-authored-by: Danny Chiao <[email protected]>
franciscojavierarceo
pushed a commit
to franciscojavierarceo/feast
that referenced
this pull request
Oct 18, 2022
* Initial implementation of BigTable online store. Signed-off-by: Abhin Chhabra <[email protected]> * Attempt to run bigtable integration tests. Currently focusing on just getting the tests running locally. I've only build python3.8 requirements. Signed-off-by: Abhin Chhabra <[email protected]> * Got the BigTable tests running in local containers Signed-off-by: Abhin Chhabra <[email protected]> * Set serialization version when computing entity ID Signed-off-by: Abhin Chhabra <[email protected]> * Switch to the recommended layout in bigtable. This was recommended by the BigTable dev team. Details of this layout will be added to the documentation in a future commit. Signed-off-by: Abhin Chhabra <[email protected]> * Minor bugfixes. - If a row is empty when fetching data, don't process it more. - If a task in the threadpool fails, bubble up that failure. - If a `created_ts` is not available, use an empty string. `None` does not automatically serialize to bytes. Signed-off-by: Abhin Chhabra <[email protected]> * Move BigTable online store out of contrib As per feedback on the PR. Signed-off-by: Abhin Chhabra <[email protected]> * Attempt to run integration tests in CI. Provide the GCP project and the bigtable instance ID for the tests to connect to. Signed-off-by: Abhin Chhabra <[email protected]> * Delete tables for entity-less feature views. Signed-off-by: Abhin Chhabra <[email protected]> * Table names should be smaller than 50 characters This is BigTable's table length limit and it's causing test failures. Signed-off-by: Abhin Chhabra <[email protected]> * Optimize bigtable reads. - Fetch all the rows in one bigtable fetch. - Get only the columns that are necessary (using a column regex filter). Signed-off-by: Abhin Chhabra <[email protected]> * dynamodb: switch to `mock_dynamodb` The latest rebuilding of requirements has upgraded the `moto` library past the `4.0.0` release, which has a couple of breaking changes. Specifically, the `mock_dynamodb2` decorator has been deprecated. See https://github.com/spulec/moto/blob/master/CHANGELOG.md#400 for more details. The actual PR (getmoto/moto#4919) mentions that it's because the `mock_dynamodb` decorator is now equivalent to the `mock_dynamodb2` decorator. Signed-off-by: Abhin Chhabra <[email protected]> * minor: rename `BigTable` to `Bigtable` This matches the GCP docs. Signed-off-by: Abhin Chhabra <[email protected]> * Wrote some Bigtable documentation. Closely mirrors the docs for the other online stores. Signed-off-by: Abhin Chhabra <[email protected]> * Bugfix: Deal with missing row keys. It looks like the bigtable client will just skip over non-existent row keys. Signed-off-by: Abhin Chhabra <[email protected]> * Fix linting issues. Signed-off-by: Abhin Chhabra <[email protected]> * Generate requirements files. - As of version `1.49`, the various python packages in the [grpc repo](https://github.com/grpc/grpc/tree/master/src/python) require `protobuf>=4.21.3`. Unfortunately, this is incompatible with all versions of `tensorflow-metadata` (see [this issue](tensorflow/metadata#37)). And since `piptools` doesn't backtrack during dependency resolution, the requirement files cannot be regenerated without adding an upper limit on these grpc libraries directly in `setup.py`. - The previous attempt to upgrade usages of the `mock_dynamodb2` decorator to the newest version failed. Since I'm not an expert in dynamodb, it made sense to just cap the test tool to the version already being used in CI. Signed-off-by: Abhin Chhabra <[email protected]> * Don't bother materializing created timestamp. Had a discussion with Danny about whether it's useful to copy this column. He agreed that there's no value to storing this in the online store. Signed-off-by: Abhin Chhabra <[email protected]> * Remove `tensorflow-metadata`. Turns out that this dependency is not required. We removed all references to it in [this PR](feast-dev#2063), but did not remove it from `setup.py`. Removing it has caused many of the restrictions imposed in previous commits to be unnecessary. Signed-off-by: Abhin Chhabra <[email protected]> * Minor fix to Bigtable documentation. Feedback from Danny mentioned that Bigtable should be able to store multiple versions of the same key and fetch the latest at read time. This makes sense and means that concurrent writes should work just fine. Signed-off-by: Abhin Chhabra <[email protected]> * update roadmap docs Signed-off-by: Danny Chiao <[email protected]> * Fix roadmap doc Signed-off-by: Danny Chiao <[email protected]> * Change link to point to roadmap page Signed-off-by: Danny Chiao <[email protected]> * change order in roadmap Signed-off-by: Danny Chiao <[email protected]> Signed-off-by: Abhin Chhabra <[email protected]> Signed-off-by: Abhin Chhabra <[email protected]> Signed-off-by: Danny Chiao <[email protected]> Co-authored-by: Danny Chiao <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
User-facing changes:
The
mock_dynamodbis now equivalent tomock_dynamodb2Users of
mock_dynamodb2receive a warning that this decorator is deprecated, and they should switch tomock_dynamodbinsteadInternal changes:
The
moto/dynamodbdirectory has been renamed tomoto/dynamodb_v20111205The
moto/dynamodb2directory has been renamed tomoto/dynamodbAll internal references to Dynamodb now point towards
moto/dynamodbNote that the old API version is still usable, but only in ServerMode. A new test has been added to verify that this works by adding a specific request header,
"X-Amz-Target": "DynamoDB_20111205.ListTables"