[storage] Preview 2 #6442
Merged
[storage] Preview 2 #6442
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
* Refactored blob shared utils * Refactored file shared utils * Refactored queue shared utils * Refactored downloads * Refactored file downloads * Started async downloads * Async Files API * Flatten copy polling * Renamed uploads * Fixes samples based on vendor feedback (#6357) * fixed import statements for sync samples * fixed secrets readme samples * fixed keys readme samples * deduped secret names across samples * Upload refactor * Release approval docs (#6361) * updating dev documentation with approval details * Updated async pipeline * Avoid surprising aiohttp with unexpected kwargs (#6355) * Add challenge authentication to azure-keyvault-keys (#6244) * Add decorator (#6299) * add opencensus impl * dont need to check for noop tracer * get rid of span and trace id * fix span instance * added documentation * added documentation and refactor names of a few variables * write test for opencensus wrapper * put opencensus in the dev requirements * initial common * only import the wrapper when necessary * add check for the exporter * rework logic and fix some settings * added initial decorator * some mroe documentation * added decorators * small change * some minor fixes * test patch happening * fix space * share a function * clearer logic for setting span context * better logic * better logic * fix environ variable * test the way opencensus does it * middle of tests * only load if opencensus has already been imported * fix spelling mistake * temp * finish writing tests for common * charles fixes * fix tests * fix test settings * to header should not take a dict * from header should be class method * initial tests * dont create trace and get rid of end_tracer * dont need to save the trace * more little fixes * some intermediatary changes * fix type annotations * rst fix types * add :class:annotations * fix line wrapping * added tests for decorator * rename opencensus wrapper * intermediate changes * use spans the right way * some formatting * some grammar * restructure settings and make tests pass * rename get_parent * fix typings * use protocol and from_headers becomes links * ramifications of opencensus wrapper being a protocol * add tests for link * added async tests * delete the unit test thing * added add_attribute * added add_attribute * added tests for add attributes * remove unused import * added docstrings * minor docstring formatting * fix pylint errors * don't rely on opencensus children to check * use exporter to not rely on parent.children * test should only propagate * made test tracing helper * decrease flakiness of test * simplify get parent * calling a decorator decorator is redundant * fix settings * more elegant code * add await for async * should only have to wait a 1/1000 of a second * fix tests spans too short * accidentally deleted setup.cfg * Added async file tests * Consolidate Key Vault shared code (#6384) * Download tests * Service property tests * No recordings * Add credential wrapping MSAL ConfidentialClientApplication (#6358) * Add policy (#6379) * add opencensus impl * dont need to check for noop tracer * get rid of span and trace id * fix span instance * added documentation * added documentation and refactor names of a few variables * write test for opencensus wrapper * put opencensus in the dev requirements * initial common * only import the wrapper when necessary * add check for the exporter * rework logic and fix some settings * added initial decorator * some mroe documentation * added decorators * small change * some minor fixes * test patch happening * fix space * share a function * clearer logic for setting span context * better logic * better logic * fix environ variable * test the way opencensus does it * middle of tests * only load if opencensus has already been imported * fix spelling mistake * temp * finish writing tests for common * charles fixes * fix tests * fix test settings * to header should not take a dict * from header should be class method * initial tests * dont create trace and get rid of end_tracer * dont need to save the trace * more little fixes * some intermediatary changes * fix type annotations * rst fix types * add :class:annotations * fix line wrapping * added tests for decorator * rename opencensus wrapper * intermediate changes * use spans the right way * some formatting * some grammar * restructure settings and make tests pass * rename get_parent * fix typings * use protocol and from_headers becomes links * ramifications of opencensus wrapper being a protocol * add tests for link * added async tests * delete the unit test thing * added add_attribute * added add_attribute * added tests for add attributes * add initial policy * remove unused import * added docstrings * minor docstring formatting * fix pylint errors * don't rely on opencensus children to check * use exporter to not rely on parent.children * added documentation and span attributes * added test tracing policy * test should only propagate * made test tracing helper * decrease flakiness of test * simplify get parent * calling a decorator decorator is redundant * middle of writing tests * fix settings * add tests * test propogation also happens * more elegant code * await async stuff * add await for async * should only have to wait a 1/1000 of a second * fix tests spans too short * accidentally deleted setup.cfg * add component * add set_http_attributes * fix span network name * fix http request types * fix more types * bryan fixes * more efficient tests * non flakey tests * make tracing only use my context * test user agent on exception * pylint formatting * delete unused import * fix spelling * pylint * adding dockerfile (#6393) * Update cheatsheet.md * Async share tests * Async directory tests * Fixed some tests * aiohttp socket timeout * Patch azure core * CI fixes * Fix async tests for py35 * Python 3.5 support * Clean pylint
* Async Queues + tests port * More tests plus changes * pytest conf * Port shared folder * remove warnings from tests * Latest shared code * change destination * some more changes * ..and some more * fix tests * pylint + mypy * some more lint * A couple of test fixes * Fixed fake settings mode * Refactored queue utils * change conf ignore * minor change * Fix for urlencoding SAS tokens
|
Can one of the admins verify this patch? |
#6454) * add decorator and policty to files also propagated context! * added decorator to pollers * finished should also be traced * polling method does not need to be decorated * add tracing policies * propagate context * no decorator for generate_shared_access_signature * added decorators to async libs
* decorators and policy added * properly clear context * dont decorate private stuff * added policy * decorated async * get rid of those that dont make netowrk calls * propagate context
* properly clear context (#6448) * added decorators to keyvault-keys (#6381) * Remove default development settings (#6376) * Trace decorator optional arguments (#6459) * add optional parameters to decorators * added type hints * Use tracing policy (#6390) * Remove config public API for KV keys (#6418) * Remove config public API for KV keys * Make create_config private * Sync _shared keys/secrets * Type annotation fix * Adapt secrets tests for no config * Update README.md * decorate polling * decorate lease * decorate container client * trace blob service client * polling method does not need to be decorated * decorate blob client * add policy * added policy to the blob * Update root README.md Updating the Contributing section according to the guidelines * get rid of util and propagate context properly. * get rid of functions that do not do a network call * propagate context * Revert "propagate context" This reverts commit 1bd07ed.
* Update root README.md Updating the Contributing section according to the guidelines * Update CODEOWNERS * more code owners * add policy and decorators for secrets (#6453) * Smoke test for Python (#6412) * Smoke Test Sample for Track 2 libraries Smoke Test for Indentity, Key Vault Secrets, Storage Blobs, Event Hubs and Cosmos DB * simpleQuery method added * Method's names updated * Create README.md * Update README.md * Commented lines deleted * README.md moved to correct folder * Create requirements.txt * Update README.md * Update README.md * Imports changed * Use of literals instead of append * Database Name variable to class level. * Use of Pythonic with statements * Update requirements.txt * Revert "Update requirements.txt" This reverts commit 4b79c6a. * Revert "Use of Pythonic with statements" This reverts commit 81adc4c. * Revert "Revert "Use of Pythonic with statements"" This reverts commit 27b2a2d. * requiriments.txt encoded as a txt file * requirements.txt as text file * Misspelling in "Key concepts" * Update .docsettings.yml to match the tittle of Smoke Test * Went trought Suyog comments * Revert "Went trought Suyog comments" This reverts commit 35c6223. * Gone trought Suyog comments * use of snake case in file names * Paging v2 (#6420) * First version of paging * Clean sync paging * Async paging * Name AsyncList without underscore * MyPy happyness * pylint * black * PageIterator as conf * Simplify ItemPaged * Use chain to link iterable * Make Paging more open * Docstring * pylint / mypy / black * Regenerate KV with latest Autorest + Paging branch (#6479) * Regenerate KV with latest Autorest + Paging branch * Remove unused import in tests * Make the async adapter more generic * Regeneration after operation mixin fix in Autorest * COmmit changes forgotten in #6418 * Fixing syntax for 2.7 * Move Storage Queue to new Paging (#6447) * Move Storage Queue to new Paging * Working Storage queue on new paging * Fix types * First shot of blob tests update * Fixing blob tests * File to new paging * Doc fix * Feedbacks from @annatisch * Fix last @annatisch comment * Simplify paging contract * Fix storage queue after new paging contract * Fixed incorrect auto-merge * Fix docstring * Port aio to new async paging * Adapt async tests * Missing by_page
lmazuel
added
Client
|
@zezha-msft It'd be great if you can have a look into this meanwhile. We'll be merging the blobs PR into this later today. |
rakshith91
requested review from
johanste,
lmazuel,
xiangyan99 and
zezha-msft
as code owners
zezha-msft
reviewed
Aug 2, 2019
| @@ -744,6 +752,7 @@ def walk_blobs( | |||
| results_per_page=results_per_page, | |||
| delimiter=delimiter) | |||
|
|
|||
| @distributed_trace | |||
There was a problem hiding this comment.
@rakshith91 Btw is the fix for the SAS URL encoding bug raised earlier in this PR?
There was a problem hiding this comment.
@zezha-msft No, this is for tracing network calls which has been implemented across different SDKs by @SuyogSoti
There was a problem hiding this comment.
Yah, I left the comment here because this method upload_blob calls self.get_blob_client which has the bug.
Do you know if the fix is done somewhere else? Should I submit a PR for it?
There was a problem hiding this comment.
@zezha-msft - yes this PR has the fix. However it was not just with upload_blob, nor get_blob_client - the issue was with all SAS URLs.
What I didn't realize is that parse_qs function used here was actually URL-decoding the query string. So in this PR we now use quote() to re-encode the SAS token on the next line.
There was a problem hiding this comment.
Thanks @annatisch, I'll try to finish looking through the PR as soon as possible.
* Updated blobs shared code * Started blob refactor * Refactoring upload * Updated shared code * Started fixing tests * Refactored sync blobs * Added blob async APIs * Some fixes * Append blob async tests * blob access async tests * Blob client async tests * encryption async tests * Patch for azure core exception * blob retry async * Retry async tests * Get blob async tests * Bug fix for clear page operation * More async tests + upload fix * Merged blobs * Updated Files shared code * Updated queue shared code * async tests pass except 2 common blob tests * adds async paging to blobs and some async tests (not all pass) * initial commit * block_blob_tests * page blob tests * fix for special chars, some tests, and recordings * add to shared storage and fix import * adding more tests/recordings * more tests/recordings * rerecord tests, fix imports * fix import again * blacklist azure-servicemanagement-legacy * get CI to run * rerecord all async tests * testing * add variable indirection for storage live tests. this is a temporary measure to create live test pipelines for this particular framework * newline * print envar * remove testing * adjust pypy testing
kristapratico
requested review from
mitchdenny,
scbedd and
weshaggard
as code owners
* initial commit * recordings * fix all tests * update queue recordings * file recordings * queue recordings * fix for encoding tests * small fix * change conftest * remove recordings * oops * new recordings * some fixes * more fixes * and more fixes * and one more fix
… 'archtype-sdk-tests'
* play * playing * some prints * some tweaks * try-345 * fix * tweak for queue and file * remove prints
* Updated change logs * Bumped versions * Some test fixes * Fix for file handle next marker * Fixed error in release notes * Updated core version * Updated azure-core shared dep * Updated release notes
zezha-msft
approved these changes
Aug 6, 2019
* Fix for queue models * Docs cleanups
* Some final tweaks * oops * comments * little better * Pylint
* Fix for queue models * Docs cleanups * Better async import error
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Updates: