Upgrade WindowsAzure.Storage to Microsoft.Azure.Cosmos.Table and Asure.Storage.Blobs #151
Conversation
…e.Microsoft.Azure.Storage.Blob (Step 1)
| true, | ||
| BlobListingDetails.Metadata, null, null, requestOptions, new OperationContext(), cts.Token); | ||
| } | ||
| var results = Container.GetBlobsAsync( |
There was a problem hiding this comment.
Get blob info list IAsyncEnumerator
|
|
||
| async Task<SelectedSnapshot> FilterAndFetch(BlobResultSegment segment) | ||
| { | ||
| if (!await pageEnumerator.MoveNextAsync()) |
There was a problem hiding this comment.
We're only interested on the first page, will not retrieve the whole list of blobs
| async Task<SelectedSnapshot> FilterAndFetch(BlobResultSegment segment) | ||
| { | ||
| if (!await pageEnumerator.MoveNextAsync()) | ||
| return null; |
There was a problem hiding this comment.
and if that fails, just return immediately
| .Where(x => FilterBlobTimestamp(criteria, x)); | ||
|
|
||
| var deleteTasks = new List<Task>(); | ||
| await foreach (var blob in filtered.WithCancellation(cts.Token)) |
There was a problem hiding this comment.
Replace the old paging code with the one provided by the SDK
| /// <summary> | ||
| /// Provides helpers to validate resource names across the Microsoft Azure Storage Services. | ||
| /// </summary> | ||
| public static class NameValidator |
There was a problem hiding this comment.
The latest Blob SDK doesn't have a name validator, so swiping it from the old SDK code
| @@ -5,6 +5,7 @@ | |||
| <PropertyGroup> | |||
| <TargetFramework>$(NetStandardLibVersion)</TargetFramework> | |||
| <Description>Akka.Persistence support for Windows Azure Table storage and Azure blob storage.</Description> | |||
| <LangVersion>8.0</LangVersion> | |||
| } | ||
|
|
||
| return containerRef; | ||
| var response = await blobClient.CreateIfNotExistsAsync( | ||
| PublicAccessType.BlobContainer, |
There was a problem hiding this comment.
Does this affect whether the blob is accessible to the public or is this just weird naming on the part of the Azure SDK?
There was a problem hiding this comment.
From the documentation:
Azure Storage supports optional anonymous public read access for containers and blobs. By default, anonymous access to your data is never permitted. Unless you explicitly enable anonymous access, all requests to a container and its blobs must be authorized. When you configure a container's public access level setting to permit anonymous access, clients can read data in that container without authorizing the request.
So the code allows access to the blobs and containers to anonymous connections.
Its there to replace the original code await containerRef.CreateIfNotExistsAsync(BlobContainerPublicAccessType.Container, new BlobRequestOptions(), op, cts.Token) which does the exact same thing.
There was a problem hiding this comment.
Didn't we just merge a PR in a few days ago that explicitly disabled this (because disallowing anonymous read access to application data by default makes sense) ?
There was a problem hiding this comment.
I don't see that merge, can you link the PR?
There was a problem hiding this comment.
It was this issue here: #144
There was a problem hiding this comment.
Looks like they never changed the default setting when auto-initialize = on, but it shouldn't be public by default
There was a problem hiding this comment.
Lets address that in another PR
| async Task<SelectedSnapshot> FilterAndFetch(BlobResultSegment segment) | ||
| { | ||
| if (!await pageEnumerator.MoveNextAsync()) | ||
| return null; |
| /// <summary> | ||
| /// Provides helpers to validate resource names across the Microsoft Azure Storage Services. | ||
| /// </summary> | ||
| public static class NameValidator |
|
The runtime of the test suite increased from about 2.5 minutes on Linux to 3.25 minutes, 3.5 minutes on Linux to 5.25 minutes here. Is there anything this SDK changes, or the way we're using it, that would have made the queries run for much longer than previously? |
|
Since everything is handled by the API |
|
Can we pin-point the cause of the performance degradation? |
I'm just looking at the data from Azure Pipelines. Can you investigate? |
|
I'll try and give it a look-see |
|
Oh, the performance test only tests for journaling, which uses Azure Tables, which is upgraded to There is a replacement for |
|
Ok, we'll see if anything comes up from user reports down the stretch. |
No description provided.