Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Does Azure Storage's strong consistency extend to listing operations? #105331

Closed
teo-tsirpanis opened this issue Feb 16, 2023 · 7 comments
Closed

Comments

@teo-tsirpanis
Copy link
Contributor

Our code uses Azure Storage and requires blob/container creations/deletions to be immediately visible in listing operations. Currently we make sure of that by looping until the blob/container's properties can be retrieved. Do we actually need to do that or is it covered under Azure Storage's strong consistency guarantees?


Document Details

Do not edit this section. It is required for learn.microsoft.com ➟ GitHub issue linking.

@Naveenommi-MSFT
Copy link
Contributor

@teo-tsirpanis
Thanks for your feedback! We will investigate and update as appropriate.

@SaibabaBalapur-MSFT
Copy link
Contributor

@teo-tsirpanis
Yes, Azure Storage provides strong consistency guarantees for read operations, including listing operations. This means that after a write operation, read operations immediately thereafter will reflect the latest state of the data, including listing operations. However, there may be a small delay between the time a write operation is completed and the time it becomes visible in a listing operation. To ensure that a blob or container is immediately visible in a listing operation, you can use the GetBlobReference or GetContainerReference method to retrieve the blob or container and then check its properties. If the properties are not yet available, you can wait a short period of time and then try again. This process can be repeated until the properties are available.

Alternatively, you can use the BlobRequestOptions class to set the MaximumExecutionTime property to a value that is appropriate for your application. This property specifies the maximum amount of time that the client will wait for a response from the server. If the response is not received within the specified time, the client will throw an exception. This can be useful in ensuring that your application does not wait indefinitely for a response from the server.

In summary, Azure Storage provides strong consistency guarantees for read operations, including listing operations, but there may be a small delay between the time a write operation is completed and the time it becomes visible in a listing operation. To ensure immediate visibility, you can use the GetBlobReference or GetContainerReference method to retrieve the blob or container and then check its properties, or you can use the BlobRequestOptions class to set the MaximumExecutionTime property.

If there are any further questions regarding the documentation, please tag me in your reply and we will be happy to continue the conversation.

@teo-tsirpanis
Copy link
Contributor Author

Thanks for the timely response @SaibabaBalapur-MSFT.

Azure Storage provides strong consistency guarantees for […] listing operations

there may be a small delay between the time a write operation is completed and the time it becomes visible in a listing operation

Aren't these two phrases contradictory? Isn't the consistency model for listing operations eventual instead of strong?

@SaibabaBalapur-MSFT
Copy link
Contributor

@teo-tsirpanis
The consistency model for listing operations in Azure Storage is eventually consistent. (https://learn.microsoft.com/en-us/azure/cosmos-db/consistency-levels) This means that there may be a small delay between the time a write operation is completed and the time it becomes visible in a listing operation. However, the consistency model for reading and writing data in Azure Storage is strong consistency, which means that read operations always return the most recent version of an object.

I'd recommend working leverage our Q&A forum by posting your issue there so our community, and MVPs can further assist you in troubleshooting this issue or finding potential workarounds.[Teams Q&A forum] (https://docs.microsoft.com/en-us/answers/topics/46488/office-teams-windows-itpro.html) .

@teo-tsirpanis
Copy link
Contributor Author

I understand, thanks @SaibabaBalapur-MSFT. Can you update the documentation to mention eventual consistency for listing operations?

@SaibabaBalapur-MSFT
Copy link
Contributor

@teo-tsirpanis
Thanks for your feedback.
I'm going to assign this to the document author so they can take a look at it accordingly.

@pauljewellmsft
Can you please check and add your comments on this doc update request as applicable.

@pauljewellmsft
Copy link
Contributor

pauljewellmsft commented Mar 1, 2023

Thanks so much for this feedback. I confirmed with the engineering team that the List Blobs operation follows a strong consistency model. The doc has been updated to clarify that read and list operations are strongly consistent. Doc changes should go live later today.

Thanks again for contributing!

#please-close

teo-tsirpanis added a commit to teo-tsirpanis/TileDB that referenced this issue Mar 3, 2023
teo-tsirpanis added a commit to teo-tsirpanis/TileDB that referenced this issue Mar 8, 2023
teo-tsirpanis added a commit to teo-tsirpanis/TileDB that referenced this issue Mar 9, 2023
teo-tsirpanis added a commit to teo-tsirpanis/TileDB that referenced this issue Mar 10, 2023
teo-tsirpanis added a commit to teo-tsirpanis/TileDB that referenced this issue Mar 22, 2023
teo-tsirpanis added a commit to teo-tsirpanis/TileDB that referenced this issue Mar 28, 2023
teo-tsirpanis added a commit to teo-tsirpanis/TileDB that referenced this issue Apr 4, 2023
davisp pushed a commit to TileDB-Inc/TileDB that referenced this issue Apr 5, 2023
teo-tsirpanis added a commit to teo-tsirpanis/TileDB that referenced this issue Apr 11, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

4 participants