Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[FEATURE REQ] BlobClient.upload(InputStream, ...) shouldn't require a known length #20567

Closed
jaschrep-msft opened this issue Apr 12, 2021 · 1 comment
Labels
Client This issue points to a problem in the data-plane of the library. feature-request This issue requires a new behavior in the product in order be resolved. Storage Storage Service (Queues, Blobs, Files)

Comments

@jaschrep-msft
Copy link
Member

There are no sync upload methods which take an InputStream and don't require the user to submit a known length of that InputStream. However, the async equivalents don't need this. They don't even allow it to be set because the code wouldn't use it; once the library has a Flux a known length is irrelevant. Sync upload needs this because com.azure.storage.common.Utility.convertStreamToByteBuffer() needs a length to function with it's current implementation.

The method currently requires a length because (deferred until subscription on the return type) it opens out the conversion with Flux.range(0, (int) Math.ceil((double) length / (double) blockSize)).map(i -> i * blockSize) and then transforms that flux into InputStream reads of that size. We should be able to use Flux.push() or a similar method to return a Flux where subscription triggers a synchronous function that consumes the stream into ByteBuffers and pushes those out to subscribers. Care would need to be taken in writing the synchronous stream consumption such that each call doesn't read too much of the InputStream, building up memory and blocking other reactive steps from taking place (when Reactor is operating on one thread). With this, the stream is just read until completion and converted into the Flux the SDK already know how to handle indefinitely.

@jaschrep-msft jaschrep-msft added Storage Storage Service (Queues, Blobs, Files) Client This issue points to a problem in the data-plane of the library. feature-request This issue requires a new behavior in the product in order be resolved. labels Apr 12, 2021
@jaschrep-msft
Copy link
Member Author

Solved with #22218

azure-sdk pushed a commit to azure-sdk/azure-sdk-for-java that referenced this issue Oct 3, 2022
[Hub Generated] Review request for Microsoft.RecoveryServices to add version preview/2022-09-01-preview (Azure#20567)

* Adds base for updating Microsoft.RecoveryServices from version preview/2022-06-01-preview to version 2022-09-01-preview

* Updates readme

* Updates API version in new specs and examples

* Adding changes for container soft delete functionality

* Fixing lintdiff issues

* Updating URL with correct prefix
@github-actions github-actions bot locked and limited conversation to collaborators Apr 12, 2023
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
Client This issue points to a problem in the data-plane of the library. feature-request This issue requires a new behavior in the product in order be resolved. Storage Storage Service (Queues, Blobs, Files)
Projects
None yet
Development

No branches or pull requests

1 participant