Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[CI] AzureBlobStoreRepositoryTests failures #66385

Closed
fcofdez opened this issue Dec 15, 2020 · 1 comment · Fixed by #66489
Closed

[CI] AzureBlobStoreRepositoryTests failures #66385

fcofdez opened this issue Dec 15, 2020 · 1 comment · Fixed by #66489
Assignees
@fcofdez
Labels
:Distributed Coordination/Snapshot/Restore Anything directly related to the `_snapshot/*` APIs Team:Distributed (Obsolete) Meta label for distributed team (obsolete). Replaced by Distributed Indexing/Coordination. >test-failure Triaged test failures from CI v7.11.0 v8.0.0-alpha1

Comments

@fcofdez
Copy link
Contributor

fcofdez commented Dec 15, 2020

In some cases these tests are failing due to high memory usage.
@fcofdez fcofdez added :Distributed Coordination/Snapshot/Restore Anything directly related to the `_snapshot/*` APIs >test-failure Triaged test failures from CI v8.0.0 Team:Distributed (Obsolete) Meta label for distributed team (obsolete). Replaced by Distributed Indexing/Coordination. v7.11.0 labels Dec 15, 2020
@fcofdez fcofdez self-assigned this Dec 15, 2020
@elasticmachine
Copy link
Collaborator

Pinging @elastic/es-distributed (Team:Distributed)

This was referenced Dec 15, 2020
Merged
Merged
fcofdez added a commit that referenced this issue Dec 17, 2020
@fcofdez
Reduce memory usage on Azure repository implementation (#66489)
02ac68e 
This commit moves the upload logic to the repository itself
instead of delegating into the SDK.
Multi-block uploads are done sequentially instead of in parallel
that allows to bound the outstanding memory.
Additionally the number of i/o threads and heap arenas have been 
reduced to 1, to reduce the memory overhead.

Closes #66385
fcofdez added a commit to fcofdez/elasticsearch that referenced this issue Dec 17, 2020
@fcofdez
Reduce memory usage on Azure repository implementation
62d8251 
This commit moves the upload logic to the repository itself
instead of delegating into the SDK.
Multi-block uploads are done sequentially instead of in parallel
that allows to bound the outstanding memory.
Additionally the number of i/o threads and heap arenas have been
reduced to 1, to reduce the memory overhead.

Closes elastic#66385

Backport of elastic#66489
@fcofdez fcofdez mentioned this issue Dec 17, 2020
Merged
fcofdez added a commit to fcofdez/elasticsearch that referenced this issue Dec 17, 2020
@fcofdez
Reduce memory usage on Azure repository implementation
727faf1 
This commit moves the upload logic to the repository itself
instead of delegating into the SDK.
Multi-block uploads are done sequentially instead of in parallel
that allows to bound the outstanding memory.
Additionally the number of i/o threads and heap arenas have been
reduced to 1, to reduce the memory overhead.

Closes elastic#66385

Backport of  elastic#66489
@fcofdez fcofdez mentioned this issue Dec 17, 2020
Merged
fcofdez added a commit that referenced this issue Dec 17, 2020
@fcofdez
[7.x] Reduce memory usage on Azure repository implementation (#66499)
237265e 
This commit moves the upload logic to the repository itself
instead of delegating into the SDK.
Multi-block uploads are done sequentially instead of in parallel
that allows to bound the outstanding memory.
Additionally the number of i/o threads and heap arenas have been
reduced to 1, to reduce the memory overhead.

Closes #66385

Backport of #66489
fcofdez added a commit that referenced this issue Dec 17, 2020
@fcofdez
[7.11] Reduce memory usage on Azure repository implementation (#66500)
2737f03 
This commit moves the upload logic to the repository itself
instead of delegating into the SDK.
Multi-block uploads are done sequentially instead of in parallel
that allows to bound the outstanding memory.
Additionally the number of i/o threads and heap arenas have been
reduced to 1, to reduce the memory overhead.

Closes #66385

Backport of  #66489
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@fcofdez fcofdez

Labels
:Distributed Coordination/Snapshot/Restore Anything directly related to the `_snapshot/*` APIs Team:Distributed (Obsolete) Meta label for distributed team (obsolete). Replaced by Distributed Indexing/Coordination. >test-failure Triaged test failures from CI v7.11.0 v8.0.0-alpha1
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Reduce memory usage on Azure repository implementation fcofdez/elasticsearch
3 participants
@jakelandis @fcofdez @elasticmachine