Skip to content
This repository has been archived by the owner on Oct 11, 2024. It is now read-only.

GC: Add routines for large operations #362

Closed
1 of 2 tasks
dadams39 opened this issue Jul 19, 2022 · 1 comment
Closed
1 of 2 tasks

GC: Add routines for large operations #362

dadams39 opened this issue Jul 19, 2022 · 1 comment
Assignees
@dadams39
Labels
enhancement New feature or request tech-debt Non-feature, non-bug improvements to the codebase.

Comments

@dadams39
Copy link
Contributor

dadams39 commented Jul 19, 2022 •
edited
Loading

Related to Issue #360 and #356
Large requests can exceed the amount of time set aside for testing and general customer satisfaction. Need to add the ability to implement a divide-and-conquer abstraction on large tasks.

  • Add a cap for the number of allowable thread workers
  • Ensure that all Collections retrieval operations are parallelized as much as possible to reduce the are given separate workers to prevent deadlocking when possible
@dadams39 dadams39 added the enhancement New feature or request label Jul 19, 2022
@dadams39 dadams39 self-assigned this Jul 19, 2022
@ryanfkeepers
Copy link
Contributor

Traverse work batches in an order that can be replicated by the KW

"Can be replicated" or "are guaranteed to be replicated"? I think the latter might be a requirement.

@dadams39 dadams39 mentioned this issue Jul 19, 2022
Closed
dadams39 added a commit that referenced this issue Jul 19, 2022
@dadams39
Issue #362: Sort optimization
7dc7505 
Added sort and sort test for DataCollections
@dadams39 dadams39 mentioned this issue Nov 28, 2022
Merged
9 tasks
aviator-app bot pushed a commit that referenced this issue Dec 1, 2022
@meain
Speed up Exchange backup by parallelizing url fetch (#1608)
784db9e 
## Description

From rough numbers we can speedup an account with ~3000 emails, ~1000 contacts and ~1000 events from ~18m to <3m.

## Type of change

<!--- Please check the type of change your PR introduces: --->
- [ ] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Test
- [ ] 💻 CI/Deployment
- [ ] 🐹 Trivial/Minor

## Issue(s)

<!-- Can reference multiple issues. Use one of the following "magic words" - "closes, fixes" to auto-close the Github issue. -->
* #362
* #1595
* #1607

## Test Plan

<!-- How will this be tested prior to merging.-->
- [ ] 💪 Manual
- [x] ⚡ Unit test
- [x] 💚 E2E
@dadams39 dadams39 added the tech-debt Non-feature, non-bug improvements to the codebase. label Jan 19, 2023
@gmatev gmatev closed this as not planned Won't fix, can't repro, duplicate, stale Mar 30, 2023
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Assignees

@dadams39 dadams39

Labels
enhancement New feature or request tech-debt Non-feature, non-bug improvements to the codebase.
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@dadams39 @gmatev @ryanfkeepers