curio: feat: break trees task into TreeD(prefetch) and TreeRC #11895
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
@magik6k I was tested the new Tasks and they are working fine. Can you please review the logic once? |
magik6k
reviewed
Apr 19, 2024
|
Sealing pipeline test after last commit 
|
magik6k
reviewed
Apr 29, 2024
LexLuthr
force-pushed
the
feat/break-treestask
branch
from
a10d1ef
to
1e1d305
Compare
|
@snadrus This is ready for your review. |
snadrus
requested changes
Apr 29, 2024
magik6k
reviewed
Apr 30, 2024
LexLuthr
force-pushed
the
feat/break-treestask
branch
from
e7da516
to
d4d92cf
Compare
magik6k
approved these changes
May 2, 2024
snadrus
added a commit
that referenced
this pull request
May 2, 2024
* Fixing dead links (#11907) * ci: ci: create gh workflow that updates sorted pr checks (#11861) * ci: create gh workflow that updates sorted pr checks * ci: use grouped_by_result template for pr checks sticky comment * chore: apply pr review suggestion * Avoid cfg lookup on chain remove since unenabled splitstore delete is noop anyway (#11916) Co-authored-by: zenground0 <[email protected]> * Fix mismatched method names in comments (#11913) Signed-off-by: forcedebug <[email protected]> * release: v1.26.3 (#11908) (#11915) (#11922) * deps: update dependencies to address migration memory bloat to address memory concerns during a heavy migration Ref: filecoin-project/go-state-types#260 Ref: whyrusleeping/cbor-gen#96 Ref: filecoin-project/go-amt-ipld#90 * release: prep v1.26.3 patch Prep v1.26.3 patch release: - Update changelog, version and make gen + make docsgen-cli * deps: update cbor-gen to tagged version deps: update cbor-gen to tagged version * deps: update go-state-types to tagged version deps: update go-state-types to tagged version v0.13.2 * chore: deps: update go-state-types to v0.13.3 Fixes a panic when we have fewer than 1k proposals. --------- Co-authored-by: Phi-rjan <[email protected]> Co-authored-by: Rod Vagg <[email protected]> Co-authored-by: Steven Allen <[email protected]> * Refactor `LookupID*` APIs in `StateManager` and `StateTree` The naming of `LookupID` can cause confusion when resolving actor IDs vs ID addresses. To avoid this: * Refactor `StateTree` `LookupID` to `LookupIDAddress`, because it returns ID address. * Refactor `StateManager` `LookupID` to `LookupIDAddress` because it also returns ID address via a chain call to `StateTree`. * Introduce a new API `StateManager` dedicated to resolving address to actor ID, called `LookupID` which returns `abi.ActorID`. For context, see: * #11723 (comment) * Add v13 support to invariants-checker (#11931) Add v13 support to invariants-checker * chore: docs: nv-skeleton documentation (#11065) * nv-skeleton documentation Add a tutorial for how one can create a nv-skeleton in Lotus * Add footnote for `Add migration` step Add footnote for `Add migration` step * Indent migration-code Indent migration-code to make it show properly as a footnote. * Add ref-fvm and filecoin-ffi checklist Add ref-fvm and filecoin-ffi checklist * Add Filecoin-FFI steps Add Filecoin-FFI steps * Add step to params_butterfly.go Add step to params_butterfly.go * Fix typo Fix typo * Add links to reference PRs Add links to reference PRs * Update ref-fvm list Update ref-fvm list * feat: curio: add StorageInit api (#11918) * feat: add StorageInit api * remove unused variables * fix gen check * feat: curio: simpler reservation release logic (#11900) * simpler release logic * oops, plus simpler * simpler * fix NewLine (#11893) * fix(events): check for sync-in-progress (#11932) * feat(events): adjust indexes in event index db to match query patterns Introduces a v4 migration that just adjusts indexes. Copies some improvements from #11723 Closes: #11909 * fix(pipeline): should return if error occurred when get network version (#11902) * fix(events): correct log msg for v4 events index db migration * chore: remove duplicate words in strings and comments * fix(events): register events index db migration v4 * fix: curio seal: Failed commit retry strategy (#11870) * ffi: improved-error-handling * curio seal: Failed commit retry strategy * use master ffi * mod tidy * fix: curio: Update pgx imports, fix db_storage alloc * feat: curioweb: Improve task_history indexes (#11911) * mod tidy * Event index should be unique for tipsets (#11952) * event index should be unique for tipsets * fix formatting * migrate to version 5 * chore: bump build version in master (#11946) * Bump version Bump version in master branch in preperation for cutting v1.27.0-rc1 * chore: bump build-version chore: bump build-version * feat: curioweb: Show piece info on the sector page (#11955) * curio: feat: break trees task into TreeD(prefetch) and TreeRC (#11895) * break trees task * fix TreeD reservation * fix nil pointer err * apply suggestions * fix allocate file types * fix dbIndex inserts * set resource, move release func * refactor func(), update memory * remove extra release --------- Signed-off-by: forcedebug <[email protected]> Co-authored-by: parthshah1 <[email protected]> Co-authored-by: Piotr Galar <[email protected]> Co-authored-by: ZenGround0 <[email protected]> Co-authored-by: zenground0 <[email protected]> Co-authored-by: forcedebug <[email protected]> Co-authored-by: Jiaying Wang <[email protected]> Co-authored-by: Phi-rjan <[email protected]> Co-authored-by: Rod Vagg <[email protected]> Co-authored-by: Steven Allen <[email protected]> Co-authored-by: Masih H. Derkani <[email protected]> Co-authored-by: Lee <[email protected]> Co-authored-by: Andrew Jackson (Ajax) <[email protected]> Co-authored-by: beck <[email protected]> Co-authored-by: 0x5459 <[email protected]> Co-authored-by: Łukasz Magiera <[email protected]> Co-authored-by: Łukasz Magiera <[email protected]> Co-authored-by: Aarsh Shah <[email protected]>
magik6k
added a commit
that referenced
this pull request
May 4, 2024
* fix: curio: Update pgx imports, fix db_storage alloc * feat: curioweb: Improve task_history indexes (#11911) * 1 * relatable * add and delete layer * chore: curio: merge master (#11956) * Fixing dead links (#11907) * ci: ci: create gh workflow that updates sorted pr checks (#11861) * ci: create gh workflow that updates sorted pr checks * ci: use grouped_by_result template for pr checks sticky comment * chore: apply pr review suggestion * Avoid cfg lookup on chain remove since unenabled splitstore delete is noop anyway (#11916) Co-authored-by: zenground0 <[email protected]> * Fix mismatched method names in comments (#11913) Signed-off-by: forcedebug <[email protected]> * release: v1.26.3 (#11908) (#11915) (#11922) * deps: update dependencies to address migration memory bloat to address memory concerns during a heavy migration Ref: filecoin-project/go-state-types#260 Ref: whyrusleeping/cbor-gen#96 Ref: filecoin-project/go-amt-ipld#90 * release: prep v1.26.3 patch Prep v1.26.3 patch release: - Update changelog, version and make gen + make docsgen-cli * deps: update cbor-gen to tagged version deps: update cbor-gen to tagged version * deps: update go-state-types to tagged version deps: update go-state-types to tagged version v0.13.2 * chore: deps: update go-state-types to v0.13.3 Fixes a panic when we have fewer than 1k proposals. --------- Co-authored-by: Phi-rjan <[email protected]> Co-authored-by: Rod Vagg <[email protected]> Co-authored-by: Steven Allen <[email protected]> * Refactor `LookupID*` APIs in `StateManager` and `StateTree` The naming of `LookupID` can cause confusion when resolving actor IDs vs ID addresses. To avoid this: * Refactor `StateTree` `LookupID` to `LookupIDAddress`, because it returns ID address. * Refactor `StateManager` `LookupID` to `LookupIDAddress` because it also returns ID address via a chain call to `StateTree`. * Introduce a new API `StateManager` dedicated to resolving address to actor ID, called `LookupID` which returns `abi.ActorID`. For context, see: * #11723 (comment) * Add v13 support to invariants-checker (#11931) Add v13 support to invariants-checker * chore: docs: nv-skeleton documentation (#11065) * nv-skeleton documentation Add a tutorial for how one can create a nv-skeleton in Lotus * Add footnote for `Add migration` step Add footnote for `Add migration` step * Indent migration-code Indent migration-code to make it show properly as a footnote. * Add ref-fvm and filecoin-ffi checklist Add ref-fvm and filecoin-ffi checklist * Add Filecoin-FFI steps Add Filecoin-FFI steps * Add step to params_butterfly.go Add step to params_butterfly.go * Fix typo Fix typo * Add links to reference PRs Add links to reference PRs * Update ref-fvm list Update ref-fvm list * feat: curio: add StorageInit api (#11918) * feat: add StorageInit api * remove unused variables * fix gen check * feat: curio: simpler reservation release logic (#11900) * simpler release logic * oops, plus simpler * simpler * fix NewLine (#11893) * fix(events): check for sync-in-progress (#11932) * feat(events): adjust indexes in event index db to match query patterns Introduces a v4 migration that just adjusts indexes. Copies some improvements from #11723 Closes: #11909 * fix(pipeline): should return if error occurred when get network version (#11902) * fix(events): correct log msg for v4 events index db migration * chore: remove duplicate words in strings and comments * fix(events): register events index db migration v4 * fix: curio seal: Failed commit retry strategy (#11870) * ffi: improved-error-handling * curio seal: Failed commit retry strategy * use master ffi * mod tidy * fix: curio: Update pgx imports, fix db_storage alloc * feat: curioweb: Improve task_history indexes (#11911) * mod tidy * Event index should be unique for tipsets (#11952) * event index should be unique for tipsets * fix formatting * migrate to version 5 * chore: bump build version in master (#11946) * Bump version Bump version in master branch in preperation for cutting v1.27.0-rc1 * chore: bump build-version chore: bump build-version * feat: curioweb: Show piece info on the sector page (#11955) * curio: feat: break trees task into TreeD(prefetch) and TreeRC (#11895) * break trees task * fix TreeD reservation * fix nil pointer err * apply suggestions * fix allocate file types * fix dbIndex inserts * set resource, move release func * refactor func(), update memory * remove extra release --------- Signed-off-by: forcedebug <[email protected]> Co-authored-by: parthshah1 <[email protected]> Co-authored-by: Piotr Galar <[email protected]> Co-authored-by: ZenGround0 <[email protected]> Co-authored-by: zenground0 <[email protected]> Co-authored-by: forcedebug <[email protected]> Co-authored-by: Jiaying Wang <[email protected]> Co-authored-by: Phi-rjan <[email protected]> Co-authored-by: Rod Vagg <[email protected]> Co-authored-by: Steven Allen <[email protected]> Co-authored-by: Masih H. Derkani <[email protected]> Co-authored-by: Lee <[email protected]> Co-authored-by: Andrew Jackson (Ajax) <[email protected]> Co-authored-by: beck <[email protected]> Co-authored-by: 0x5459 <[email protected]> Co-authored-by: Łukasz Magiera <[email protected]> Co-authored-by: Łukasz Magiera <[email protected]> Co-authored-by: Aarsh Shah <[email protected]> * linter oops * gen cleanup * fix * named returns are confusing --------- Signed-off-by: forcedebug <[email protected]> Co-authored-by: Łukasz Magiera <[email protected]> Co-authored-by: Łukasz Magiera <[email protected]> Co-authored-by: LexLuthr <[email protected]> Co-authored-by: parthshah1 <[email protected]> Co-authored-by: Piotr Galar <[email protected]> Co-authored-by: ZenGround0 <[email protected]> Co-authored-by: zenground0 <[email protected]> Co-authored-by: forcedebug <[email protected]> Co-authored-by: Jiaying Wang <[email protected]> Co-authored-by: Phi-rjan <[email protected]> Co-authored-by: Rod Vagg <[email protected]> Co-authored-by: Steven Allen <[email protected]> Co-authored-by: Masih H. Derkani <[email protected]> Co-authored-by: Lee <[email protected]> Co-authored-by: beck <[email protected]> Co-authored-by: 0x5459 <[email protected]> Co-authored-by: Aarsh Shah <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Related Issues
Proposed Changes
Additional Info
Checklist
Before you mark the PR ready for review, please make sure that:
<PR type>: <area>: <change being made>fix: mempool: Introduce a cache for valid signaturesPR type: fix, feat, build, chore, ci, docs, perf, refactor, revert, style, testarea, e.g. api, chain, state, market, mempool, multisig, networking, paych, proving, sealing, wallet, deps