Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

sealing: Recover sectors after failed AddPiece #7444

Merged
merged 1 commit into from
Oct 7, 2021
Merged

Conversation

magik6k
Copy link
Contributor

@magik6k magik6k commented Oct 4, 2021

No description provided.

@magik6k magik6k requested a review from a team as a code owner October 4, 2021 18:00
@jennijuju jennijuju added this to the v1.13.1 milestone Oct 5, 2021
@codecov
Copy link

codecov bot commented Oct 5, 2021

Codecov Report

Merging #7444 (cadbd00) into master (fc10281) will increase coverage by 0.00%.
The diff coverage is 20.00%.

Impacted file tree graph

@@           Coverage Diff           @@
##           master    #7444   +/-   ##
=======================================
  Coverage   39.54%   39.54%           
=======================================
  Files         616      616           
  Lines       65295    65297    +2     
=======================================
+ Hits        25818    25819    +1     
- Misses      34980    34988    +8     
+ Partials     4497     4490    -7     
Impacted Files Coverage Δ
extern/storage-sealing/fsm.go 58.56% <0.00%> (+1.13%) ⬆️
extern/storage-sealing/fsm_events.go 54.16% <0.00%> (-0.46%) ⬇️
extern/storage-sealing/input.go 55.35% <0.00%> (+0.19%) ⬆️
extern/storage-sealing/sector_state.go 92.30% <100.00%> (ø)
chain/actors/builtin/miner/diff.go 48.52% <0.00%> (-10.30%) ⬇️
extern/sector-storage/sched_resources.go 75.00% <0.00%> (-9.38%) ⬇️
extern/sector-storage/manager_calltracker.go 57.70% <0.00%> (-4.85%) ⬇️
chain/exchange/client.go 52.96% <0.00%> (-1.70%) ⬇️
itests/kit/blockminer.go 93.65% <0.00%> (-1.59%) ⬇️
extern/sector-storage/sched_worker.go 78.38% <0.00%> (-0.74%) ⬇️
... and 14 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update fc10281...cadbd00. Read the comment docs.

@@ -245,9 +245,7 @@ func (m *Sealing) handleAddPiece(ctx statemachine.Context, sector SectorInfo) er
}

func (m *Sealing) handleAddPieceFailed(ctx statemachine.Context, sector SectorInfo) error {
log.Errorf("No recovery plan for AddPiece failing")
// todo: cleanup sector / just go retry (requires adding offset param to AddPiece in sector-storage for this to be safe)
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review note: we do have offset in AddPiece already, calculated from the existingPieces thing which only has valid pieces

@magik6k magik6k merged commit 1993efe into master Oct 7, 2021
@magik6k magik6k deleted the feat/ap-fail-retry branch October 7, 2021 02:18
@neondragon
Copy link

neondragon commented Oct 14, 2021

Seems to work, perhaps not optimally but it saved a deal from being lost.

A piece failed to add to a sector, the sector ended up as a CC sector:

Event Log:
0.	2021-10-12 16:41:00 +0000 UTC:	[event;sealing.SectorStart]	{"User":{"ID":4510,"SectorType":8}}
1.	2021-10-12 16:41:00 +0000 UTC:	[event;sealing.SectorAddPiece]	{"User":{}}
2.	2021-10-12 16:44:50 +0000 UTC:	[event;sealing.SectorAddPieceFailed]	{"User":{}}
	writing piece: storage call error 0: pr read error: read tcp 1.1.1.1:3056->2.2.2.2:55220: read: connection timed out
3.	2021-10-12 16:44:50 +0000 UTC:	[event;sealing.SectorRetryWaitDeals]	{"User":{}}
4.	2021-10-12 17:41:00 +0000 UTC:	[event;sealing.SectorStartPacking]	{"User":{}}
5.	2021-10-12 17:46:13 +0000 UTC:	[event;sealing.SectorPacked]	{"User":{"FillerPieces":[{"Size":34359738368,"PieceCID":{"/":"baga6ea4seaqao7s73y24kcutaosvacpdjgfe5pw76ooefnyqw4ynr3d2y6x2mpq"}}]}}

The piece was added to another sector, sealed, and the storage deal is StorageDealActive.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants