bug: supernode cannot re-download a uncompleted task which has header parameter

[lowzj]

Ⅰ. Issue Description

as described in the title

Ⅱ. Describe what happened

supernode downloads the file always failed before deleting it manually or in GC-procedure.

Ⅲ. Describe what you expected to happen

The client should download successfully after retrying.

Ⅳ. How to reproduce it (as minimally and precisely as possible)

1. use dfget to download a large file with non-empty --header flag

   dfget -u 'http://127.0.0.1:8001/a.test' -o /tmp/a.test --header 'User-Agent:dfget' --notbs

2. restart supernode before downloading completed
3. use dfget to download the same file again before the supernode delete the task in GC-procedure, it will get failed

Ⅴ. Anything else we need to know?

dfclient.log

2020-06-05 17:01:51.393 INFO sign:87141-1591347711.391 : get cmd params:["./bin/darwin_amd64/dfget" "-u" "http://127.0.0.1:8003/a.test" "-o" "/tmp/a.test" "--console" "--notbs" "--header" "User-Agent:dfget"]
2020-06-05 17:01:51.393 INFO sign:87141-1591347711.391 : get init config:{"url":"http://127.0.0.1:8003/a.test","output":"/tmp/a.test","pattern":"p2p","header":["User-Agent:dfget"],"notbs":true,"console":true,"localLimit":"20MB","minRate":"64KB","clientQueueSize":6,"workHome":"/Users/zj/.small-dragonfly","logConfig":{"maxSize":0,"maxBackups":0,"path":"/Users/zj/.small-dragonfly/logs/dfclient.log"}}
2020-06-05 17:01:51.393 INFO sign:87141-1591347711.391 : target file path:/tmp/a.test
2020-06-05 17:01:51.394 INFO sign:87141-1591347711.391 : runtimeVariable: {"MetaPath":"/Users/zj/.small-dragonfly/meta/host.meta","SystemDataDir":"/Users/zj/.small-dragonfly/data","DataDir":"/Users/zj/.small-dragonfly/data","RealTarget":"/tmp/a.test","StreamMode":false,"TargetDir":"/tmp","TempTarget":"/tmp/dfget-87141-1591347711.391.tmp-051994928","Cid":"127.0.0.1-87141-1591347711.391","TaskURL":"http://127.0.0.1:8003/a.test","TaskFileName":"a.test-87141-1591347711.391","LocalIP":"127.0.0.1","PeerPort":0,"FileLength":-1,"DataExpireTime":180000000000,"ServerAliveTime":300000000000}
2020-06-05 17:01:51.455 INFO sign:87141-1591347711.391 : local http result:a.test-87141-1591347711.391 err:<nil>, port:19492 path:/check/
2020-06-05 17:01:51.455 INFO sign:87141-1591347711.391 : use peer server on port:19492
2020-06-05 17:01:51.455 INFO sign:87141-1591347711.391 : do register to one of [127.0.0.1:8002 127.0.0.1:8002]
2020-06-05 17:01:51.458 INFO sign:87141-1591347711.391 : do register to 127.0.0.1:8002, res:{"code":200,"msg":"success","data":{"taskId":"ba4b024dca33f94d083a70a660c05ebe5ca04053437ebebc063280583ae30fa5","fileLength":522190433,"pieceSize":6291456,"cdnSource":"supernode","asSeed":false,"seedTaskID":""}} error:<nil>
2020-06-05 17:01:51.459 INFO sign:87141-1591347711.391 : do register result:{"code":200,"msg":"success","data":{"taskId":"ba4b024dca33f94d083a70a660c05ebe5ca04053437ebebc063280583ae30fa5","fileLength":522190433,"pieceSize":6291456,"cdnSource":"supernode","asSeed":false,"seedTaskID":""}} and cost:0.004s
2020-06-05 17:01:51.460 INFO sign:87141-1591347711.391 : downloading piece:{"taskID":"ba4b024dca33f94d083a70a660c05ebe5ca04053437ebebc063280583ae30fa5","superNode":"127.0.0.1:8002","dstCid":"","range":"","result":502,"status":700,"pieceSize":0,"pieceNum":0}
2020-06-05 17:01:51.528 ERRO sign:87141-1591347711.391 : pull piece task fail:{"code":500,"msg":"taskID: ba4b024dca33f94d083a70a660c05ebe5ca04053437ebebc063280583ae30fa5: {\"Code\":7,\"Msg\":\"cdn status is fail\"}"} and will migrate
2020-06-05 17:01:51.528 INFO sign:87141-1591347711.391 : do register to one of [127.0.0.1:8002]
2020-06-05 17:01:51.528 WARN sign:87141-1591347711.391 : the last registered node is the same(127.0.0.1:8002)
2020-06-05 17:01:51.528 ERRO sign:87141-1591347711.391 : register fail:{"Code":-100,"Msg":"empty response, unknown error"}
2020-06-05 17:01:51.528 ERRO sign:87141-1591347711.391 : failed to download piece: {"Code":-100,"Msg":"empty response, unknown error"}
2020-06-05 17:01:51.529 ERRO sign:87141-1591347711.391 : failed to download by dragonfly: failed to download with p2p pattern, reason: 3, and start try to download from source
2020-06-05 17:01:51.529 INFO sign:87141-1591347711.391 : download FAIL cost:0.139s length:522190433 reason:1003 error:failed to download file from source: download fail and not back source: 1003

supernode log: app.log

2020-06-05 17:01:51.458 INFO sign:87132 : success to init cdn node or taskID ba4b024dca33f94d083a70a660c05ebe5ca04053437ebebc063280583ae30fa5
2020-06-05 17:01:51.458 INFO sign:87132 : success to start cdn trigger for taskID: ba4b024dca33f94d083a70a660c05ebe5ca04053437ebebc063280583ae30fa5
2020-06-05 17:01:51.458 DEBU sign:87132 : success to register task &{CID:127.0.0.1-87141-1591347711.391 CallSystem: Dfdaemon:false FileLength:0 Filter:[] Headers:map[User-Agent:dfget] Identifier: Md5: Path:/peer/file/a.test-87141-15913
47711.391 PeerID:ali-186590dee46f.local-127.0.0.1-1591347711456805000 RawURL:http://127.0.0.1:8003/a.test SupernodeIP:127.0.0.1 TaskID: TaskURL:http://127.0.0.1:8003/a.test}
2020-06-05 17:01:51.458 DEBU sign:87132 : success to read metadata: &{TaskID:ba4b024dca33f94d083a70a660c05ebe5ca04053437ebebc063280583ae30fa5 URL:http://127.0.0.1:8003/a.test PieceSize:6291456 HTTPFileLen:522190433 Identifier: AccessTi
me:1591347698179 Interval:0 FileLength:0 Md5: RealMd5: LastModified:1560503022000 ETag:"5d0362ee-1f1ffe61" Finish:false Success:false} for taskID: ba4b024dca33f94d083a70a660c05ebe5ca04053437ebebc063280583ae30fa5
2020-06-05 17:01:51.458 DEBU sign:87132 : check same File for taskID(ba4b024dca33f94d083a70a660c05ebe5ca04053437ebebc063280583ae30fa5) get result: true
2020-06-05 17:01:51.459 DEBU sign:87132 : success to get expired result: false for taskID(ba4b024dca33f94d083a70a660c05ebe5ca04053437ebebc063280583ae30fa5)
2020-06-05 17:01:51.460 INFO sign:87132 : taskID: ba4b024dca33f94d083a70a660c05ebe5ca04053437ebebc063280583ae30fa5, detects cache breakNum: 0
2020-06-05 17:01:51.460 INFO sign:87132 : reset repo for taskID: ba4b024dca33f94d083a70a660c05ebe5ca04053437ebebc063280583ae30fa5
2020-06-05 17:01:51.461 INFO sign:87132 : start to download for taskId(ba4b024dca33f94d083a70a660c05ebe5ca04053437ebebc063280583ae30fa5) with fileUrl: http://127.0.0.1:8003/a.test header: map[If-Modified-Since:Fri, 14 Jun 2019 09:03:42
 GMT If-None-Match:"5d0362ee-1f1ffe61" Range:bytes=0-0 User-Agent:dfget] checkCode: [200 206]
2020-06-05 17:01:51.461 DEBU sign:87132 : get pieces request: &{DfgetTaskStatus:STARTED DstPID: PieceRange: PieceResult:INVALID} with taskID(ba4b024dca33f94d083a70a660c05ebe5ca04053437ebebc063280583ae30fa5) and clientID(127.0.0.1-87141
-1591347711.391)
2020-06-05 17:01:51.461 DEBU sign:87132 : success to get dfgetTask: &{CID:127.0.0.1-87141-1591347711.391 CallSystem: Dfdaemon:false Path:/peer/file/a.test-87141-1591347711.391 PeerID:ali-186590dee46f.local-127.0.0.1-1591347711456805000
 PieceSize:6291456 Status:WAITING SupernodeIP:127.0.0.1 TaskID:ba4b024dca33f94d083a70a660c05ebe5ca04053437ebebc063280583ae30fa5}
2020-06-05 17:01:51.461 DEBU sign:87132 : success to get task: &{ID:ba4b024dca33f94d083a70a660c05ebe5ca04053437ebebc063280583ae30fa5 AsSeed:false CdnStatus:RUNNING FileLength:0 Headers:map[If-Modified-Since:Fri, 14 Jun 2019 09:03:42 GM
T If-None-Match:"5d0362ee-1f1ffe61" Range:bytes=0-0 User-Agent:dfget] HTTPFileLength:522190433 Identifier: Md5: PieceSize:6291456 PieceTotal:83 RawURL:http://127.0.0.1:8003/a.test RealMd5: TaskURL:http://127.0.0.1:8003/a.test}
2020-06-05 17:01:51.461 DEBU sign:87132 : start to process task(ba4b024dca33f94d083a70a660c05ebe5ca04053437ebebc063280583ae30fa5) start
2020-06-05 17:01:51.462 INFO sign:87132 : success update dfgetTask status to RUNNING with taskID: ba4b024dca33f94d083a70a660c05ebe5ca04053437ebebc063280583ae30fa5 clientID: 127.0.0.1-87141-1591347711.391
2020-06-05 17:01:51.462 ERRO sign:87132 : taskID(ba4b024dca33f94d083a70a660c05ebe5ca04053437ebebc063280583ae30fa5) trigger cdn get error: unexpected status code: 304
2020-06-05 17:01:51.462 INFO sign:87132 : success to update task cdn &{ID: AsSeed:false CdnStatus:FAILED FileLength:0 Headers:map[] HTTPFileLength:0 Identifier: Md5: PieceSize:0 PieceTotal:0 RawURL: RealMd5: TaskURL:}

Ⅵ. Environment:

• dragonfly version: 1.0.2
• OS (e.g. from /etc/os-release):
• Kernel (e.g. uname -a):
• Install tools:
• Others: