Skip to content
This repository has been archived by the owner on Feb 27, 2023. It is now read-only.

bugfix: modify two supernode api's bug #682

Merged
merged 1 commit into from
Jul 31, 2019

Conversation

yunfeiyanggzq
Copy link
Member

@yunfeiyanggzq yunfeiyanggzq commented Jul 18, 2019

Signed-off-by: yunfeiyangbuaa [email protected]

Ⅰ. Describe what this PR did

  • First bug:when supernode send response to dfget's reportpiece and servicedown request,it will return a nil response.So when the dfget do with the response with json.Unmarshal(body, resp) ,it will produce an err unexpected end of JSON input err.I add a status code info to the supernode's response to resolve these bugs and do with the err and status code
  • Second bug:when dfget send pulltask request to supernode.if the supernode is off,the dfget will crash instead of find another supernode to finish its work.because the pulltask will retuen a {} instead of nil,the coder use nil to check the res,so it will happen a invalid memory address or nil pointer dereference err .i make the return be nil when the supernode is off

Ⅱ. Does this pull request fix one issue?

fixes #680

Ⅲ. Why don't you add test cases (unit test/integration test)? (你真的觉得不需要加测试吗?)

Ⅳ. Describe how to verify it

you can add a fmt.println() to the previous code's api return err,it will report an err.and my code won't

Ⅴ. Special notes for reviews

@pouchrobot pouchrobot added kind/bug This is bug report for project size/S labels Jul 18, 2019
@allencloud
Copy link
Contributor

Please carefully check the CI failures. Thanks a lot. @yunfeiyangbuaa

@yunfeiyanggzq
Copy link
Member Author

Please carefully check the CI failures. Thanks a lot. @yunfeiyangbuaa

Done

@codecov-io
Copy link

Codecov Report

Merging #682 into master will decrease coverage by 0.12%.
The diff coverage is 9.52%.

Impacted file tree graph

@@            Coverage Diff             @@
##           master     #682      +/-   ##
==========================================
- Coverage   46.86%   46.74%   -0.13%     
==========================================
  Files         108      108              
  Lines        6226     6240      +14     
==========================================
- Hits         2918     2917       -1     
- Misses       3075     3085      +10     
- Partials      233      238       +5
Impacted Files Coverage Δ
...et/core/downloader/p2p_downloader/client_writer.go 7.44% <0%> (-0.34%) ⬇️
supernode/server/0.3_bridge.go 0% <0%> (ø) ⬆️
dfget/core/uploader/peer_server.go 84.3% <20%> (-2.54%) ⬇️
supernode/daemon/mgr/scheduler/manager.go 24.06% <0%> (-0.76%) ⬇️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 7689d60...bab1b04. Read the comment docs.

4 similar comments
@codecov-io
Copy link

Codecov Report

Merging #682 into master will decrease coverage by 0.12%.
The diff coverage is 9.52%.

Impacted file tree graph

@@            Coverage Diff             @@
##           master     #682      +/-   ##
==========================================
- Coverage   46.86%   46.74%   -0.13%     
==========================================
  Files         108      108              
  Lines        6226     6240      +14     
==========================================
- Hits         2918     2917       -1     
- Misses       3075     3085      +10     
- Partials      233      238       +5
Impacted Files Coverage Δ
...et/core/downloader/p2p_downloader/client_writer.go 7.44% <0%> (-0.34%) ⬇️
supernode/server/0.3_bridge.go 0% <0%> (ø) ⬆️
dfget/core/uploader/peer_server.go 84.3% <20%> (-2.54%) ⬇️
supernode/daemon/mgr/scheduler/manager.go 24.06% <0%> (-0.76%) ⬇️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 7689d60...bab1b04. Read the comment docs.

@codecov-io
Copy link

Codecov Report

Merging #682 into master will decrease coverage by 0.12%.
The diff coverage is 9.52%.

Impacted file tree graph

@@            Coverage Diff             @@
##           master     #682      +/-   ##
==========================================
- Coverage   46.86%   46.74%   -0.13%     
==========================================
  Files         108      108              
  Lines        6226     6240      +14     
==========================================
- Hits         2918     2917       -1     
- Misses       3075     3085      +10     
- Partials      233      238       +5
Impacted Files Coverage Δ
...et/core/downloader/p2p_downloader/client_writer.go 7.44% <0%> (-0.34%) ⬇️
supernode/server/0.3_bridge.go 0% <0%> (ø) ⬆️
dfget/core/uploader/peer_server.go 84.3% <20%> (-2.54%) ⬇️
supernode/daemon/mgr/scheduler/manager.go 24.06% <0%> (-0.76%) ⬇️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 7689d60...bab1b04. Read the comment docs.

@codecov-io
Copy link

Codecov Report

Merging #682 into master will decrease coverage by 0.12%.
The diff coverage is 9.52%.

Impacted file tree graph

@@            Coverage Diff             @@
##           master     #682      +/-   ##
==========================================
- Coverage   46.86%   46.74%   -0.13%     
==========================================
  Files         108      108              
  Lines        6226     6240      +14     
==========================================
- Hits         2918     2917       -1     
- Misses       3075     3085      +10     
- Partials      233      238       +5
Impacted Files Coverage Δ
...et/core/downloader/p2p_downloader/client_writer.go 7.44% <0%> (-0.34%) ⬇️
supernode/server/0.3_bridge.go 0% <0%> (ø) ⬆️
dfget/core/uploader/peer_server.go 84.3% <20%> (-2.54%) ⬇️
supernode/daemon/mgr/scheduler/manager.go 24.06% <0%> (-0.76%) ⬇️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 7689d60...bab1b04. Read the comment docs.

@codecov-io
Copy link

Codecov Report

Merging #682 into master will decrease coverage by 0.12%.
The diff coverage is 9.52%.

Impacted file tree graph

@@            Coverage Diff             @@
##           master     #682      +/-   ##
==========================================
- Coverage   46.86%   46.74%   -0.13%     
==========================================
  Files         108      108              
  Lines        6226     6240      +14     
==========================================
- Hits         2918     2917       -1     
- Misses       3075     3085      +10     
- Partials      233      238       +5
Impacted Files Coverage Δ
...et/core/downloader/p2p_downloader/client_writer.go 7.44% <0%> (-0.34%) ⬇️
supernode/server/0.3_bridge.go 0% <0%> (ø) ⬆️
dfget/core/uploader/peer_server.go 84.3% <20%> (-2.54%) ⬇️
supernode/daemon/mgr/scheduler/manager.go 24.06% <0%> (-0.76%) ⬇️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 7689d60...bab1b04. Read the comment docs.

@codecov-io
Copy link

codecov-io commented Jul 18, 2019

Codecov Report

Merging #682 into master will decrease coverage by 0.1%.
The diff coverage is 20%.

Impacted file tree graph

@@            Coverage Diff             @@
##           master     #682      +/-   ##
==========================================
- Coverage   47.39%   47.28%   -0.11%     
==========================================
  Files         109      109              
  Lines        6294     6306      +12     
==========================================
- Hits         2983     2982       -1     
- Misses       3078     3086       +8     
- Partials      233      238       +5
Impacted Files Coverage Δ
supernode/server/0.3_bridge.go 0% <0%> (ø) ⬆️
dfget/core/api/supernode_api.go 85.07% <28.57%> (-14.93%) ⬇️
supernode/daemon/mgr/scheduler/manager.go 24.06% <0%> (-0.76%) ⬇️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update f620104...cbe57dc. Read the comment docs.

@codecov-io
Copy link

Codecov Report

Merging #682 into master will decrease coverage by 0.12%.
The diff coverage is 9.52%.

Impacted file tree graph

@@            Coverage Diff             @@
##           master     #682      +/-   ##
==========================================
- Coverage   46.86%   46.74%   -0.13%     
==========================================
  Files         108      108              
  Lines        6226     6240      +14     
==========================================
- Hits         2918     2917       -1     
- Misses       3075     3085      +10     
- Partials      233      238       +5
Impacted Files Coverage Δ
...et/core/downloader/p2p_downloader/client_writer.go 7.44% <0%> (-0.34%) ⬇️
supernode/server/0.3_bridge.go 0% <0%> (ø) ⬆️
dfget/core/uploader/peer_server.go 84.3% <20%> (-2.54%) ⬇️
supernode/daemon/mgr/scheduler/manager.go 24.06% <0%> (-0.76%) ⬇️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 7689d60...bab1b04. Read the comment docs.

@yunfeiyanggzq yunfeiyanggzq force-pushed the supernodeAPI branch 2 times, most recently from b013df3 to 200ed54 Compare July 18, 2019 13:08
dfget/core/downloader/p2p_downloader/client_writer.go Outdated Show resolved Hide resolved
dfget/core/uploader/peer_server.go Outdated Show resolved Hide resolved
dfget/core/uploader/peer_server.go Outdated Show resolved Hide resolved
@yunfeiyanggzq yunfeiyanggzq force-pushed the supernodeAPI branch 2 times, most recently from dfc9b7e to b4ecc66 Compare July 19, 2019 11:16
Copy link
Member Author

@yunfeiyanggzq yunfeiyanggzq left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@starnop @allencloud I have modified the code according to your advice

dfget/core/uploader/peer_server.go Outdated Show resolved Hide resolved
@yunfeiyanggzq yunfeiyanggzq changed the title bugfix:modify supernode's response to dfget to resolve the json err bugfix:modify two supernode api's bug Jul 23, 2019
@allencloud allencloud changed the title bugfix:modify two supernode api's bug bugfix: modify two supernode api's bug Jul 23, 2019
@@ -90,7 +93,9 @@ func (api *supernodeAPI) PullPieceTask(node string, req *types.PullPieceTaskRequ
api.Scheme, node, peerPullPieceTaskPath, util.ParseQuery(req))

resp = new(types.PullPieceTaskResponse)
e = api.get(url, resp)
if e = api.get(url, resp); e != nil {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why to change this? I do not see any difference after the change actually.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

if res == nil || (res.Code != constants.CodePeerContinue &&

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

if the supernode is off,the resp={} in the previous code.and res.code will make the dfget happen a a invalid memory address or nil pointer dereference err

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I return a nil if the supernode if off so the the program will go on

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It is so tricky. I would like to see you change the function definition. Please do not define the variables in the function declaration. Just define them in the implementation block.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It is so tricky. I would like to see you change the function definition. Please do not define the variables in the function declaration. Just define them in the implementation block.
In fact I can't understand your comment very clearly,Can in more details?.But I think the coder want to return a nil when a err happen such as

if code, body, e = api.HTTPClient.PostJSON(url, req, api.Timeout); e != nil {
.
and he just ignore this place in the previous implement .

resp = new(types.BaseResponse)
e = api.get(url, resp)
if e != nil {
logrus.Infof("failed to report piece{taskid:%s,range:%s},err: %v", req.TaskID, req.PieceRange, e)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please confirm that here is an Infof level log, or Errorf level log?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@starnop #682 (comment)
Error or Info ?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am sure that Info level is improper. Both Error and Warn are better.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am sure that Info level is improper. Both Error and Warn are better.
I am agree with you.I will modify it

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done

dfget/core/api/supernode_api.go Outdated Show resolved Hide resolved
dfget/core/api/supernode_api.go Outdated Show resolved Hide resolved
Copy link
Member

@lowzj lowzj left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Method Register also need to be fixed.

@yunfeiyanggzq
Copy link
Member Author

Method Register also need to be fixed.

done

@starnop
Copy link
Contributor

starnop commented Jul 29, 2019

LGTM. @yunfeiyangbuaa And could you please make a rebase?

@yunfeiyanggzq
Copy link
Member Author

LGTM. @yunfeiyangbuaa And could you please make a rebase?

Done

resp = new(types.BaseResponse)
e = api.get(url, resp)
if e = api.get(url, resp); e != nil {
logrus.Errorf("failed to report piece{taskid:%s,range:%s},err: %v", req.TaskID, req.PieceRange, e)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just return error in API implementation rather than logging them. Then the caller can choose to log the error themselves.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

dfgte haven't check all the err and response.so should I and a check ? @lowzj

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

#682 (comment)
I added a check before. and I think it is a good idea to add checks in supernode api

@starnop
Copy link
Contributor

starnop commented Jul 31, 2019

LGTM.

@starnop starnop merged commit c86ccf5 into dragonflyoss:master Jul 31, 2019
starnop added a commit to starnop/Dragonfly that referenced this pull request Nov 27, 2019
inoc603 pushed a commit to inoc603/Dragonfly that referenced this pull request Dec 23, 2019
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
kind/bug This is bug report for project size/M
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[bug]dfget's supernode API return err
6 participants