Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

share/befp: add benchmarks for fraud proof creation and validation #2400

Merged
merged 7 commits into from
Jul 6, 2023
Merged

share/befp: add benchmarks for fraud proof creation and validation #2400

merged 7 commits into from
Jul 6, 2023

Conversation

vgonkivs
Copy link
Member

@vgonkivs vgonkivs commented Jun 26, 2023 •
edited
Loading

Overview

Add benchmarks for BEFP creation and validation
BEFP was created for the first row.

Checklist

  • New and updated code has appropriate documentation
  • New and updated code has new and/or updated testing
  • Required CI checks are passing
  • Visual proof for any user facing features like CLI or documentation updates
  • Linked issues closed with keywords

@vgonkivs vgonkivs added area:fraud kind:testing Related to unit tests labels Jun 26, 2023
@vgonkivs vgonkivs requested a review from renaynay as a code owner June 26, 2023 12:17
@vgonkivs vgonkivs self-assigned this Jun 26, 2023
@codecov-commenter
Copy link

codecov-commenter commented Jun 26, 2023 •
edited
Loading

Codecov Report

Merging #2400 (41327fc) into main (1a37094) will increase coverage by 0.14%.
The diff coverage is 0.00%.

@@            Coverage Diff             @@
##             main    #2400      +/-   ##
==========================================
+ Coverage   52.93%   53.08%   +0.14%     
==========================================
  Files         156      156              
  Lines        9915     9915              
==========================================
+ Hits         5249     5263      +14     
+ Misses       4206     4198       -8     
+ Partials      460      454       -6
Impacted Files Coverage Δ
header/headertest/testing.go 72.48% <0.00%> (ø)

... and 6 files with indirect coverage changes

distractedm1nd
distractedm1nd previously approved these changes Jun 26, 2023
View reviewed changes
@vgonkivs
Copy link
Member Author

vgonkivs commented Jun 26, 2023 •
edited
Loading

Top memory consuming operations(odsSize=32)

Type: alloc_space
Showing nodes accounting for 969.56MB, 64.67% of 1499.16MB total
Dropped 122 nodes (cum <= 7.50MB)
Showing top 10 nodes out of 110
      flat  flat%   sum%        cum   cum%
  172.53MB 11.51% 11.51%   172.53MB 11.51%  github.com/ipfs/go-ipfs-ds-help.NewKeyFromBinary
  151.02MB 10.07% 21.58%   151.02MB 10.07%  github.com/ipfs/go-datastore.Key.Child
  139.51MB  9.31% 30.89%   140.01MB  9.34%  github.com/klauspost/reedsolomon.(*leopardFF8).encode
  117.56MB  7.84% 38.73%   117.56MB  7.84%  github.com/celestiaorg/nmt.(*Hasher).HashLeaf
  106.56MB  7.11% 45.84%   241.62MB 16.12%  github.com/celestiaorg/celestia-app/pkg/wrapper.(*ErasuredNamespacedMerkleTree).Push
   77.51MB  5.17% 51.01%    77.51MB  5.17%  github.com/ipfs/go-cid.Cid.Bytes
   60.01MB  4.00% 55.01%   165.51MB 11.04%  github.com/celestiaorg/nmt.(*Hasher).HashNode
   50.51MB  3.37% 58.38%    50.51MB  3.37%  strings.(*Builder).grow
      48MB  3.20% 61.58%       48MB  3.20%  github.com/celestiaorg/nmt.MaxNamespace (inline)
   46.36MB  3.09% 64.67%    46.36MB  3.09%  github.com/ipfs/go-datastore.(*MapDatastore).Put

mem32.pdf
cpu32.pdf

@vgonkivs
Copy link
Member Author

vgonkivs commented Jun 26, 2023 •
edited
Loading

Top memory consuming operations(odsSize=64)

Type: alloc_space
Showing nodes accounting for 1810.65MB, 64.21% of 2819.99MB total
Dropped 119 nodes (cum <= 14.10MB)
Showing top 10 nodes out of 105
      flat  flat%   sum%        cum   cum%
  370.56MB 13.14% 13.14%   370.56MB 13.14%  github.com/ipfs/go-ipfs-ds-help.NewKeyFromBinary
  251.54MB  8.92% 22.06%   251.54MB  8.92%  github.com/ipfs/go-datastore.Key.Child
  232.61MB  8.25% 30.31%   232.61MB  8.25%  github.com/celestiaorg/nmt.(*Hasher).HashLeaf
  209.11MB  7.42% 37.72%   470.78MB 16.69%  github.com/celestiaorg/celestia-app/pkg/wrapper.(*ErasuredNamespacedMerkleTree).Push
  176.24MB  6.25% 43.97%   176.74MB  6.27%  github.com/klauspost/reedsolomon.(*leopardFF8).encode
  168.52MB  5.98% 49.95%   168.52MB  5.98%  github.com/ipfs/go-cid.Cid.Bytes
  117.52MB  4.17% 54.12%   339.53MB 12.04%  github.com/celestiaorg/nmt.(*Hasher).HashNode
   99.49MB  3.53% 57.65%    99.49MB  3.53%  github.com/ipfs/go-datastore.(*MapDatastore).Put
   95.06MB  3.37% 61.02%    95.06MB  3.37%  github.com/celestiaorg/rsmt2d.deepCopy
      90MB  3.19% 64.21%       90MB  3.19%  github.com/celestiaorg/nmt.MinNamespace (inline)

mem64.pdf
cpu64.pdf

@vgonkivs
Copy link
Member Author

Top memory consuming operations(odsSize=128)

Type: alloc_space
Showing nodes accounting for 2071.79MB, 64.22% of 3226.29MB total
Dropped 108 nodes (cum <= 16.13MB)
Showing top 10 nodes out of 99
      flat  flat%   sum%        cum   cum%
  394.56MB 12.23% 12.23%   394.56MB 12.23%  github.com/ipfs/go-ipfs-ds-help.NewKeyFromBinary
  340.67MB 10.56% 22.79%   340.67MB 10.56%  github.com/celestiaorg/nmt.(*Hasher).HashLeaf
  287.54MB  8.91% 31.70%   287.54MB  8.91%  github.com/ipfs/go-datastore.Key.Child
  276.15MB  8.56% 40.26%   682.54MB 21.16%  github.com/celestiaorg/celestia-app/pkg/wrapper.(*ErasuredNamespacedMerkleTree).Push
  169.02MB  5.24% 45.50%   169.02MB  5.24%  github.com/ipfs/go-cid.Cid.Bytes
  157.52MB  4.88% 50.38%   461.04MB 14.29%  github.com/celestiaorg/nmt.(*Hasher).HashNode
  122.50MB  3.80% 54.18%   122.50MB  3.80%  github.com/celestiaorg/nmt.MinNamespace (inline)
     116MB  3.60% 57.77%      116MB  3.60%  github.com/celestiaorg/nmt.MaxNamespace (inline)
  107.07MB  3.32% 61.09%   107.07MB  3.32%  github.com/celestiaorg/rsmt2d.deepCopy
  100.75MB  3.12% 64.22%   100.75MB  3.12%  github.com/klauspost/reedsolomon.(*leopardFF8).encode

mem128.pdf
cpu128.pdf

renaynay
renaynay reviewed Jul 3, 2023
View reviewed changes
header/headertest/testing.go Outdated Show resolved Hide resolved
share/eds/retriever_test.go Outdated Show resolved Hide resolved
share/eds/retriever_test.go Outdated Show resolved Hide resolved
@vgonkivs
Copy link
Member Author

vgonkivs commented Jul 3, 2023

New numbers after a small rework:

goos: darwin
goarch: amd64
pkg: github.com/celestiaorg/celestia-node/share/eds
cpu: Intel(R) Core(TM) i7-9750H CPU @ 2.60GHz
BenchmarkBEFP
BenchmarkBEFP/ods_size:2
BenchmarkBEFP/ods_size:2-12         	     554	   2079172 ns/op	  810339 B/op	    5721 allocs/op
BenchmarkBEFP/ods_size:4
BenchmarkBEFP/ods_size:4-12         	     390	   3090171 ns/op	 2489607 B/op	   17741 allocs/op
BenchmarkBEFP/ods_size:16
BenchmarkBEFP/ods_size:16-12        	      75	  17233645 ns/op	26165536 B/op	  202487 allocs/op
BenchmarkBEFP/ods_size:32
BenchmarkBEFP/ods_size:32-12        	      24	  63073970 ns/op	96808118 B/op	  763386 allocs/op
BenchmarkBEFP/ods_size:64
BenchmarkBEFP/ods_size:64-12        	       7	 192229117 ns/op	343801670 B/op	 2923090 allocs/op
BenchmarkBEFP/ods_size:128
BenchmarkBEFP/ods_size:128-12       	       1	1023359137 ns/op	1502869104 B/op	13061865 allocs/op

@vgonkivs
Copy link
Member Author

vgonkivs commented Jul 4, 2023

results after small rework:

goos: darwin
goarch: amd64
pkg: github.com/celestiaorg/celestia-node/share/eds
cpu: Intel(R) Core(TM) i7-9750H CPU @ 2.60GHz
BenchmarkBEFP
BenchmarkBEFP/ods_size:2
BenchmarkBEFP/ods_size:2-12         	    7406	    161065 ns/op	  216139 B/op	    1508 allocs/op
BenchmarkBEFP/ods_size:4
BenchmarkBEFP/ods_size:4-12         	    3030	    364199 ns/op	  526127 B/op	    4237 allocs/op
BenchmarkBEFP/ods_size:16
BenchmarkBEFP/ods_size:16-12        	     367	   2784798 ns/op	 4114565 B/op	   38679 allocs/op
BenchmarkBEFP/ods_size:32
BenchmarkBEFP/ods_size:32-12        	     148	   9151226 ns/op	13657661 B/op	  132728 allocs/op
BenchmarkBEFP/ods_size:64
BenchmarkBEFP/ods_size:64-12        	      50	  23106186 ns/op	38860387 B/op	  391383 allocs/op
BenchmarkBEFP/ods_size:128
BenchmarkBEFP/ods_size:128-12       	      20	 117521981 ns/op	164281557 B/op	 1649290 allocs/op

Wondertan
Wondertan previously approved these changes Jul 4, 2023
View reviewed changes
Copy link
Member

@Wondertan Wondertan left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The good practise it to comment the benchmark results in the comment about it. LGTM other than that.

renaynay
renaynay previously approved these changes Jul 5, 2023
View reviewed changes
walldiss
walldiss requested changes Jul 5, 2023
View reviewed changes
share/eds/retriever_test.go Outdated Show resolved Hide resolved
share/eds/retriever_test.go Show resolved Hide resolved
@vgonkivs vgonkivs dismissed stale reviews from renaynay and Wondertan via 0c171be July 5, 2023 10:24
@vgonkivs vgonkivs enabled auto-merge July 6, 2023 08:21
@vgonkivs vgonkivs disabled auto-merge July 6, 2023 08:45
@vgonkivs vgonkivs enabled auto-merge July 6, 2023 08:46
@vgonkivs vgonkivs added this pull request to the merge queue Jul 6, 2023
Merged via the queue into celestiaorg:main with commit 387c160 Jul 6, 2023
@vgonkivs vgonkivs deleted the bencmarks_for_the_BEFP branch July 6, 2023 10:12
@faddat faddat mentioned this pull request Feb 22, 2024
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@walldiss walldiss walldiss approved these changes

@Wondertan Wondertan Wondertan approved these changes

@distractedm1nd distractedm1nd Awaiting requested review from distractedm1nd

@renaynay renaynay Awaiting requested review from renaynay renaynay is a code owner

Assignees

@vgonkivs vgonkivs

Labels
area:fraud kind:testing Related to unit tests
Projects
No open projects
Celestia Node
Status: Done
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
6 participants
@vgonkivs @codecov-commenter @walldiss @distractedm1nd @Wondertan @renaynay