Skip to content

Commit

Permalink
Ensure C# autogen bindings are up-to-date (#1780)
Browse files Browse the repository at this point in the history
  • Loading branch information
@RReverser
RReverser authored Oct 1, 2024
1 parent a18ef7b commit 3c9952a
Showing 1 changed file with 5 additions and 0 deletions.
5 changes: 5 additions & 0 deletions .github/workflows/ci.yml
View file
Original file line number Diff line number Diff line change
Expand Up @@ -80,6 +80,11 @@ jobs:
- name: Run cargo test
run: cargo test --all

- name: Ensure C# autogen bindings are up-to-date
run: |
cargo run --example regen-csharp-moduledef
git diff --exit-code -- crates/bindings-csharp
- name: C# bindings tests
working-directory: crates/bindings-csharp
run: dotnet test -warnaserror
Expand Down

2 comments on commit 3c9952a

@github-actions
Copy link

@github-actions github-actions bot commented on 3c9952a Oct 1, 2024 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Criterion benchmark results

Criterion benchmark report

YOU SHOULD PROBABLY IGNORE THESE RESULTS.

Criterion is a wall time based benchmarking system that is extremely noisy when run on CI. We collect these results for longitudinal analysis, but they are not reliable for comparing individual PRs.

Go look at the callgrind report instead.

empty

db on disk new latency old latency new throughput old throughput
sqlite 💿 412.6±1.87ns 416.4±4.43ns - -
sqlite 🧠 410.7±3.80ns 407.8±4.10ns - -
stdb_raw 💿 624.5±1.19ns 628.6±1.50ns - -
stdb_raw 🧠 624.8±0.80ns 629.5±1.58ns - -

insert_1

db on disk schema indices preload new latency old latency new throughput old throughput

insert_bulk

db on disk schema indices preload count new latency old latency new throughput old throughput
sqlite 💿 u32_u64_str btree_each_column 2048 256 591.8±32.64µs 585.0±0.99µs 1689 tx/sec 1709 tx/sec
sqlite 💿 u32_u64_str unique_0 2048 256 151.3±0.41µs 149.4±0.49µs 6.5 Ktx/sec 6.5 Ktx/sec
sqlite 💿 u32_u64_u64 btree_each_column 2048 256 467.1±0.57µs 468.3±0.77µs 2.1 Ktx/sec 2.1 Ktx/sec
sqlite 💿 u32_u64_u64 unique_0 2048 256 135.9±0.32µs 134.8±0.65µs 7.2 Ktx/sec 7.2 Ktx/sec
sqlite 🧠 u32_u64_str btree_each_column 2048 256 445.3±1.54µs 447.1±1.07µs 2.2 Ktx/sec 2.2 Ktx/sec
sqlite 🧠 u32_u64_str unique_0 2048 256 122.8±0.56µs 122.6±0.85µs 8.0 Ktx/sec 8.0 Ktx/sec
sqlite 🧠 u32_u64_u64 btree_each_column 2048 256 362.6±0.30µs 365.5±0.43µs 2.7 Ktx/sec 2.7 Ktx/sec
sqlite 🧠 u32_u64_u64 unique_0 2048 256 104.3±0.42µs 102.0±0.48µs 9.4 Ktx/sec 9.6 Ktx/sec
stdb_raw 💿 u32_u64_str btree_each_column 2048 256 601.6±19.55µs 530.2±61.81µs 1662 tx/sec 1885 tx/sec
stdb_raw 💿 u32_u64_str unique_0 2048 256 491.0±27.77µs 494.5±24.47µs 2036 tx/sec 2022 tx/sec
stdb_raw 💿 u32_u64_u64 btree_each_column 2048 256 378.0±10.04µs 351.5±28.04µs 2.6 Ktx/sec 2.8 Ktx/sec
stdb_raw 💿 u32_u64_u64 unique_0 2048 256 351.8±6.14µs 322.9±29.99µs 2.8 Ktx/sec 3.0 Ktx/sec
stdb_raw 🧠 u32_u64_str btree_each_column 2048 256 307.1±0.48µs 306.1±0.29µs 3.2 Ktx/sec 3.2 Ktx/sec
stdb_raw 🧠 u32_u64_str unique_0 2048 256 237.2±0.18µs 235.4±0.39µs 4.1 Ktx/sec 4.1 Ktx/sec
stdb_raw 🧠 u32_u64_u64 btree_each_column 2048 256 246.0±0.13µs 243.3±0.23µs 4.0 Ktx/sec 4.0 Ktx/sec
stdb_raw 🧠 u32_u64_u64 unique_0 2048 256 217.6±0.14µs 216.4±0.06µs 4.5 Ktx/sec 4.5 Ktx/sec

iterate

db on disk schema indices new latency old latency new throughput old throughput
sqlite 💿 u32_u64_str unique_0 22.6±0.19µs 22.1±0.08µs 43.1 Ktx/sec 44.2 Ktx/sec
sqlite 💿 u32_u64_u64 unique_0 20.9±0.11µs 21.1±0.16µs 46.7 Ktx/sec 46.3 Ktx/sec
sqlite 🧠 u32_u64_str unique_0 19.7±0.15µs 19.6±0.34µs 49.7 Ktx/sec 49.8 Ktx/sec
sqlite 🧠 u32_u64_u64 unique_0 18.6±0.06µs 18.5±0.18µs 52.5 Ktx/sec 52.7 Ktx/sec
stdb_raw 💿 u32_u64_str unique_0 4.7±0.00µs 4.7±0.00µs 206.3 Ktx/sec 206.2 Ktx/sec
stdb_raw 💿 u32_u64_u64 unique_0 4.6±0.00µs 4.6±0.00µs 211.2 Ktx/sec 210.8 Ktx/sec
stdb_raw 🧠 u32_u64_str unique_0 4.7±0.00µs 4.7±0.00µs 206.4 Ktx/sec 206.1 Ktx/sec
stdb_raw 🧠 u32_u64_u64 unique_0 4.6±0.00µs 4.6±0.00µs 211.2 Ktx/sec 210.8 Ktx/sec

find_unique

db on disk key type preload new latency old latency new throughput old throughput

filter

db on disk key type index strategy load count new latency old latency new throughput old throughput
sqlite 💿 string index 2048 256 66.2±0.22µs 67.1±0.39µs 14.7 Ktx/sec 14.6 Ktx/sec
sqlite 💿 u64 index 2048 256 64.3±0.20µs 65.7±0.17µs 15.2 Ktx/sec 14.9 Ktx/sec
sqlite 🧠 string index 2048 256 63.2±0.12µs 63.1±0.10µs 15.4 Ktx/sec 15.5 Ktx/sec
sqlite 🧠 u64 index 2048 256 58.4±0.12µs 58.8±0.27µs 16.7 Ktx/sec 16.6 Ktx/sec
stdb_raw 💿 string index 2048 256 4.9±0.00µs 4.8±0.00µs 198.6 Ktx/sec 202.0 Ktx/sec
stdb_raw 💿 u64 index 2048 256 4.7±0.00µs 4.8±0.00µs 208.2 Ktx/sec 202.1 Ktx/sec
stdb_raw 🧠 string index 2048 256 4.9±0.00µs 4.8±0.00µs 198.8 Ktx/sec 201.9 Ktx/sec
stdb_raw 🧠 u64 index 2048 256 4.7±0.00µs 4.8±0.00µs 208.1 Ktx/sec 202.2 Ktx/sec

serialize

schema format count new latency old latency new throughput old throughput
u32_u64_str bflatn_to_bsatn_fast_path 100 3.3±0.01µs 3.3±0.08µs 28.9 Mtx/sec 28.8 Mtx/sec
u32_u64_str bflatn_to_bsatn_slow_path 100 3.0±0.01µs 3.0±0.01µs 31.5 Mtx/sec 31.7 Mtx/sec
u32_u64_str bsatn 100 2.4±0.08µs 2.3±0.01µs 39.3 Mtx/sec 40.9 Mtx/sec
u32_u64_str bsatn 100 39.9±0.09ns 39.9±0.08ns 2.3 Gtx/sec 2.3 Gtx/sec
u32_u64_str json 100 5.7±0.05µs 5.8±0.11µs 16.8 Mtx/sec 16.5 Mtx/sec
u32_u64_str json 100 8.4±0.07µs 8.4±0.01µs 11.4 Mtx/sec 11.4 Mtx/sec
u32_u64_str product_value 100 1018.2±0.76ns 1023.5±0.73ns 93.7 Mtx/sec 93.2 Mtx/sec
u32_u64_u64 bflatn_to_bsatn_fast_path 100 1169.1±31.32ns 1119.9±4.86ns 81.6 Mtx/sec 85.2 Mtx/sec
u32_u64_u64 bflatn_to_bsatn_slow_path 100 2.5±0.00µs 2.5±0.00µs 38.0 Mtx/sec 38.1 Mtx/sec
u32_u64_u64 bsatn 100 1699.9±35.23ns 1706.3±34.62ns 56.1 Mtx/sec 55.9 Mtx/sec
u32_u64_u64 bsatn 100 39.8±0.07ns 29.5±0.03ns 2.3 Gtx/sec 3.2 Gtx/sec
u32_u64_u64 json 100 3.3±0.03µs 3.2±0.03µs 29.1 Mtx/sec 30.1 Mtx/sec
u32_u64_u64 json 100 4.9±0.00µs 4.9±0.03µs 19.5 Mtx/sec 19.4 Mtx/sec
u32_u64_u64 product_value 100 1015.5±1.32ns 1017.4±1.80ns 93.9 Mtx/sec 93.7 Mtx/sec
u64_u64_u32 bflatn_to_bsatn_fast_path 100 892.5±2.07ns 898.0±7.18ns 106.8 Mtx/sec 106.2 Mtx/sec
u64_u64_u32 bflatn_to_bsatn_slow_path 100 2.5±0.01µs 2.5±0.01µs 38.0 Mtx/sec 38.1 Mtx/sec
u64_u64_u32 bsatn 100 1734.4±36.77ns 1745.8±29.01ns 55.0 Mtx/sec 54.6 Mtx/sec
u64_u64_u32 bsatn 100 750.4±0.38ns 749.4±0.27ns 127.1 Mtx/sec 127.3 Mtx/sec
u64_u64_u32 json 100 3.4±0.03µs 3.2±0.03µs 28.3 Mtx/sec 29.9 Mtx/sec
u64_u64_u32 json 100 5.0±0.03µs 5.0±0.02µs 19.1 Mtx/sec 19.1 Mtx/sec
u64_u64_u32 product_value 100 1015.3±0.48ns 1021.0±0.87ns 93.9 Mtx/sec 93.4 Mtx/sec

stdb_module_large_arguments

arg size new latency old latency new throughput old throughput
64KiB 112.5±12.61µs 113.8±7.74µs - -

stdb_module_print_bulk

line count new latency old latency new throughput old throughput
1 56.5±4.80µs 56.1±5.29µs - -
100 597.5±6.30µs 595.6±6.17µs - -
1000 3.7±0.64ms 4.0±0.88ms - -

remaining

name new latency old latency new throughput old throughput
special/db_game/circles/load=10 296.3±3.91µs 293.1±3.05µs - -
special/db_game/circles/load=100 294.2±3.31µs 291.9±2.91µs - -
special/db_game/ia_loop/load=10 0.0±0.00ns 0.0±0.00ns - -
special/db_game/ia_loop/load=100 0.0±0.00ns 0.0±0.00ns - -
sqlite/💿/update_bulk/u32_u64_str/unique_0/load=2048/count=256 54.9±0.17µs 54.8±0.23µs 17.8 Ktx/sec 17.8 Ktx/sec
sqlite/💿/update_bulk/u32_u64_u64/unique_0/load=2048/count=256 47.2±0.26µs 46.5±0.10µs 20.7 Ktx/sec 21.0 Ktx/sec
sqlite/🧠/update_bulk/u32_u64_str/unique_0/load=2048/count=256 40.2±0.08µs 40.1±0.25µs 24.3 Ktx/sec 24.3 Ktx/sec
sqlite/🧠/update_bulk/u32_u64_u64/unique_0/load=2048/count=256 36.7±0.09µs 36.8±0.11µs 26.6 Ktx/sec 26.6 Ktx/sec
stdb_module/💿/update_bulk/u32_u64_str/unique_0/load=2048/count=256 1292.9±8.86µs 1268.2±13.63µs 773 tx/sec 788 tx/sec
stdb_module/💿/update_bulk/u32_u64_u64/unique_0/load=2048/count=256 1047.1±6.67µs 1031.4±5.74µs 955 tx/sec 969 tx/sec
stdb_raw/💿/update_bulk/u32_u64_str/unique_0/load=2048/count=256 625.7±46.80µs 634.8±22.81µs 1598 tx/sec 1575 tx/sec
stdb_raw/💿/update_bulk/u32_u64_u64/unique_0/load=2048/count=256 495.0±9.45µs 494.8±7.45µs 2020 tx/sec 2020 tx/sec
stdb_raw/🧠/update_bulk/u32_u64_str/unique_0/load=2048/count=256 368.9±0.37µs 371.3±0.24µs 2.6 Ktx/sec 2.6 Ktx/sec
stdb_raw/🧠/update_bulk/u32_u64_u64/unique_0/load=2048/count=256 335.8±0.24µs 335.6±0.40µs 2.9 Ktx/sec 2.9 Ktx/sec

@github-actions
Copy link

@github-actions github-actions bot commented on 3c9952a Oct 1, 2024 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Callgrind benchmark results

Callgrind Benchmark Report

These benchmarks were run using callgrind,
an instruction-level profiler. They allow comparisons between sqlite (sqlite), SpacetimeDB running through a module (stdb_module), and the underlying SpacetimeDB data storage engine (stdb_raw). Callgrind emulates a CPU to collect the below estimates.

Measurement changes larger than five percent are in bold.

In-memory benchmarks

callgrind: empty transaction

db total reads + writes old total reads + writes Δrw estimated cycles old estimated cycles Δcycles
stdb_raw 5408 5408 0.00% 5458 5458 0.00%
sqlite 5509 5509 0.00% 5959 5959 0.00%

callgrind: filter

db schema indices count preload _column data_type total reads + writes old total reads + writes Δrw estimated cycles old estimated cycles Δcycles
stdb_raw u32_u64_str no_index 64 128 2 string 117812 117812 0.00% 118376 118380 -0.00%
stdb_raw u32_u64_str no_index 64 128 1 u64 75401 75401 0.00% 75839 75843 -0.01%
stdb_raw u32_u64_str btree_each_column 64 128 2 string 24068 24084 -0.07% 24540 24548 -0.03%
stdb_raw u32_u64_str btree_each_column 64 128 1 u64 23036 23036 0.00% 23430 23422 0.03%
sqlite u32_u64_str no_index 64 128 2 string 144619 144619 0.00% 145967 145967 0.00%
sqlite u32_u64_str no_index 64 128 1 u64 124017 124023 -0.00% 125245 125251 -0.00%
sqlite u32_u64_str btree_each_column 64 128 2 string 134418 134418 0.00% 136018 136022 -0.00%
sqlite u32_u64_str btree_each_column 64 128 1 u64 131334 131334 0.00% 132676 132664 0.01%

callgrind: insert bulk

db schema indices count preload total reads + writes old total reads + writes Δrw estimated cycles old estimated cycles Δcycles
stdb_raw u32_u64_str unique_0 64 128 900871 899082 0.20% 974127 972210 0.20%
stdb_raw u32_u64_str btree_each_column 64 128 1049187 1052564 -0.32% 1135695 1139578 -0.34%
sqlite u32_u64_str unique_0 64 128 398156 398156 0.00% 415492 415492 0.00%
sqlite u32_u64_str btree_each_column 64 128 983483 983483 0.00% 1025293 1025297 -0.00%

callgrind: iterate

db schema indices count total reads + writes old total reads + writes Δrw estimated cycles old estimated cycles Δcycles
stdb_raw u32_u64_str unique_0 1024 152694 152694 0.00% 152772 152772 0.00%
stdb_raw u32_u64_str unique_0 64 15719 15719 0.00% 15789 15789 0.00%
sqlite u32_u64_str unique_0 1024 1046659 1046659 0.00% 1050135 1050135 0.00%
sqlite u32_u64_str unique_0 64 74799 74799 0.00% 75843 75843 0.00%

callgrind: serialize_product_value

count format total reads + writes old total reads + writes Δrw estimated cycles old estimated cycles Δcycles
64 json 47374 47374 0.00% 49890 49890 0.00%
64 bsatn 25716 25716 0.00% 27960 27960 0.00%
16 json 12126 12126 0.00% 13860 13860 0.00%
16 bsatn 8117 8117 0.00% 9477 9477 0.00%

callgrind: update bulk

db schema indices count preload total reads + writes old total reads + writes Δrw estimated cycles old estimated cycles Δcycles
stdb_raw u32_u64_str unique_0 1024 1024 20666428 20665642 0.00% 21496554 21495592 0.00%
stdb_raw u32_u64_str unique_0 64 128 1304407 1304463 -0.00% 1361953 1362085 -0.01%
sqlite u32_u64_str unique_0 1024 1024 1802101 1802101 0.00% 1811463 1811463 0.00%
sqlite u32_u64_str unique_0 64 128 128447 128447 0.00% 131187 131187 0.00%
On-disk benchmarks

callgrind: empty transaction

db total reads + writes old total reads + writes Δrw estimated cycles old estimated cycles Δcycles
stdb_raw 5418 5418 0.00% 5472 5468 0.07%
sqlite 5551 5551 0.00% 6065 6065 0.00%

callgrind: filter

db schema indices count preload _column data_type total reads + writes old total reads + writes Δrw estimated cycles old estimated cycles Δcycles
stdb_raw u32_u64_str no_index 64 128 2 string 117822 117822 0.00% 118378 118330 0.04%
stdb_raw u32_u64_str no_index 64 128 1 u64 75411 75411 0.00% 75801 75801 0.00%
stdb_raw u32_u64_str btree_each_column 64 128 2 string 24079 24079 0.00% 24511 24511 0.00%
stdb_raw u32_u64_str btree_each_column 64 128 1 u64 23046 23046 0.00% 23424 23400 0.10%
sqlite u32_u64_str no_index 64 128 2 string 146540 146540 0.00% 148184 148184 0.00%
sqlite u32_u64_str no_index 64 128 1 u64 125928 125928 0.00% 127434 127438 -0.00%
sqlite u32_u64_str btree_each_column 64 128 2 string 136540 136540 0.00% 138550 138550 0.00%
sqlite u32_u64_str btree_each_column 64 128 1 u64 133420 133420 0.00% 135248 135244 0.00%

callgrind: insert bulk

db schema indices count preload total reads + writes old total reads + writes Δrw estimated cycles old estimated cycles Δcycles
stdb_raw u32_u64_str unique_0 64 128 849232 849610 -0.04% 922702 922272 0.05%
stdb_raw u32_u64_str btree_each_column 64 128 998476 1000099 -0.16% 1055278 1087215 -2.94%
sqlite u32_u64_str unique_0 64 128 415693 415693 0.00% 432423 432439 -0.00%
sqlite u32_u64_str btree_each_column 64 128 1021734 1021734 0.00% 1063938 1063930 0.00%

callgrind: iterate

db schema indices count total reads + writes old total reads + writes Δrw estimated cycles old estimated cycles Δcycles
stdb_raw u32_u64_str unique_0 1024 152704 152704 0.00% 152774 152774 0.00%
stdb_raw u32_u64_str unique_0 64 15729 15729 0.00% 15799 15799 0.00%
sqlite u32_u64_str unique_0 1024 1049721 1049721 0.00% 1053497 1053497 0.00%
sqlite u32_u64_str unique_0 64 76571 76571 0.00% 77819 77819 0.00%

callgrind: serialize_product_value

count format total reads + writes old total reads + writes Δrw estimated cycles old estimated cycles Δcycles
64 json 47374 47374 0.00% 49890 49890 0.00%
64 bsatn 25716 25716 0.00% 27960 27960 0.00%
16 json 12126 12126 0.00% 13860 13860 0.00%
16 bsatn 8117 8117 0.00% 9477 9477 0.00%

callgrind: update bulk

db schema indices count preload total reads + writes old total reads + writes Δrw estimated cycles old estimated cycles Δcycles
stdb_raw u32_u64_str unique_0 1024 1024 19370075 19362970 0.04% 20278387 20272968 0.03%
stdb_raw u32_u64_str unique_0 64 128 1257337 1257490 -0.01% 1345081 1344956 0.01%
sqlite u32_u64_str unique_0 1024 1024 1809652 1809652 0.00% 1818162 1818162 0.00%
sqlite u32_u64_str unique_0 64 128 132563 132563 0.00% 135419 135419 0.00%

Please sign in to comment.