Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add Close, Fanout and RollbackAndForward to our MBT #1296

Merged
merged 23 commits into from
Feb 12, 2024
Merged

Conversation

v0d1ch
Copy link
Contributor

@v0d1ch v0d1ch commented Feb 8, 2024

In order to improve our MBT Close, Fanout and RollbackAndForward are added as the actions. Now the MockChain is only rolling forward and only rolls back when appropriate action is produced.

There is also new test case fanout contains whole confirmed UTxO that should further assure us that the Fanout UTxO matches with whatever we transacted in the open Head.

NOTE:
Currently we are not able to compare equality between Model representation of the UTxO ([(CardanoSigningKey, Value)]) with the real UTxO . Even if the value and the address matches between the two the input Ix we generate when converting starts always from zero and we can't just sort TxOuts and compare that since there is no Ord instance.

This is related to #1260 as it will help reproduce this bug and preparatory work for #1057


  • CHANGELOG updated or not needed
  • Documentation updated or not needed
  • Haddocks updated or not needed
  • No new TODOs introduced or explained herafter

@v0d1ch v0d1ch requested a review from a team February 8, 2024 10:17
Copy link

github-actions bot commented Feb 8, 2024

Transactions Costs

Sizes and execution budgets for Hydra protocol transactions. Note that unlisted parameters are currently using arbitrary values and results are not fully deterministic and comparable to previous runs.

Metadata
Generated at 2024-02-12 12:44:06.102001557 UTC
Max. memory units 14000000
Max. CPU units 10000000000
Max. tx size (kB) 16384

Script summary

Name Hash Size (Bytes)
νInitial 985245919fcc6c0c5cd116023cd2c947c43e80dcbb5075fe12433fbb 4072
νCommit 7cb20fa71eb4c563ca283566ebe0aa65859d96c3f8cba35c52c181fd 2043
νHead 7a36661f5c15e9f1783aeaab890812c59b7286cbbc6de762d3110772 8816
μHead 8b111ac12274e46314769295a1c5dcab1d260096fc469fd698065463* 3851
  • The minting policy hash is only usable for comparison. As the script is parameterized, the actual script is unique per Head.

Cost of Init Transaction

Parties Tx size % max Mem % max CPU Min fee ₳
1 4374 10.48 4.05 0.46
2 4575 12.66 4.87 0.49
3 4774 14.68 5.63 0.52
5 5177 19.22 7.35 0.59
10 6185 30.54 11.65 0.75
41 12417 99.06 37.61 1.77

Cost of Commit Transaction

This is using ada-only outputs for better comparability.

UTxO Tx size % max Mem % max CPU Min fee ₳
1 531 11.37 4.44 0.30
2 723 15.04 6.07 0.35
3 912 18.85 7.75 0.40
5 1282 26.90 11.27 0.51
10 2209 49.55 20.97 0.80
19 3910 99.43 41.75 1.43

Cost of CollectCom Transaction

Parties UTxO (bytes) Tx size % max Mem % max CPU Min fee ₳
1 56 479 21.12 8.27 0.40
2 114 590 34.16 13.45 0.55
3 169 704 46.24 18.40 0.69
4 226 814 63.49 25.35 0.88
5 284 920 78.15 31.46 1.05

Cost of Close Transaction

Parties Tx size % max Mem % max CPU Min fee ₳
1 528 16.68 7.67 0.36
2 693 18.00 8.97 0.39
3 872 19.90 10.65 0.43
5 1171 23.03 13.55 0.49
10 2013 30.73 20.92 0.64
50 7837 89.94 76.45 1.77

Cost of Contest Transaction

Parties Tx size % max Mem % max CPU Min fee ₳
1 598 20.27 8.99 0.41
2 758 22.26 10.69 0.44
3 834 23.34 11.67 0.46
5 1191 27.47 15.15 0.53
10 1883 35.74 22.31 0.68
47 7752 99.48 78.18 1.86

Cost of Abort Transaction

Some variation because of random mixture of still initial and already committed outputs.

Parties Tx size % max Mem % max CPU Min fee ₳
1 4222 15.19 6.26 0.50
2 4376 27.20 11.61 0.65
3 4511 41.15 17.75 0.81
4 4705 63.56 27.84 1.07
5 4882 84.12 37.00 1.31
6 4835 96.53 42.22 1.45

Cost of FanOut Transaction

Involves spending head output and burning head tokens. Uses ada-only UTxO for better comparability.

Parties UTxO UTxO (bytes) Tx size % max Mem % max CPU Min fee ₳
5 0 0 4206 7.84 3.28 0.42
5 1 57 4239 8.86 3.96 0.44
5 5 285 4376 13.98 7.09 0.51
5 10 569 4544 19.92 10.81 0.59
5 20 1136 4881 32.01 18.35 0.76
5 30 1707 5224 44.52 26.07 0.93
5 40 2277 5564 57.04 33.79 1.10
5 50 2843 5901 69.35 41.43 1.27
5 74 4211 6717 98.71 59.69 1.67

End-To-End Benchmark Results

This page is intended to collect the latest end-to-end benchmarks results produced by Hydra's Continuous Integration system from the latest master code.

Please take those results with a grain of salt as they are currently produced from very limited cloud VMs and not controlled hardware. Instead of focusing on the absolute results, the emphasis should be on relative results, eg. how the timings for a scenario evolve as the code changes.

Generated at 2024-02-12 12:47:49.03681458 UTC

Baseline Scenario

Number of nodes 3
Number of txs 9000
Avg. Confirmation Time (ms) 22.790790930
P99 54.693139060000085ms
P95 31.839321199999983ms
P50 20.5493885ms
Number of Invalid txs 0

Baseline Scenario

Number of nodes 1
Number of txs 3000
Avg. Confirmation Time (ms) 5.014760475
P99 9.479096359999986ms
P95 7.1884885999999995ms
P50 4.6688235ms
Number of Invalid txs 0

Copy link

github-actions bot commented Feb 8, 2024

Test Results

411 tests  +2   403 ✅ +1   12m 45s ⏱️ +19s
138 suites ±0     8 💤 +1 
  5 files   ±0     0 ❌ ±0 

Results for commit 88fee05. ± Comparison against base commit efb8d11.

♻️ This comment has been updated with latest results.

hydra-node/test/Hydra/Model.hs Outdated Show resolved Hide resolved
hydra-node/test/Hydra/Model.hs Outdated Show resolved Hide resolved
hydra-node/test/Hydra/Model.hs Outdated Show resolved Hide resolved
hydra-node/test/Hydra/Model.hs Show resolved Hide resolved
hydra-node/test/Hydra/Model.hs Outdated Show resolved Hide resolved
hydra-node/test/Hydra/ModelSpec.hs Show resolved Hide resolved
hydra-node/test/Hydra/ModelSpec.hs Outdated Show resolved Hide resolved
@v0d1ch v0d1ch self-assigned this Feb 8, 2024
Copy link
Member

@ch1bo ch1bo left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good if it works, but a bit surprising that we don't need to wait for ReadyToFanout?

Also consider my "Should" suggestions, please.

hydra-node/src/Hydra/Chain/Direct/TimeHandle.hs Outdated Show resolved Hide resolved
hydra-node/test/Hydra/Model.hs Show resolved Hide resolved
hydra-node/test/Hydra/Model.hs Show resolved Hide resolved
hydra-node/test/Hydra/Model.hs Show resolved Hide resolved
hydra-node/test/Hydra/Model.hs Outdated Show resolved Hide resolved
hydra-node/test/Hydra/Model.hs Show resolved Hide resolved
hydra-node/test/Hydra/Model.hs Outdated Show resolved Hide resolved
hydra-node/test/Hydra/Model.hs Outdated Show resolved Hide resolved
hydra-node/test/Hydra/ModelSpec.hs Show resolved Hide resolved
hydra-node/test/Hydra/ModelSpec.hs Show resolved Hide resolved
@v0d1ch v0d1ch force-pushed the model-test-fanout branch 5 times, most recently from ec2932b to 2ffccf4 Compare February 12, 2024 12:10
There was a change of plans for the postcondition check so I removed
CheckFanoutUTxO and GetConfirmedUTxO since we can check directly
the Fanout utxo against the result in postcondition.
Instead of doing rollbacks automatically in the MockChain we modelled
the Action to do the same from the Model.
We are not able to check equality between a real UTxO and the
Model representation. For now we just check that the Fanout
value matches.
Also use indefinite Horizon
Add this check to the MockChain and generate more blocks
in the Model
Fix precondition to always be able to rollback
Currently the rollback in the Init can easily go back past the
current block so for now we can ommit it. Check in the MockChain
is needed to make this proper since the one I added does not work.
Just wait to see the appropriate server output or throw
the error after some time.
@v0d1ch v0d1ch merged commit 3f3eb25 into master Feb 12, 2024
21 checks passed
@v0d1ch v0d1ch deleted the model-test-fanout branch February 12, 2024 12:53
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants