Incremental decommit #1344

ch1bo · 2024-03-08T10:10:35Z

This PR adds "incremental decommits" to the Hydra Head protocol, which allows users to take funds out of an open Head.

New API endpoint /decommit which accepts a "Decommit transaction", that spends some UTxO and whatever outputs it produces will be made available on the L1. This can be also done through a new Decommit client input and new server outputs DecommitRequested, DecommitApproved and DecommitFinalized, as well as DecommitInvalid to inform about status of the decommit.
Decommits are first approved in a snapshot on L2 via a new network message ReqDec, before a new decrementTx can be posted and observed on-chain.
Only one decommit can be processed at a given time.
Update documentation and added how-to about how to use this.
Acknowledged specification changes by "clearing" of $\textcolor{red}{\red}$ areas covered by this implementation in the specification.
End-to-end test covering the main scenario of decommitting funds.
Added mutation tests for Decrement, Close and Contest to cover all on-chain-verification changes.
Enhanced TxTrace tests to test decrements with various snapshots and their interaction with close/contest and fanout of a head.

CHANGELOG updated
Documentation updated
Haddocks updated
New TODOs explained hereafter

Two FIXMEs covered by Tune snapshot generation in TxTrace tests #1524
TODO in HeadLogic coverd by Prevent empty decommits #1502
TODO in tx-cost how we could improve the benchmark output (not crucial)
TODO in head logic about rollbacks .. actually something we need to consider with Incremental commit #199 too

github-actions · 2024-03-11T15:44:59Z

Transaction costs

Sizes and execution budgets for Hydra protocol transactions. Note that unlisted parameters are currently using arbitrary values and results are not fully deterministic and comparable to previous runs.

Metadata
Generated at	2024-07-22 19:57:51.46247528 UTC
Max. memory units	14000000
Max. CPU units	10000000000
Max. tx size (kB)	16384

Script summary

Name	Hash	Size (Bytes)
νInitial	2fac819a1f4f14e29639d1414220d2a18b6abd6b8e444d88d0dda8ff	3799
νCommit	2043a9f1a685bcf491413a5f139ee42e335157c8c6bc8d9e4018669d	1743
νHead	7ae23bc9f0833a5689b9fc812dd92fbe9dac881a632f14b28e8eb8db	10193
μHead	3ebfb5b268e0c94200e2c8a8eeebf704aabf303a769af602edb3603a*	4607

The minting policy hash is only usable for comparison. As the script is parameterized, the actual script is unique per head.

`Init` transaction costs

Parties	Tx size	% max Mem	% max CPU	Min fee ₳
1	5192	5.61	2.21	0.44
2	5389	6.99	2.76	0.46
3	5591	8.37	3.30	0.49
5	5993	11.21	4.43	0.54
10	6999	18.20	7.20	0.66
56	16244	81.61	32.28	1.76

`Commit` transaction costs

This uses ada-only outputs for better comparability.

UTxO	Tx size	% max Mem	% max CPU	Min fee ₳
1	557	10.52	4.15	0.29
2	747	13.86	5.65	0.34
3	931	17.33	7.20	0.38
5	1313	24.65	10.44	0.48
10	2241	45.22	19.36	0.75
20	4120	95.99	40.76	1.40

`CollectCom` transaction costs

Parties	UTxO (bytes)	Tx size	% max Mem	% max CPU	Min fee ₳
1	56	548	21.46	8.41	0.41
2	114	659	32.17	12.77	0.53
3	170	769	47.54	18.98	0.71
4	227	879	61.73	24.86	0.87
5	283	994	72.95	29.77	1.00
6	337	1100	96.60	39.37	1.26

Cost of Decrement Transaction

Parties	Tx size	% max Mem	% max CPU	Min fee ₳
1	631	17.96	7.89	0.38
2	763	18.57	8.79	0.40
3	1039	22.11	10.94	0.45
5	1110	21.93	12.31	0.47
10	2055	32.66	20.08	0.65
48	7671	96.39	72.63	1.79

`Close` transaction costs

Parties	Tx size	% max Mem	% max CPU	Min fee ₳
1	630	21.02	9.41	0.42
2	777	22.45	10.79	0.44
3	963	24.26	12.50	0.48
5	1355	27.96	16.00	0.55
10	1932	34.62	22.54	0.68
50	8005	98.56	83.62	1.90

`Contest` transaction costs

Parties	Tx size	% max Mem	% max CPU	Min fee ₳
1	665	27.16	11.67	0.48
2	800	28.98	13.20	0.51
3	995	31.06	15.01	0.55
5	1261	34.61	18.04	0.62
10	1864	43.02	25.23	0.76
39	6351	99.46	73.37	1.76

`Abort` transaction costs

There is some variation due to the random mixture of initial and already committed outputs.

Parties	Tx size	% max Mem	% max CPU	Min fee ₳
1	5074	17.47	7.60	0.57
2	5202	28.50	12.49	0.70
3	5341	42.73	18.90	0.87
4	5526	59.49	26.45	1.06
5	5479	72.53	32.12	1.21
6	5596	88.18	39.08	1.39

`FanOut` transaction costs

Involves spending head output and burning head tokens. Uses ada-only UTxO for better comparability.

Parties	UTxO	UTxO (bytes)	Tx size	% max Mem	% max CPU	Min fee ₳
5	0	0	5022	7.75	3.28	0.46
5	1	57	5057	8.88	3.99	0.47
5	5	284	5191	13.41	6.84	0.54
5	10	568	5360	19.26	10.48	0.62
5	20	1139	5701	30.58	17.60	0.78
5	30	1709	6043	41.90	24.72	0.93
5	40	2277	6381	53.03	31.76	1.09
5	50	2849	6724	64.37	38.89	1.25
5	81	4607	7768	99.53	61.01	1.74

End-to-end benchmark results

This page is intended to collect the latest end-to-end benchmark results produced by Hydra's continuous integration (CI) system from the latest master code.

Please note that these results are approximate as they are currently produced from limited cloud VMs and not controlled hardware. Rather than focusing on the absolute results, the emphasis should be on relative results, such as how the timings for a scenario evolve as the code changes.

Generated at 2024-07-22 20:01:47.176659373 UTC

Baseline Scenario

Number of nodes	1
Number of txs	3000
Avg. Confirmation Time (ms)	3.917242528
P99	6.177633689999866ms
P95	4.6298655ms
P50	3.7702235ms
Number of Invalid txs	0

Three local nodes

Number of nodes	3
Number of txs	9000
Avg. Confirmation Time (ms)	23.691672297
P99	115.74061088ms
P95	32.598588599999985ms
P50	21.08962ms
Number of Invalid txs	0

github-actions · 2024-03-19T22:50:54Z

Test Results

464 tests 452 ✅ 15m 53s ⏱️
148 suites 12 💤
5 files 0 ❌

Results for commit a70a781.

♻️ This comment has been updated with latest results.

This includes a re-ordering of clauses in checkDecrement, which did not change semantics (but script hashes).

By having versions in the ModelSnapshot, we can correctly test the head validator with contest transactions that try to set wrong versions (see removed TODO).

github-actions · 2024-07-23T08:48:10Z

Transaction costs

Sizes and execution budgets for Hydra protocol transactions. Note that unlisted parameters are currently using arbitrary values and results are not fully deterministic and comparable to previous runs.

Metadata
Generated at	2024-07-23 08:45:17.066031221 UTC
Max. memory units	14000000
Max. CPU units	10000000000
Max. tx size (kB)	16384

Script summary

Name	Hash	Size (Bytes)
νInitial	2fac819a1f4f14e29639d1414220d2a18b6abd6b8e444d88d0dda8ff	3799
νCommit	2043a9f1a685bcf491413a5f139ee42e335157c8c6bc8d9e4018669d	1743
νHead	7ae23bc9f0833a5689b9fc812dd92fbe9dac881a632f14b28e8eb8db	10193
μHead	3ebfb5b268e0c94200e2c8a8eeebf704aabf303a769af602edb3603a*	4607

The minting policy hash is only usable for comparison. As the script is parameterized, the actual script is unique per head.

`Init` transaction costs

Parties	Tx size	% max Mem	% max CPU	Min fee ₳
1	5188	5.79	2.29	0.44
2	5390	6.99	2.76	0.46
3	5590	8.46	3.34	0.49
5	5993	11.12	4.39	0.54
10	7003	18.16	7.18	0.66
56	16244	81.53	32.25	1.76

`Commit` transaction costs

This uses ada-only outputs for better comparability.

UTxO	Tx size	% max Mem	% max CPU	Min fee ₳
1	559	10.52	4.15	0.29
2	749	13.86	5.65	0.34
3	928	17.33	7.20	0.38
5	1313	24.65	10.44	0.48
10	2243	45.22	19.36	0.75
20	4107	95.99	40.76	1.40

`CollectCom` transaction costs

Parties	UTxO (bytes)	Tx size	% max Mem	% max CPU	Min fee ₳
1	57	549	22.17	8.68	0.42
2	114	659	32.17	12.76	0.53
3	170	769	46.22	18.48	0.69
4	227	879	58.24	23.54	0.83
5	282	989	73.85	30.08	1.01
6	339	1100	91.27	37.38	1.21

Cost of Decrement Transaction

Parties	Tx size	% max Mem	% max CPU	Min fee ₳
1	626	17.93	7.87	0.38
2	790	18.57	8.79	0.40
3	948	20.77	10.36	0.43
5	1326	26.41	14.07	0.52
10	2028	34.20	20.74	0.67
50	7945	98.54	74.86	1.84

`Close` transaction costs

Parties	Tx size	% max Mem	% max CPU	Min fee ₳
1	642	20.98	9.39	0.42
2	735	22.07	10.47	0.44
3	914	23.88	12.18	0.47
5	1328	28.15	16.05	0.55
10	1962	34.97	22.70	0.68
49	8148	99.68	83.99	1.92

`Contest` transaction costs

Parties	Tx size	% max Mem	% max CPU	Min fee ₳
1	658	27.18	11.67	0.48
2	763	28.48	12.83	0.51
3	937	30.68	14.68	0.55
5	1199	34.23	17.70	0.61
10	2053	44.58	26.55	0.79
38	6268	98.01	72.23	1.74

`Abort` transaction costs

There is some variation due to the random mixture of initial and already committed outputs.

Parties	Tx size	% max Mem	% max CPU	Min fee ₳
1	5093	17.43	7.59	0.57
2	5136	27.59	12.01	0.69
3	5289	41.46	18.27	0.85
4	5414	56.42	24.97	1.02
5	5560	71.55	31.65	1.20
6	5564	80.33	35.38	1.30

`FanOut` transaction costs

Involves spending head output and burning head tokens. Uses ada-only UTxO for better comparability.

Parties	UTxO	UTxO (bytes)	Tx size	% max Mem	% max CPU	Min fee ₳
5	0	0	5022	7.56	3.20	0.46
5	1	57	5057	8.88	3.99	0.47
5	5	285	5193	13.80	7.01	0.54
5	10	567	5359	19.26	10.48	0.62
5	20	1135	5697	30.38	17.51	0.77
5	30	1705	6040	41.70	24.63	0.93
5	40	2276	6381	53.03	31.76	1.09
5	50	2845	6720	64.56	38.97	1.25
5	81	4615	7777	99.53	61.01	1.74

End-to-end benchmark results

This page is intended to collect the latest end-to-end benchmark results produced by Hydra's continuous integration (CI) system from the latest master code.

Please note that these results are approximate as they are currently produced from limited cloud VMs and not controlled hardware. Rather than focusing on the absolute results, the emphasis should be on relative results, such as how the timings for a scenario evolve as the code changes.

Generated at 2024-07-23 08:47:59.164953195 UTC

Baseline Scenario

Number of nodes	1
Number of txs	3000
Avg. Confirmation Time (ms)	4.819395379
P99	8.700532639999995ms
P95	6.492272999999997ms
P50	4.542225500000001ms
Number of Invalid txs	0

Three local nodes

Number of nodes	3
Number of txs	9000
Avg. Confirmation Time (ms)	23.642216640
P99	58.78958378000018ms
P95	32.28759689999999ms
P50	21.48567ms
Number of Invalid txs	0

All three were doing the same thing and the third one was also not a "single hydra head" (as the category said).

By using zero-fee decommit tx, the amounts line up better.

The TxTraceSpec does test exactly that

One per validator code path is enough

github-actions · 2024-07-23T08:56:07Z

Transaction costs

Sizes and execution budgets for Hydra protocol transactions. Note that unlisted parameters are currently using arbitrary values and results are not fully deterministic and comparable to previous runs.

Metadata
Generated at	2024-07-23 08:53:23.953120817 UTC
Max. memory units	14000000
Max. CPU units	10000000000
Max. tx size (kB)	16384

Script summary

Name	Hash	Size (Bytes)
νInitial	2fac819a1f4f14e29639d1414220d2a18b6abd6b8e444d88d0dda8ff	3799
νCommit	2043a9f1a685bcf491413a5f139ee42e335157c8c6bc8d9e4018669d	1743
νHead	7ae23bc9f0833a5689b9fc812dd92fbe9dac881a632f14b28e8eb8db	10193
μHead	3ebfb5b268e0c94200e2c8a8eeebf704aabf303a769af602edb3603a*	4607

The minting policy hash is only usable for comparison. As the script is parameterized, the actual script is unique per head.

`Init` transaction costs

Parties	Tx size	% max Mem	% max CPU	Min fee ₳
1	5189	5.81	2.30	0.44
2	5389	7.21	2.85	0.47
3	5590	8.37	3.30	0.49
5	5994	11.22	4.43	0.54
10	6998	18.11	7.16	0.66
56	16244	81.53	32.25	1.76

`Commit` transaction costs

This uses ada-only outputs for better comparability.

UTxO	Tx size	% max Mem	% max CPU	Min fee ₳
1	556	10.52	4.15	0.29
2	745	13.86	5.65	0.34
3	934	17.33	7.20	0.38
5	1304	24.65	10.44	0.48
10	2244	45.22	19.36	0.75
20	4105	95.99	40.76	1.40

`CollectCom` transaction costs

Parties	UTxO (bytes)	Tx size	% max Mem	% max CPU	Min fee ₳
1	57	549	21.46	8.41	0.41
2	114	659	32.95	13.05	0.54
3	171	769	46.26	18.50	0.69
4	228	879	58.83	23.77	0.84
5	282	989	78.00	31.62	1.05
6	338	1100	88.75	36.46	1.18

Cost of Decrement Transaction

Parties	Tx size	% max Mem	% max CPU	Min fee ₳
1	626	17.71	7.79	0.38
2	788	19.29	9.14	0.41
3	926	20.65	10.41	0.43
5	1213	23.97	13.14	0.49
10	1911	31.86	19.86	0.64
50	7770	98.46	74.95	1.83

`Close` transaction costs

Parties	Tx size	% max Mem	% max CPU	Min fee ₳
1	649	21.02	9.41	0.42
2	773	22.45	10.79	0.44
3	973	24.18	12.49	0.48
5	1270	27.46	15.59	0.54
10	2047	36.14	23.64	0.70
50	8052	99.15	83.88	1.91

`Contest` transaction costs

Parties	Tx size	% max Mem	% max CPU	Min fee ₳
1	672	27.14	11.66	0.48
2	831	28.86	13.17	0.52
3	994	31.17	15.06	0.55
5	1367	35.40	18.71	0.63
10	2083	44.61	26.59	0.79
39	6481	99.16	73.62	1.77

`Abort` transaction costs

There is some variation due to the random mixture of initial and already committed outputs.

Parties	Tx size	% max Mem	% max CPU	Min fee ₳
1	5060	17.32	7.54	0.57
2	5223	29.46	12.98	0.71
3	5401	43.93	19.50	0.88
4	5408	56.90	25.15	1.03
5	5527	74.66	33.09	1.24
6	5767	96.60	43.08	1.49

`FanOut` transaction costs

Involves spending head output and burning head tokens. Uses ada-only UTxO for better comparability.

Parties	UTxO	UTxO (bytes)	Tx size	% max Mem	% max CPU	Min fee ₳
5	0	0	5022	7.56	3.20	0.46
5	1	57	5056	8.69	3.91	0.47
5	5	284	5191	13.41	6.84	0.54
5	10	570	5363	19.46	10.56	0.62
5	20	1140	5702	30.58	17.60	0.78
5	30	1704	6038	41.90	24.72	0.93
5	40	2275	6379	53.32	31.88	1.09
5	50	2844	6718	64.56	38.97	1.25
5	81	4609	7770	99.53	61.01	1.74

End-to-end benchmark results

This page is intended to collect the latest end-to-end benchmark results produced by Hydra's continuous integration (CI) system from the latest master code.

Please note that these results are approximate as they are currently produced from limited cloud VMs and not controlled hardware. Rather than focusing on the absolute results, the emphasis should be on relative results, such as how the timings for a scenario evolve as the code changes.

Generated at 2024-07-23 08:55:53.15345429 UTC

Baseline Scenario

Number of nodes	1
Number of txs	3000
Avg. Confirmation Time (ms)	5.453022671
P99	10.855603189999991ms
P95	7.904597699999999ms
P50	5.025929ms
Number of Invalid txs	0

Three local nodes

Number of nodes	3
Number of txs	9000
Avg. Confirmation Time (ms)	24.035840538
P99	65.54344090000006ms
P95	32.16640219999997ms
P50	21.8496265ms
Number of Invalid txs	0

ch1bo changed the title ~~incremental decommit~~ Incremental decommit Mar 8, 2024

ch1bo linked an issue Mar 8, 2024 that may be closed by this pull request

Incremental decommit #1057

Closed

16 tasks

v0d1ch force-pushed the feature/incremental-decommit branch 3 times, most recently from 23eea99 to 9d493fd Compare March 11, 2024 15:35

v0d1ch force-pushed the feature/incremental-decommit branch 5 times, most recently from d31cf15 to cd6ec4f Compare March 13, 2024 08:24

ch1bo force-pushed the feature/incremental-decommit branch 2 times, most recently from 3fd0f4d to d5ac680 Compare March 13, 2024 09:57

v0d1ch force-pushed the feature/incremental-decommit branch 14 times, most recently from 603d1c7 to 0beee6c Compare March 19, 2024 22:42

v0d1ch force-pushed the feature/incremental-decommit branch 2 times, most recently from 81c15fd to 5913388 Compare March 21, 2024 11:39

ch1bo added 5 commits July 23, 2024 10:40

Review and unmark red on-chain spec areas

2bbdc8a

This includes a re-ordering of clauses in checkDecrement, which did not change semantics (but script hashes).

Clear gaps and red markup in offchain

2601312

Fix TxTraceSpec by modelling snapshot versions

c813a88

By having versions in the ModelSnapshot, we can correctly test the head validator with contest transactions that try to set wrong versions (see removed TODO).

Model: genPayment to any hydra party

8243c8f

Revert changes on ObserveConfirmedTx

2138af9

ch1bo force-pushed the feature/incremental-decommit branch from 5600db5 to 105ff76 Compare July 23, 2024 08:41

ch1bo mentioned this pull request Jul 23, 2024

No processing of transactions during a decommit #1526

Closed

ch1bo added 17 commits July 23, 2024 10:50

Small refactor in genOpenActions

b6d5c49

Remove redundant e2e tests

476de1d

All three were doing the same thing and the third one was also not a "single hydra head" (as the category said).

Simplify aliceAddress calculation

fca6839

Simplify decommit e2e scenario to only one decommit

cb267c2

Further reduce complexity of decommit e2e test

ed02fdb

By using zero-fee decommit tx, the amounts line up better.

Rename a mutation enumeration to clarify

0536b8d

Drop redundant test in TxSpec

33c0cbc

The TxTraceSpec does test exactly that

Regenerate plutus scripts after rebase

d8b9bf7

Update genDecrement to also have sometimes empty commits

f261d37

Fix warnings

6d81570

Property test and implement splitUTxO differently

0915b5e

Remove redundant Fanout mutation tests

839e77f

One per validator code path is enough

Start tuning TxTraceSpec

df51ee9

Reduce discards of fanout

e2867a5

Fix logs.yaml with forgotten OnDecrementTx

98d950a

Make model aware of outdated snapshots

b4e3b33

Remove/demote TODO and XXXs

a70a781

ch1bo force-pushed the feature/incremental-decommit branch from 105ff76 to a70a781 Compare July 23, 2024 08:50

ffakenz approved these changes Jul 23, 2024

View reviewed changes

ch1bo merged commit 9c332ac into master Jul 23, 2024
21 checks passed

ch1bo deleted the feature/incremental-decommit branch July 23, 2024 10:31

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Incremental decommit #1344

Incremental decommit #1344

ch1bo commented Mar 8, 2024 •

edited

Loading

github-actions bot commented Mar 11, 2024 •

edited

Loading

github-actions bot commented Mar 19, 2024 •

edited

Loading

github-actions bot commented Jul 23, 2024

github-actions bot commented Jul 23, 2024

Incremental decommit #1344

Incremental decommit #1344

Conversation

ch1bo commented Mar 8, 2024 • edited Loading

github-actions bot commented Mar 11, 2024 • edited Loading

Transaction costs

Script summary

Init transaction costs

Commit transaction costs

CollectCom transaction costs

Cost of Decrement Transaction

Close transaction costs

Contest transaction costs

Abort transaction costs

FanOut transaction costs

End-to-end benchmark results

Baseline Scenario

Three local nodes

github-actions bot commented Mar 19, 2024 • edited Loading

Test Results

github-actions bot commented Jul 23, 2024

Transaction costs

Script summary

Init transaction costs

Commit transaction costs

CollectCom transaction costs

Cost of Decrement Transaction

Close transaction costs

Contest transaction costs

Abort transaction costs

FanOut transaction costs

End-to-end benchmark results

Baseline Scenario

Three local nodes

github-actions bot commented Jul 23, 2024

Transaction costs

Script summary

Init transaction costs

Commit transaction costs

CollectCom transaction costs

Cost of Decrement Transaction

Close transaction costs

Contest transaction costs

Abort transaction costs

FanOut transaction costs

End-to-end benchmark results

Baseline Scenario

Three local nodes

ch1bo commented Mar 8, 2024 •

edited

Loading

github-actions bot commented Mar 11, 2024 •

edited

Loading

`Init` transaction costs

`Commit` transaction costs

`CollectCom` transaction costs

`Close` transaction costs

`Contest` transaction costs

`Abort` transaction costs

`FanOut` transaction costs

github-actions bot commented Mar 19, 2024 •

edited

Loading

`Init` transaction costs

`Commit` transaction costs

`CollectCom` transaction costs

`Close` transaction costs

`Contest` transaction costs

`Abort` transaction costs

`FanOut` transaction costs

`Init` transaction costs

`Commit` transaction costs

`CollectCom` transaction costs

`Close` transaction costs

`Contest` transaction costs

`Abort` transaction costs

`FanOut` transaction costs