wire: cache the non-witness serialization of MsgTx to memoize part of… #1376

Roasbeef · 2019-01-19T00:04:02Z

… TxHash

In this commit, we add a new field to the MsgTx struct:
cachedSeralizedNoWitness. As we decode the main transaction, we use an
io.TeeReader to copy over the non-witness bytes into this new field.
As a result, we can fully cache all tx serialization when computing the
TxHash. This has been shown to show up on profiles during IBD. Caching
this value allows us to optimize TxHash calculation across the entire
daemon as a whole.

Roasbeef · 2019-01-19T03:48:03Z

There's a bug here in that it doesn't detect mutations in the underlying *wire.MsgTx so if the value is changed at all, then we don't update the serialization. In the current pipeline for block validation, this doesn't manifest, but some of our tests do this which introduces some of the new failures in Travis.

jakesylvestre · 2020-03-04T13:55:08Z

@jcvernaleo (as per #1530)

Low priority
Outdated (needs updates to be merged)

Roasbeef · 2023-03-13T23:42:53Z

Here's a recent run late in the chain:

From this we can see that when checking signatures, we actually spend most of our time just encoding the tx again to compute the sighash.

Here's a flame graph for another view:

cc @kcalvinalvin

kcalvinalvin · 2023-03-14T06:16:37Z

I need a refresher on this but if memory serves right, doing away with the channel stuff in binaryFreeList and using a sync.Pool got rid of a big chunk of the time taken.

From this we can see that when checking signatures, we actually spend most of our time just encoding the tx again to compute the sighash.

But yeah if this is something that doesn't need to happen in the first place with memoization that'd be much better.

Roasbeef · 2023-08-09T23:46:10Z

Recent observation of the impact here:

Roasbeef · 2023-08-09T23:48:05Z

Related PR re binary free list: #1426

… TxHash In this commit, we add a new field to the `MsgTx` struct: `cachedSeralizedNoWitness`. As we decode the main transaction, we use an `io.TeeReader` to copy over the non-witness bytes into this new field. As a result, we can fully cache all tx serialization when computing the TxHash. This has been shown to show up on profiles during IBD. Caching this value allows us to optimize TxHash calculation across the entire daemon as a whole.

Roasbeef · 2023-11-16T01:40:06Z

Rebased!

Roasbeef · 2023-12-05T02:39:19Z

Dug a bit, and I realize the issue is in the test itself, it mutates transactions, with stuff like replaceSpendScript. So what we need here (for the test's sake) is that we have a method to wipe the cached serialization if something is mutated. Alternatively, we have some internal map that uses the MsgTx as a value, which is consulted to see if the transaction has changed since last time.

…ests

Roasbeef · 2023-12-05T02:47:50Z

Pushed a tx implementing the above, don't think it's ideal though...worried about weird edge cases where transaction generation code uses a tx as a scratch pad and makes multiple versions to sign, with this it'll give the wrong txid after the first instance.

Also added a commit that'll re-use the non-witness serialization for SerializeNoWitness.

Crypt-iQ

I looked in both btcd and lnd to see if we modify wire.MsgTx after calculating the hash, but couldn't find any instances. The code looks good, but I think we need to be extra careful and see if there are any instances across our codebases

Crypt-iQ · 2023-12-08T16:50:26Z

wire/msgtx.go

@@ -877,6 +877,10 @@ func (msg *MsgTx) Serialize(w io.Writer) error {
 // Serialize, however even if the source transaction has inputs with witness
 // data, the old serialization format will still be used.
 func (msg *MsgTx) SerializeNoWitness(w io.Writer) error {
+	if msg.cachedSeralizedNoWitness != nil {
+		w.Write(msg.cachedSeralizedNoWitness)
+	}


missing return?

Crypt-iQ · 2023-12-08T16:51:33Z

wire/msgtx.go

+	// the rawTxTeeReader here, as these are segwit specific bytes.
+	var (
+		flag        [1]byte
+		hasWitneess bool


s/hasWitneess/hasWitness

guggero

While I like the elegance of using a io.TeeReader (TIL by the way), I'm not sure if this would break a bunch of things outside of btcd, namely in lnd and related projects.

What if we make the caching optional with an additional struct member boolean (or if memory footprint is a concern, use a bit within the Version field) that turns the caching on?
That way we could both benefit from the optimization within btcd while not breaking any assumptions for outside code.

guggero · 2023-12-12T12:12:59Z

wire/msgtx.go

+
+	// Write out the actual number of inputs as this won't be the very byte
+	// series after the versino of segwit transactions.
+	if WriteVarInt(&rawTxBuf, pver, count); err != nil {


Need to add err := WriteVarInt().

guggero · 2023-12-12T12:14:53Z

wire/msgtx.go

+	}
+
+	// Write out the actual number of inputs as this won't be the very byte
+	// series after the versino of segwit transactions.


nit: s/versino/version

guggero · 2023-12-12T12:19:22Z

wire/msgtx.go

+	// this transaction without witness data. When we decode a transaction,
+	// we'll write out the non-witness bytes to this so we can quickly
+	// calculate the TxHash later if needed.
+	cachedSeralizedNoWitness []byte


nit: s/cachedSeralizedNoWitness/cachedSerializedNoWitness/

guggero · 2023-12-12T12:22:04Z

wire/msgtx.go

+// useful to be able to get the correct txid after mutating a transaction's
+// state.
+func (msg *MsgTx) WipeCache() {
+	msg.cachedSeralizedNoWitness = nil


Don't we also need to call this in the helper methods like AddTxIn() and AddTxOut()? Or is the assumption that TxHash() would in practice only be called once the transaction is fully built, so the cache doesn't need to be invalidated?

I'm mostly worrying about uses of MsgTx outside of btcd, where I'm not sure we can 100% guarantee that we're always using this pattern...

Roasbeef · 2023-12-13T03:04:50Z

What if we make the caching optional with an additional struct member boolean (or if memory footprint is a concern, use a bit within the Version field) that turns the caching on?

Not a bad idea, agree that this could have a lot of unintended consequences. The other approach uses btcutil.Tx which gives another route to implement something like this.

Roasbeef · 2023-12-16T00:10:55Z

Closing in favor of #2023, see this comment for some details: #2023 (comment)

TL;DR: I think we need to remove/optimize the binary free list instead.

jcvernaleo added low priority outdated labels Mar 4, 2020

Roasbeef added the IBD optimization label Feb 4, 2021

Roasbeef mentioned this pull request Aug 15, 2023

chainhash, wire, btcutil, main: Memory efficient txhash #1978

Merged

Roasbeef mentioned this pull request Nov 16, 2023

blockchain, cmd, netsync, main: Add utxocache #1955

Merged

Roasbeef force-pushed the cache-raw-msg-tx branch from c73570b to 0924825 Compare November 16, 2023 01:39

Roasbeef mentioned this pull request Dec 5, 2023

btcutil: reuse serialized tx during TxHash #2023

Closed

Roasbeef added 2 commits December 4, 2023 18:44

blockchain: wipe internal tx cache after mutating txns for fullblockt…

e2a7cd1

…ests

wire: modify SerializeNoWitness to use cached version if available

d045a70

btcutil/txsort: wipe cache before assertion on mutated txn

5ba479d

Crypt-iQ reviewed Dec 8, 2023

View reviewed changes

guggero self-requested a review December 8, 2023 19:09

guggero requested changes Dec 12, 2023

View reviewed changes

Roasbeef closed this Dec 16, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

wire: cache the non-witness serialization of MsgTx to memoize part of… #1376

wire: cache the non-witness serialization of MsgTx to memoize part of… #1376

Roasbeef commented Jan 19, 2019

Roasbeef commented Jan 19, 2019

jakesylvestre commented Mar 4, 2020

Roasbeef commented Mar 13, 2023

kcalvinalvin commented Mar 14, 2023

Roasbeef commented Aug 9, 2023

Roasbeef commented Aug 9, 2023

Roasbeef commented Nov 16, 2023

Roasbeef commented Dec 5, 2023

Roasbeef commented Dec 5, 2023

Crypt-iQ left a comment

Crypt-iQ Dec 8, 2023

Crypt-iQ Dec 8, 2023

guggero left a comment

guggero Dec 12, 2023

guggero Dec 12, 2023

guggero Dec 12, 2023

guggero Dec 12, 2023

Roasbeef commented Dec 13, 2023

Roasbeef commented Dec 16, 2023

wire: cache the non-witness serialization of MsgTx to memoize part of… #1376

wire: cache the non-witness serialization of MsgTx to memoize part of… #1376

Conversation

Roasbeef commented Jan 19, 2019

Roasbeef commented Jan 19, 2019

jakesylvestre commented Mar 4, 2020

Roasbeef commented Mar 13, 2023

kcalvinalvin commented Mar 14, 2023

Roasbeef commented Aug 9, 2023

Roasbeef commented Aug 9, 2023

Roasbeef commented Nov 16, 2023

Roasbeef commented Dec 5, 2023

Roasbeef commented Dec 5, 2023

Crypt-iQ left a comment

Choose a reason for hiding this comment

Crypt-iQ Dec 8, 2023

Choose a reason for hiding this comment

Crypt-iQ Dec 8, 2023

Choose a reason for hiding this comment

guggero left a comment

Choose a reason for hiding this comment

guggero Dec 12, 2023

Choose a reason for hiding this comment

guggero Dec 12, 2023

Choose a reason for hiding this comment

guggero Dec 12, 2023

Choose a reason for hiding this comment

guggero Dec 12, 2023

Choose a reason for hiding this comment

Roasbeef commented Dec 13, 2023

Roasbeef commented Dec 16, 2023