[psbt saga part 3/4]: taprpc: add CommitVirtualPsbts and PublishAndLogTransfer RPC methods

[guggero]

Depends on #805.

This is a first draft towards a new series of RPCs that allows us to drive the send logic completely from outside of the daemon.
That means we'll likely want to bypass the freighter partially or completely, since for TAP channels not all state and/or proofs of all state updates should be reflected in the database.

We'll likely want an additional RPC that allows us to store the output of the CommitVirtualPsbts call in the database, realizing the actual entries in the transfers and assets tables.

---

This change is 

[guggero]

@GeorgeTsagk it's still in draft, needs a lot of cleanup and some decisions. But should have what you need to build the sighash swap itest.

[guggero]

@Roasbeef I have the following flow in mind for driving an asset send mostly from outside of the daemon (for the ASSET_VERSION_V1/segwit use case):

1. User calls FundVirtualPsbt or creates vPSBTs from a list of known assets they want to spend (e.g. TAP channels, the list of proofs from the tx_asset_proof funding messages).
2. User calls CommitVirtualPsbts to create the output commitments for the vPSBTs, this creates proof suffixes for each asset output (lacking the asset witness and the chain header information).
3. The user then finalizes the proofs by signing the vPSBTs and "injecting" the asset witness into the proof suffixes.
4. Optional step: for transactions that should be reflected in the TAP database, the user would call DeliverVirtualPsbts (or whatever, this method doesn't exist yet), that would then create the asset_transfer entries, wait for confirmation and then ship the proofs to recipients using the proof courier.
   1. This step would only be called for transactions that the user should be aware of. For example normal sends (interactive or non-interactive) or the funding transaction. It would however not be called for commitment transactions, as they should IMO not be reflected in the TAP database.
   2. We would probably call this step if a channel commitment transaction is broadcast due to a force-close. Then it would make sense to realize the action as a transfer and see the balances.

My current question is mostly around step 4: To transport the unfinished proof suffixes and optional proof courier addresses from step 3 to step 4 we would need to place them somewhere. I would argue that the vPSBT outputs would be a good place for that. But I distinctly remember that you were against storing proofs or proof courier addresses in the vPSBTs. Do you remember why you argued that way? Mostly due to the size of the vPSBTs?

[GeorgeTsagk]

IIUC at this point there is a strict check that the inputs amount must equal the outputs amount.

If I want to create an incomplete anchorTx, could we somehow signal that there may be an input or output missing, hence no need to perform strict checks?

[guggero]

No, the input and output sum doesn't have to be equal here. That's exactly the task for lndWallet.FundPsbt to figure out if additional inputs are required to fund the outputs.
But that does mean that all outputs need to be specified here, otherwise we wouldn't know what to fund.

[Roasbeef]

Optional step: for transactions that should be reflected in the TAP database, the user would call DeliverVirtualPsbts (or whatever, this method doesn't exist yet), that would then create the asset_transfer entries, wait for confirmation and then ship the proofs to recipients using the proof courier.

Yeah I think that makes sense. Alternatively, the user may wish to just have the set of participants side load the proofs as well. Though either way you'd want to make sure that the sender's DB is updated to reflect that those outputs are now spent. In theory, we could also just watch all the outpoints we believe to be unspent for spends. In that case though, we'd still want to know things like if a change output was created that does still store outputs.

It would however not be called for commitment transactions, as they should IMO not be reflected in the TAP database.

Agreed, as at that point the user (channel daemon) is managing all those outputs. Alternatively, this could go into a sort of "watch only" mode, where the daemon keeps track of the outputs for accounting purposes, but understands it can't unilaterally sign for them itself.

My current question is mostly around step 4: To transport the unfinished proof suffixes and optional proof courier addresses from step 3 to step 4 we would need to place them somewhere. I would argue that the vPSBT outputs would be a good place for that. But I distinctly remember that you were against storing proofs or proof courier addresses in the vPSBTs. Do you remember why you argued that way? Mostly due to the size of the vPSBTs?

Hmm, IIRC I think it was about the potential size of the vPSBTs assuming it was actually storing the entire proof. If it's just the proof suffix, then I think that's ok, as their sort of bounded, in that the largest item is usually the Bitcoin block header merkle proof.

[guggero]

I was able to re-enable the re-org itest by bumping to the latest version of btcd 🎉

[lightninglabs-deploy]

@guggero, remember to re-request review from reviewers when ready

[GeorgeTsagk]

had another look due to the many force updates, still LGTM 💯

[Roasbeef]

Reviewed 30 of 30 files at r25, 4 of 4 files at r26, 2 of 2 files at r27, 2 of 2 files at r28, 2 of 2 files at r29, 5 of 5 files at r30, 3 of 3 files at r31, 3 of 3 files at r32, 2 of 2 files at r33, 2 of 2 files at r34, 3 of 3 files at r35, 3 of 3 files at r36, 11 of 13 files at r37, 22 of 22 files at r38, all commit messages.
Reviewable status: all files reviewed, 24 unresolved discussions (waiting on @GeorgeTsagk and @guggero)