assets: implement segwit btc swap contracts #741

buck54321 · 2020-10-11T01:38:12Z

Using segwit contracts saves us something like 30% in transaction fees. This will give us a little bit of wiggle room on Bitcoin markets where on-chain fees are outrageous.

Segwit is implemented as an option for the wallet backends and server's asset backends. Bitcoin is configured to use segwit by defualt. Litecoin will too, and I've tested it, but I'm leaving it in non-segwit just to demonstrate the option. It's important to offer the option, because even though some bitcoin forks won't have segwit, our clone backend might still be useful.

chappjc

Looks good on first pass. Not tested though.

dex/networks/btc/script.go

client/asset/btc/btc.go

dex/networks/btc/script.go

chappjc · 2020-10-13T13:08:53Z

A basic perfect match (complete fill) worked as expected with the redeem txn being authored with a p2wsh contract. The redeem txn:

{
  "txid": "b26ad979b065c3d84eea6e26ccc87873aca3874ac55e183df5a5e2bc4f02ff98",
  "hash": "21264133d696be60c61bbc6de836d3f73dfedf6d47d5dcbe0e427334836c4a13",
  "version": 1,
  "size": 325,
  "vsize": 143,
  "weight": 571,
  "locktime": 0,
  "vin": [
    {
      "txid": "ee27ed270f5ccb5c35e43d84102176fe888ea3088f127ed47b683d6c57401d39",
      "vout": 0,
      "scriptSig": {
        "asm": "",
        "hex": ""
      },
      "txinwitness": [
        "3045022100a9d99864c3e57d3db3d5a9319bf8c59c63c3e5793b8266c33eeac371a0432b34022062d7bfa15c706ba82b77ca775811b6be3d20cb51cb59557ddd1ff9d7557c91f201",
        "03ee1d83aee4d8e08f1e0f0ec87ff71b8303e1bdd75058adf71801d8a10efa4c5a",
        "41b424eb7ef18172024941fd54de49151917f175c9fb20502058b164ed4bfaa1",
        "01",
        "6382012088a820cacfa4763fd1644b48283349c337ba2ce59f70197f1127e7061b7f76f36b1a228876a914fc8853dca92f44ed6d0d571995ce12195ff8163d670454bc865fb17576a914f49d7fbe561d05a0c63139786a08d5fe3ad70f296888ac"
      ],
      "sequence": 4294967294
    }
  ],
  "vout": [
    {
      "value": 1.99999354,
      "n": 0,
      "scriptPubKey": {
        "asm": "0 a5b6cf46426df6b76d4e847d0a85bd43f1ba8681",
        "hex": "0014a5b6cf46426df6b76d4e847d0a85bd43f1ba8681",
        "reqSigs": 1,
        "type": "witness_v0_keyhash",
        "addresses": [
          "bcrt1q5kmv73jzdhmtwm2ws37s4pdag0cm4p5pz8adul"
        ]
      }
    }
  ],
  "hex": "01000000000101391d40576c3d687bd47e128f08a38e88fe762110843de4355ccb5c0f27ed27ee0000000000feffffff017abfeb0b00000000160014a5b6cf46426df6b76d4e847d0a85bd43f1ba868105483045022100a9d99864c3e57d3db3d5a9319bf8c59c63c3e5793b8266c33eeac371a0432b34022062d7bfa15c706ba82b77ca775811b6be3d20cb51cb59557ddd1ff9d7557c91f2012103ee1d83aee4d8e08f1e0f0ec87ff71b8303e1bdd75058adf71801d8a10efa4c5a2041b424eb7ef18172024941fd54de49151917f175c9fb20502058b164ed4bfaa10101616382012088a820cacfa4763fd1644b48283349c337ba2ce59f70197f1127e7061b7f76f36b1a228876a914fc8853dca92f44ed6d0d571995ce12195ff8163d670454bc865fb17576a914f49d7fbe561d05a0c63139786a08d5fe3ad70f296888ac00000000"
}

Note the empty scriptSig, data pushes in txinwitness, and "vsize": 143 vs. "size": 325.

Here is the swap txn that creates the contract output:

{
  "txid": "ee27ed270f5ccb5c35e43d84102176fe888ea3088f127ed47b683d6c57401d39",                                                                                                                
  "hash": "94d9d5adbb7e7b0f88577f9c7ed5bea11cd65258e4dae6b6526c0f7f358ae314",                                                                                                                                                                                                                                                                                                                "version": 1,                                                                                                                                                                                                                                                                                                                                                                            
  "size": 257,                                                                                                                                                                               
  "vsize": 176,
  "weight": 701,                                                             
  "locktime": 0,                                                             
  "vin": [     
    {         
      "txid": "0a7f5ebebef37835ee1917742edff6257a1924ea15f3e210bd9a6a7acf4926ab",
      "vout": 1,
      "scriptSig": {
        "asm": "0014beb6767b0227fa23ac59e98d204ae0441e3e71ff",
        "hex": "160014beb6767b0227fa23ac59e98d204ae0441e3e71ff"
      },                                                                         
      "txinwitness": [
        "3044022070b91b5cacae040e2b9e8d70c7d7b09be8e2e809e6af32fca2adee6bed86704f022058b030d384fd93380c30bfa02656902a9dc8580cb335ab1a1468c20fba236cd601",
        "03fa8e1f57d57bbd018739e9017f765c131feecbaa75564d919e19af3c5adf8e26"
      ],         
      "sequence": 4294967295
    }                 
  ],                                                                                                                                                                                         
  "vout": [                                                                  
    {                                                                      
      "value": 2.00000000,
      "n": 0,                                                                                                                                                                                                                                                                                                                                                                              
      "scriptPubKey": {
        "asm": "0 95305ae9994959d450eb7b4bfcc0411a100f2f042ab36211d80a33d8e0329d84",
        "hex": "002095305ae9994959d450eb7b4bfcc0411a100f2f042ab36211d80a33d8e0329d84",
        "reqSigs": 1,
        "type": "witness_v0_scripthash",
        "addresses": [
          "bcrt1qj5c946vef9vag58t0d9leszprggq7tcy92ekyywcpgea3cpjnkzqxh4p52"
        ]    
      }                
    },                                                      
    {                                                         
      "value": 0.99999486,
      "n": 1,                        
      "scriptPubKey": {
        "asm": "0 488b13e62604d691648abf98e4195c587b9a3673",
        "hex": "0014488b13e62604d691648abf98e4195c587b9a3673",
        "reqSigs": 1,
        "type": "witness_v0_keyhash",
        "addresses": [
          "bcrt1qfz938e3xqntfzey2h7vwgx2utpae5dnndwu3hl"                                                                                                                                                                                                                                                                                                                                           ]                                                                                                                                                                                                                                                                                                                                                                                  
      }
    }                                                      
  ],
  "hex": "01000000000101ab2649cf7a6a9abd10e2f315ea24197a25f6df2e741719ee3578f3bebe5e7f0a0100000017160014beb6767b0227fa23ac59e98d204ae0441e3e71ffffffffff0200c2eb0b0000000022002095305ae9994959d450eb7b4bfcc0411a100f2f042ab36211d80a33d8e0329d84fedef50500000000160014488b13e62604d691648abf98e4195c587b9a367302473044022070b91b5cacae040e2b9e8d70c7d7b09be8e2e809e6af32fca2adee6bed86704f0
22058b030d384fd93380c30bfa02656902a9dc8580cb335ab1a1468c20fba236cd6012103fa8e1f57d57bbd018739e9017f765c131feecbaa75564d919e19af3c5adf8e2600000000"
}

Note that this swap was funded by a p2sh-p2wpkh output (bip16 p2sh wrapping segwit p2wpkh), so the size was larger than InitTxSizeSegwit (152), instead 176 (expected, and why we have the "base" size).

Going to try some partial fills next...

chappjc · 2020-10-13T14:36:24Z

Trivial conflicts with #729 now merged.

This implements segwit as an option for the wallet backends and server's asset backends. Bitcoin is configured to use segwit by defualt. Litecoin likely will too, and I've tested it, but I'm leaving it in non-segwit just to demonstrate the option. It's important to offer the option, because even though some bitcoin forks won't have segwit, our clone backend might still be able to accommodate.

chappjc · 2020-10-14T05:51:48Z

I did run into some trouble when testing with tx split enabled and fee rate at max fee rate (set max fee rate for btc in markets.json to say 3).

First issue was with a 9-lot/9-swap order funded with tx split. It got through the swaps fine up until the final swap and it wasn't able to fund the swap.

2020-10-13 22:28:48.247 [ERR] CORE: Route 'match' request handler error (DEX 127.0.0.1:7232): runMatches - {tick of order 53f4aa8df1e3928dee801fc82492336723cb2895da7611a8b735620e391d9d01 failed: 127.0.0.1:7232 tick: {swapMatches order 53f4aa8df1e3928dee801fc82492336723cb2895da7611a8b735620e391d9d01 - {error sending swap transaction: not enough funds to cover minimum fee rate. 100000351 < 100000366, raw tx: 010000000168581320430e2fd9e3aed5971d6d128a25f51d93b23c948188e277d37f1b7b1c0100000000ffffffff0100e1f5050000000022002060b5b75149c6c326734e8cc85ca02600ea5a2f67d3a459040537e94292c7a8b200000000}}}

The successful swaps that came before that looked like the following:

2020-10-13 22:26:42.215 [DBG] CORE[btc]: Change output size = 31, addr = bcrt1qv7tld3hageq8kmazmwf97h63nt8xq6mhlcrk7k
2020-10-13 22:26:42.222 [DBG] CORE[btc]: 2 signature cycles to converge on fees for tx 1c7b1b7fd377e28881943cb2931df5258a126d1d97d5aee3d92f0e4320135868: min rate = 3, actual fee rate = 3 (459 for 153 bytes), change = true
2020-10-13 22:26:42.226 [DBG] CORE[btc]: locking change coin 1c7b1b7fd377e28881943cb2931df5258a126d1d97d5aee3d92f0e4320135868:1
2020-10-13 22:26:42.229 [INF] CORE: Broadcasted transaction with 1 swap contracts for order 53f4aa8df1e3928dee801fc82492336723cb2895da7611a8b735620e391d9d01. Fee rate = 3. Receipts (btc): [1c7b1b7fd377e28881943cb2931df5258a126d1d97d5aee3d92f0e4320135868:0]

It's unclear why 459 and 366 don't agree, but either way it only funded for 351 in fees.

Second was a single lot order funded with tx split. This time it got to the "impossible place":

[ERR] CORE[btc]: reached the impossible place. in = 100000375, out = 100000000, reqFee = 459, lastFee = 459, raw tx = 01000000000101e76d286efcfceda7db40ddeacaf279ec286d53c11bb5de28741d55ad079b613b0000000000ffffffff0200e1f5050000000022002027fa2271ff3cc384c3495c87994b5b4270f97c32308324d300dc0aaff5f0bb9cacffffffffffffff1600143f696f97b353af8273ab7bc25cef1f5cc403c97d024730440220722ebf96fa74aa4e3d1f1ce31ac183ed6c659cf6a3c238155850fd71b2f0313102202dd5b61d4459ce5c4bdb7766182847e14f0110d5705b2c0772f701c3ab6cc8cb01210337ebe04511e3c08ac2ca6515940b2d14e5d87ddd4cd5598c128f1adee6d250ee00000000
[ERR] CORE: notify: |ERROR| (order) Swap error - Error encountered sending a swap output(s) worth 1.00000000 btc on order 51b39f02 - Order: 51b39f0245dd2965e0e3136c54a1601e0d69265238874f8cb960a54b62bb5e6b
[ERR] CORE: Route 'match' request handler error (DEX 127.0.0.1:7232): runMatches - {tick of order 51b39f0245dd2965e0e3136c54a1601e0d69265238874f8cb960a54b62bb5e6b failed: 127.0.0.1:7232 tick: {swapMatches order 51b39f0245dd2965e0e3136c54a1601e0d69265238874f8cb960a54b62bb5e6b - {error sending swap transaction: change error}}}

The root of these issues with max fee rate + tx split may very well be on master though...

buck54321 · 2020-10-14T13:26:18Z

in = 100000375, out = 100000000

Is this the simnet harness? Did you use a rate of 1.0? Regardless, 375 sats to fund a swap with a p2wpkh-spending input (152 bytes) is nonsense. How did FundOrder mess that up though? I'm still working on it.

{
  "txid": "8fe44ccbbee15e7f3ce006bfe057603e6f4a1114b839f36571ee459dc3306507",
  "hash": "6ef38fef18625f158042799d74b073cb2436c2976d14a3365e153dcb18522273",
  "version": 1,
  "size": 234,
  "vsize": 153,
  "weight": 609,
  "locktime": 0,
  "vin": [
    {
      "txid": "3b619b07ad551d7428deb51bc1536d28ec79f2caeadd40dba7edfcfc6e286de7",
      "vout": 0,
      "scriptSig": {
        "asm": "",
        "hex": ""
      },
      "txinwitness": [
        "30440220722ebf96fa74aa4e3d1f1ce31ac183ed6c659cf6a3c238155850fd71b2f0313102202dd5b61d4459ce5c4bdb7766182847e14f0110d5705b2c0772f701c3ab6cc8cb01",
        "0337ebe04511e3c08ac2ca6515940b2d14e5d87ddd4cd5598c128f1adee6d250ee"
      ],
      "sequence": 4294967295
    }
  ],
  "vout": [
    {
      "value": 1.00000000,
      "n": 0,
      "scriptPubKey": {
        "asm": "0 27fa2271ff3cc384c3495c87994b5b4270f97c32308324d300dc0aaff5f0bb9c",
        "hex": "002027fa2271ff3cc384c3495c87994b5b4270f97c32308324d300dc0aaff5f0bb9c",
        "reqSigs": 1,
        "type": "witness_v0_scripthash",
        "addresses": [
          "bcrt1qylazyu0l8npcfs6ftjrejj6mgfc0jlpjxzpjf5cqms92la0shwwql8eh95"
        ]
      }
    },
    {
      "value": -0.00000084,
      "n": 1,
      "scriptPubKey": {
        "asm": "0 3f696f97b353af8273ab7bc25cef1f5cc403c97d",
        "hex": "00143f696f97b353af8273ab7bc25cef1f5cc403c97d",
        "reqSigs": 1,
        "type": "witness_v0_keyhash",
        "addresses": [
          "bcrt1q8a5kl9an2whcyuat00p9emcltnzq8jtarpxx7y"
        ]
      }
    }
  ]
}

buck54321 · 2020-10-15T12:10:30Z

I'd missed some accounting, but I think I've got it all now. The impossible place errors were junk in -> junk out, but should be good now.

I've added tests for split tx funding edges. I did a 10-lot, 1-lot per match chain with no issue at the end, so hopefully what you saw earlier was fixed with the accounting fixes.

buck54321 · 2020-10-15T21:05:50Z

dex/networks/btc/script.go

@@ -157,7 +168,7 @@ const (
 	//   - 1 wu compact int encoding value 33
 	//   - 33 wu serialized compressed pubkey
 	// NOTE: witness data is not script.
-	RedeemP2WPKHInputWitnessWeight = 1 + DERSigLength + 1 + 33 // 108
+	RedeemP2WPKHInputWitnessWeight = 1 + 1 + DERSigLength + 1 + 33 // 109


Pre-existing bug here wasn't helping anything.

buck54321 · 2020-10-15T21:10:24Z

dex/networks/btc/script.go

+	// 41 vbytes base tx input
+	// 109wu witness +  2wu segwit marker and flag = 28 vbytes
+	// total = 153 vbytes
+	InitTxSizeSegwit = InitTxSizeBaseSegwit + RedeemP2WPKHInputSize + ((RedeemP2WPKHInputWitnessWeight + 2 + 3) / 4)


We have to add the 2 marker and flag weight units and take the final vsize calculation for the transaction at the same time. We can't, say, assign a variable P2WPHKInputVBytes = RedeemP2WPKHInputSize + ((RedeemP2WPKHInputWitnessWeight + 2 + 3) / 4), because if there were multiple inputs, it would count the marker and flag for each input. We also can't create P2WPHKInputVBytes without the marker and flag bytes. because the bytes must be added in before rounding.

buck54321 · 2020-10-16T01:34:04Z

dex/networks/btc/script.go

-	return txOut.Value*1000/(3*int64(totalSize)) < int64(minRelayTxFee)
+	return txOut.Value/(3*int64(totalSize)) < int64(minRelayTxFee)


Was bug. I know there's some motivation to move to a per kB unit throughout, but that would require quite an effort. For now, I'll just bring this function in-line with our current practice of using sats/vbyte.

chappjc

Working great now. Order funding is exact now, and the lack of a change output on the final swap provides a sort of implicit buffer to the fees since order funding still assumes a change output on the final swap. This is good IMO.

Ran multi-lot partial maker fill swaps with max fee rate and tx split, with both cases of DCR and BTC makers. Both succeeded.

chappjc requested changes Oct 12, 2020

View reviewed changes

dex/networks/btc/script.go Outdated Show resolved Hide resolved

dex/networks/btc/script.go Show resolved Hide resolved

client/asset/btc/btc.go Outdated Show resolved Hide resolved

client/asset/btc/btc.go Outdated Show resolved Hide resolved

dex/networks/btc/script.go Show resolved Hide resolved

buck54321 added 2 commits October 13, 2020 11:59

review followup 1

4a4d285

buck54321 force-pushed the segwit branch from f6553eb to 4a4d285 Compare October 13, 2020 18:57

missed accounting

8a56671

chappjc added the high priority label Oct 15, 2020

do better

cf74288

buck54321 commented Oct 15, 2020

View reviewed changes

fix dust check and test dust edges

ebbbe41

buck54321 commented Oct 16, 2020

View reviewed changes

chappjc approved these changes Oct 16, 2020

View reviewed changes

chappjc merged commit 6d51169 into decred:master Oct 16, 2020

This was referenced Oct 16, 2020

reconfigure wallet panic #751

Closed

btc: author and verify P2WSH (native segwit) contract outputs #733

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

assets: implement segwit btc swap contracts #741

assets: implement segwit btc swap contracts #741

buck54321 commented Oct 11, 2020

chappjc left a comment

chappjc commented Oct 13, 2020 •

edited

Loading

chappjc commented Oct 13, 2020

chappjc commented Oct 14, 2020 •

edited

Loading

buck54321 commented Oct 14, 2020

buck54321 commented Oct 15, 2020

buck54321 Oct 15, 2020

buck54321 Oct 15, 2020

buck54321 Oct 16, 2020

chappjc left a comment

		return txOut.Value1000/(3int64(totalSize)) < int64(minRelayTxFee)
		return txOut.Value/(3*int64(totalSize)) < int64(minRelayTxFee)

assets: implement segwit btc swap contracts #741

assets: implement segwit btc swap contracts #741

Conversation

buck54321 commented Oct 11, 2020

chappjc left a comment

Choose a reason for hiding this comment

chappjc commented Oct 13, 2020 • edited Loading

chappjc commented Oct 13, 2020

chappjc commented Oct 14, 2020 • edited Loading

buck54321 commented Oct 14, 2020

buck54321 commented Oct 15, 2020

buck54321 Oct 15, 2020

Choose a reason for hiding this comment

buck54321 Oct 15, 2020

Choose a reason for hiding this comment

buck54321 Oct 16, 2020

Choose a reason for hiding this comment

chappjc left a comment

Choose a reason for hiding this comment

chappjc commented Oct 13, 2020 •

edited

Loading

chappjc commented Oct 14, 2020 •

edited

Loading