Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Get details from transaction differences in getrawmempool #6035

Merged
merged 8 commits into from
Feb 1, 2023
Merged

Conversation

oxarbitrage
Copy link
Contributor

Motivation

We know there are differences sometimes in the zebrad mempool and the zcashd one. We want to know what are the differences by getting transaction content among other data (tx hash, height).

This is for #5935

Solution

  • Commit d5a28bc is a bit unrelated, it adds the zcashd cookie file path as an env var. This makes my testing easier but i will not mind removing it if other people think is not a good idea.
  • Commit de68004 actually adds the extra code for retrieving transaction data for each transaction in both mempools.

Review

This has a problem. When a transaction is too big, the call to decoderawtransaction will fail with Argument list too long. This is an OS limitation and it seems not trivial to change the argument limit in linux, i am not sure if zcash-cli accepts a file as argument and if this can solve the problem. Researching also xargs but unsure if will work either.

The PR is a draft until i can fix that, ideas are welcome.

Reviewer Checklist

  • Will the PR name make sense to users?
    • Does it need extra CHANGELOG info? (new features, breaking changes, large changes)
  • Are the PR labels correct?
  • Does the code do what the ticket and PR says?
    • Does it change concurrent code, unsafe code, or consensus rules?
  • How do you know it works? Does it have tests?

Follow Up Work

@oxarbitrage
Copy link
Contributor Author

The mempool part will not be executed if there are no differences so you need to execute it several times to catch a diff and then get the full data. For example:

$ ./zebra-utils/zcash-rpc-diff 6666 getrawmempool
Checking first node release info...
Checking second node release info...
Connected to zebrad (port 6666) and zcashd (/media/oxarbitrage/4eb53770-4e4f-4d80-a830-0914f3f5b89a/zcash/zcash/src/zcash-cli zcash.conf port).

Checking zebrad network and tip height...
Checking zcashd network and tip height...

Request:
getrawmempool

Querying zebrad main chain at height >=1961221...

real	0m0,005s
user	0m0,000s
sys	0m0,004s

Querying zcashd main chain at height >=1961221...

real	0m0,004s
user	0m0,000s
sys	0m0,003s


Response diff between zebrad and zcashd:
--- /tmp/tmp.Tg1Etuy2AC.rpc-diff/zebrad-main-1961221-getrawmempool.json	2023-01-25 17:51:52.179333048 -0300
+++ /tmp/tmp.Tg1Etuy2AC.rpc-diff/zcashd-main-1961221-getrawmempool.json	2023-01-25 17:51:52.183332818 -0300
@@ -1,5 +1,4 @@
 [
-  "91f1b526fe7693308d57e5a73583499cf42cff13e9ef96ddd3d413d45eda1014",
   "38595c1249988d45a9368106ec12c4c88f1ead94ccca052a1f3fc5739407e18a",
   "73a4ab71f1b22323a594d06034fc5851d2a91529b2e161be481c741f2a9433d1",
   "344fe0d07ef089d07840af722dbdc3622d140b7ade81ac47c371841f2153c66a",

Cross-checking request:
getrawmempool

Querying zebrad main chain at height >=1961221...
Querying zcashd main chain at height >=1961221...

getrawmempool diff between zebrad and zcashd:
--- /tmp/tmp.Tg1Etuy2AC.rpc-diff/zebrad-main-1961221-getrawmempool.json	2023-01-25 17:51:52.187332586 -0300
+++ /tmp/tmp.Tg1Etuy2AC.rpc-diff/zcashd-main-1961221-getrawmempool.json	2023-01-25 17:51:52.191332356 -0300
@@ -1,5 +1,4 @@
 [
-  "91f1b526fe7693308d57e5a73583499cf42cff13e9ef96ddd3d413d45eda1014",
   "38595c1249988d45a9368106ec12c4c88f1ead94ccca052a1f3fc5739407e18a",
   "73a4ab71f1b22323a594d06034fc5851d2a91529b2e161be481c741f2a9433d1",
   "344fe0d07ef089d07840af722dbdc3622d140b7ade81ac47c371841f2153c66a",

Obtaining transaction details

## Displaying transaction 91f1b526fe7693308d57e5a73583499cf42cff13e9ef96ddd3d413d45eda1014

{
  "txid": "91f1b526fe7693308d57e5a73583499cf42cff13e9ef96ddd3d413d45eda1014",
  "authdigest": "ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff",
  "size": 244,
  "overwintered": true,
  "version": 4,
  "versiongroupid": "892f2085",
  "locktime": 0,
  "expiryheight": 0,
  "hex": "0400008085202f89019d1ad2de2fd64d55f5965644ba431ff19b207d356e991279f24365faeff86978150000006a473044022032c424d7060b6e2267a4fc286b7e83d407c0f6855575ac91d25a22e1b372face022060f7f92c6b447c382f740e916bd0ac65274623fbdce3b7013064ad0228207e45012103df5ea22f66fb093dab14c24095421f4a87e848939eae920a669e52064c7cda93ffffffff022d162706000000001976a9144ff173f2cf866a485a350d677838e8730bd86a2b88ac11f59101000000001976a914d49ab90c939c1cd3388a8c3470bf832f3317867288ac00000000000000000000000000000000000000",
  "vin": [
    {
      "txid": "7869f8effa6543f27912996e357d209bf11f43ba445696f5554dd62fded21a9d",
      "vout": 21,
      "scriptSig": {
        "asm": "3044022032c424d7060b6e2267a4fc286b7e83d407c0f6855575ac91d25a22e1b372face022060f7f92c6b447c382f740e916bd0ac65274623fbdce3b7013064ad0228207e45[ALL] 03df5ea22f66fb093dab14c24095421f4a87e848939eae920a669e52064c7cda93",
        "hex": "473044022032c424d7060b6e2267a4fc286b7e83d407c0f6855575ac91d25a22e1b372face022060f7f92c6b447c382f740e916bd0ac65274623fbdce3b7013064ad0228207e45012103df5ea22f66fb093dab14c24095421f4a87e848939eae920a669e52064c7cda93"
      },
      "sequence": 4294967295
    }
  ],
  "vout": [
    {
      "value": 1.03224877,
      "valueZat": 103224877,
      "valueSat": 103224877,
      "n": 0,
      "scriptPubKey": {
        "asm": "OP_DUP OP_HASH160 4ff173f2cf866a485a350d677838e8730bd86a2b OP_EQUALVERIFY OP_CHECKSIG",
        "hex": "76a9144ff173f2cf866a485a350d677838e8730bd86a2b88ac",
        "reqSigs": 1,
        "type": "pubkeyhash",
        "addresses": [
          "t1RAJdw6KpksLQNEbpkyVrhDSys4g53LAbB"
        ]
      }
    },
    {
      "value": 0.26342673,
      "valueZat": 26342673,
      "valueSat": 26342673,
      "n": 1,
      "scriptPubKey": {
        "asm": "OP_DUP OP_HASH160 d49ab90c939c1cd3388a8c3470bf832f33178672 OP_EQUALVERIFY OP_CHECKSIG",
        "hex": "76a914d49ab90c939c1cd3388a8c3470bf832f3317867288ac",
        "reqSigs": 1,
        "type": "pubkeyhash",
        "addresses": [
          "t1dFkcLR9t53dcwXC2z5KEJekvt3fi8xdWf"
        ]
      }
    }
  ],
  "vjoinsplit": [],
  "valueBalance": 0,
  "valueBalanceZat": 0,
  "vShieldedSpend": [],
  "vShieldedOutput": []
}

## Displaying transaction 38595c1249988d45a9368106ec12c4c88f1ead94ccca052a1f3fc5739407e18a

{
  "txid": "38595c1249988d45a9368106ec12c4c88f1ead94ccca052a1f3fc5739407e18a",
  "authdigest": "ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff",
  "size": 1095,
  "overwintered": true,
  "version": 4,
  "versiongroupid": "892f2085",
  "locktime": 1961210,
  "expiryheight": 1961261,
  "hex": "0400008085202f890194049fb4a69b4346d6c6ea83cef0f246de49946b4b80371ce5b3133eae5d2a6e0f0000006b483045022100efcfa68f04570c2ed76ce49c2a59beb318c03af3b5eade4716286556965c70e00220261f13224a007e5d36467a2e490f7e262a7ea62ae70ad064885c542ecefbe9f401210200689c8be2ce53e5bd80debc6513615569b7071a0866a93f6cb7da94d4a4fe66feffffff1b48170000000000001976a914bcd5241b39ceecb75f44c43b48b402d2281cdb5d88acda200000000000001976a91492c39cfcc4653a0d68b3e2d1d561ddc9dec09b2388ac386e0000000000001976a91447ec9f3b2f5d5706e48659af4d5a567469d70bd988ac80710000000000001976a9141ffaa9d8665c8140b33895cb8fa5b816b828637a88acc0030000000000001976a9144337792f0e9c9be470eb0042417c71c6076c983788ac4e110000000000001976a9149f3c3cb0b75652d265f258691f4a458aa156a26488ac1c020000000000001976a91492682838d37a3204d631ae91d3fbc524c72f646488acbc480000000000001976a914319ca992c65e8ada32c56644fb7ea861b0b1261188ac090b0000000000001976a914cec5f934e3367c4a897230ac3a1acd56e85f18c088ac26200000000000001976a9142bada3b13dfa56fbf98e10069bf027223978a95f88ac4a1a0000000000001976a914eb42f2a9acf273eece9aa513f44ceadfe83a689488ac12160000000000001976a914c65cec079be1f13bf35dfc4b897a15113e61417788ac7a170000000000001976a91480191a04ae8b6e3733602948c5943931b388e8e488ac58020000000000001976a914a798ea879ad6150dce2cb08a986ded51a5cf82e688ac34030000000000001976a9145bd6751b5485cc0ce8cbbbcc2cc510b4fd6c5a7b88ac5e150000000000001976a914ed75a69eeaa1d3719fa199d327fb7f75f7294c9a88acf4010000000000001976a914fc0ae12fea7536d88545d296f088cb555553074f88ac30430000000000001976a914afa531dce3cf85abe528561b3831c1b27363776d88ac84120000000000001976a914052e531bd56259f738ad075ad807c797de4b77e788acd01b0000000000001976a9143eb0935c20f5d808c87291d16c981c15566ea94288ac76020000000000001976a914176e7493f201165c727e013499b09dd19e9622cb88ac7c470000000000001976a91498f361e73153e74e31ef6f0139ad6982621e14b288ac26020000000000001976a9147b3db654e8ced4fcfb87e79818b080b4528cd95488ac362e0000000000001976a9149ea5bc18c0295a519fb87a51235ff9e85ea0aa9c88acf7577e4b000000001976a9141c9cb11e96585daa0f253eda69132c95fbc35a2088acea010000000000001976a914c477a46e13e72808b7168da0f82fb181381a3c5388ac2a3a0000000000001976a9147c5ebecaf0f1bb2bb6862a536f39a5e674d2f81c88acfaec1d002ded1d000000000000000000000000",
  "vin": [
    {
      "txid": "6e2a5dae3e13b3e51c37804b6b9449de46f2f0ce83eac6d646439ba6b49f0494",
      "vout": 15,
      "scriptSig": {
        "asm": "3045022100efcfa68f04570c2ed76ce49c2a59beb318c03af3b5eade4716286556965c70e00220261f13224a007e5d36467a2e490f7e262a7ea62ae70ad064885c542ecefbe9f4[ALL] 0200689c8be2ce53e5bd80debc6513615569b7071a0866a93f6cb7da94d4a4fe66",
        "hex": "483045022100efcfa68f04570c2ed76ce49c2a59beb318c03af3b5eade4716286556965c70e00220261f13224a007e5d36467a2e490f7e262a7ea62ae70ad064885c542ecefbe9f401210200689c8be2ce53e5bd80debc6513615569b7071a0866a93f6cb7da94d4a4fe66"
      },
      "sequence": 4294967294
    }
  ],
  "vout": [
    {
      "value": 5.96e-05,
      "valueZat": 5960,
      "valueSat": 5960,
      "n": 0,
      "scriptPubKey": {
        "asm": "OP_DUP OP_HASH160 bcd5241b39ceecb75f44c43b48b402d2281cdb5d OP_EQUALVERIFY OP_CHECKSIG",
        "hex": "76a914bcd5241b39ceecb75f44c43b48b402d2281cdb5d88ac",
        "reqSigs": 1,
        "type": "pubkeyhash",
        "addresses": [
          "t1b64NhV41tL9gTomY9gDC8wwC1w4KXmPP4"
        ]
      }
    },
    {
      "value": 8.41e-05,
      "valueZat": 8410,
      "valueSat": 8410,
      "n": 1,
      "scriptPubKey": {
        "asm": "OP_DUP OP_HASH160 92c39cfcc4653a0d68b3e2d1d561ddc9dec09b23 OP_EQUALVERIFY OP_CHECKSIG",
        "hex": "76a91492c39cfcc4653a0d68b3e2d1d561ddc9dec09b2388ac",
        "reqSigs": 1,
        "type": "pubkeyhash",
        "addresses": [
          "t1XFczngbBbDJAN4Rh1Qaxa5j1zMH1RBELw"
        ]
      }
    },
    {
      "value": 0.00028216,
      "valueZat": 28216,
      "valueSat": 28216,
      "n": 2,
      "scriptPubKey": {
        "asm": "OP_DUP OP_HASH160 47ec9f3b2f5d5706e48659af4d5a567469d70bd9 OP_EQUALVERIFY OP_CHECKSIG",
        "hex": "76a91447ec9f3b2f5d5706e48659af4d5a567469d70bd988ac",
        "reqSigs": 1,
        "type": "pubkeyhash",
        "addresses": [
          "t1QRuSjsxymqmb6fyR9GGgTXe2BrD6qf9wJ"
        ]
      }
    },
    {
      "value": 0.00029056,
      "valueZat": 29056,
      "valueSat": 29056,
      "n": 3,
      "scriptPubKey": {
        "asm": "OP_DUP OP_HASH160 1ffaa9d8665c8140b33895cb8fa5b816b828637a OP_EQUALVERIFY OP_CHECKSIG",
        "hex": "76a9141ffaa9d8665c8140b33895cb8fa5b816b828637a88ac",
        "reqSigs": 1,
        "type": "pubkeyhash",
        "addresses": [
          "t1LnhEgfgmEm96Btnyyj7ofxFAVekYYpqdh"
        ]
      }
    },
    {
      "value": 9.6e-06,
      "valueZat": 960,
      "valueSat": 960,
      "n": 4,
      "scriptPubKey": {
        "asm": "OP_DUP OP_HASH160 4337792f0e9c9be470eb0042417c71c6076c9837 OP_EQUALVERIFY OP_CHECKSIG",
        "hex": "76a9144337792f0e9c9be470eb0042417c71c6076c983788ac",
        "reqSigs": 1,
        "type": "pubkeyhash",
        "addresses": [
          "t1Q11jYcSCtn3GJMnXhGoHcNrRbdkrs5Sab"
        ]
      }
    },
    {
      "value": 4.43e-05,
      "valueZat": 4430,
      "valueSat": 4430,
      "n": 5,
      "scriptPubKey": {
        "asm": "OP_DUP OP_HASH160 9f3c3cb0b75652d265f258691f4a458aa156a264 OP_EQUALVERIFY OP_CHECKSIG",
        "hex": "76a9149f3c3cb0b75652d265f258691f4a458aa156a26488ac",
        "reqSigs": 1,
        "type": "pubkeyhash",
        "addresses": [
          "t1YPZcBVYR7x5Uq9smW1cvDE6dP74PRcYEJ"
        ]
      }
    },
    {
      "value": 5.4e-06,
      "valueZat": 540,
      "valueSat": 540,
      "n": 6,
      "scriptPubKey": {
        "asm": "OP_DUP OP_HASH160 92682838d37a3204d631ae91d3fbc524c72f6464 OP_EQUALVERIFY OP_CHECKSIG",
        "hex": "76a91492682838d37a3204d631ae91d3fbc524c72f646488ac",
        "reqSigs": 1,
        "type": "pubkeyhash",
        "addresses": [
          "t1XDjSJiGZo2k9AKrrutPVRMK5ktihyQ7BS"
        ]
      }
    },
    {
      "value": 0.0001862,
      "valueZat": 18620,
      "valueSat": 18620,
      "n": 7,
      "scriptPubKey": {
        "asm": "OP_DUP OP_HASH160 319ca992c65e8ada32c56644fb7ea861b0b12611 OP_EQUALVERIFY OP_CHECKSIG",
        "hex": "76a914319ca992c65e8ada32c56644fb7ea861b0b1261188ac",
        "reqSigs": 1,
        "type": "pubkeyhash",
        "addresses": [
          "t1NPvnq4E1AFHoZrze2KWF7Ski2vdVj62AN"
        ]
      }
    },
    {
      "value": 2.825e-05,
      "valueZat": 2825,
      "valueSat": 2825,
      "n": 8,
      "scriptPubKey": {
        "asm": "OP_DUP OP_HASH160 cec5f934e3367c4a897230ac3a1acd56e85f18c0 OP_EQUALVERIFY OP_CHECKSIG",
        "hex": "76a914cec5f934e3367c4a897230ac3a1acd56e85f18c088ac",
        "reqSigs": 1,
        "type": "pubkeyhash",
        "addresses": [
          "t1civNHmt5BVUjc3VFvFGvJor7dSVc4yBGb"
        ]
      }
    },
    {
      "value": 8.23e-05,
      "valueZat": 8230,
      "valueSat": 8230,
      "n": 9,
      "scriptPubKey": {
        "asm": "OP_DUP OP_HASH160 2bada3b13dfa56fbf98e10069bf027223978a95f OP_EQUALVERIFY OP_CHECKSIG",
        "hex": "76a9142bada3b13dfa56fbf98e10069bf027223978a95f88ac",
        "reqSigs": 1,
        "type": "pubkeyhash",
        "addresses": [
          "t1MrZ5GCioxgKawN3q5vbbSKaFcZMSq2bHk"
        ]
      }
    },
    {
      "value": 6.73e-05,
      "valueZat": 6730,
      "valueSat": 6730,
      "n": 10,
      "scriptPubKey": {
        "asm": "OP_DUP OP_HASH160 eb42f2a9acf273eece9aa513f44ceadfe83a6894 OP_EQUALVERIFY OP_CHECKSIG",
        "hex": "76a914eb42f2a9acf273eece9aa513f44ceadfe83a689488ac",
        "reqSigs": 1,
        "type": "pubkeyhash",
        "addresses": [
          "t1fKZ138KMpmuaHNR9vEmCLo1X7VVx6MUzY"
        ]
      }
    },
    {
      "value": 5.65e-05,
      "valueZat": 5650,
      "valueSat": 5650,
      "n": 11,
      "scriptPubKey": {
        "asm": "OP_DUP OP_HASH160 c65cec079be1f13bf35dfc4b897a15113e614177 OP_EQUALVERIFY OP_CHECKSIG",
        "hex": "76a914c65cec079be1f13bf35dfc4b897a15113e61417788ac",
        "reqSigs": 1,
        "type": "pubkeyhash",
        "addresses": [
          "t1bxT7eiexgpR58ADP1fsVqw2hrHbHWHdDA"
        ]
      }
    },
    {
      "value": 6.01e-05,
      "valueZat": 6010,
      "valueSat": 6010,
      "n": 12,
      "scriptPubKey": {
        "asm": "OP_DUP OP_HASH160 80191a04ae8b6e3733602948c5943931b388e8e4 OP_EQUALVERIFY OP_CHECKSIG",
        "hex": "76a91480191a04ae8b6e3733602948c5943931b388e8e488ac",
        "reqSigs": 1,
        "type": "pubkeyhash",
        "addresses": [
          "t1VYvZyucYVHevgXZjt3sZo98EYTJAmXCxj"
        ]
      }
    },
    {
      "value": 6e-06,
      "valueZat": 600,
      "valueSat": 600,
      "n": 13,
      "scriptPubKey": {
        "asm": "OP_DUP OP_HASH160 a798ea879ad6150dce2cb08a986ded51a5cf82e6 OP_EQUALVERIFY OP_CHECKSIG",
        "hex": "76a914a798ea879ad6150dce2cb08a986ded51a5cf82e688ac",
        "reqSigs": 1,
        "type": "pubkeyhash",
        "addresses": [
          "t1Z9n3AqKHb4K4xKrTf831Y231yfCYhsqNt"
        ]
      }
    },
    {
      "value": 8.2e-06,
      "valueZat": 820,
      "valueSat": 820,
      "n": 14,
      "scriptPubKey": {
        "asm": "OP_DUP OP_HASH160 5bd6751b5485cc0ce8cbbbcc2cc510b4fd6c5a7b OP_EQUALVERIFY OP_CHECKSIG",
        "hex": "76a9145bd6751b5485cc0ce8cbbbcc2cc510b4fd6c5a7b88ac",
        "reqSigs": 1,
        "type": "pubkeyhash",
        "addresses": [
          "t1SFCQZFbhy7AHZjV7jq4jKAJZCBvizXPyB"
        ]
      }
    },
    {
      "value": 5.47e-05,
      "valueZat": 5470,
      "valueSat": 5470,
      "n": 15,
      "scriptPubKey": {
        "asm": "OP_DUP OP_HASH160 ed75a69eeaa1d3719fa199d327fb7f75f7294c9a OP_EQUALVERIFY OP_CHECKSIG",
        "hex": "76a914ed75a69eeaa1d3719fa199d327fb7f75f7294c9a88ac",
        "reqSigs": 1,
        "type": "pubkeyhash",
        "addresses": [
          "t1fXB6KnFozpSXqEzk8ZCUQiRQEdJmWLEKe"
        ]
      }
    },
    {
      "value": 5e-06,
      "valueZat": 500,
      "valueSat": 500,
      "n": 16,
      "scriptPubKey": {
        "asm": "OP_DUP OP_HASH160 fc0ae12fea7536d88545d296f088cb555553074f OP_EQUALVERIFY OP_CHECKSIG",
        "hex": "76a914fc0ae12fea7536d88545d296f088cb555553074f88ac",
        "reqSigs": 1,
        "type": "pubkeyhash",
        "addresses": [
          "t1grHKcBDYiH12FPSpvLRy3Vmciw4EwbS4a"
        ]
      }
    },
    {
      "value": 0.000172,
      "valueZat": 17200,
      "valueSat": 17200,
      "n": 17,
      "scriptPubKey": {
        "asm": "OP_DUP OP_HASH160 afa531dce3cf85abe528561b3831c1b27363776d OP_EQUALVERIFY OP_CHECKSIG",
        "hex": "76a914afa531dce3cf85abe528561b3831c1b27363776d88ac",
        "reqSigs": 1,
        "type": "pubkeyhash",
        "addresses": [
          "t1ZtL9qgjEVeycmooktaBqPmtZ9zcNXZSr6"
        ]
      }
    },
    {
      "value": 4.74e-05,
      "valueZat": 4740,
      "valueSat": 4740,
      "n": 18,
      "scriptPubKey": {
        "asm": "OP_DUP OP_HASH160 052e531bd56259f738ad075ad807c797de4b77e7 OP_EQUALVERIFY OP_CHECKSIG",
        "hex": "76a914052e531bd56259f738ad075ad807c797de4b77e788ac",
        "reqSigs": 1,
        "type": "pubkeyhash",
        "addresses": [
          "t1JLzszDpKRcsxrC4JFVG3GmVGrj2j3ri68"
        ]
      }
    },
    {
      "value": 7.12e-05,
      "valueZat": 7120,
      "valueSat": 7120,
      "n": 19,
      "scriptPubKey": {
        "asm": "OP_DUP OP_HASH160 3eb0935c20f5d808c87291d16c981c15566ea942 OP_EQUALVERIFY OP_CHECKSIG",
        "hex": "76a9143eb0935c20f5d808c87291d16c981c15566ea94288ac",
        "reqSigs": 1,
        "type": "pubkeyhash",
        "addresses": [
          "t1Pb5RvEDwhg3Z6hZ4bUDDLQA1mVAaz3Hqa"
        ]
      }
    },
    {
      "value": 6.3e-06,
      "valueZat": 630,
      "valueSat": 630,
      "n": 20,
      "scriptPubKey": {
        "asm": "OP_DUP OP_HASH160 176e7493f201165c727e013499b09dd19e9622cb OP_EQUALVERIFY OP_CHECKSIG",
        "hex": "76a914176e7493f201165c727e013499b09dd19e9622cb88ac",
        "reqSigs": 1,
        "type": "pubkeyhash",
        "addresses": [
          "t1L1VsKe7wEhK2ExRDjkGn383bZy5fwSU4y"
        ]
      }
    },
    {
      "value": 0.000183,
      "valueZat": 18300,
      "valueSat": 18300,
      "n": 21,
      "scriptPubKey": {
        "asm": "OP_DUP OP_HASH160 98f361e73153e74e31ef6f0139ad6982621e14b2 OP_EQUALVERIFY OP_CHECKSIG",
        "hex": "76a91498f361e73153e74e31ef6f0139ad6982621e14b288ac",
        "reqSigs": 1,
        "type": "pubkeyhash",
        "addresses": [
          "t1XpLH14E2vydhVWqRAEeKyND3aAFrBU94z"
        ]
      }
    },
    {
      "value": 5.5e-06,
      "valueZat": 550,
      "valueSat": 550,
      "n": 22,
      "scriptPubKey": {
        "asm": "OP_DUP OP_HASH160 7b3db654e8ced4fcfb87e79818b080b4528cd954 OP_EQUALVERIFY OP_CHECKSIG",
        "hex": "76a9147b3db654e8ced4fcfb87e79818b080b4528cd95488ac",
        "reqSigs": 1,
        "type": "pubkeyhash",
        "addresses": [
          "t1V7F3mikMTHWwFAiTdBgQmd7JymnWZXoJ7"
        ]
      }
    },
    {
      "value": 0.0001183,
      "valueZat": 11830,
      "valueSat": 11830,
      "n": 23,
      "scriptPubKey": {
        "asm": "OP_DUP OP_HASH160 9ea5bc18c0295a519fb87a51235ff9e85ea0aa9c OP_EQUALVERIFY OP_CHECKSIG",
        "hex": "76a9149ea5bc18c0295a519fb87a51235ff9e85ea0aa9c88ac",
        "reqSigs": 1,
        "type": "pubkeyhash",
        "addresses": [
          "t1YLTK6bVxMZrTbyKPzJ6RchfPYnsvN8wrN"
        ]
      }
    },
    {
      "value": 12.66571255,
      "valueZat": 1266571255,
      "valueSat": 1266571255,
      "n": 24,
      "scriptPubKey": {
        "asm": "OP_DUP OP_HASH160 1c9cb11e96585daa0f253eda69132c95fbc35a20 OP_EQUALVERIFY OP_CHECKSIG",
        "hex": "76a9141c9cb11e96585daa0f253eda69132c95fbc35a2088ac",
        "reqSigs": 1,
        "type": "pubkeyhash",
        "addresses": [
          "t1LUtdrCPKYZKeRXkfM3Q3BvWZrPAB1xkxA"
        ]
      }
    },
    {
      "value": 4.9e-06,
      "valueZat": 490,
      "valueSat": 490,
      "n": 25,
      "scriptPubKey": {
        "asm": "OP_DUP OP_HASH160 c477a46e13e72808b7168da0f82fb181381a3c53 OP_EQUALVERIFY OP_CHECKSIG",
        "hex": "76a914c477a46e13e72808b7168da0f82fb181381a3c5388ac",
        "reqSigs": 1,
        "type": "pubkeyhash",
        "addresses": [
          "t1bnRmpwdodSKd579yDqFesqQprVYM94JzH"
        ]
      }
    },
    {
      "value": 0.0001489,
      "valueZat": 14890,
      "valueSat": 14890,
      "n": 26,
      "scriptPubKey": {
        "asm": "OP_DUP OP_HASH160 7c5ebecaf0f1bb2bb6862a536f39a5e674d2f81c OP_EQUALVERIFY OP_CHECKSIG",
        "hex": "76a9147c5ebecaf0f1bb2bb6862a536f39a5e674d2f81c88ac",
        "reqSigs": 1,
        "type": "pubkeyhash",
        "addresses": [
          "t1VDDJ96DsdMHKkywMqxw1cV8v1Fyqqk8tZ"
        ]
      }
    }
  ],
  "vjoinsplit": [],
  "valueBalance": 0,
  "valueBalanceZat": 0,
  "vShieldedSpend": [],
  "vShieldedOutput": []
}

## Displaying transaction 73a4ab71f1b22323a594d06034fc5851d2a91529b2e161be481c741f2a9433d1

{
  "txid": "73a4ab71f1b22323a594d06034fc5851d2a91529b2e161be481c741f2a9433d1",
  "authdigest": "ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff",
  "size": 245,
  "overwintered": true,
  "version": 4,
  "versiongroupid": "892f2085",
  "locktime": 0,
  "expiryheight": 0,
  "hex": "0400008085202f89013f8cadd80ef4b238a493db96697363d43c6ee45cf70c9fb17e8e6da523cf51ad010000006b483045022100a180689b4ad571042681e6a4e926e7a3b81f5a9cb2d1b6fc00133342b06a5d6002206ada61c5fbced5fcaa1ae7e84cd6eff1067f943aa9ba2eba41ddd98459d9db49012103ec57e53d52e48f6e4bec3728f70df6e9e6c7c30bee96b0cf1819468346a2a81bffffffff02302c9e00000000001976a9146f440500ac35fc1ceabc67f913831a26e9b28e1c88ac3e290000000000001976a914e878fb14bed2358621f76f3b3c36df2dd992797b88ac00000000000000000000000000000000000000",
  "vin": [
    {
      "txid": "ad51cf23a56d8e7eb19f0cf75ce46e3cd463736996db93a438b2f40ed8ad8c3f",
      "vout": 1,
      "scriptSig": {
        "asm": "3045022100a180689b4ad571042681e6a4e926e7a3b81f5a9cb2d1b6fc00133342b06a5d6002206ada61c5fbced5fcaa1ae7e84cd6eff1067f943aa9ba2eba41ddd98459d9db49[ALL] 03ec57e53d52e48f6e4bec3728f70df6e9e6c7c30bee96b0cf1819468346a2a81b",
        "hex": "483045022100a180689b4ad571042681e6a4e926e7a3b81f5a9cb2d1b6fc00133342b06a5d6002206ada61c5fbced5fcaa1ae7e84cd6eff1067f943aa9ba2eba41ddd98459d9db49012103ec57e53d52e48f6e4bec3728f70df6e9e6c7c30bee96b0cf1819468346a2a81b"
      },
      "sequence": 4294967295
    }
  ],
  "vout": [
    {
      "value": 0.10366,
      "valueZat": 10366000,
      "valueSat": 10366000,
      "n": 0,
      "scriptPubKey": {
        "asm": "OP_DUP OP_HASH160 6f440500ac35fc1ceabc67f913831a26e9b28e1c OP_EQUALVERIFY OP_CHECKSIG",
        "hex": "76a9146f440500ac35fc1ceabc67f913831a26e9b28e1c88ac",
        "reqSigs": 1,
        "type": "pubkeyhash",
        "addresses": [
          "t1U1vVi6Wv1xCzmXuRJXifDBm4KD696Wcwf"
        ]
      }
    },
    {
      "value": 0.00010558,
      "valueZat": 10558,
      "valueSat": 10558,
      "n": 1,
      "scriptPubKey": {
        "asm": "OP_DUP OP_HASH160 e878fb14bed2358621f76f3b3c36df2dd992797b OP_EQUALVERIFY OP_CHECKSIG",
        "hex": "76a914e878fb14bed2358621f76f3b3c36df2dd992797b88ac",
        "reqSigs": 1,
        "type": "pubkeyhash",
        "addresses": [
          "t1f4ohjSbYLpXHGMVs5hyPrNXkU8aCaRXf4"
        ]
      }
    }
  ],
  "vjoinsplit": [],
  "valueBalance": 0,
  "valueBalanceZat": 0,
  "vShieldedSpend": [],
  "vShieldedOutput": []
}

## Displaying transaction 344fe0d07ef089d07840af722dbdc3622d140b7ade81ac47c371841f2153c66a

{
  "txid": "344fe0d07ef089d07840af722dbdc3622d140b7ade81ac47c371841f2153c66a",
  "authdigest": "ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff",
  "size": 210,
  "overwintered": true,
  "version": 4,
  "versiongroupid": "892f2085",
  "locktime": 0,
  "expiryheight": 1961261,
  "hex": "0400008085202f890194049fb4a69b4346d6c6ea83cef0f246de49946b4b80371ce5b3133eae5d2a6e000000006a4730440220411329af23e974592453405fa62c6ded17cda4c85cfd3d71e5803c4934f3ba6c02204a0ca76baada579ef204ae0d7d4ba29da9990638b7d92cba155d3f5d33ed1499012102ad710bc073aa7f86e70ab91b8dd04eb3badf99ca8ab60544c180a7e2c09abe47ffffffff01bc020000000000001976a9143f7b36091b40004ad23ba459f56be8d361a2d0a988ac000000002ded1d000000000000000000000000",
  "vin": [
    {
      "txid": "6e2a5dae3e13b3e51c37804b6b9449de46f2f0ce83eac6d646439ba6b49f0494",
      "vout": 0,
      "scriptSig": {
        "asm": "30440220411329af23e974592453405fa62c6ded17cda4c85cfd3d71e5803c4934f3ba6c02204a0ca76baada579ef204ae0d7d4ba29da9990638b7d92cba155d3f5d33ed1499[ALL] 02ad710bc073aa7f86e70ab91b8dd04eb3badf99ca8ab60544c180a7e2c09abe47",
        "hex": "4730440220411329af23e974592453405fa62c6ded17cda4c85cfd3d71e5803c4934f3ba6c02204a0ca76baada579ef204ae0d7d4ba29da9990638b7d92cba155d3f5d33ed1499012102ad710bc073aa7f86e70ab91b8dd04eb3badf99ca8ab60544c180a7e2c09abe47"
      },
      "sequence": 4294967295
    }
  ],
  "vout": [
    {
      "value": 7e-06,
      "valueZat": 700,
      "valueSat": 700,
      "n": 0,
      "scriptPubKey": {
        "asm": "OP_DUP OP_HASH160 3f7b36091b40004ad23ba459f56be8d361a2d0a9 OP_EQUALVERIFY OP_CHECKSIG",
        "hex": "76a9143f7b36091b40004ad23ba459f56be8d361a2d0a988ac",
        "reqSigs": 1,
        "type": "pubkeyhash",
        "addresses": [
          "t1PfGBGzdUF66hF42Pc99obfZyCG7WMDvmF"
        ]
      }
    }
  ],
  "vjoinsplit": [],
  "valueBalance": 0,
  "valueBalanceZat": 0,
  "vShieldedSpend": [],
  "vShieldedOutput": []
}

## Displaying transaction b18063362e72717855d8df6e614bc9c1e75974f87e481822451e8fdfedf77c52

./zebra-utils/zcash-rpc-diff: line 221: /media/oxarbitrage/4eb53770-4e4f-4d80-a830-0914f3f5b89a/zcash/zcash/src/zcash-cli: Argument list too long
$ 

So here we have a transaction which is in zcashd but not in zebrad that we could investigate:

{
  "txid": "91f1b526fe7693308d57e5a73583499cf42cff13e9ef96ddd3d413d45eda1014",
  "authdigest": "ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff",
  "size": 244,
  "overwintered": true,
  "version": 4,
  "versiongroupid": "892f2085",
  "locktime": 0,
  "expiryheight": 0,
  "hex": "0400008085202f89019d1ad2de2fd64d55f5965644ba431ff19b207d356e991279f24365faeff86978150000006a473044022032c424d7060b6e2267a4fc286b7e83d407c0f6855575ac91d25a22e1b372face022060f7f92c6b447c382f740e916bd0ac65274623fbdce3b7013064ad0228207e45012103df5ea22f66fb093dab14c24095421f4a87e848939eae920a669e52064c7cda93ffffffff022d162706000000001976a9144ff173f2cf866a485a350d677838e8730bd86a2b88ac11f59101000000001976a914d49ab90c939c1cd3388a8c3470bf832f3317867288ac00000000000000000000000000000000000000",
  "vin": [
    {
      "txid": "7869f8effa6543f27912996e357d209bf11f43ba445696f5554dd62fded21a9d",
      "vout": 21,
      "scriptSig": {
        "asm": "3044022032c424d7060b6e2267a4fc286b7e83d407c0f6855575ac91d25a22e1b372face022060f7f92c6b447c382f740e916bd0ac65274623fbdce3b7013064ad0228207e45[ALL] 03df5ea22f66fb093dab14c24095421f4a87e848939eae920a669e52064c7cda93",
        "hex": "473044022032c424d7060b6e2267a4fc286b7e83d407c0f6855575ac91d25a22e1b372face022060f7f92c6b447c382f740e916bd0ac65274623fbdce3b7013064ad0228207e45012103df5ea22f66fb093dab14c24095421f4a87e848939eae920a669e52064c7cda93"
      },
      "sequence": 4294967295
    }
  ],
  "vout": [
    {
      "value": 1.03224877,
      "valueZat": 103224877,
      "valueSat": 103224877,
      "n": 0,
      "scriptPubKey": {
        "asm": "OP_DUP OP_HASH160 4ff173f2cf866a485a350d677838e8730bd86a2b OP_EQUALVERIFY OP_CHECKSIG",
        "hex": "76a9144ff173f2cf866a485a350d677838e8730bd86a2b88ac",
        "reqSigs": 1,
        "type": "pubkeyhash",
        "addresses": [
          "t1RAJdw6KpksLQNEbpkyVrhDSys4g53LAbB"
        ]
      }
    },
    {
      "value": 0.26342673,
      "valueZat": 26342673,
      "valueSat": 26342673,
      "n": 1,
      "scriptPubKey": {
        "asm": "OP_DUP OP_HASH160 d49ab90c939c1cd3388a8c3470bf832f33178672 OP_EQUALVERIFY OP_CHECKSIG",
        "hex": "76a914d49ab90c939c1cd3388a8c3470bf832f3317867288ac",
        "reqSigs": 1,
        "type": "pubkeyhash",
        "addresses": [
          "t1dFkcLR9t53dcwXC2z5KEJekvt3fi8xdWf"
        ]
      }
    }
  ],
  "vjoinsplit": [],
  "valueBalance": 0,
  "valueBalanceZat": 0,
  "vShieldedSpend": [],
  "vShieldedOutput": []
}

The output also shows one of the transactions (b18063362e72717855d8df6e614bc9c1e75974f87e481822451e8fdfedf77c52) is big enough to fail being a command line argument to decoderawtransaction.

@teor2345
Copy link
Contributor

The output also shows one of the transactions (b18063362e72717855d8df6e614bc9c1e75974f87e481822451e8fdfedf77c52) is big enough to fail being a command line argument to decoderawtransaction.

Here's how you can read the transaction data from a file, to fix that issue:

# read the proposal data from a file, to avoid command-line length limits
cat "$PROPOSAL_DATA" | \
$TIME $ZCASH_CLI -rpcport="$PORT" -stdin getblocktemplate \
> "$PROPOSAL_NODE_PROPOSAL_RESPONSE" || \
echo "$ZCASH_CLI -rpcport=$PORT exited with an error"

Each extra argument needs to be on its own line, in this case you just have one argument - the transaction data:

-stdin
Read extra arguments from standard input, one per line until EOF/Ctrl-D (recommended for sensitive information such as passphrases). If first extra argument is walletpassphrase then the first line(password) will not be echoed.

https://man.archlinux.org/man/zcash-cli.1.en

Copy link
Contributor

@teor2345 teor2345 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks great, thanks for putting it together!

I have some suggestions for making the command-line argument changes more flexible and compatible with existing setups.

I also think we might need to get transaction data from zcashd to diagnose some mempool differences.

zebra-utils/zcash-rpc-diff Outdated Show resolved Hide resolved
zebra-utils/zcash-rpc-diff Outdated Show resolved Hide resolved
zebra-utils/zcash-rpc-diff Outdated Show resolved Hide resolved
@teor2345
Copy link
Contributor

These changes look good, but I haven't tested them yet.

@oxarbitrage oxarbitrage marked this pull request as ready for review January 29, 2023 22:02
@oxarbitrage oxarbitrage requested a review from a team as a code owner January 29, 2023 22:02
@oxarbitrage oxarbitrage requested review from dconnolly and removed request for a team January 29, 2023 22:02
@mpguerra mpguerra removed the request for review from dconnolly January 30, 2023 13:45
@teor2345 teor2345 self-requested a review January 30, 2023 21:43
@teor2345
Copy link
Contributor

I tested these changes using this shell script:

SAME=0; while zcash-rpc-diff 28232 getrawmempool; do SAME=$((SAME + 1)); echo "$SAME identical mempools"; sleep 10; date; done

I found a difference, but it seems like zcashd was just slow to verify the transaction, because it was mined into a block 3 minutes later:
https://explorer.zcha.in/transactions/a25f1f79e1b5ccf8930861c620af96643df29d80d35e3873bc3c36be8d65f18b

Here is what I got, with some long data removed:

Checking first node release info...                                                                                                                                                                        Checking second node release info...                                                                                                                                                                       Connected to zebrad (port 28232) and zcashd (zcash-cli zcash.conf port).                                                                                                                                                                                                                                                                                                                                              Checking zebrad network and tip height...                                                                                                                                                                  Checking zcashd network and tip height...                                                                                                                                                                                                                                                                                                                                                                             Request:                                                                                                                                                                                                   getrawmempool                                                                                                                                                                                                                                                                                                                                                                                                         Querying zebrad main chain at height >=1967143...                                                                                                                                                                                                                                                                                                                                                                     real    0m0.007s                                                                                                                                                                                           user    0m0.004s                                                                                                                                                                                           sys     0m0.003s                                                                                                                                                                                                                                                                                                                                                                                                      Querying zcashd main chain at height >=1967143...                                                                                                                                                                                                                                                                                                                                                                     real    0m0.007s                                                                                                                                                                                           user    0m0.003s                                                                                                                                                                                           sys     0m0.003s                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                 Response diff between zebrad and zcashd:                                                                                                                                                                   --- /run/user/1000/tmp.x1NRZLUBcB.rpc-diff/zebrad-main-1967143-getrawmempool.json       2023-01-31 10:42:23.974510941 +1000                                                                                +++ /run/user/1000/tmp.x1NRZLUBcB.rpc-diff/zcashd-main-1967143-getrawmempool.json       2023-01-31 10:42:23.981510866 +1000                                                                                @@ -1,3 +1,2 @@                                                                                                                                                                                             [                                                                                                                                                                                                         -  "a25f1f79e1b5ccf8930861c620af96643df29d80d35e3873bc3c36be8d65f18b"                                                                                                                                       ]                                                                                                                                                                                                                                                                                                                                                                                                                    Cross-checking request:                                                                                                                                                                                    getrawmempool                                                                                                                                                                                                                                                                                                                                                                                                         Querying zebrad main chain at height >=1967143...                                                                                                                                                          Querying zcashd main chain at height >=1967143...                                                                                                                                                                                                                                                                                                                                                                     
getrawmempool diff between zebrad and zcashd:                                                                                                                                                              
--- /run/user/1000/tmp.x1NRZLUBcB.rpc-diff/zebrad-main-1967143-getrawmempool.json       2023-01-31 10:42:23.991510760 +1000                                                                                
+++ /run/user/1000/tmp.x1NRZLUBcB.rpc-diff/zcashd-main-1967143-getrawmempool.json       2023-01-31 10:42:23.997510695 +1000                                                                                
@@ -1,3 +1,2 @@                                                                                                                                                                                            
 [                                                                                                                                                                                                         
-  "a25f1f79e1b5ccf8930861c620af96643df29d80d35e3873bc3c36be8d65f18b"                                                                                                                                      
 ]                                                                    

# Dumping transactions from zebrad mempool                                                                                                                                                                 
                                                                                                                                                                                                           
## Displaying transaction a25f1f79e1b5ccf8930861c620af96643df29d80d35e3873bc3c36be8d65f18b from zebrad                                                                                                     
                                                                                                                                                                                                           
{                                                                                                                                                                                                          
  "txid": "a25f1f79e1b5ccf8930861c620af96643df29d80d35e3873bc3c36be8d65f18b",                                                                                                                              
  "authdigest": "eb6e9ed61134df901c4f8cfe00683c7104c6eeeb8da5341f24ead0cafdf5db38",                                                                                                                        
  "size": 15511,                                                                                                                                                                                           
  "overwintered": true,                                                                                                                                                                                    
  "version": 5,                                                                                                                                                                                            
  "versiongroupid": "26a7270a",                                                                                                                                                                            
  "locktime": 0,                                                                                                                                                                                           
  "expiryheight": 1967182,                                                                                                                                                                                 
  "hex": "...",
  "vin": [],                                                                                                                                                                                               
  "vout": [
    {
      "value": 64.51318018,
      "valueZat": 6451318018,
      "valueSat": 6451318018,
      "n": 0,
      "scriptPubKey": {
        "asm": "OP_DUP OP_HASH160 46c3e2226ffbb4dbfc0834973f99e8a7f2968e16 OP_EQUALVERIFY OP_CHECKSIG",
        "hex": "76a91446c3e2226ffbb4dbfc0834973f99e8a7f2968e1688ac",
        "reqSigs": 1,
        "type": "pubkeyhash",
        "addresses": [
          "t1QKmxybeTpoBWPMCtmPztdk8C1UMiTu1S5"
        ]
      }
    }
  ],
  "vjoinsplit": [],
  "valueBalance": 0,
  "valueBalanceZat": 0,
  "vShieldedSpend": [],
  "vShieldedOutput": [],
  "orchard": {
    "actions": [
      {
        "cv": "d5de224418682f5b153b0247b9ebea1028eae8af0450a9da5d8a95c723f8a492",
        "nullifier": "5caa7cbce8a368b13f71fba2368315a18f29739b469ae7cb7d35bb7de246b338",
        "rk": "cf41febb3bf5f0ef032d9f6e99007640501b72dc4d8e2d70dd85993a0940bc18",
        "cmx": "36f345b4471efb78fa16e138cf31e846d7c3dede825aa3cd8ca49bfe8f506f1c",
        "ephemeralKey": "173ad869fc8c61bbdb7256f2d35d0b1d77aa8ccdac2cfbb2dbca9af1c7d80404",
        "encCiphertext": "588980278e03e6bd06349cd1b8f34a035d6d98f58dea1e14fffa65fd20c1209d6d459dc87a89ef8528de1d5f86412c656d0e35f2ef4777408d2ce0c7df257a8336dd62016ae782465bca7af286c440d30951ad021d8453105
b80f8221a8298c4d3c166299f631d4bfde46e982a5c902d45e4e6cbe8fd2348e7457df0f887be9ce3fd8e15e8be61139e5d728205e3a8d2e1b22ea24aa5853228fe6a0d93b91d2b34704f901cabf63464c08489b41ef72e6cad8bff09f4a0681c4e839c16df
5a13eae853d8457a2012ba53ac125afc695569a5c1651697fb161713dc4e098435389db3d5d497ce414f452aa0e85bfe2b61b197705cf5704589e253ac3ee825ebd54ef716bfb77da748b06e622d1a5d49239813270b1cc4fe11ff40d72a62b569f52e581ce
a68cf13135c34054b4de631bafe9c567f5839e20e18e4159c6a63ab9c9dc53b7911ad129216a86a3f961a32b0bf667b839bc6d4f85a90c06a8d5df2e9748c078e0db1446a4155e3dcfa7d44f722f69a73faea9d8a5fedd9e917d9eb928e963597734a1d1879
58c6fc7746bf94867470ec170921b408822792a856a247e2a535683fff34d1491779746993ebe6222d05753bddc325fd00e6a21f4f0ef144bb54b652b73a0292b7eaedf8e95e68e4963c25b897f8c82882439fd73e694899751d993be022c3f0df9c6956dda
250e154c71e4a3cc0cdb976b555810e677c0635506bebeb8eaf05953ac0b506694e57d4c25b8e4534a0762364a3f83f22723a70844ad47bf045323fa0dfe7eee723802e9f22e00224183ddf842bba77c7ccf0b00c76",
        "outCiphertext": "b3e0252d0125d3952d1e4045fa3b0657289418b36a9a4bd284654d2eb2e3fa47c8bd386fc89454fa7f3f88622251acbaef66aa61220600bc56db58c664c9776e3975c9ebd427a8035545d33fca25dfce",
        "spendAuthSig": "706feeab36e43c6a20ef751372fe8814157a687a9b954cf6f468450c18812f3cf4677fee45cffc0d3119c49908abd16b58b04a05afe58f2a7f432e9c76623d08"
      },
      {                                                                                                                                                                                          [175/9239]
        "cv": "89a236fc7ea2712e04915f2ed9bac295c4270ae04add32eef4025aed7b0b69ac",
        "nullifier": "37c7e249c32994bb8e5c379089dfbc558239579919ecf38739d4739786d9de15",
        "rk": "0f3597698880641e9c97b8effa56ab060d45ad24ab80fccd21c8708677dd3dbb",
        "cmx": "da77ac659e9c4065c625aa45bb99c6f17a62156995e929c47b8a84559b772436",
        "ephemeralKey": "322802279d037195af66e6d36e726ed68cebb3fed1875b0fd3641e564ec2cca8",
        "encCiphertext": "c16e3c8aae0d73cef65d31283ce1e84d4cceb3e6c240965e00a700e8860e705154f15669b542c2ad4efebfa3ec4a60656cf2dde705e739ece99198a952b0740033d3c8a308c570c091e26b133dfa95b9b999e07482c97ee52
f679cf474253b1c2096d86bc61303dcbaf65e671d5a1033079eb58cce8d2b3bb76cd62845a68fa2910ac383eaa60afbf0d5aeacad4b46593d2c94f2796912f2d6eca9838b9f5799eaaf7f541000dbc948b7b72aa6c325c9720afe9917165e780a08374c0a16
870028ba428dbb91a2787afb60f4f985b472bea4215b2e73fba24e4801dae512183f2d89b3d35c99d211e0369eb1853e9eaa7466a6816fcad3b52bcf1af71385f5e14aee5f17d198401711eda62f649ae74d16a1734b683719f1f7f5e5bfa89cb56499bf4f7
67b8faaa64f45a79f45f9275b332d421e4cfc8abbd11e1131585066abfacae5410e843652ce0c4c614edba4c0a28e4f128becde4e495f5e1b1f6e0a38207434fb678dd7feedc2d8372ed88e22ce255ee453d613ef7385228a39ac0f8899e7aa6056b7feabfb
3dc17490b93e4c5056acbf8901686dc57e45e21c8caea1167f42d59ea635b7e399f1b7cf7617c7695033b6c17c9d5d03305c6eaaf48b91aa719ede44e368b15cc170212f2e8c9a2a512858e1c35b607437e91555158cbb216cd41248746216de42c90c4aa40
b06947e8d7f3677bc73f6e3e63c12d126d329fdf0f770b9ecddfa457fa0dab544b34adf00f759cfa0fca8da697e06dcef091bb924e0a3d6276f9ef1d2705ac7eecbc468346ebc3101a4d4373b3ea6180679f9dd0bb6",
        "outCiphertext": "2aca52c65ff201a4b142b43cd9e7f93b095dec7308292d5c939ef092e792b27aa053a41e6de3aa45894d7dba07dc377684fa2c11025523dcc7ceb251a67e46b2a97a95afbda436d5ca05fb6865540da6",
        "spendAuthSig": "560351c2b5ab85b9835c15fa5ca4296b78b0df1891532341d1d3630228d448b7447fc956794aaed788eea71d519bdf20eeda5c268ea0dc4d2384cdf92173ee06"
      },
      {
        "cv": "8b99345693548d1409528320b7ff30232fe40c1881f48eca2276e3a65c92e535",
        "nullifier": "5f99b326f9fea92f4148f518c64103cfc471c06f635d1d67099f49ca31c69327",
        "rk": "26a5f5eef0a8b7d0727efe78e8af38963b0d4d992d0a14a5279f841f4986b89b",
        "cmx": "a8207d062bce48ac1c0bed7b28c806946e7f00332b3785a649e98d818ee8f929",
        "ephemeralKey": "17358b367a4fb5d0fb758e66fe97bff8263a5a7f3286abec5bd767fccb63279f",
        "encCiphertext": "9ed820859da5d32821c265b6c99f605569a110f53bfc1bb53c8ebe3e265affcfe03e7ee6093cad06179525b48d7bf8752a7ae4f3a0a2f809589266db4fa315e9658f29ebc5ff50ef79f179a79f3d1d17dabf16c5e5e2a385a
c4368040ac54519bb87bf3a9a01c247161c74ea932153bfd96aabb190aaa81938515207089e28ae672aeb8898d3df1ec449243fe6ada07a2d7d6883fe2cd41672bdb53b35d1322a42e84ecdb83690cc51e85ce8f852c44cd30a3f4ee223943a8fe4be33ec21
bb1605f8ffc8813935a8946658f08cca9fe0ae4f8d8b5f61ae465a989509fa39ba50424cbef13b2469e3b3f9e0d1a1811d8f67e7e0018f7cf6531fe40b67b0a29c25cd17486ab71175835556d5ed7d60df8d13d1d940d5a09ffd318c70631ed8268dc652b66
45d36672fb195e4ba067493ffaf74e895fafa6bd30fa4937db58867ce6fce76673acb5cacd60f2d3b1e4833aa2ec53be1862a9576a3ba27004c473665535f5ec1f6071a0c28b7636b73bf63da3f9c7a4f99181105924668ee21b8161120fe79914f7c2bcaed
3732225df071ef39aa0a655e20181b380945bedd72f6c58c7439d98ae175bc82561a36530519638c8441b049c62e3060c3db217c56b342f838d6da479f0545ca4174b4dc6cc53e24f337ce8c03ecb53dd5626af81593d1d68aa6a3a752c461cb2ce54f77036
c9f8326f8f7e9f31cef846846254129817aee0bf8dc374f6835124c57cac63a75232cd486fab7a7aad7999561b1f31224ba949bc7ed96d380fb4383ad69949a1c2a3de1519a784cece42af4d0cc52e221ab4143b3cd",
        "outCiphertext": "1e2bd9029e75053a1d42d39e6376882d93948cc94f96694b0b287d69abfaa54b03e74ecb631f387b0edc60cdaaeeb83be3d403d5d5b4ff033ab879525c68eb87b1d55007b5d78a3b62a66613c8675cc7",
        "spendAuthSig": "9655f4aadbe11ae771bcef418f6d3886cb98fea2224210584df74261a3d1ab25f1295e29e86826545099655f74152053eae7975705a2905cb3683ca8bf615d13"
      },
      {
        "cv": "df8c11c87b8bbf61bddf36093f08f077baa1b36ffcb59e5d755a5c521b050c00",
        "nullifier": "a91bd0ef2e20a505bfcf9abeee71a6d4228558e61d782ba2be621c968980ca01",
        "rk": "d5e8bd3a260dc837855da5543fe77244e7afbd847d8b2e626311532b834ec007",
        "cmx": "680b6fae69ac8a779622380017760286fccafe221144693cdc46de0794a33910",
        "ephemeralKey": "ee2a96a243ce2bf1742bf092f59963aec9dfde0017d7641805ea9780f1a6f5b5",
        "encCiphertext": "602b279ce7396fda01a76ff7eb20f0686db358f4375ace42e52be3a2278bee9d0257f8fb3f5340e179e65978a2f67e0e9187485b6fde1171e26daf7143ca4c59a1e99b157347c2683243b0755a60a003267fbba11d09d9483
e00a685b5ea4e95858faf5ffa1e0cbd3949a5e9201e5c77840b6dfc20d498530c2bad51de087502d68225181e6ea5150c4713e3f6d02dcbceaaef8e36e4c5cc83104d6208fee240d6514528613365f123576da7855dbd7c259421a4ab76c611c3606266c29a
d7f7001684adddca4bbe3d9380a7f46a3b60c1c3364f65c0e9a89e903944c46e6d8faecfe192a0b21c4d65b61f961074687a3b27f9305d6cce3bbeacd3d8f784411c1d87e8f031b6596dbd9bc4eab1fa299039a20a4c6425b50d0a6e8f5ec8c00a5a963c052
dad2402deed271697130ce3a61857c7aad06d692a88077d5a7b5f98e0c52b36730311bd7d04a6617767e5877384b39eff6e3ee51ace5ce07cd4ec39762a240247bf2aeb8787479fde146ae7c1a016087d6535771cc3dda30e0a2724e78de017eec94399adae
b6893cc4e3ffb80c0a7d07f741f1fcf3a7274f861271f3042b059c5d056221b0bf1b2e0aad5c3ee6cbfbea2bd9f996ed0f5b32b6eaee2c09c8075dbbd85ecb0b616a3ffe9435e8530f59d0b6f58b110e0ac2470086869d9d17dda20dc90e3b2eccb6f510cd6
cd2e899e2408755a352414a537ed9114b8ab34c2d942215b02198823566a984b926ccf5559d8d1dbdabb41fd69bee98a0f14d29fd7def2c052b7ca2defb009aaef9a24511815e5cdd0eda9a842df6af8e97c44a5427",
        "outCiphertext": "4019a07487d9a246c9218ddc3e15c9efb277c6ed87a7ad367009c9268865ee3c7916bd387142413daba2d3f7ded72eb541a53756192a7d8a6533325212fef77717b849cfd623e856df7f008ec8181e75",
        "spendAuthSig": "22d672caf627be7fa5d20e58d431f568d1b792389299136171c1fc8a57712d2ac39429774977b82bc107d9c8db3430b308bfd5560bd72c51ce563958f0a22c2a"
      }
    ],
    "valueBalance": 64.51319018,
    "valueBalanceZat": 6451319018,
    "flags": {
      "enableSpends": true,
      "enableOutputs": true
    },
    "anchor": "22cdbff0868c23fe42b698dc27c89d3abea0c0e7141d8620547e82b96ae2c62c",
    "proof": "...",
    "bindingSig": "3fe220a05c12eabccca0dad4aaf26c9a281b3d48626d3fadb1ca12a36b79beb2b46198e5ca4d66e1d90fc87c889c141e4bff594dfe7df47599282c8200bbf326"
  }
}


# Dumping transactions from zcashd mempool


Here is the full transaction data:
zebrad-main-1967143-TRANSACTION_HEX-a25f1f79e1b5ccf8930861c620af96643df29d80d35e3873bc3c36be8d65f18b.json.zip

teor2345
teor2345 previously approved these changes Jan 31, 2023
Copy link
Contributor

@teor2345 teor2345 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks great, all my suggestions are optional, the script works well!

@teor2345
Copy link
Contributor

teor2345 commented Jan 31, 2023

What I am getting on testnet from that script:
(There are almost no transactions on testnet, so I'd just use mainnet for testing mempools.)

Checking first node release info...
Checking second node release info...
Connected to zebrad (port 38232) and zcashd (zcash-cli-testnet zcash.conf port).

Checking zebrad network and tip height...
Checking zcashd network and tip height...

Request:
getrawmempool

Querying zebrad test chain at height >=2211202...

real    0m0.011s
user    0m0.006s
sys     0m0.005s

Querying zcashd test chain at height >=2211202...

real    0m0.011s
user    0m0.006s
sys     0m0.005s


Response diff between zebrad and zcashd:
RPC responses were identical

/run/user/1000/tmp.xXGWUjxMOp.rpc-diff/zebrad-test-2211202-getrawmempool.json:
[
]

Cross-checking request:
getrawmempool

Querying zebrad test chain at height >=2211202...
Querying zcashd test chain at height >=2211202...

getrawmempool diff between zebrad and zcashd:
RPC check responses were identical

/run/user/1000/tmp.xXGWUjxMOp.rpc-diff/zebrad-test-2211202-getrawmempool.json:
[
]
1110 identical mempools

@teor2345
Copy link
Contributor

Another difference from mainnet, Zebra had this transaction, zcashd didn't, but it got mined into a block a few seconds later:
https://explorer.zcha.in/transactions/20bd4e40f20d7ae33309e331a75f6bbf65b79907bf39a17921cdde97ef0125ce

This appears to be a timing issue, but I have attached the full transaction content in case we want to do further analysis. (I deleted an identical 3.6 MB transaction from zebrad to get under the GitHub size limit.)

tmp.GDppLSARlE.rpc-diff.zip

@teor2345
Copy link
Contributor

Same for my third difference, looks like zcashd was just slow to pick this one up on my machine:
https://explorer.zcha.in/transactions/3c83f015932f864885e43f0076777723256604845ce3a63ca10f60595605ecbc

Checking first node release info...                                                                                                                                                                        
Checking second node release info...                                                                                                                                                                       
Connected to zebrad (port 28232) and zcashd (zcash-cli zcash.conf port).                                                                                                                                   
                                                                                                                                                                                                           
Checking zebrad network and tip height...                                                                                                                                                                  
Checking zcashd network and tip height...                                                                                                                                                                  
                                                                                                                                                                                                           
Request:                                                                                                                                                                                                   
getrawmempool                                                                                                                                                                                              
                                                                                                                                                                                                           
Querying zebrad main chain at height >=1967295...                                                                                                                                                          
                                                                                                                                                                                                           
real    0m0.007s                                                                                                                                                                                           
user    0m0.005s                                                                                                                                                                                           
sys     0m0.002s                                                                                                                                                                                           
                                                                                                                                                                                                           
Querying zcashd main chain at height >=1967295...                                                                                                                                                          
                                                                                                                                                                                                           
real    0m0.007s                                                                                                                                                                                           
user    0m0.004s                                                                                                                                                                                           
sys     0m0.003s                                                                                                                                                                                           
                                                                                                                                                                                                           
                                                                                                                                                                                                           
Response diff between zebrad and zcashd:                                                                                                                                                                   
--- /run/user/1000/tmp.GUKh8WGJFz.rpc-diff/zebrad-main-1967295-getrawmempool.json       2023-01-31 14:22:39.173027083 +1000                                                                                
+++ /run/user/1000/tmp.GUKh8WGJFz.rpc-diff/zcashd-main-1967295-getrawmempool.json       2023-01-31 14:22:39.180027019 +1000                                                                                
@@ -1,5 +1,4 @@                                                                                                                                                                                            
 [                                                                                                                                                                                                         
-  "3c83f015932f864885e43f0076777723256604845ce3a63ca10f60595605ecbc",                                                                                                                                     
   "755d016da17696c45c685764d8f666f9870ed9af272287b1f735162181d4f403",                                                                                                                                     
   "c6a230d46443b2187d31a4c63464cfbac1067a2be66b1411659fd939579fb313",                                                                                                                                     
   "45992ea4bb144d342a8b15cc528b09fe61d48d96763463578fa332cb5b66b9b6"                                                                                                                                      
                                                                                                                                                                                                           
Cross-checking request:                                                                                                                                                                                    
getrawmempool                                                                                                                                                                                              
                                                                                                                                                                                                           
Querying zebrad main chain at height >=1967295...                                                                                                                                                          
Querying zcashd main chain at height >=1967295...                                                                                                                                                          
                                                                                                                                                                                                           
getrawmempool diff between zebrad and zcashd:                                                                                                                                                              
--- /run/user/1000/tmp.GUKh8WGJFz.rpc-diff/zebrad-main-1967295-getrawmempool.json       2023-01-31 14:22:39.188026947 +1000                                                                                
+++ /run/user/1000/tmp.GUKh8WGJFz.rpc-diff/zcashd-main-1967295-getrawmempool.json       2023-01-31 14:22:39.194026892 +1000                                                                                
@@ -1,5 +1,4 @@                                                                                                                                                                                            
 [                                                                                                                                                                                                         
-  "3c83f015932f864885e43f0076777723256604845ce3a63ca10f60595605ecbc",                                                                                                                                     
   "755d016da17696c45c685764d8f666f9870ed9af272287b1f735162181d4f403",                                                                                                                                     
   "c6a230d46443b2187d31a4c63464cfbac1067a2be66b1411659fd939579fb313",                                                                                                                                     
   "45992ea4bb144d342a8b15cc528b09fe61d48d96763463578fa332cb5b66b9b6"                                                                                                                                      
                                                                                                                                                                                                           
# Dumping transactions from zebrad mempool

## Displaying transaction 3c83f015932f864885e43f0076777723256604845ce3a63ca10f60595605ecbc from zebrad                                                                                                     

{
  "txid": "3c83f015932f864885e43f0076777723256604845ce3a63ca10f60595605ecbc",
  "authdigest": "fdd35b26083248df8f70e5dcfb7e1fcd11037bbda28395809bd12595d0ec72bd",
  "size": 241,
  "overwintered": true,
  "version": 5,
  "versiongroupid": "26a7270a",
  "locktime": 0,
  "expiryheight": 1967336,
  "hex": "050000800a27a726b4d0d6c200000000e8041e0001f04a571cf8e59b2b979a2a81a7b714b2a4e5579f468e7e116c7a9dc79134c5c7000000006b483045022100f5f13fc957739a1ceb2cae491c36f074056d56b0170aa00e58bbdb4ceed3119d0
2204f4623df3d44ac1df7c5d64348b286ba8d4dafed485ab0a69e54c52359594aa70121035286c8e3c8112ab306fd1eba32d917d67265740cabfb1b622b595125ea128f36ffffffff02c0175c3f000000001976a914ac4b5b445d68c7bec9beee5faaac0828
33e4a6fc88ac3bb6d40b000000001976a91402dc707474d1b05bfe25c830396f1898e6c7f3f988ac000000",
  "vin": [
    {
      "txid": "c7c53491c79d7a6c117e8e469f57e5a4b214b7a7812a9a972b9be5f81c574af0",
      "vout": 0,
      "scriptSig": {
        "asm": "3045022100f5f13fc957739a1ceb2cae491c36f074056d56b0170aa00e58bbdb4ceed3119d02204f4623df3d44ac1df7c5d64348b286ba8d4dafed485ab0a69e54c52359594aa7[ALL] 035286c8e3c8112ab306fd1eba32d917d672657
40cabfb1b622b595125ea128f36",
        "hex": "483045022100f5f13fc957739a1ceb2cae491c36f074056d56b0170aa00e58bbdb4ceed3119d02204f4623df3d44ac1df7c5d64348b286ba8d4dafed485ab0a69e54c52359594aa70121035286c8e3c8112ab306fd1eba32d917d672657
40cabfb1b622b595125ea128f36"
      },
      "sequence": 4294967295
    }
  ],
  "vout": [
    {
      "value": 10.63,
      "valueZat": 1063000000,
      "valueSat": 1063000000,
      "n": 0,
      "scriptPubKey": {
        "asm": "OP_DUP OP_HASH160 ac4b5b445d68c7bec9beee5faaac082833e4a6fc OP_EQUALVERIFY OP_CHECKSIG",
        "hex": "76a914ac4b5b445d68c7bec9beee5faaac082833e4a6fc88ac",
        "reqSigs": 1,
        "type": "pubkeyhash",
        "addresses": [
          "t1ZacWC8KnY2d5YKZbLa5YdU3SQfAJWpdmh"
        ]
      }
    },
    {
      "value": 1.98489659,
      "valueZat": 198489659,
      "valueSat": 198489659,
      "n": 1,
      "scriptPubKey": {
        "asm": "OP_DUP OP_HASH160 02dc707474d1b05bfe25c830396f1898e6c7f3f9 OP_EQUALVERIFY OP_CHECKSIG",
        "hex": "76a91402dc707474d1b05bfe25c830396f1898e6c7f3f988ac",
        "reqSigs": 1,
        "type": "pubkeyhash",
        "addresses": [
          "t1J8jS7FyrDqAYhRMGyusKP7n6dgkCprgyf"
        ]
      }
    }
  ],
  "vjoinsplit": [],
  "valueBalance": 0,
  "valueBalanceZat": 0,
  "vShieldedSpend": [],
  "vShieldedOutput": [],
  "orchard": {
    "actions": [],
    "valueBalance": 0,
    "valueBalanceZat": 0
  }
}

...

Full RPC output is in /run/user/1000/tmp.GUKh8WGJFz.rpc-diff

tmp.GUKh8WGJFz.rpc-diff.zip

@teor2345 teor2345 added P-Medium ⚡ C-testing Category: These are tests A-rpc Area: Remote Procedure Call interfaces labels Feb 1, 2023
@teor2345 teor2345 added the A-compatibility Area: Compatibility with other nodes or wallets, or standard rules label Feb 1, 2023
mergify bot added a commit that referenced this pull request Feb 1, 2023
@mergify mergify bot merged commit ed22dff into main Feb 1, 2023
@mergify mergify bot deleted the issue5935 branch February 1, 2023 13:22
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
A-compatibility Area: Compatibility with other nodes or wallets, or standard rules A-rpc Area: Remote Procedure Call interfaces C-testing Category: These are tests
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants