[Pre-Production] - TX Fee Deducted Twice in Emerynet agoric-upgrade-14-rc0 testpass

[ivanlei]

Describe the bug

The Finoa team has reported an anomalous transaction that appears to deduct the tx fee twice, exceeding the amount specified by the signed transaction.

We noticed that after upgrading to agoric-upgrade-14-rc0, that the TX fee is paid twice.

Example of TX:
https://emerynet.explorer.agoric.net/agoric/tx/F661B7A8E25737AD047B296CFB7EF8624EF786B0E6457B84D16DD3AD09056051

This is a transfer of 1 BLD from agoric10d8rfalktk8n77zzxnqjkrhkw7ka24aw85zlu9 to agoric1ensvnvv03uwedczf5x7y9dcrngk2aaznjgjmrk with a desired fee payment of 0.005 BLD.

But instead of 2 transfer events, it has 3:
0.005 BLD was transfered as a fee to agoric17xpfvakm2amg962yls6f84z3kell8c5lzyplmt
0.005 BLD was transfered as a fee to agoric1ae0lmtzlgrcnla9xjkpaarq5d5dfez63h3nucl
The desired transfer amount 1 BLD was transfered to the desired destination wallet agoric1ensvnvv03uwedczf5x7y9dcrngk2aaznjgjmrk.

The transaction details in the explorer shows that the fee was 0.01 BLD, which is the total of 2x 0.005 BLD.
This problem is noticed for the first with Emerynet testnet and was never noticed with Mainnet or other cosmos based chains.

[ivanlei]

quick query and event decode using decoded_events.py

./bin/agd --node https://emerynet.rpc.agoric.net:443 query tx F661B7A8E25737AD047B296CFB7EF8624EF786B0E6457B84D16DD3AD09056051 -o json | python3 ~/bin/decoded_events.py | jq -C .
{
  "height": "3396361",
  "txhash": "F661B7A8E25737AD047B296CFB7EF8624EF786B0E6457B84D16DD3AD09056051",
  "codespace": "",
  "code": 0,
  "data": "0A1E0A1C2F636F736D6F732E62616E6B2E763162657461312E4D736753656E64",
  "raw_log": "[{\"events\":[{\"type\":\"coin_received\",\"attributes\":[{\"key\":\"receiver\",\"value\":\"agoric1ensvnvv03uwedczf5x7y9dcrngk2aaznjgjmrk\"},{\"key\":\"amount\",\"value\":\"10000ubld\"}]},{\"type\":\"coin_spent\",\"attributes\":[{\"key\":\"spender\",\"value\":\"agoric10d8rfalktk8n77zzxnqjkrhkw7ka24aw85zlu9\"},{\"key\":\"amount\",\"value\":\"10000ubld\"}]},{\"type\":\"message\",\"attributes\":[{\"key\":\"action\",\"value\":\"/cosmos.bank.v1beta1.MsgSend\"},{\"key\":\"sender\",\"value\":\"agoric10d8rfalktk8n77zzxnqjkrhkw7ka24aw85zlu9\"},{\"key\":\"module\",\"value\":\"bank\"}]},{\"type\":\"transfer\",\"attributes\":[{\"key\":\"recipient\",\"value\":\"agoric1ensvnvv03uwedczf5x7y9dcrngk2aaznjgjmrk\"},{\"key\":\"sender\",\"value\":\"agoric10d8rfalktk8n77zzxnqjkrhkw7ka24aw85zlu9\"},{\"key\":\"amount\",\"value\":\"10000ubld\"}]}]}]",
  "logs": [
    {
      "msg_index": 0,
      "log": "",
      "events": [
        {
          "type": "coin_received",
          "attributes": [
            {
              "key": "receiver",
              "value": "agoric1ensvnvv03uwedczf5x7y9dcrngk2aaznjgjmrk"
            },
            {
              "key": "amount",
              "value": "10000ubld"
            }
          ]
        },
        {
          "type": "coin_spent",
          "attributes": [
            {
              "key": "spender",
              "value": "agoric10d8rfalktk8n77zzxnqjkrhkw7ka24aw85zlu9"
            },
            {
              "key": "amount",
              "value": "10000ubld"
            }
          ]
        },
        {
          "type": "message",
          "attributes": [
            {
              "key": "action",
              "value": "/cosmos.bank.v1beta1.MsgSend"
            },
            {
              "key": "sender",
              "value": "agoric10d8rfalktk8n77zzxnqjkrhkw7ka24aw85zlu9"
            },
            {
              "key": "module",
              "value": "bank"
            }
          ]
        },
        {
          "type": "transfer",
          "attributes": [
            {
              "key": "recipient",
              "value": "agoric1ensvnvv03uwedczf5x7y9dcrngk2aaznjgjmrk"
            },
            {
              "key": "sender",
              "value": "agoric10d8rfalktk8n77zzxnqjkrhkw7ka24aw85zlu9"
            },
            {
              "key": "amount",
              "value": "10000ubld"
            }
          ]
        }
      ]
    }
  ],
  "info": "",
  "gas_wanted": "200000",
  "gas_used": "71652",
  "tx": {
    "@type": "/cosmos.tx.v1beta1.Tx",
    "body": {
      "messages": [
        {
          "@type": "/cosmos.bank.v1beta1.MsgSend",
          "from_address": "agoric10d8rfalktk8n77zzxnqjkrhkw7ka24aw85zlu9",
          "to_address": "agoric1ensvnvv03uwedczf5x7y9dcrngk2aaznjgjmrk",
          "amount": [
            {
              "denom": "ubld",
              "amount": "10000"
            }
          ]
        }
      ],
      "memo": "",
      "timeout_height": "0",
      "extension_options": [],
      "non_critical_extension_options": []
    },
    "auth_info": {
      "signer_infos": [
        {
          "public_key": {
            "@type": "/cosmos.crypto.secp256k1.PubKey",
            "key": "A3XhNtSc1fMAWu5TrP+w2uG7y3BImxYNs545g74jJRyI"
          },
          "mode_info": {
            "single": {
              "mode": "SIGN_MODE_DIRECT"
            }
          },
          "sequence": "53"
        }
      ],
      "fee": {
        "amount": [
          {
            "denom": "ubld",
            "amount": "10000"
          }
        ],
        "gas_limit": "200000",
        "payer": "",
        "granter": ""
      },
      "tip": null
    },
    "signatures": [
      "J8eDvJVMhAoIAhclFfUvixXJ21oUDYCUcjBHeFP0YC1g5rvwM72VuIivfFq0T1SUDTIkOY+5+xXUy7r+ST2+wg=="
    ]
  },
  "timestamp": "2024-01-17T14:39:47Z",
  "events": [
    {
      "type": "coin_spent",
      "attributes": [
        {
          "key": "b'spender'",
          "value": "b'agoric10d8rfalktk8n77zzxnqjkrhkw7ka24aw85zlu9'",
          "index": true
        },
        {
          "key": "b'amount'",
          "value": "b'10000ubld'",
          "index": true
        }
      ]
    },
    {
      "type": "coin_received",
      "attributes": [
        {
          "key": "b'receiver'",
          "value": "b'agoric1ae0lmtzlgrcnla9xjkpaarq5d5dfez63h3nucl'",
          "index": true
        },
        {
          "key": "b'amount'",
          "value": "b'10000ubld'",
          "index": true
        }
      ]
    },
    {
      "type": "transfer",
      "attributes": [
        {
          "key": "b'recipient'",
          "value": "b'agoric1ae0lmtzlgrcnla9xjkpaarq5d5dfez63h3nucl'",
          "index": true
        },
        {
          "key": "b'sender'",
          "value": "b'agoric10d8rfalktk8n77zzxnqjkrhkw7ka24aw85zlu9'",
          "index": true
        },
        {
          "key": "b'amount'",
          "value": "b'10000ubld'",
          "index": true
        }
      ]
    },
    {
      "type": "message",
      "attributes": [
        {
          "key": "b'sender'",
          "value": "b'agoric10d8rfalktk8n77zzxnqjkrhkw7ka24aw85zlu9'",
          "index": true
        }
      ]
    },
    {
      "type": "tx",
      "attributes": [
        {
          "key": "b'fee'",
          "value": "b'10000ubld'",
          "index": true
        }
      ]
    },
    {
      "type": "tx",
      "attributes": [
        {
          "key": "b'acc_seq'",
          "value": "b'agoric10d8rfalktk8n77zzxnqjkrhkw7ka24aw85zlu9/53'",
          "index": true
        }
      ]
    },
    {
      "type": "tx",
      "attributes": [
        {
          "key": "b'signature'",
          "value": "b'J8eDvJVMhAoIAhclFfUvixXJ21oUDYCUcjBHeFP0YC1g5rvwM72VuIivfFq0T1SUDTIkOY+5+xXUy7r+ST2+wg=='",
          "index": true
        }
      ]
    },
    {
      "type": "message",
      "attributes": [
        {
          "key": "b'action'",
          "value": "b'/cosmos.bank.v1beta1.MsgSend'",
          "index": true
        }
      ]
    },
    {
      "type": "coin_spent",
      "attributes": [
        {
          "key": "b'spender'",
          "value": "b'agoric10d8rfalktk8n77zzxnqjkrhkw7ka24aw85zlu9'",
          "index": true
        },
        {
          "key": "b'amount'",
          "value": "b'10000ubld'",
          "index": true
        }
      ]
    },
    {
      "type": "coin_received",
      "attributes": [
        {
          "key": "b'receiver'",
          "value": "b'agoric1ensvnvv03uwedczf5x7y9dcrngk2aaznjgjmrk'",
          "index": true
        },
        {
          "key": "b'amount'",
          "value": "b'10000ubld'",
          "index": true
        }
      ]
    },
    {
      "type": "transfer",
      "attributes": [
        {
          "key": "b'recipient'",
          "value": "b'agoric1ensvnvv03uwedczf5x7y9dcrngk2aaznjgjmrk'",
          "index": true
        },
        {
          "key": "b'sender'",
          "value": "b'agoric10d8rfalktk8n77zzxnqjkrhkw7ka24aw85zlu9'",
          "index": true
        },
        {
          "key": "b'amount'",
          "value": "b'10000ubld'",
          "index": true
        }
      ]
    },
    {
      "type": "message",
      "attributes": [
        {
          "key": "b'sender'",
          "value": "b'agoric10d8rfalktk8n77zzxnqjkrhkw7ka24aw85zlu9'",
          "index": true
        }
      ]
    },
    {
      "type": "message",
      "attributes": [
        {
          "key": "b'module'",
          "value": "b'bank'",
          "index": true
        }
      ]
    }
  ]
}

[ivanlei]

Now let's compare against a /cosmos.bank.v1beta1.MsgSend on mainnet

curl -s "https://main.rpc.agoric.net/tx_search?query=\"message.action='/cosmos.bank.v1beta1.MsgSend'\"&prove=false&page=1&per_page=1&order_by=\"desc\"&match_events=true" -H "accept: application/json" | jq -M -c '.result.txs[0].tx_result'  | python3 ~/bin/decoded_events.py
{
  "code": 0,
  "data": "Ch4KHC9jb3Ntb3MuYmFuay52MWJldGExLk1zZ1NlbmQ=",
  "log": "[{\"events\":[{\"type\":\"coin_received\",\"attributes\":[{\"key\":\"receiver\",\"value\":\"agoric158n9q2dxer2aejrxk88v39slum0mckt2ejng6d\"},{\"key\":\"amount\",\"value\":\"2512488103ubld\"}]},{\"type\":\"coin_spent\",\"attributes\":[{\"key\":\"spender\",\"value\":\"agoric155svs6sgxe55rnvs6ghprtqu0mh69keh3wdjs4\"},{\"key\":\"amount\",\"value\":\"2512488103ubld\"}]},{\"type\":\"message\",\"attributes\":[{\"key\":\"action\",\"value\":\"/cosmos.bank.v1beta1.MsgSend\"},{\"key\":\"sender\",\"value\":\"agoric155svs6sgxe55rnvs6ghprtqu0mh69keh3wdjs4\"},{\"key\":\"module\",\"value\":\"bank\"}]},{\"type\":\"transfer\",\"attributes\":[{\"key\":\"recipient\",\"value\":\"agoric158n9q2dxer2aejrxk88v39slum0mckt2ejng6d\"},{\"key\":\"sender\",\"value\":\"agoric155svs6sgxe55rnvs6ghprtqu0mh69keh3wdjs4\"},{\"key\":\"amount\",\"value\":\"2512488103ubld\"}]}]}]",
  "info": "",
  "gas_wanted": "200000",
  "gas_used": "75131",
  "events": [
    {
      "type": "coin_spent",
      "attributes": [
        {
          "key": "b'spender'",
          "value": "b'agoric155svs6sgxe55rnvs6ghprtqu0mh69keh3wdjs4'",
          "index": true
        },
        {
          "key": "b'amount'",
          "value": "b'5000ubld'",
          "index": true
        }
      ]
    },
    {
      "type": "coin_received",
      "attributes": [
        {
          "key": "b'receiver'",
          "value": "b'agoric1ae0lmtzlgrcnla9xjkpaarq5d5dfez63h3nucl'",
          "index": true
        },
        {
          "key": "b'amount'",
          "value": "b'5000ubld'",
          "index": true
        }
      ]
    },
    {
      "type": "transfer",
      "attributes": [
        {
          "key": "b'recipient'",
          "value": "b'agoric1ae0lmtzlgrcnla9xjkpaarq5d5dfez63h3nucl'",
          "index": true
        },
        {
          "key": "b'sender'",
          "value": "b'agoric155svs6sgxe55rnvs6ghprtqu0mh69keh3wdjs4'",
          "index": true
        },
        {
          "key": "b'amount'",
          "value": "b'5000ubld'",
          "index": true
        }
      ]
    },
    {
      "type": "message",
      "attributes": [
        {
          "key": "b'sender'",
          "value": "b'agoric155svs6sgxe55rnvs6ghprtqu0mh69keh3wdjs4'",
          "index": true
        }
      ]
    },
    {
      "type": "tx",
      "attributes": [
        {
          "key": "b'fee'",
          "value": "b'5000ubld'",
          "index": true
        }
      ]
    },
    {
      "type": "tx",
      "attributes": [
        {
          "key": "b'acc_seq'",
          "value": "b'agoric155svs6sgxe55rnvs6ghprtqu0mh69keh3wdjs4/2251'",
          "index": true
        }
      ]
    },
    {
      "type": "tx",
      "attributes": [
        {
          "key": "b'signature'",
          "value": "b'gUpBMbuZ/jQPs5JuMhU2EUwSVWI+/49DNPK9nR8kPRoXmfVEadGHtqF1AqtixYvN+2RJCDYCcL9NRvu9o9MT8w=='",
          "index": true
        }
      ]
    },
    {
      "type": "message",
      "attributes": [
        {
          "key": "b'action'",
          "value": "b'/cosmos.bank.v1beta1.MsgSend'",
          "index": true
        }
      ]
    },
    {
      "type": "coin_spent",
      "attributes": [
        {
          "key": "b'spender'",
          "value": "b'agoric155svs6sgxe55rnvs6ghprtqu0mh69keh3wdjs4'",
          "index": true
        },
        {
          "key": "b'amount'",
          "value": "b'2512488103ubld'",
          "index": true
        }
      ]
    },
    {
      "type": "coin_received",
      "attributes": [
        {
          "key": "b'receiver'",
          "value": "b'agoric158n9q2dxer2aejrxk88v39slum0mckt2ejng6d'",
          "index": true
        },
        {
          "key": "b'amount'",
          "value": "b'2512488103ubld'",
          "index": true
        }
      ]
    },
    {
      "type": "transfer",
      "attributes": [
        {
          "key": "b'recipient'",
          "value": "b'agoric158n9q2dxer2aejrxk88v39slum0mckt2ejng6d'",
          "index": true
        },
        {
          "key": "b'sender'",
          "value": "b'agoric155svs6sgxe55rnvs6ghprtqu0mh69keh3wdjs4'",
          "index": true
        },
        {
          "key": "b'amount'",
          "value": "b'2512488103ubld'",
          "index": true
        }
      ]
    },
    {
      "type": "message",
      "attributes": [
        {
          "key": "b'sender'",
          "value": "b'agoric155svs6sgxe55rnvs6ghprtqu0mh69keh3wdjs4'",
          "index": true
        }
      ]
    },
    {
      "type": "message",
      "attributes": [
        {
          "key": "b'module'",
          "value": "b'bank'",
          "index": true
        }
      ]
    }
  ],
  "codespace": ""
}

[michaelfig]

This issue is caused by the accidental inclusion in the ante handler of both

1. the Cosmos SDK default ante.NewDeductFeeDecorator which sends fees to a hardcoded module account to be distributed amongst validators, and
2. our Agoric custom NewDeductFeeDecorator which sends fees to a configurable account (currently the reserve module account).

The solution is to reincorporate the Cosmos decorator into our custom decorator, and use only that one in the ante handler.

[gibson042]

Update: this would be fixed by #9045 or its #9049 replacement, but the current plan is to instead apply our "configurable fee collector account" logic at the cosmos-sdk layer in our fork: agoric-labs/cosmos-sdk#407 , #9051.

This will eliminate the need for Agoric/agoric-sdk to maintain its own modified copy of DeductFeeDecorator, allowing future maintenance of that functionality to better track upstream changes.