Unable to get transaction rerunning information: error processing transaction: user error: gas mismatch between receipt and actual gas used, message: Rerun gas mismatch for transaction

[achiko]

Scenario :

Local ganache in forked mode.
Ganache Command :

ganache-cli -f <my rpc url> -i 1

Yaml Config

account_id: <my account id>
project_slug: <my project name> 

exports: # running tenderly export will export local transaction to the provided project
  my-network:
    project_slug: <my project name>
    rpc_address: 127.0.0.1:8545 // local ganache node in forked mode
    protocol: http
    forked_network: mainnet
    chain_config:
      homestead_block: 0 # (default 0)
      eip150_block: 0 # (default 0)
      eip150_hash: 0x0 # (default 0x0)
      eip155_block: 0 # (default 0)
      eip158_block: 0 # (default 0)
      byzantium_block: 0 # (default 0)
      constantinople_block: 0 # (default 0)
      petersburg_block: 0 # (default 0)
      istanbul_block: 0 # (default 0)

Command :
tenderly export 0x2f226311b066a0361e1505319e5061087247199bce513f5f02577f87978c6138 --debug

Error :

Collecting network information...
Collecting transaction information...

Unable to get transaction rerunning information: error processing transaction: user error: gas mismatch between receipt and actual gas used, message: Rerun gas mismatch for transaction 0x2f226311b066a0361e1505319e5061087247199bce513f5f02577f87978c6138. This can happen when the chain config is incorrect or the local node is not running the latest version.

Please check which hardfork is active on your local node. If you are not running the newest fork, comment out the forks block in tenderly.yaml

[BogdanHabic]

Hey @achiko,

There is an issue in ganache-core triggering this behavior. This is an edge-case that occurs only when --fork is used.

We plan on fixing this issue soon, and once that's merged, this issue should be fixed as well.

I'll notify you here once the fix in ganache is live!

[BogdanHabic]

Hey @achiko, the Ganache team did a lot of work on the forking feature. Can you try exporting the transaction now?

[pyggie]

I am seeing the same failure with the Hardhat Tenderly plugin.

Environment:
NodeJS 10.21
Tenderly CLI 0.9.4

Selected packages from package.json:

    "@nomiclabs/hardhat-ethers": "^2.0.0",
    "@nomiclabs/hardhat-waffle": "^2.0.0",
    "@nomiclabs/hardhat-web3": "^2.0.0",
    "@openzeppelin/cli": "^2.8.2",
    "@tenderly/hardhat-tenderly": "^1.0.3",
    "hardhat": "^2.0.2",
    "hardhat-deploy": "^0.7.0-beta.28",
    "hardhat-typechain": "^0.3.3",

Note that I am not using OZ or truffle to compile/deploy. I am using hardhat and the hardhat-deploy plugin.

I have configured the Hardhat EVM to run as a mainnet fork. From hardhat.config.ts:

networks: {
    hardhat: {
      forking: {
        url: <a mainnet archive node>,
        blockNumber: 11190000
      }
    ...

After following these steps:

tenderly login
tenderly export init
npx hardhat node

The tenderly.yaml file:

$ cat tenderly.yaml
account_id: ""
exports:
  hardhat:
    project_slug: .../...
    rpc_address: 127.0.0.1:8545              [this refers to the hardhat evm]
    protocol: ""
    forked_network: Mainnet
    chain_config:
      homestead_block: 0
      eip150_block: 0
      eip150_hash: "0x0000000000000000000000000000000000000000000000000000000000000000"
      eip155_block: 0
      eip158_block: 0
      byzantium_block: 0
      constantinople_block: 0
      petersburg_block: 0
      istanbul_block: 0
project_slug: ""

Now I want to export one of the transactions that was executed during a test case.

npx hardhat test test/SomeTestCase...
tenderly export --debug 0x....
Collecting network information...

Collecting transaction information...

Unable to get transaction rerunning information: error processing transaction: user error: gas mismatch between receipt and actual gas used, message: Rerun gas mismatch for transaction 0x.... This can happen when the chain config is incorrect or the local node is not running the latest version.

Please check which hardfork is active on your local node. If you are not running the newest fork, comment out the forks block in tenderly.yaml.

Transaction processing failed. To see more info about this error, please run this command with the --debug flag.

[BogdanHabic]

Just to have a trace here as well, we're discussing the issue over Discord.

[pyggie]

I tried to create a reproduction on a near-empty repo. Instead of replicating the error above, I ran into two other problems, earlier in the process. I saved each environment with a README to a public repo here:

https://github.com/pyggie/hardhat-tests/tree/tenderly-16 - without openzeppelin, fails looking for config

https://github.com/pyggie/hardhat-tests/tree/tenderly-16-oz - uses openzeppelin, finds config but fails with unable to apply message: insufficient balance to pay for gas

[Malcador]

Hey @pyggie, we've looked into this and it seems like there are a few problems, so I'll address them here.

For this repo, the issue is that we don't support ts files as we can't parse them on our client. We will look into implementing this. I've worked around this by making a hardhat.config.js file (we use this to check if the current directory is of a format and provider we support, and to extract some meta information like the solidity compiler version).

For the second repo, it seems like there are a few issues with the Hardhat rpc node. One of them (the one you are experiencing) is related to this issue: NomicFoundation/hardhat#1004
In short the balance returned by the node is 0 when it should be higher, causing the error you get.

I can offer a workaround, as this seems to only happen on the first block, if you simply repeated the tests and used that hash, without restarting the node, this error wouldn't happen.

This exports the transaction to our platform properly, however doing this I've encountered another issue with the node returning an invalid nonce for the deployed contract, causing the exported transaction to fail when rerun by our backend.
(NomicFoundation/hardhat#1005)

We're still fiddling around with this and i will keep you posted, and I will link any further relevant issues.

Thanks for the detailed examples!

[sebastiantf]

Hey there. I've started seeing this now with hardhat mainnet fork. Any updates on this?

[nebojsa94]

Both NomicFoundation/hardhat#1004 and NomicFoundation/hardhat#1005 issues are resolved.

The export command is fully compatible with the hardhat node when forking is enabled, you might want to ensure that chain_config in tenderly.yaml is up to date

[trajan0x]

We're seeing the same issue. As soon as this module is used, all subsequent transactions on that account return an invalid nonce. Do the evm.NewProcesser calls actually re-broadcast the transaction/otherwise affect the rpc server state?

[0xdavinchee]

This is not a real solution, but I've been able to get it to work by calling tenderly export <TXN-HASH> --force. Seems to still properly display all the information I need in the tenderly dashboard.

[nebojsa94]

Hey @0xNero, sorry for the late response,

Tenderly CLI doesn't perform any write requests to the RPC node, it is just querying the state and reprocessing the transaction internally. When you say returning an invalid nonce, do you mean metamask?

[nebojsa94]

Hey @0xdavinchee,

Force flag will ignore some checks that are performed in the export command, this can sometimes result in an invalid trace on the dashboard, I'd strongly suggest trying to resolve the error first. It is usually due to incorrect chain config in tenderly.yaml. Can you try adding berlin_block: 0?

[0xdavinchee]

Hey @nebojsa94, that did the trick, thank you! Just curious why does this fix it?

[nebojsa94]

Hey @0xdavinchee,

The Berlin HF introduces gas repricing for SLOAD/SSTORE opcodes, if your local node is running on berlin HF and the CLI on Istanbul HF then in most cases there is going to be a gas usage mismatch between these two transaction executions. Unfortunately, there is no easy way to ask a node which fork is active, therefore we need this info in tenderly/.yaml

P.S. CLI version 1.1.1 adds berlin_block: 0 in default chain config when running tenderly export init