rpc eth_getTransactionByHash and eth_getTransactionReceipt inconsistency after reorgs

[NickKelly1]

System information

Geth version: geth version 1.13.11-stable-8f7eb9cc
CL client & version: beacon-chain version Prysm/v4.2.1/59b310a2216c57fcf67ea0fdec739dad07aeec8b.
OS & Version: Linux Ubuntu 22.04.3 LTS amd64
Commit hash : (not develop)

Initially synced with --syncmode snap, switched to --syncmode full and --gcmode archive after initial sync completed. Always --state.scheme hash and --db.engine pebble.

I have an older Geth node synced the same way running version 1.12.0-stable-e501b3b0 that does not experience this issue.

Expected behaviour

eth_getTransactionByHash and eth_getTransactionReceipt should always return the correct data, including after a reorg.

Actual behaviour

Immediately & shortly after a reorg eth_getTransactionByHash and eth_getTransactionReceipt are incorrect. In some circumstances they remain incorrect forever. eth_getBlockByNumber and eth_getBlockReceipts return the correct data even when eth_getTransactionByHash and eth_getTransactionReceipt return incorrect data.

Last week I snap synced a new Geth node on version 1.13.11-stable-8f7eb9cc. After switching my application to use it and I started noticing the above inconsistencies. Yesterday I implemented extra checks that restart my application and rewind 10 blocks when the issue occurs and everything seemed fine after that (the correct data would be available from eth_getTransactionReceipt when requested after my application restarted). Moments ago I was digging deeper to understand the problem, hitting the node with lots of eth_getBlockReceipts requests after a reorg, and it appears to have caused the bad data to stick permanently.

For reference, Etherscan keeps a list of recent reorgs here https://etherscan.io/blocks_forked

Steps to reproduce the behaviour

Subscribe to new blocks and check their receipts via eth_getBlockReceipts to see whether the block hashes match.

For a minimal reproduction see https://github.com/NickKelly1/geth-rpc-tx-receipt-issue-reproduction.

[NickKelly1]

Looks like it might be the same issue as #28938, #28918, #28885 which has a fix merged here #28865

Nothing about it in the latest release notes for Edolus (v1.13.12) from a week ago. Upgrading to that version to find out if it's fixed.

[lightclient]

This was fixed in 1.13.12, you can see it included in the full list of milestones. Please reopen if you continue seeing an issue.

[NickKelly1]

This was fixed in 1.13.12, you can see it included in the full list of milestones. Please reopen if you continue seeing an issue.

Thanks! Confirmed as fixed in 1.13.12