eth_syncing returns 'false' while Geth node is still syncing

[crystalgarden]

System information

Geth version: 1.10.21-stable
OS & Version: Linux

Geth is paired with Prysm v2.1.4

Expected behaviour

While geth node is still syncing, eth_syncing API should return accurate syncing status with currentBlock and highestBlock info

Actual behaviour

eth_syncing returns 'false', while eth_blockNumber is still behind the latest block.

Steps to reproduce the behaviour

Stop Prysm/Geth to let the node behind the blockchain. Restart Prysm/Geth, curl eth_syncing and eth_blockNumber APIs.

We tested in both Goerli testnet and mainnet, and observed the same behavior. Is this an expected behavior post merge? If yes, what's the reliable mechanism to determine if Geth is fully in sync with the blockchain?

[holiman]

The eth_syncing is set to true by the downloader, when it downloads the N blocks it needs from peers. If you have fallen behind, that can be a pretty quick thing.
Just because "a higher block exists somewhere", that doesn't mean eth_syncing becomes set to true. We look only at the local behaviour: are we in a sync-cycle?

[crystalgarden]

I did one more testing this morning on Goerli. My Geth/Prysm was stopped overnight and restarted in the morning. Etherscan block height was at 7426914 when I started the testing. Please see below output from Geth JavaScript console. The eth.blockNumber showed the geth node was in syncing, while eth_syncing constantly returned false. Did I miss anything?

$ geth attach /data/goerli/geth.ipc
Welcome to the Geth JavaScript console!

instance: Geth/v1.10.21-stable-67109427/linux-amd64/go1.18.4
at block: 7421567 (Wed Aug 17 2022 14:31:48 GMT+0000 (UTC))
datadir: /data/goerli
modules: admin:1.0 clique:1.0 debug:1.0 engine:1.0 eth:1.0 miner:1.0 net:1.0 personal:1.0 rpc:1.0 txpool:1.0 web3:1.0

To exit, press ctrl-d or type exit

eth.syncing
false
admin.peers
[{
caps: ["eth/66", "eth/67", "snap/1"],
enode: "enode://0ff6abad39d72fc6bd901d29535f136ba6752466d6a8f33066f9a6e9e824460386a8f770ea1bdf746532c41b1621364a08c438ed8b8664fe3f1ec7598fbf4724@18.183.253.187:30303",
id: "da9fa107e2e72927045fb08c1d647a35a81cf7477fa18bafb29ff5f3a8e1a5dc",
name: "Geth/v1.10.21-stable/linux-amd64/go1.18.4",
network: {
inbound: false,
localAddress: "10.244.57.7:34532",
remoteAddress: "18.183.253.187:30303",
static: false,
trusted: false
},
protocols: {
eth: {
difficulty: 10790000,
head: "0x1802040fd27064383790ce687a46505cff4217e543629d87c36141a13aeb07a7",
version: 67
},
snap: {
version: 1
}
}
}]

eth.blockNumber
7421924
eth.blockNumber
7421977
eth.blockNumber
7422030
eth.syncing
false
eth.blockNumber
7422085
eth.syncing
false
eth.blockNumber
7422137
eth.syncing
false

[karalabe]

Post merge there's a weird new quirk. If the beacon client tells us "hey, sync your chain to this new HEAD", then we will spin up a new sync cycle and do all the usual reportings. However, what usually happens is that the beacon client retrieves the (beacon) blocks 1 by 1, unpacks the payload and feeds it to us, 1 by 1. In that case, we're never really syncing, we're just receiving 1 new block each time, which we import.

Perhaps we could (after the merge) visit this issue and extend the newpayload API to feed some more metadata from the beacon client to the execution so we might also know what's happening.