StateGetBeaconEntry uses drand client to look up round instead of using tipset data #12414
Comments
@rvagg I was getting different results here on every round, not just null: https://github.com/data-preservation-programs/spade/blob/22dc7dc0ba81b/webapi/auth.go#L268-L280 |
|
Retested it now out of curiosity. It might be that I was tripped up by Test with recent null as of writing this (to be pasted into terminal with local running API) Result: |
|
Behavior on calibnet with multiple consecutive |
|
There's a whole infrastructure in the FVM randomness syscall package for picking out the beacon from the chain, entering here and forking depending on network version: Lines 180 to 190 in dbef5de We just need a bit of historical knowledge of this API to inform the decision here. @Stebalien @Kubuxu might know whether it's safe to switch from getting the beacon from drand directly to getting it off what we have on the chain. |
|
Some historical explanation for the NV13 special casing: #3613 |
|
We discussed this offline - I think this api is implemented wrongly and I’d expect this to return what’s (beacon entries) in the block header. |
|
Some additional context:
The blocking behaviour for future randomness means that it is easier to implement as a call directly into drand. But we end up with the property that the beacon entry returned from this will be different from historical beacon entries in a number of epoch ranges where we fetch them off chain, as used in |
|
A wrinkle: Lines 271 to 280 in 4a4ddaa The block mining process relies on So, even if we decide to "fix" this API for historical epochs, we need it to do query drand for But maybe that also doesn't matter if we're able to fetch the correct round from the tipset anyway, we know it's "available" and that's all the call is doing in this case, it's not even getting the value. |
|
Results of spending far too much time figuring out the behaviour of the beacon-from-chain behaviour: Filecoin Drand Beacon Randomness from ChainAs used in the FVM syscall BackgroundFor each tipset, we expect all blocks to have the same One special case in code is that when there is a change of beacons (drand networks) and the new beacon is "chained" (i.e. not quicknet, because Filecoin no longer tracks connected chain of rounds), an entry will be recorded for both the previous and the new beacons in the new block header. Since quicknet (Phoenix @ 3855480 (2024-04-11T15:00:00Z)), there is not a 1:1 correspondence between drand rounds and filecoin epochs since drand moves in 3s intervals and filecoin in 30s intervals. This means that we no longer store the full "chain" of drand rounds. It also means that any applications relying on specific drand rounds to be available via Network versions 0-12: < Hyperdrive(13) @ 892800 (2021-06-30T22:00:00Z)
This results in a mismatch between the epoch and the beacon round for null rounds since we only look backward on the chain. Network version 13: >= Hyperdrive(13) @ 892800 (2021-06-30T22:00:00Z) < Chocolate(14) 1231620 (2021-10-26T13:30:00Z)
Additional context:
This still results in a mismatch between the epoch and the beacon round for null rounds since we only fetch the last Network version 14: >= Chocolate(14) 1231620 (2021-10-26T13:30:00Z)
|
|
Some more comments @ filecoin-project/FIPs#1051 (comment) I would like to test this if we end up implementing this call as chain-beacon-lookup using the existing logic:
-- I think the path forward here is twofold:
The outstanding question I think is whether to use |
|
Continuing my comment @ filecoin-project/FIPs#1051 (comment) about the |
Ref: #12414 Previously StateGetBeaconEntry would always try and use a drand beacon to get the appropriate round. But as drand has shut down old beacons and we've removed client details from Lotus, it has stopped working for historical beacons. This fix restores historical beacon entries by using the on-chain lookup, however it now follows the rules used by StateGetRandomnessFromBeacon and the get_beacon_randomness syscall which has some quirks with null rounds prior to nv14. See #12414 (comment) for specifics. StateGetBeaconEntry still blocks for future epochs and uses live drand beacon clients to wait for and fetch rounds as they are available.
Fixes: #12414 Previously StateGetBeaconEntry would always try and use a drand beacon to get the appropriate round. But as drand has shut down old beacons and we've removed client details from Lotus, it has stopped working for historical beacons. This fix restores historical beacon entries by using the on-chain lookup, however it now follows the rules used by StateGetRandomnessFromBeacon and the get_beacon_randomness syscall which has some quirks with null rounds prior to nv14. See #12414 (comment) for specifics. StateGetBeaconEntry still blocks for future epochs and uses live drand beacon clients to wait for and fetch rounds as they are available.
…pochs Fixes: #12414 Previously StateGetBeaconEntry would always try and use a drand beacon to get the appropriate round. But as drand has shut down old beacons and we've removed client details from Lotus, it has stopped working for historical beacons. This fix restores historical beacon entries by using the on-chain lookup, however it now follows the rules used by StateGetRandomnessFromBeacon and the get_beacon_randomness syscall which has some quirks with null rounds prior to nv14. See #12414 (comment) for specifics. StateGetBeaconEntry still blocks for future epochs and uses live drand beacon clients to wait for and fetch rounds as they are available.
|
This will be closed when I can get a review to get #12428 landed |
…pochs Fixes: #12414 Previously StateGetBeaconEntry would always try and use a drand beacon to get the appropriate round. But as drand has shut down old beacons and we've removed client details from Lotus, it has stopped working for historical beacons. This fix restores historical beacon entries by using the on-chain lookup, however it now follows the rules used by StateGetRandomnessFromBeacon and the get_beacon_randomness syscall which has some quirks with null rounds prior to nv14. See #12414 (comment) for specifics. StateGetBeaconEntry still blocks for future epochs and uses live drand beacon clients to wait for and fetch rounds as they are available.
…pochs Fixes: #12414 Previously StateGetBeaconEntry would always try and use a drand beacon to get the appropriate round. But as drand has shut down old beacons and we've removed client details from Lotus, it has stopped working for historical beacons. This fix restores historical beacon entries by using the on-chain lookup, however it now follows the rules used by StateGetRandomnessFromBeacon and the get_beacon_randomness syscall which has some quirks with null rounds prior to nv14. See #12414 (comment) for specifics. StateGetBeaconEntry still blocks for future epochs and uses live drand beacon clients to wait for and fetch rounds as they are available.
…pochs Fixes: #12414 Previously StateGetBeaconEntry would always try and use a drand beacon to get the appropriate round. But as drand has shut down old beacons and we've removed client details from Lotus, it has stopped working for historical beacons. This fix restores historical beacon entries by using the on-chain lookup, however it now follows the rules used by StateGetRandomnessFromBeacon and the get_beacon_randomness syscall which has some quirks with null rounds prior to nv14. See #12414 (comment) for specifics. StateGetBeaconEntry still blocks for future epochs and uses live drand beacon clients to wait for and fetch rounds as they are available.
…pochs Fixes: #12414 Previously StateGetBeaconEntry would always try and use a drand beacon to get the appropriate round. But as drand has shut down old beacons and we've removed client details from Lotus, it has stopped working for historical beacons. This fix restores historical beacon entries by using the on-chain lookup, however it now follows the rules used by StateGetRandomnessFromBeacon and the get_beacon_randomness syscall which has some quirks with null rounds prior to nv14. See #12414 (comment) for specifics. StateGetBeaconEntry still blocks for future epochs and uses live drand beacon clients to wait for and fetch rounds as they are available.
…pochs (#12428) * fix(drand): `StateGetBeaconEntry` uses chain beacons for historical epochs Fixes: #12414 Previously StateGetBeaconEntry would always try and use a drand beacon to get the appropriate round. But as drand has shut down old beacons and we've removed client details from Lotus, it has stopped working for historical beacons. This fix restores historical beacon entries by using the on-chain lookup, however it now follows the rules used by StateGetRandomnessFromBeacon and the get_beacon_randomness syscall which has some quirks with null rounds prior to nv14. See #12414 (comment) for specifics. StateGetBeaconEntry still blocks for future epochs and uses live drand beacon clients to wait for and fetch rounds as they are available. * fixup! fix(drand): `StateGetBeaconEntry` uses chain beacons for historical epochs * fixup! fix(drand): `StateGetBeaconEntry` uses chain beacons for historical epochs
…pochs (#12428) * fix(drand): `StateGetBeaconEntry` uses chain beacons for historical epochs Fixes: #12414 Previously StateGetBeaconEntry would always try and use a drand beacon to get the appropriate round. But as drand has shut down old beacons and we've removed client details from Lotus, it has stopped working for historical beacons. This fix restores historical beacon entries by using the on-chain lookup, however it now follows the rules used by StateGetRandomnessFromBeacon and the get_beacon_randomness syscall which has some quirks with null rounds prior to nv14. See #12414 (comment) for specifics. StateGetBeaconEntry still blocks for future epochs and uses live drand beacon clients to wait for and fetch rounds as they are available. * fixup! fix(drand): `StateGetBeaconEntry` uses chain beacons for historical epochs * fixup! fix(drand): `StateGetBeaconEntry` uses chain beacons for historical epochs
…pochs (#12428) * fix(drand): `StateGetBeaconEntry` uses chain beacons for historical epochs Fixes: #12414 Previously StateGetBeaconEntry would always try and use a drand beacon to get the appropriate round. But as drand has shut down old beacons and we've removed client details from Lotus, it has stopped working for historical beacons. This fix restores historical beacon entries by using the on-chain lookup, however it now follows the rules used by StateGetRandomnessFromBeacon and the get_beacon_randomness syscall which has some quirks with null rounds prior to nv14. See #12414 (comment) for specifics. StateGetBeaconEntry still blocks for future epochs and uses live drand beacon clients to wait for and fetch rounds as they are available. * fixup! fix(drand): `StateGetBeaconEntry` uses chain beacons for historical epochs * fixup! fix(drand): `StateGetBeaconEntry` uses chain beacons for historical epochs
Reported by @LesnyRumcajs:
In more detail, running calibnet comparisons with forest on RPC calls:
I believe this is due to removal of clients for looking up historic beacons.
Mainnet uses 3 different Drand networks: incentinet, mainnet and quicknet. We switched to quicknet in nv22. Calibnet only uses mainnet and quicknet. Client details for incentinet were removed in #10476, mainnet were removed more recently, in #12272.
I don't believe drand are on the hook for maintaining ancient beacons, incentinet is entirely gone afaik so you couldn't even look that far back. Mainnet probably too someday.
What confuses me about this API is that it takes an epoch and then does a remote look-up even though we should have it in the chainstore hanging off each block.
TipSet->BlockHeader[]->BeaconEntries[]. I'm just not clear on which would we would return if we looked it up from there. Logic to build it is here:lotus/chain/beacon/beacon.go
Line 115 in dbef5de
Perhaps there's null-round implications of trying to use the tipsets for this?
The text was updated successfully, but these errors were encountered: