chainSpec_lightSyncState: Provide light-client sync state

[lexnv]

What would be the most appropriate way to fetch the chain spec needed for the light-client with the new V2 APIs?

Although there's been some discussion about a standard format for this checkpoint

• Light sync state needs a proper format polkadot-sdk#60
• Support syncing from LightSyncState polkadot-sdk#12

Ideally, we would stabilize the format of the sync-state before including it in the chainSpec class of functions.

Opening this to gather your opinion since we are also moving to stabilize the V1 of the chainSpec in

• Stabilize chainSpec-prefixed functions #59

Some advantages will be that

• we'll have the functionality needed to migrate the ecosystem to the RPC-V2 methods (Are you aware of any other methods we'd need 100% to keep parity with the old APIs?)
• the chainSpec will be versioned and V1 will always return the same result (would avoid breaking changes for smoldot as in - Warp sync part II substrate#9284)

While in constant change, we might needlessly burn multiple chainSpec versions until we find a final syncState

// @paritytech/subxt-team

[tomaka]

This function wouldn't belong to the chainSpec prefix.
The chainSpec provides information that is found in the chain spec. The "sync state" isn't found in the chain spec, it's generated by the node. If you query the "sync state" for example every 30 seconds, you'll get a different result every 30 seconds.

[jsdw]

Would it make sense to add an "unstable" method to the new JSON-RPC spec which obtains the chain spec in whatever format currently exists*. This way, we could have a "new" RPC for doing this but leave things open to the chain spec format being properly specified and potentially changing?

* ie one that both Smoldot and Substrate nodes could produce which happens to work at present when handed to Smoldot

[tomaka]

Smoldot isn't really able to generate a chain spec checkpoint in the format that currently exists, because that format is a hack and contains additional unnecessary data that smoldot can't really obtain in a sane way.

[lexnv]

I did like the checkpoint idea because it was agnostic of the consensus algorithm.

Do you think there's a better format which smoldot could easily construct from the up-to-date parameters of the chain? (cc paritytech/polkadot-sdk#60 (comment))?

And if smoldot could not generate the checkpoint without trusting some nodes, do you think it would be still appropriate as a first unstable version (cc paritytech/polkadot-sdk#3271)?
I'm a bit concerned that the current substrate chainSpec will grow without a proper spec 🤔

[tomaka]

Do you think there's a better format which smoldot could easily construct from the up-to-date parameters of the chain? (cc paritytech/polkadot-sdk#60 (comment))?

This is a question for paritytech/polkadot-sdk#60 that I think has already been answered in paritytech/polkadot-sdk#60.

And if smoldot could not generate the checkpoint without trusting some nodes

I didn't mention anywhere that smoldot had to trust some nodes. paritytech/polkadot-sdk#60 (comment) mentions having to download something from full nodes, which in itself is problematic because that makes generating the chain spec an asynchronous operation that can potentially fail, instead of an immediate operation that always succeeds which would be way more convenient.