Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add substrate connect support to staking dashboard #166

Merged
merged 20 commits into from
Aug 15, 2022

Conversation

wirednkod
Copy link

@wirednkod wirednkod commented Jul 9, 2022

Add light client (substrate connect) support to available chains (Polkadot, Kusama, Westend);
Fixes #149;

Some screenshots that show the design result as well:

Screenshot 2022-07-10 at 3 06 59 AM

Screenshot 2022-07-10 at 3 07 39 AM

@wirednkod wirednkod requested a review from rossbulat July 9, 2022 23:50
@wirednkod
Copy link
Author

wirednkod commented Jul 10, 2022

I can see that there are some numbers (e.g. Minimum Active Bond, Nominations (active, Inactive, list), Active Validators number) that is not returned to the dashboard when light client is used.
Im not sure if thats a light client's issue or the way PolkadotJS is implemented (and we use it with ScProbiver).

I'll add a reminder to myself for that - and we should probably put some "experimental" asterisk when light client is used (as we did with polkadotJS/apps)

@rossbulat
Copy link
Collaborator

rossbulat commented Jul 10, 2022

I can see that there are some numbers (e.g. Minimum Active Bond, Nominations (active, Inactive, list), Active Validators number) that is not returned to the dashboard when light client is used. Im not sure if thats a light client's issue or the way PolkadotJS is implemented (and we use it with ScProbiver).

I'll add a reminder to myself for that - and we should probably put some "experimental" asterisk when light client is used (as we did with polkadotJS/apps)

These stats are derived from erasStakers, which is passed to a web worker to loop through the exposures and accumulate the stats.

The fetching is done here, and the callback defined here.

Is the light client fetching erasStakers successfully?

const _exposures = await api.query.staking.erasStakers.entries(
   metrics.activeEra.index
);

@wirednkod
Copy link
Author

I can see that there are some numbers (e.g. Minimum Active Bond, Nominations (active, Inactive, list), Active Validators number) that is not returned to the dashboard when light client is used. Im not sure if thats a light client's issue or the way PolkadotJS is implemented (and we use it with ScProbiver).
I'll add a reminder to myself for that - and we should probably put some "experimental" asterisk when light client is used (as we did with polkadotJS/apps)

These stats are derived from erasStakers, which is passed to a web worker to loop through the exposures and accumulate the stats.

The fetching is done here, and the callback defined here.

Is the light client fetching erasStakers successfully?

const _exposures = await api.query.staking.erasStakers.entries(
   metrics.activeEra.index
);

This fixes the issue. Next version of substrate-connect that will be released today will have that in.

@wirednkod
Copy link
Author

wirednkod commented Jul 27, 2022

Added latest version of rpc-provider and stylised accordingly the light client based on the left menubar.

Indication of "light" next to sidebar logo when light client is active have been added.

Screenshot 2022-07-27 at 6 46 40 PM

In addition localStorage has a new item called isLightClient in order to keep the latest choice of user.

@rossbulat
Copy link
Collaborator

Hi Nikos, I am having issues running this in Brave browser. The syncing hangs with 0 peers. Do we know what is causing this? On Chrome it is working well. Screenshots for context:

light1

Screenshot 2022-08-09 at 14 33 12

Latest logs:

...
14:33:59 427 Debug - json-rpc-polkadot JSON-RPC <= {"jsonrpc":"2.0","id":"health-check:96422","result":{"isSyncing":false,"peers":0,"shouldHavePeers":t…
14:33:59 430 Debug - json-rpc-ksmcc3 JSON-RPC <= {"jsonrpc":"2.0","id":"health-check:96419","result":{"isSyncing":false,"peers":0,"shouldHavePeers":t…
14:33:59 430 Debug - json-rpc-rococo_v2_2 JSON-RPC <= {"jsonrpc":"2.0","id":"health-check:96420","result":{"isSyncing":false,"peers":0,"shouldHavePeers":t…
14:33:59 431 Debug - json-rpc-westend2 JSON-RPC <= {"jsonrpc":"2.0","id":"health-check:96421","result":{"isSyncing":false,"peers":0,"shouldHavePeers":t…

@rossbulat
Copy link
Collaborator

rossbulat commented Aug 9, 2022

@wirednkod Perhaps a separate issue, I am also getting the following console errors on yarn start:

WARNING in ./node_modules/@substrate/connect-extension-protocol/dist/mjs/index.js
Module Warning (from ./node_modules/source-map-loader/dist/cjs.js):
Failed to parse source map from '/Users/macbookpro/polkadot/polkadot-staking-dashboard/node_modules/@substrate/connect-extension-protocol/src/index.ts' file: Error: ENOENT: no such file or directory, open '/Users/macbookpro/polkadot/polkadot-staking-dashboard/node_modules/@substrate/connect-extension-protocol/src/index.ts'
 @ ./node_modules/@substrate/connect/dist/mjs/connector/index.js 3:0-71 22:141-155
 @ ./node_modules/@substrate/connect/dist/mjs/index.js 86:0-37 86:0-37
 @ ./node_modules/@polkadot/rpc-provider/substrate-connect/ScProvider.js 3:0-68 14:46-60 16:0-26 67:78-92
 @ ./node_modules/@polkadot/rpc-provider/substrate-connect/index.js 3:0-32 3:0-32
 @ ./node_modules/@polkadot/rpc-provider/bundle.js 6:0-58 6:0-58
 @ ./node_modules/@polkadot/rpc-provider/index.js 4:0-28 4:0-28
 @ ./node_modules/@polkadot/api/bundle.js 5:0-66 5:0-66 5:0-66
 @ ./node_modules/@polkadot/api/index.js 4:0-28 4:0-28
 @ ./src/contexts/Api/index.tsx 10:0-55 70:21-31 112:26-36
 @ ./src/Providers.tsx 14:0-53 57:6-12 77:20-26 81:39-50
 @ ./src/App.tsx 8:0-38 21:35-44
 @ ./src/index.tsx 8:0-22 16:34-37

WARNING in ./node_modules/@substrate/connect/dist/mjs/WellKnownChain.js
Module Warning (from ./node_modules/source-map-loader/dist/cjs.js):
Failed to parse source map from '/Users/macbookpro/polkadot/polkadot-staking-dashboard/node_modules/@substrate/connect/src/WellKnownChain.ts' file: Error: ENOENT: no such file or directory, open '/Users/macbookpro/polkadot/polkadot-staking-dashboard/node_modules/@substrate/connect/src/WellKnownChain.ts'
 @ ./node_modules/@substrate/connect/dist/mjs/index.js 85:0-53 85:0-53
 @ ./node_modules/@polkadot/rpc-provider/substrate-connect/ScProvider.js 3:0-68 14:46-60 16:0-26 67:78-92
 @ ./node_modules/@polkadot/rpc-provider/substrate-connect/index.js 3:0-32 3:0-32
 @ ./node_modules/@polkadot/rpc-provider/bundle.js 6:0-58 6:0-58
 @ ./node_modules/@polkadot/rpc-provider/index.js 4:0-28 4:0-28
 @ ./node_modules/@polkadot/api/bundle.js 5:0-66 5:0-66 5:0-66
 @ ./node_modules/@polkadot/api/index.js 4:0-28 4:0-28
 @ ./src/contexts/Api/index.tsx 10:0-55 70:21-31 112:26-36
 @ ./src/Providers.tsx 14:0-53 57:6-12 77:20-26 81:39-50
 @ ./src/App.tsx 8:0-38 21:35-44
 @ ./src/index.tsx 8:0-22 16:34-37

WARNING in ./node_modules/@substrate/connect/dist/mjs/connector/extension.js
Module Warning (from ./node_modules/source-map-loader/dist/cjs.js):
Failed to parse source map from '/Users/macbookpro/polkadot/polkadot-staking-dashboard/node_modules/@substrate/connect/src/connector/extension.ts' file: Error: ENOENT: no such file or directory, open '/Users/macbookpro/polkadot/polkadot-staking-dashboard/node_modules/@substrate/connect/src/connector/extension.ts'
 @ ./node_modules/@substrate/connect/dist/mjs/connector/index.js 2:0-69 31:51-68
 @ ./node_modules/@substrate/connect/dist/mjs/index.js 86:0-37 86:0-37
 @ ./node_modules/@polkadot/rpc-provider/substrate-connect/ScProvider.js 3:0-68 14:46-60 16:0-26 67:78-92
 @ ./node_modules/@polkadot/rpc-provider/substrate-connect/index.js 3:0-32 3:0-32
 @ ./node_modules/@polkadot/rpc-provider/bundle.js 6:0-58 6:0-58
 @ ./node_modules/@polkadot/rpc-provider/index.js 4:0-28 4:0-28
 @ ./node_modules/@polkadot/api/bundle.js 5:0-66 5:0-66 5:0-66
 @ ./node_modules/@polkadot/api/index.js 4:0-28 4:0-28
 @ ./src/contexts/Api/index.tsx 10:0-55 70:21-31 112:26-36
 @ ./src/Providers.tsx 14:0-53 57:6-12 77:20-26 81:39-50
 @ ./src/App.tsx 8:0-38 21:35-44
 @ ./src/index.tsx 8:0-22 16:34-37

WARNING in ./node_modules/@substrate/connect/dist/mjs/connector/index.js
Module Warning (from ./node_modules/source-map-loader/dist/cjs.js):
Failed to parse source map from '/Users/macbookpro/polkadot/polkadot-staking-dashboard/node_modules/@substrate/connect/src/connector/index.ts' file: Error: ENOENT: no such file or directory, open '/Users/macbookpro/polkadot/polkadot-staking-dashboard/node_modules/@substrate/connect/src/connector/index.ts'
 @ ./node_modules/@substrate/connect/dist/mjs/index.js 86:0-37 86:0-37
 @ ./node_modules/@polkadot/rpc-provider/substrate-connect/ScProvider.js 3:0-68 14:46-60 16:0-26 67:78-92
 @ ./node_modules/@polkadot/rpc-provider/substrate-connect/index.js 3:0-32 3:0-32
 @ ./node_modules/@polkadot/rpc-provider/bundle.js 6:0-58 6:0-58
 @ ./node_modules/@polkadot/rpc-provider/index.js 4:0-28 4:0-28
 @ ./node_modules/@polkadot/api/bundle.js 5:0-66 5:0-66 5:0-66
 @ ./node_modules/@polkadot/api/index.js 4:0-28 4:0-28
 @ ./src/contexts/Api/index.tsx 10:0-55 70:21-31 112:26-36
 @ ./src/Providers.tsx 14:0-53 57:6-12 77:20-26 81:39-50
 @ ./src/App.tsx 8:0-38 21:35-44
 @ ./src/index.tsx 8:0-22 16:34-37

WARNING in ./node_modules/@substrate/connect/dist/mjs/connector/smoldot-light.js
Module Warning (from ./node_modules/source-map-loader/dist/cjs.js):
Failed to parse source map from '/Users/macbookpro/polkadot/polkadot-staking-dashboard/node_modules/@substrate/connect/src/connector/smoldot-light.ts' file: Error: ENOENT: no such file or directory, open '/Users/macbookpro/polkadot/polkadot-staking-dashboard/node_modules/@substrate/connect/src/connector/smoldot-light.ts'
 @ ./node_modules/@substrate/connect/dist/mjs/connector/index.js 1:0-71 32:9-24
 @ ./node_modules/@substrate/connect/dist/mjs/index.js 86:0-37 86:0-37
 @ ./node_modules/@polkadot/rpc-provider/substrate-connect/ScProvider.js 3:0-68 14:46-60 16:0-26 67:78-92
 @ ./node_modules/@polkadot/rpc-provider/substrate-connect/index.js 3:0-32 3:0-32
 @ ./node_modules/@polkadot/rpc-provider/bundle.js 6:0-58 6:0-58
 @ ./node_modules/@polkadot/rpc-provider/index.js 4:0-28 4:0-28
 @ ./node_modules/@polkadot/api/bundle.js 5:0-66 5:0-66 5:0-66
 @ ./node_modules/@polkadot/api/index.js 4:0-28 4:0-28
 @ ./src/contexts/Api/index.tsx 10:0-55 70:21-31 112:26-36
 @ ./src/Providers.tsx 14:0-53 57:6-12 77:20-26 81:39-50
 @ ./src/App.tsx 8:0-38 21:35-44
 @ ./src/index.tsx 8:0-22 16:34-37

WARNING in ./node_modules/@substrate/connect/dist/mjs/connector/specs/index.js
Module Warning (from ./node_modules/source-map-loader/dist/cjs.js):
Failed to parse source map from '/Users/macbookpro/polkadot/polkadot-staking-dashboard/node_modules/@substrate/connect/src/connector/specs/index.ts' file: Error: ENOENT: no such file or directory, open '/Users/macbookpro/polkadot/polkadot-staking-dashboard/node_modules/@substrate/connect/src/connector/specs/index.ts'
 @ ./node_modules/@substrate/connect/dist/mjs/connector/smoldot-light.js 1:0-43 146:25-32
 @ ./node_modules/@substrate/connect/dist/mjs/connector/index.js 1:0-71 32:9-24
 @ ./node_modules/@substrate/connect/dist/mjs/index.js 86:0-37 86:0-37
 @ ./node_modules/@polkadot/rpc-provider/substrate-connect/ScProvider.js 3:0-68 14:46-60 16:0-26 67:78-92
 @ ./node_modules/@polkadot/rpc-provider/substrate-connect/index.js 3:0-32 3:0-32
 @ ./node_modules/@polkadot/rpc-provider/bundle.js 6:0-58 6:0-58
 @ ./node_modules/@polkadot/rpc-provider/index.js 4:0-28 4:0-28
 @ ./node_modules/@polkadot/api/bundle.js 5:0-66 5:0-66 5:0-66
 @ ./node_modules/@polkadot/api/index.js 4:0-28 4:0-28
 @ ./src/contexts/Api/index.tsx 10:0-55 70:21-31 112:26-36
 @ ./src/Providers.tsx 14:0-53 57:6-12 77:20-26 81:39-50
 @ ./src/App.tsx 8:0-38 21:35-44
 @ ./src/index.tsx 8:0-22 16:34-37

WARNING in ./node_modules/@substrate/connect/dist/mjs/connector/types.js
Module Warning (from ./node_modules/source-map-loader/dist/cjs.js):
Failed to parse source map from '/Users/macbookpro/polkadot/polkadot-staking-dashboard/node_modules/@substrate/connect/src/connector/types.ts' file: Error: ENOENT: no such file or directory, open '/Users/macbookpro/polkadot/polkadot-staking-dashboard/node_modules/@substrate/connect/src/connector/types.ts'
 @ ./node_modules/@substrate/connect/dist/mjs/connector/index.js 4:0-27 4:0-27
 @ ./node_modules/@substrate/connect/dist/mjs/index.js 86:0-37 86:0-37
 @ ./node_modules/@polkadot/rpc-provider/substrate-connect/ScProvider.js 3:0-68 14:46-60 16:0-26 67:78-92
 @ ./node_modules/@polkadot/rpc-provider/substrate-connect/index.js 3:0-32 3:0-32
 @ ./node_modules/@polkadot/rpc-provider/bundle.js 6:0-58 6:0-58
 @ ./node_modules/@polkadot/rpc-provider/index.js 4:0-28 4:0-28
 @ ./node_modules/@polkadot/api/bundle.js 5:0-66 5:0-66 5:0-66
 @ ./node_modules/@polkadot/api/index.js 4:0-28 4:0-28
 @ ./src/contexts/Api/index.tsx 10:0-55 70:21-31 112:26-36
 @ ./src/Providers.tsx 14:0-53 57:6-12 77:20-26 81:39-50
 @ ./src/App.tsx 8:0-38 21:35-44
 @ ./src/index.tsx 8:0-22 16:34-37

WARNING in ./node_modules/@substrate/connect/dist/mjs/index.js
Module Warning (from ./node_modules/source-map-loader/dist/cjs.js):
Failed to parse source map from '/Users/macbookpro/polkadot/polkadot-staking-dashboard/node_modules/@substrate/connect/src/index.ts' file: Error: ENOENT: no such file or directory, open '/Users/macbookpro/polkadot/polkadot-staking-dashboard/node_modules/@substrate/connect/src/index.ts'
 @ ./node_modules/@polkadot/rpc-provider/substrate-connect/ScProvider.js 3:0-68 14:46-60 16:0-26 67:78-92
 @ ./node_modules/@polkadot/rpc-provider/substrate-connect/index.js 3:0-32 3:0-32
 @ ./node_modules/@polkadot/rpc-provider/bundle.js 6:0-58 6:0-58
 @ ./node_modules/@polkadot/rpc-provider/index.js 4:0-28 4:0-28
 @ ./node_modules/@polkadot/api/bundle.js 5:0-66 5:0-66 5:0-66
 @ ./node_modules/@polkadot/api/index.js 4:0-28 4:0-28
 @ ./src/contexts/Api/index.tsx 10:0-55 70:21-31 112:26-36
 @ ./src/Providers.tsx 14:0-53 57:6-12 77:20-26 81:39-50
 @ ./src/App.tsx 8:0-38 21:35-44
 @ ./src/index.tsx 8:0-22 16:34-37

@wirednkod
Copy link
Author

wirednkod commented Aug 9, 2022

@wirednkod Perhaps a separate issue, I am also getting the following console errors on yarn start:

WARNING in ./node_modules/@substrate/connect-extension-protocol/dist/mjs/index.js
Module Warning (from ./node_modules/source-map-loader/dist/cjs.js):
Failed to parse source map from '/Users/macbookpro/polkadot/polkadot-staking-dashboard/node_modules/@substrate/connect-extension-protocol/src/index.ts' file: Error: ENOENT: no such file or directory, open '/Users/macbookpro/polkadot/polkadot-staking-dashboard/node_modules/@substrate/connect-extension-protocol/src/index.ts'

We should open an issue for this.. Im not sure what is really triggering it... We have made some adjustments that may help warnings disappear in next version.
I read that this is a Webpack 5.x issue that they have not resolved yet. I need to dig bit more into that though - to have a clear answer. In any case - that is still just a warning - so I hope its not a blocker

@wirednkod
Copy link
Author

Hi Nikos, I am having issues running this in Brave browser. The syncing hangs with 0 peers. Do we know what is causing this? On Chrome it is working well.

There is some peculiar decision making with Brave since 1.36. In substrate connect extension we prepare a similar "note" on that for Brave users, and we have added also a note on the landing page.

TLDR; As of Brave v1.36, extensions and web pages are limited to a maximum of 10 active WebSocket connections

@rossbulat
Copy link
Collaborator

Hi Nikos, I am having issues running this in Brave browser. The syncing hangs with 0 peers. Do we know what is causing this? On Chrome it is working well.

There is some peculiar decision making with Brave since 1.36. In substrate connect extension we prepare a similar "note" on that for Brave users, and we have added also a note on the landing page.

TLDR; As of Brave v1.36, extensions and web pages are limited to a maximum of 10 active WebSocket connections

Got it, thank you for the context. Would a good solution be some browser detection in-app and disable light client support where browsers are being problematic?

@wirednkod
Copy link
Author

Well we can do that, and its an easy solution, but its a pity to lose Brave users, due to a setting. IMO its better to throw a note for Brave users, that there is a setting that may create issues with websockets, rather than disable light client support. In addition, keep in mind that this is a temp solution, as we have in our roadmap the integration with WebRTC that will overcome these kind of issues

…oard into nik-add-substrate-connect-support
@wirednkod
Copy link
Author

Latest commit adds also the Brave browser notice concerning light clients and websocket limits that brave introduced

image

@@ -24,6 +24,10 @@ export interface Networks {

export interface Network {
name: string;
endpoints: {
rpc: string;
lightClient: WellKnownChain;
Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You should now always assume that a light client will exist.
This should be:

    lightClient?: WellKnownChain;

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nope, we are assuming they exist as all supported chains have a light client

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is there a case where they would not exist?

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is there a case where they would not exist?

The only case is either a custom chain that supports staking or in case we do not want to include light client for a known chain (e.g. polkadot) inside the dashboard. This is of course something that can be altered on a later PR (current PR implementation supports the network UI when light client does not exist [e.g. shows full width button instead of split to 50/50 of RPC/Light client)

let _provider: WsProvider | ScProvider;
if (_isLightClient && nodeEndpoint.lightClientEndpoint) {
_provider = new ScProvider(nodeEndpoint.lightClientEndpoint);
if (_isLightClient && endpoints.lightClient) {
Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same for the optional lightClient parameter

    if (_isLightClient && endpoints?.lightClient) {

@rossbulat
Copy link
Collaborator

Ready to merge. Please let me know if the optional light client flag is totally necessary, It is probably wise not to add complexity if not needed.

@rossbulat rossbulat merged commit f8d5f68 into master Aug 15, 2022
@rossbulat rossbulat deleted the nik-add-substrate-connect-support branch August 20, 2022 19:20
@jonathanpdunne
Copy link

fwiw Talisman also has a similar flag in our wallet. If there's a way to mirror copy and present a consistent message to users of both products, I'd be open to changing what we have in the wallet.

image

@wirednkod
Copy link
Author

Same for substrate connect extension. I will open a discussion about that so we can track it better. Dont know though where would be the best place to do so

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Integrate Substrate Connect as connection option
3 participants