feat: `externally_connectable` CAIP delivery and enveloping #25075

jiexi · 2024-06-05T17:39:25Z

Description

Implements initial Delivery CAIP specs.

Adds BARAD_DUR feature flag
Adds http://* and https://* wildcards to the externally_connectable manifest property behind feature flag
Enables externally_connectable CAIP envelope handling from webpage connections (leaves extension connections as is) behind feature flag
Bifurcates the EIP1993 and CAIP RPC pipelines
- CAIP RPC Pipeline always responds with error CAIP RPC Pipeline not yet implemented.

Related issues

Fixes: https://github.com/MetaMask/MetaMask-planning/issues/2528
See: jiexi/CAIPs#1

Manual testing steps

BARAD_DUR=1 yarn start
Use your local extension build on Chrome
Go to a webpage
In console enter the following, replacing EXTENSION_ID with your local extension ID

const EXTENSION_ID = 'nonfpcflonapegmnfeafnddgdniflbnk';
const extensionPort = chrome.runtime.connect(EXTENSION_ID)
extensionPort.onMessage.addListener((msg) => console.log('extensionPort on message', msg))
extensionPort.postMessage({type: 'caip-x', data: {method: 'eth_chainId'}})

There should be a log with the error response CAIP RPC Pipeline not yet implemented.

Screenshots/Recordings

Before

After

Pre-merge author checklist

I’ve followed MetaMask Coding Standards.
I've completed the PR template to the best of my ability
I’ve included tests if applicable
I’ve documented my code using JSDoc format if applicable
I’ve applied the right labels on the PR (see labeling guidelines). Not required for external contributors.

Pre-merge reviewer checklist

I've manually tested the PR (e.g. pull and build branch, run the app, test code being changed).
I confirm that this PR addresses all acceptance criteria described in the ticket it closes and includes the necessary testing evidence such as recordings and or screenshots.

…nveloping

github-actions · 2024-06-05T17:39:39Z

CLA Signature Action: All authors have signed the CLA. You may need to manually re-run the blocking PR check if it doesn't pass in a few minutes.

…nveloping

jiexi · 2024-06-06T18:29:18Z

app/scripts/inpage.js

the changes in this file need to be moved into the test-dapp. Only here for local testing/verification and to demonstrate how the CAIP Stream pipeline in the inpage provider is exactly the same one used in background

TODO: Revert this before merge

relevant poc test-dapp that enables externally_connectable without caip-x enveloping

https://github.com/MetaMask/test-dapp/pull/317/files

jiexi · 2024-06-06T18:34:22Z

shared/modules/caip-stream.ts

where should this file live? My first thought is @metamask/providers, but it's more JSON-RPC pipeline aligned. Yet it doesn't quite belong in json-rpc-middleware-stream either, but I think that's the best place to park this. Thoughts?

wonder if it should be its own new package in core caip-stream? Otherwise I think json-rpc-middleware-stream could be a decent place to park it and we could consider renaming that package to be a bit more generic? @mcmire @Gudahtt

Are we thinking of extracting this to a package because we need to share it?

If so, I tend to like small packages that do one thing, so from a responsibility perspective I'm leaning slightly toward the option of making a new package called caip-stream instead of bundling it with json-rpc-middleware-stream and renaming that. I could be convinced of the other option, though.

How this would be used in test dapp MetaMask/test-dapp#342

core PR here: MetaMask/core#4399

this logic has been simplified quite a bit now and it's questionable whether this requires a new package again. If we build out a new lean provider in test-dapp to handle externally_connectable, then we don't. If we reuse our existing provider in test-dapp to handle externally_connectable, then we probably do.

jiexi · 2024-06-06T18:34:35Z

shared/modules/caip-stream.ts

+export class SplitStream extends Duplex {
+  substream: Duplex;
+
+  constructor(substream?: SplitStream) {
+    super({ objectMode: true });
+    this.substream = substream ?? new SplitStream(this);
+  }
+
+  _read() {
+    return undefined;
+  }
+
+  _write(
+    value: unknown,
+    _encoding: BufferEncoding,
+    callback: (error?: Error | null) => void,
+  ) {
+    this.substream.push(value);
+    callback();
+  }
+}


similar to https://github.com/MetaMask/core/blob/main/packages/json-rpc-middleware-stream/src/createEngineStream.ts#L16

metamaskbot · 2024-06-25T19:35:10Z

Builds ready [d27526a]

builds: chrome, firefox
builds (beta): chrome
builds (flask): chrome, firefox
builds (MMI): chrome, firefox
builds (test): chrome, firefox
builds (test-flask): chrome, firefox
build viz: Build System
mv3: Background Module Init Stats
mv3: UI Init Stats
mv3: Module Load Stats
mv3: Bundle Size Stats
mv2: E2e Actions Stats
code coverage: Report
storybook: Storybook
typescript migration: Dashboard
all artifacts
bundle viz:
- background: 0, 1, 2, 3, 4, 5
- common: 0, 1, 2, 3, 4, 5, 6, 7, 8, 9
- content-script: 0
- offscreen: 0
- ui: 0, 1, 10, 11, 2, 3, 4, 5, 6, 7, 8, 9

Page Load Metrics (225 ± 248 ms)

Platform	Page	Metric	Min (ms)	Max (ms)	Average (ms)	StandardDeviation (ms)	MarginOfError (ms)
Chrome	Home	firstPaint	64	139	88	16	8
		domContentLoaded	9	35	13	5	2
		load	41	1931	225	517	248
		domInteractive	9	35	13	5	2

Bundle size diffs [🚨 Warning! Bundle size has increased!]

background: 2.58 KiB (0.07%)
ui: 0 Bytes (0.00%)
common: 0 Bytes (0.00%)

adonesky1

LGTM! Also tested and works well 🥳

…nveloping

metamaskbot · 2024-06-25T22:36:56Z

Builds ready [1ced8f3]

builds: chrome, firefox
builds (beta): chrome
builds (flask): chrome, firefox
builds (MMI): chrome, firefox
builds (test): chrome, firefox
builds (test-flask): chrome, firefox
build viz: Build System
mv3: Background Module Init Stats
mv3: UI Init Stats
mv3: Module Load Stats
mv3: Bundle Size Stats
mv2: E2e Actions Stats
code coverage: Report
storybook: Storybook
typescript migration: Dashboard
all artifacts
bundle viz:
- background: 0, 1, 2, 3, 4, 5
- common: 0, 1, 2, 3, 4, 5, 6, 7, 8, 9
- content-script: 0
- offscreen: 0
- ui: 0, 1, 10, 11, 2, 3, 4, 5, 6, 7, 8, 9

Page Load Metrics (131 ± 171 ms)

Platform	Page	Metric	Min (ms)	Max (ms)	Average (ms)	StandardDeviation (ms)	MarginOfError (ms)
Chrome	Home	firstPaint	65	104	84	10	5
		domContentLoaded	9	26	11	4	2
		load	39	1681	131	356	171
		domInteractive	9	26	11	4	2

Bundle size diffs [🚨 Warning! Bundle size has increased!]

background: 2.58 KiB (0.08%)
ui: 0 Bytes (0.00%)
common: 0 Bytes (0.00%)

…nveloping # Conflicts: # app/scripts/background.js

…nveloping

jiexi · 2024-06-26T18:49:21Z

Including stream bifurcation in this PR as discussed with @shanejonas

metamaskbot · 2024-06-26T18:49:33Z

Builds ready [323ab1e]

builds: chrome, firefox
builds (beta): chrome
builds (flask): chrome, firefox
builds (MMI): chrome, firefox
builds (test): chrome, firefox
builds (test-flask): chrome, firefox
build viz: Build System
mv3: Background Module Init Stats
mv3: UI Init Stats
mv3: Module Load Stats
mv3: Bundle Size Stats
mv2: E2e Actions Stats
code coverage: Report
storybook: Storybook
typescript migration: Dashboard
all artifacts
bundle viz:
- background: 0, 1, 2, 3, 4, 5
- common: 0, 1, 2, 3, 4, 5, 6, 7, 8, 9
- content-script: 0
- offscreen: 0
- ui: 0, 1, 10, 11, 2, 3, 4, 5, 6, 7, 8, 9

Page Load Metrics (54 ± 13 ms)

Platform	Page	Metric	Min (ms)	Max (ms)	Average (ms)	StandardDeviation (ms)	MarginOfError (ms)
Chrome	Home	firstPaint	64	292	95	58	28
		domContentLoaded	9	11	10	1	0
		load	38	149	54	28	13
		domInteractive	9	11	10	1	0

Bundle size diffs [🚨 Warning! Bundle size has increased!]

background: 2.58 KiB (0.08%)
ui: 0 Bytes (0.00%)
common: 0 Bytes (0.00%)

…nveloping

metamaskbot · 2024-06-26T20:22:17Z

Builds ready [3c8b832]

builds: chrome, firefox
builds (beta): chrome
builds (flask): chrome, firefox
builds (MMI): chrome, firefox
builds (test): chrome, firefox
builds (test-flask): chrome, firefox
build viz: Build System
mv3: Background Module Init Stats
mv3: UI Init Stats
mv3: Module Load Stats
mv3: Bundle Size Stats
mv2: E2e Actions Stats
code coverage: Report
storybook: Storybook
typescript migration: Dashboard
all artifacts
bundle viz:
- background: 0, 1, 2, 3, 4, 5
- common: 0, 1, 2, 3, 4, 5, 6, 7, 8, 9
- content-script: 0
- offscreen: 0
- ui: 0, 1, 10, 11, 2, 3, 4, 5, 6, 7, 8, 9

Page Load Metrics (149 ± 207 ms)

Platform	Page	Metric	Min (ms)	Max (ms)	Average (ms)	StandardDeviation (ms)	MarginOfError (ms)
Chrome	Home	firstPaint	63	101	80	9	4
		domContentLoaded	9	20	12	3	1
		load	40	2024	149	430	207
		domInteractive	9	20	12	3	1

Bundle size diffs [🚨 Warning! Bundle size has increased!]

background: 3.54 KiB (0.11%)
ui: 0 Bytes (0.00%)
common: 0 Bytes (0.00%)

shanejonas · 2024-06-26T20:57:26Z

app/scripts/background.js

+  if (port.sender.tab?.id && process.env.BARAD_DUR) {
+    connectExternalCaip(...args);
+  } else {
+    connectExternalExtension(...args);


i think we need the legacy API in all cases

To be clear, there are 3 APIs now. Legacy, Current/EIP1193, CAIP

When you say Legacy, I assume you are referring to the Current/EIP1193.

I don't believe we are serving EIP1193 over externally_connectable connections from webpages. @adonesky1

yeah. @shanejonas just to clarify, this browser.runtime.onConnectExternal.addListener(... block only for externally_connectable connections and we don't need to serve the EIP1193 provider to any consumers over that transport

app/scripts/metamask-controller.js

Co-authored-by: Shane <[email protected]>

…le-caip-enveloping' into jl/mmp-2528/externally_connectable-caip-enveloping

adonesky1 · 2024-06-27T01:08:59Z

app/scripts/metamask-controller.js

+      },
+    );
+
+    // Used to show wallet liveliness to the provider


total nit

Suggested change

// Used to show wallet liveliness to the provider

// Used to signal wallet liveness to the provider

adonesky1

LGTM!

metamaskbot · 2024-06-27T01:25:18Z

Builds ready [c8ce2f8]

builds: chrome, firefox
builds (beta): chrome
builds (flask): chrome, firefox
builds (MMI): chrome, firefox
builds (test): chrome, firefox
builds (test-flask): chrome, firefox
build viz: Build System
mv3: Background Module Init Stats
mv3: UI Init Stats
mv3: Module Load Stats
mv3: Bundle Size Stats
mv2: E2e Actions Stats
code coverage: Report
storybook: Storybook
typescript migration: Dashboard
all artifacts
bundle viz:
- background: 0, 1, 2, 3, 4, 5
- common: 0, 1, 2, 3, 4, 5, 6, 7, 8, 9
- content-script: 0
- offscreen: 0
- ui: 0, 1, 10, 11, 2, 3, 4, 5, 6, 7, 8, 9

Page Load Metrics (50 ± 8 ms)

Platform	Page	Metric	Min (ms)	Max (ms)	Average (ms)	StandardDeviation (ms)	MarginOfError (ms)
Chrome	Home	firstPaint	69	138	81	15	7
		domContentLoaded	8	12	10	1	0
		load	39	121	50	17	8
		domInteractive	8	12	10	1	0

Bundle size diffs [🚨 Warning! Bundle size has increased!]

background: 3.37 KiB (0.10%)
ui: 0 Bytes (0.00%)
common: 0 Bytes (0.00%)

jiexi added 6 commits June 3, 2024 16:04

WIP

e0c7edb

Merge branch 'develop' into jl/mmp-2528/externally_connectable-caip-e…

370dc06

…nveloping

WIP

c1ae3db

WIP PortStream bypass sanity check (working)

61e9697

WIP wrapped stream (working)

1f8cfd2

cleanup inpage

5db9a97

jiexi changed the title ~~Jl/mmp 2528/externally connectable caip enveloping~~ xternally_connectable caip enveloping Jun 5, 2024

jiexi changed the title ~~xternally_connectable caip enveloping~~ externally_connectable CAIP delivery and enveloping Jun 5, 2024

jiexi added 13 commits June 5, 2024 14:54

DRY caip stream

6ef86a4

Merge branch 'develop' into jl/mmp-2528/externally_connectable-caip-e…

97469f9

…nveloping

Rename. WIP spec

1f58e7e

add SplitStream specs

a9b741f

add CaipToMultiplexStream, MultiplexToCaipStream specs

7df64f3

lint

7db9d75

WIP createCaipStream spec

65c9d68

add createCaipStream specs

c8b66b5

lint

bbb33ae

Merge branch 'develop' into jl/mmp-2528/externally_connectable-caip-e…

edab0fe

…nveloping

add BARAD_DUR flag

040d300

dry background trackDappView

2acb719

move externally_connectable manifest wildcard behind BARAD_DUR

9792594

jiexi commented Jun 6, 2024

View reviewed changes

jiexi added team-extension-platform team-wallet-api-platform labels Jun 6, 2024

jsdoc

7372604

jiexi marked this pull request as ready for review June 6, 2024 18:56

jiexi requested a review from kumavis as a code owner June 6, 2024 18:56

adonesky1 previously approved these changes Jun 25, 2024

View reviewed changes

jiexi requested review from Gudahtt June 25, 2024 20:58

jiexi added 2 commits June 25, 2024 14:13

Merge branch 'develop' into jl/mmp-2528/externally_connectable-caip-e…

571cf9e

…nveloping

Merge branch 'develop' into jl/mmp-2528/externally_connectable-caip-e…

1ced8f3

…nveloping

Merge branch 'develop' into jl/mmp-2528/externally_connectable-caip-e…

487b9ca

…nveloping # Conflicts: # app/scripts/background.js

jiexi dismissed adonesky1’s stale review via 487b9ca June 26, 2024 17:40

Merge branch 'develop' into jl/mmp-2528/externally_connectable-caip-e…

323ab1e

…nveloping

jiexi added 3 commits June 26, 2024 12:13

Bifurcate streams

7d3d5f6

Merge branch 'develop' into jl/mmp-2528/externally_connectable-caip-e…

2258ee7

…nveloping

fix specs

3c8b832

shanejonas reviewed Jun 26, 2024

View reviewed changes

app/scripts/metamask-controller.js Outdated Show resolved Hide resolved

jiexi and others added 4 commits June 26, 2024 14:07

Update app/scripts/metamask-controller.js

439f27e

Co-authored-by: Shane <[email protected]>

remove requestAccountTabIds for CAIP

9f35203

Merge remote-tracking branch 'origin/jl/mmp-2528/externally_connectab…

fa34c32

…le-caip-enveloping' into jl/mmp-2528/externally_connectable-caip-enveloping

lint

c8ce2f8

adonesky1 reviewed Jun 27, 2024

View reviewed changes

adonesky1 approved these changes Jun 27, 2024

View reviewed changes

shanejonas approved these changes Jun 27, 2024

View reviewed changes

jiexi merged commit 932c725 into develop Jun 27, 2024
70 checks passed

jiexi deleted the jl/mmp-2528/externally_connectable-caip-enveloping branch June 27, 2024 15:42

github-actions bot locked and limited conversation to collaborators Jun 27, 2024

metamaskbot added the release-12.1.0 Issue or pull request that will be included in release 12.1.0 label Jun 27, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat: `externally_connectable` CAIP delivery and enveloping #25075

feat: `externally_connectable` CAIP delivery and enveloping #25075

jiexi commented Jun 5, 2024 •

edited

Loading

github-actions bot commented Jun 5, 2024

jiexi Jun 6, 2024

jiexi Jun 6, 2024

jiexi Jun 6, 2024

jiexi Jun 6, 2024

adonesky1 Jun 7, 2024

mcmire Jun 7, 2024 •

edited

Loading

jiexi Jun 10, 2024

jiexi Jun 10, 2024

jiexi Jun 11, 2024

jiexi Jun 6, 2024

metamaskbot commented Jun 25, 2024

adonesky1 left a comment •

edited

Loading

metamaskbot commented Jun 25, 2024

jiexi commented Jun 26, 2024

metamaskbot commented Jun 26, 2024

metamaskbot commented Jun 26, 2024

shanejonas Jun 26, 2024

jiexi Jun 26, 2024

adonesky1 Jun 26, 2024

adonesky1 Jun 27, 2024

adonesky1 left a comment

metamaskbot commented Jun 27, 2024

	// Used to show wallet liveliness to the provider
	// Used to signal wallet liveness to the provider

feat: externally_connectable CAIP delivery and enveloping #25075

feat: externally_connectable CAIP delivery and enveloping #25075

Conversation

jiexi commented Jun 5, 2024 • edited Loading

Description

Related issues

Manual testing steps

Screenshots/Recordings

Before

After

Pre-merge author checklist

Pre-merge reviewer checklist

github-actions bot commented Jun 5, 2024

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

mcmire Jun 7, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

metamaskbot commented Jun 25, 2024

adonesky1 left a comment • edited Loading

Choose a reason for hiding this comment

metamaskbot commented Jun 25, 2024

jiexi commented Jun 26, 2024

metamaskbot commented Jun 26, 2024

metamaskbot commented Jun 26, 2024

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

adonesky1 left a comment

Choose a reason for hiding this comment

metamaskbot commented Jun 27, 2024

feat: `externally_connectable` CAIP delivery and enveloping #25075

feat: `externally_connectable` CAIP delivery and enveloping #25075

jiexi commented Jun 5, 2024 •

edited

Loading

mcmire Jun 7, 2024 •

edited

Loading

adonesky1 left a comment •

edited

Loading