-
Notifications
You must be signed in to change notification settings - Fork 2.3k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Native IPFS support in Brave #10220
Comments
What's the current status of this implementation? Can we expect native IPFS support and/or verification of IPFS hashes that appear in trusted gateways URLs any time soon? On a sidenote, it would be great to pair this with native ENS domains support, in such a way that after searching for "website.eth" resolving to an IPFS hash, the address bar would keep displaying "website.eth" instead of the unreadable IPFS has. |
Most of this is implemented, you can test it out by enabling it here brave://flags/#brave-ipfs |
Thank you @bbondy I shall try this myself soon. Is this a full IPFS integration, just trusted gateway IPFS hash verification or both? Also, could you point me to the issue where ENS support is discussed? Thanks. |
@experiencedft you can set it to use a public gateway behind the scenes or a local gateway. In either case the CID will be verified against the content hash before the content is served. The ENS issue is tracked here: |
By local gateway, it is meant that IPFS must be installed separately right? My understanding of fully native IPFS integration on Brave would be that the browser itself would integrate an IPFS node. Is that the case? Sorry about all the questions I'm just very excited about the potential of a decentralized and transparent transfer protocol and looking forward to it being seemless for non tech users. |
@experiencedft, please, read the very first message of this thread:
And also look at IPFS onboarding experience section of the first message in this thread, it has a screenshot. IPFS daemon will be installed and launched by Brave, no need to install it yourself. |
Thank you @ilyaigpetrov. I understand that this is indeed a goal in the context of this issue as per the first message of this thread -- my inquiry was about the current state of the implementation seeing as bbondy said "most of it" was implemented without explaining precisely the extent, which obviously I understand considering the team has a lot on their plates. Sorry if this is obvious to others. The best way to go would probably be to test it myself from that point and wait for further releases. Thanks everyone for the replies! |
No manual installs needed, but if you pick to use a local node it will silently setup a for Brave use only go-ipfs node, it's quite easy to use. Just enable it on brave://flags/#brave-ipfs and then visit an IPFS resource like: |
Please, fix #12015 before shipping native IPFS with the Release channel. |
Verification passed using builds below:
SettingsPrior to install, confirmed no ipfs component exists on brave://components. After install, component is added: Confirmed setting "Method to resolve IPFS resources" to "Gateway" prior to install did not show infobar when visiting a page such as Confirmed setting "Method to resolve IPFS resources" to "Disabled" and visiting a ipfs:// URL such as Additional testing of "Method to resolve IPFS resources" can be found under #12978 (comment) Confirmed after IPFS is installed via infobar, I am able to visit both Confirmed when "Method to resolve IPFS resources" is "Local node" and I visit a URL with ipfs:// scheme, it is preserved. Additionally, clicking on links from this page preserves ipfs:// in URL bar. Encountered logged issue #13171 while testing. Basic ChecksUsing clean profile, confirmed navigating to an ipfs:// URL prompts to install via infobar. Using clean profile, confirmed navigating to an ipns:// URL prompts to install via infobar. Using clean profile, confirmed navigating to an https:// URL with /ipfs/[CID] prompts to install on the dweb.link IPFS gateway Using clean profile, confirmed navigating to an https:// URL with /ipns/[CID] prompts to install on the dweb.link IPFS gateway Using clean profile, confirmed navigating to an https:// URL with Confirmed Guest, Private, and Tor windows cannot load ipfs URLs. See additional testing notes listed under #12978 (comment). Confirmed loading an ipfs:// URL works correctly after IPFS is installed Confirmed loading an ipns:// URL works correctly after IPFS is installed Confirmed session restore works correctly when a single tab is left open, active with an ipfs:// URL. Confirmed session restore works correctly when a single tab is left open, in the background with an ipfs:// URL. Confirmed session restore works correctly when a single tab is left open, active with an ipns:// URL. Confirmed session restore works correctly when a single tab is left open, in the background with an ipns:// URL. Encountered and logged #13519 - if you have multiple ipfs:// or ipns:// tabs open in background (not active) on relaunch, usually at least one of them will not connect. Refresh of the affected tab fixes this. Confirmed able to use ipfs:// local node using Beta, Dev, Nightly channels simultaneously. Confirmed different ports used for each channel:
WebUI pageConfirmed chrome://ipfs, about://ipfs resolve to brave://ipfs Confirmed this page shows status.
Lazy LoadingConfirmed restarting the browser does not load IPFS process in Task Manager or Activity Monitor:
Confirmed navigating to IPFS resource loads process in Task Manager and Activity Monitor:
IPFS CompanionConfirmed with ipfs installed (Method to resolve = local node) I was able to install IPFS Companion and visit ipfs:// URLs without issue. Testing N/A for: P3A, API per #10220 (comment) Subresource testingCreated test.html file per directions in txt file under Subresource testing. Closed Brave.
Note - my component version, go-ipfs version, and port differ from the example and are reflected above. Go back to Dev and load Confirmed that for "Method to resolve IPFS resources" = Local Node, the above page resolves: Confirmed that for "Method to resolve IPFS resources" = Gateway, the above page resolves:
Linked issue testing
From Step 4, confirmed interstitial page shown (note, used different link and confirmed with Jocelyn this is ok).
Steps 7 & 8 could not be tested on 1.19.82 as they regressed (they do work on 1.19.81). Logged #13544 for this.
From Step 4, confirmed daemon is launched on brave://ipfs. From Step 5, confirmed origin on page as
From Step 7, confirmed From Step 8, confirmed origin on page as
Talking to the IPFS nodeCreated test.txt file with some text in it:
Added file to IPFS node:
Loaded ipfs://QmQb54jr5pCE3HXsp54kPRR7wCx8NfdJAxFGby3VjgFQYE Published IPFS CID to IPNS:
Loaded ipns://k51qzi5uqu5dgdsdwd2txphi4ptl7f9arqled71uuke9l0yfw7ul835kytawe3/
Confirmed able to use ipfs:// local node using Beta, Dev, Nightly channels simultaneously. Confirmed different ports used for each channel
Reduced checks done on macOS M1 laptop using build below:
SettingsPrior to install, confirmed no ipfs component exists on brave://components. After install, component is added: Confirmed after IPFS is installed via infobar, I am able to visit both Confirmed when "Method to resolve IPFS resources" is "Local node" and I visit a URL with ipfs:// scheme, it is preserved. Additionally, clicking on links from this page preserves ipfs:// in URL bar. Basic ChecksUsing clean profile, confirmed navigating to an ipfs:// URL prompts to install via infobar. Using clean profile, confirmed navigating to an ipns:// URL prompts to install via infobar. Using clean profile, confirmed navigating to an https:// URL with /ipfs/[CID] prompts to install on the dweb.link IPFS gateway Using clean profile, confirmed navigating to an https:// URL with /ipfs/[CID] on non default gateway does not prompt for install via infobar. Confirmed loading an ipfs:// URL works correctly after IPFS is installed Confirmed loading an ipns:// URL works correctly after IPFS is installed Confirmed session restore works correctly when a single tab is left open, in the background with an ipfs:// URL. Confirmed session restore works correctly when a single tab is left open, active with an ipfs:// URL. Lazy LoadingConfirmed restarting the browser does not load IPFS process in Task Manager or Activity Monitor when IPFS is installed. Confirmed navigating to IPFS resource loads process in Task Manager and Activity Monitor when IPFS is installed. Verification passed on
SettingsPrior to install, confirmed no ipfs component exists on brave://components. After install, component is added: Confirmed setting "Method to resolve IPFS resources" to "Gateway" prior to install did not show infobar when visiting a page such as Confirmed setting "Method to resolve IPFS resources" to "Disabled" and visiting a ipfs:// URL such as Additional testing of "Method to resolve IPFS resources" can be found under #12978 (comment) Confirmed after IPFS is installed via infobar, I am able to visit both Confirmed when "Method to resolve IPFS resources" is "Local node" and I visit a URL with ipfs:// scheme, it is preserved. Additionally, clicking on links from this page preserves ipfs:// in URL bar. Encountered logged issue #13171 while testing. Basic ChecksUsing clean profile, confirmed navigating to an ipfs:// URL prompts to install via infobar. Using clean profile, confirmed navigating to an ipns:// URL prompts to install via infobar. Using clean profile, confirmed navigating to an https:// URL with /ipfs/[CID] prompts to install on the dweb.link IPFS gateway Using clean profile, confirmed navigating to an https:// URL with /ipns/[CID] prompts to install on the dweb.link IPFS gateway Using clean profile, confirmed navigating to an https:// URL with Confirmed Guest, Private, and Tor windows cannot load ipfs URLs. See additional testing notes listed under #12978 (comment). Confirmed loading an ipfs:// URL works correctly after IPFS is installed Confirmed loading an ipns:// URL works correctly after IPFS is installed Confirmed session restore works correctly when a single tab is left open, active with an ipfs:// URL. Confirmed session restore works correctly when a single tab is left open, in the background with an ipfs:// URL. Confirmed session restore works correctly when a single tab is left open, active with an ipns:// URL. Confirmed session restore works correctly when a single tab is left open, in the background with an ipns:// URL. Encountered #13519 Confirmed able to use ipfs:// local node using Beta, Dev, Nightly channels simultaneously. Confirmed different ports used for each channel:
WebUI pageConfirmed chrome://ipfs, about://ipfs resolve to brave://ipfs Confirmed this page shows status.
Lazy LoadingConfirmed restarting the browser does not load IPFS process in Task Manager or Activity Monitor IPFS CompanionConfirmed with ipfs installed (Method to resolve = local node) I was able to install IPFS Companion and visit ipfs:// URLs without issue. Subresource testingCreated test.html file per directions in txt file under Subresource testing. Closed Brave.
Note - my component version, go-ipfs version, and port differ from the example and are reflected above. Load Confirmed that for "Method to resolve IPFS resources" = Local Node, the above page resolves: Confirmed that for "Method to resolve IPFS resources" = Gateway, the above page resolves: Linked issue testing
confirmed interstitial page shown Verified clicking Verified after enabling network connection, all worked as expected:
confirmed daemon is launched on brave://ipfs. confirmed origin on page as confirmed confirmed Talking to the IPFS nodeCreated test.txt file with some text in it: Added file to IPFS node:
Loaded ipfs://QmefFkHXEcSY8qe3MpqSxzKYtr1rV42QYJZa5geus4tdJ4 Published IPFS CID to IPNS:
Loaded ipns://k51qzi5uqu5djsjzx6s4i5knccxz13o5hc9qn63avlqmi3uclibp5ghpi9g7q9/ Verification passed on
SettingsPrior to install, confirmed no ipfs component exists on brave://components. After install, component is added: Confirmed setting "Method to resolve IPFS resources" to "Gateway" prior to install did not show infobar when visiting a page such as Confirmed setting "Method to resolve IPFS resources" to "Disabled" and visiting a ipfs:// URL such as Additional testing of "Method to resolve IPFS resources" can be found under #12978 (comment) Confirmed after IPFS is installed via infobar, I am able to visit both Confirmed when "Method to resolve IPFS resources" is "Local node" and I visit a URL with ipfs:// scheme, it is preserved. Additionally, clicking on links from this page preserves ipfs:// in URL bar. Encountered logged issue #13171 while testing.
Basic ChecksUsing clean profile, confirmed navigating to an ipfs:// URL prompts to install via infobar. Using clean profile, confirmed navigating to an ipns:// URL prompts to install via infobar. Using clean profile, confirmed navigating to an https:// URL with /ipfs/[CID] prompts to install on the dweb.link IPFS gateway Using clean profile, confirmed navigating to an https:// URL with /ipns/[CID] prompts to install on the dweb.link IPFS gateway Using clean profile, confirmed navigating to an https:// URL with Confirmed Guest, Private, and Tor windows cannot load ipfs URLs. See additional testing notes listed under #12978 (comment). Confirmed loading an ipfs:// URL works correctly after IPFS is installed Confirmed loading an ipns:// URL works correctly after IPFS is installed Confirmed session restore works correctly when a single tab is left open, active with an ipfs:// URL. Confirmed session restore works correctly when a single tab is left open, in the background with an ipfs:// URL. Confirmed session restore works correctly when a single tab is left open, active with an ipns:// URL. Confirmed session restore works correctly when a single tab is left open, in the background with an ipns:// URL. Encountered #13519 - if you have multiple ipfs:// or ipns:// tabs open in background (not active) on relaunch, usually at least one of them will not connect. Refresh of the affected tab fixes this. Confirmed able to use ipfs:// local node using Beta, Dev, Nightly channels simultaneously. Confirmed different ports used for each channel:
WebUI pageConfirmed chrome://ipfs, about://ipfs resolve to brave://ipfs Confirmed this page shows status.
Lazy LoadingConfirmed restarting the browser does not load IPFS process in Task Manager or Activity Monitor IPFS CompanionConfirmed with ipfs installed (Method to resolve = local node) I was able to install IPFS Companion and visit ipfs:// URLs without issue. Subresource testingCreated test.html file per directions in txt file under Subresource testing. Closed Brave.
Note - my component version, go-ipfs version, and port differ from the example and are reflected above. Go back to Brave and load ipfs://QmZcCnvw52Xxmfi6s3En6qChja35ewEmrv8qtVTqKX1yAG Confirmed that for "Method to resolve IPFS resources" = Local Node, the above page resolves: Confirmed that for "Method to resolve IPFS resources" = Gateway, the above page resolves: Linked issue testingVerified test plan from brave/brave-core#6966,
Talking to the IPFS nodeCreated a test.txt file with some text in it: Added file to IPFS node:
Loaded ipfs://QmbCCQpVkfpCVKcdKqA2JwC6XbcXwrCH24SZB1gVT4GT6G Published IPFS CID to IPNS:
Loaded ipns://k51qzi5uqu5dizw1qhkv818p5phofi8q8cdscooo79xsx3u50h8zpptmiz3nf3 |
Overview
Before the work described in this issue, Brave supports IPFS through the installation of the IPFS Companion extension. A user can install this extension from the Chrome Web Store, or by toggling a switch in chrome://settings in the Extensions section.
Once IPFS Companion is installed, the user can go to IPFS Companion’s settings and make it use “External + chrome.sockets (experimental)”. This avoids the user needing to manually install a local daemon, and uses js-ipfs instead; however, js-ipfs is behind go-ipfs on several fronts. It would be ideal to use go-ipfs instead of js-ipfs.
Our objective is to seamlessly connect Brave users to the IPFS network, including allowing Brave users to easily use dApps that run on IPFS. This spec describes the handling and installation of the go-ipfs daemon via use of the Brave component update server. The functionality will feel built-in and not require the user to separately install a daemon for full IPFS support out of the box.
Targeted Platforms
Desktop (Windows, macOS, Linux)
Description
The work can be broken down into the following sections:
Native protocol support
The following 2 protocols will be handled by Brave automatically:
ipfs://
andipns://
When the user navigates to a URL such as:
ipfs://bafybeiemxf5abjwjbikoz4mc3a3dla6ual3jsgpdr4cjr3oz3evfyavhwq/wiki/Vincent_van_Gogh.html
If IPFS is not yet configured, the user will have the IPFS page loaded through the gateway
https://dweb.link
. The IPFS onboarding experience mentioned below will ask the user if they’d like to install the full node and serve it through a local gateway instead. Theipfs://
andipns://
URLs in the URL bar will not be preserved unless a local node is used.If the IPFS full node is installed, the content will be loaded through that node.
Other URL types which would activate this same UI are described here:
https://github.com/ipfs/in-web-browsers/blob/master/ADDRESSING.md
When a user requests via a base58 encoding:
ipfs://QmbWqxBEKC3P8tqsKc98xmWNzrzDtRLMiMPL8wBuTGsMnR
It gets redirected to gateway path:
https://dweb.link/ipfs/QmbWqxBEKC3P8tqsKc98xmWNzrzDtRLMiMPL8wBuTGsMnR
And the gateway does the DNS-safe conversion and redirects to base32:
https://bafybeigdyrzt5sfp7udm7hu76uh7y26nf3efuylqabf3oclgtqy55fbzdi.ipfs.dweb.link/
The base32 conversion can be done directly as an optimization.
IPFS onboarding experience
An infobar will be displayed when the user navigates to a gateway URL with URL pattern
*/ipfs/*
or*/ipns/*
:If the user selects to “Enable IPFS” the go-ipfs component will be installed and the daemon will be used for all future requests. After installation, the infobar goes away and future URL loads will load through the local node.
If the user selects “Settings”, they will be redirected to chrome://settings and they will see the advanced settings available such as using a gateway.
Installing and updating go-ipfs daemon component
The go-ipfs daemon binary will be packaged into a crx file and installable via the component updater. There will be a different package per platform (Windows, macOS, and Linux x64) each with a different component extension ID.
Updates will occur as part of the normal component update flow in Brave described here: https://github.com/brave/brave-browser/wiki/Component-Extensions
If it is installed, you can see it installed here:
brave://components/
Use find in page for IPFS.When a new go-ipfs version is released, it doesn’t need to always be updated in Brave. We’ll start with having a process of Protocol Labs asking Brave to update. When Brave receives a request to update, we’ll run a Jenkins job which packages, signs, and publishes a new component. The binary will be pulled from Protocol Labs at https://dist.ipfs.io/go-ipfs.
ipfs.bravesoftware.com should be used so that if an antivirus wants to block IPFS, then it can do so without blocking Brave completely.
Code for brave-core-crx-packager can be found here: https://github.com/brave/brave-core-crx-packager/search?q=ipfs
IPFS keyed service and proxy configuration
A new Chromium keyed service and related code will be added inside of
brave/components/ipfs/
to manage the go-ipfs daemon.A SOCKS5 proxy is not needed, we instead map URL traffic to
http://{cid}.ipfs.localhost:<port>
Default ports are changed as to not conflict with a previously installed go-ipfs node on a user's machine.
Processes
There are 3 processes that come into play:
All of the Brave code that is added is browser process level code.
Brave will also start the utility process which is sandboxed and who's sole purpose is to manage the go-ipfs process.
Both the Utility process and go-ipfs process will only be started when IPFS is being used. It will be lazily started.
Similar to how Tor is handled, there is handling for if go-ipfs crashes.
Port numbers
go-ipfs starts 3 different TCP listening sockets:
Brave
Nightly
,Dev
,Beta
, andRelease
each use different port numbers for each of those 3 sockets. So they should all be able to be used at the same time. The ports used are also not default, so they shouldn't conflict with a user that already has go-ipfs installed manually.Non top level navigations
Subresources are allowed to use the
ipfs://
andipns://
schemes, and the handling can be found here.Subresources with the
ipfs
andipns
scheme can only be loaded from within a top level scheme that is alsoipfs
andipns
. So for example, anhttp
page cannot load anipfs
scheme image. However anipfs
scheme can loadhttp
resources.Settings
In addition to the toggle that already exists in brave://settings, we will have:
ipfs
andipns
URLS. The user can select betweenAsk
,Gateway
,Local node
,Disabled
. TheLocal node
option only shows up when it is installed.Diagnostic page - brave://ipfs
To help troubleshoot IPFS in Brave, and to see daemon status, a new Chromium WebUI page is added.
This page will show if the IPFS daemon is started, and allow a user to start / stop the IPFS daemon.
It will also show the API, gateway, and swarm addresses.
A user will be able to get at standard output from the go-ipfs daemon inside this page. (Not implemented yet)
The user will have the ability to pass environment variables the user specifies, so that things like log verbosity can be changed. (Not implemented yet)
This page will also get API info from the local API endpoint:
https://docs-beta.ipfs.io/reference/http/api/
In particular for example, the number of peers:
https://docs-beta.ipfs.io/reference/http/api/#api-v0-swarm-peers
HTTP gateway fallback
If the user chooses in chrome://settings to use a gateway instead of a local go-ipfs node, then all requests to ipfs:// and similar will be redirected to the configured IPFS gateway.
Interstitial page when there are no peers
If there are no peers but an IPFS or IPNS URL is accessed, an interstitial page (such as bad SSL certificate) will be displayed telling the user that the node doesn’t seem to be started or there are no connected peers. An option will be given on the page to use the default gateway fallback and to continue to do so automatically in the future when there are no peers.
Admin policy
A top level IPFS enabled setting will be added and tied to an admin policy to completely disable all IPFS detection and functionality.
Browser tests will be added to
browser/policy/brave_policy_browsertest.cc
The role of IPFS Companion and new chrome.ipfs API
IPFS Companion will be compatible but not required to be installed. We’ll leave it to Protocol Labs to optionally enhance IPFS Companion but we’ll assist via adding APIs that would be helpful.
Installing IPFS Companion will be a separate action and will not be bundled with the installation of the go-ipfs binary. Users will continue to get updates from the Chrome Web Store and can continue to install IPFS Companion from chrome://settings.
The following new API have been exposed to IPFS Companion only:
Landing and shipping code behind a flag
In order to land code and not wait for the entire feature to be implemented, we’ll start by adding a flag in chrome://flags which will allow you to toggle on the experimental IPFS support.
Privacy-Preserving Product Analytics (P3A)
Pending review approval by the privacy team:
Guest windows, Tor windows, and incognito windows
To avoid leaking IPs and other potential privacy issues, we've disable this feature in Tor windows.
Guest windows and Incognito windows also should not load IPFS and IPNS URIs on the local node since it would store content that is accessed.
Cache clearing (1.20.x and beyond)
When clearing cache in Brave, we automatically run
ipfs repo gc
.Since this does not support time based deletions, we will clear the cache no matter which time interval is used.
Pinned items will be retained.
QA Plan
Settings:
Basic checks:
WebUI page:
Lazy loading:
Subresource testing:
IPFS Companion:
P3A:
API:
There are 3 issues that landed in past milestones, but they were originally marked as
QA/No
because the feature was disabled.These should be tested according to the following test plans:
Here are some example gateway URLs:
ipfs://bafybeiemxf5abjwjbikoz4mc3a3dla6ual3jsgpdr4cjr3oz3evfyavhwq/wiki/Vincent_van_Gogh.html
(not a link because github markdown doesn't allow ipfs://)ipns://Qmc9xb2cgWKgz8rCBihag6fmvvEu7B7nbmYPxD6RcFXe1Z
Talking to the IPFS node:
Create a test.txt file with "hello" inside of it.
Add that file to your IPFS node
Note: Check which API port in chrome://ipfs and adjust accordingly.
You can now load this file in the browser, or from anywhere on IPFS by loading:
ipfs://QmZULkCELmmk5XNfCgTnCyFgAVxBRBXyDHGGMVoLFLiXEN
It will be slower if you are loading from a different node.
You can find this binary inside your profile directory. The profile dir and component ID folder name differs depending on which platform you are on.
You can publish an IPFS CID to IPNS using a command like this:
Then you can load it with
ipns://Qmc9xb2cgWKgz8rCBihag6fmvvEu7B7nbmYPxD6RcFXe1Z
More go-ipfs command line info can be learnt here: https://docs.ipfs.io/reference/cli/
The text was updated successfully, but these errors were encountered: