chore: update libp2p to 0.30 #3427

vasco-santos · 2020-11-27T12:08:48Z

Integrates [email protected] release.

This PR changes needed types accordingly.

Needs:

A note should be included in the next release notes regarding the websockets update to anticipate possible problems for local environment tests of IPFS users

vasco-santos · 2020-12-10T20:52:00Z

packages/ipfs-core/src/components/index.js

@@ -70,6 +70,8 @@ class IPFS {
    })
    const dns = createDNSAPI()
    const isOnline = createIsOnlineAPI({ network })
+    // @ts-ignore This type check fails as options.
+    // libp2p can be a function, while IPNS router config expects libp2p config


This is not an easy fix. I recommend we do it individually

vasco-santos · 2020-12-10T20:52:34Z

packages/ipfs-core/src/components/key/export.js

-  const exportKey = (name, password, options) =>
-    keychain.exportKey(name, password, options)
+  const exportKey = (name, password) =>
+    keychain.exportKey(name, password)


We do not support Aborts/options in keychain.

vasco-santos · 2020-12-10T20:54:24Z

packages/ipfs-core/src/components/pubsub.js

@@ -41,6 +41,7 @@ module.exports = ({ network, config }) => {
   */
  async function subscribe (topic, handler, options) {
    const { libp2p } = await network.use(options)
+    // @ts-ignore Libp2p Pubsub is deprecating the handler, using the EventEmitter


At some point we might get https://github.com/libp2p/js-libp2p/blob/0.30.x/src/pubsub-adapter.js this out. I can add in a follow up PR the handlers behaviour for IPFS

vasco-santos · 2020-12-10T20:54:47Z

packages/ipfs-core/src/components/pubsub.js

@@ -111,7 +113,7 @@ module.exports = ({ network, config }) => {
   */
  async function ls (options) {
    const { libp2p } = await network.use(options)
-    return libp2p.pubsub.getTopics(options)
+    return libp2p.pubsub.getTopics()


Pubsub has no options for getTopics + getSubscribers

vasco-santos · 2020-12-10T20:55:34Z

packages/ipfs-core/src/components/storage.js

@@ -43,6 +43,8 @@ class Storage {

    const { peerId, keychain, isNew } = await loadRepo(repo, options)

+    // TODO: throw error?


Keychain is not available in init IIRC. That is why it might exist or not

So there was / is this code:

js-ipfs/packages/ipfs-core/src/components/storage.js

Lines 137 to 156 in 2bafc88

const libp2p = createLibP2P({

options: undefined,

multiaddrs: undefined,

peerId,

repo,

config,

keychainConfig: {

pass: options.pass

}

})

if (libp2p.keychain && libp2p.keychain.opts) {

await libp2p.loadKeychain()

await repo.config.set('Keychain', {

dek: libp2p.keychain.opts.dek

})

}

return { peerId, keychain: libp2p.keychain }

And this code path

js-ipfs/packages/ipfs-core/src/components/storage.js

Lines 214 to 230 in 2bafc88

const libp2p = createLibP2P({

options: undefined,

multiaddrs: undefined,

peerId,

repo,

config: changed,

keychainConfig: {

pass,

...changed.Keychain

}

})

if (libp2p.keychain) {

await libp2p.loadKeychain()

}

return { peerId, keychain: libp2p.keychain }

And I'm still not sure why libp2p is created in two different ways to be honest. It would be good to get these cleaned up (in some followup). I'd also happily pair on this as lack of uncertainty here really bothered me.

I agree with you on this, it also confused me. We should improve this on a followup

packages/ipfs-core/package.json

packages/ipfs-core/src/components/storage.js

Co-authored-by: Hugo Dias <[email protected]>

Gozala

Thanks for the pull @vasco-santos, looks good to me.

Gozala · 2020-12-29T06:18:37Z

packages/ipfs-core/src/components/storage.js

@@ -43,6 +43,8 @@ class Storage {

    const { peerId, keychain, isNew } = await loadRepo(repo, options)

+    // TODO: throw error?


So there was / is this code:

js-ipfs/packages/ipfs-core/src/components/storage.js

Lines 137 to 156 in 2bafc88

const libp2p = createLibP2P({

options: undefined,

multiaddrs: undefined,

peerId,

repo,

config,

keychainConfig: {

pass: options.pass

}

})

if (libp2p.keychain && libp2p.keychain.opts) {

await libp2p.loadKeychain()

await repo.config.set('Keychain', {

dek: libp2p.keychain.opts.dek

})

}

return { peerId, keychain: libp2p.keychain }

And this code path

js-ipfs/packages/ipfs-core/src/components/storage.js

Lines 214 to 230 in 2bafc88

const libp2p = createLibP2P({

options: undefined,

multiaddrs: undefined,

peerId,

repo,

config: changed,

keychainConfig: {

pass,

...changed.Keychain

}

})

if (libp2p.keychain) {

await libp2p.loadKeychain()

}

return { peerId, keychain: libp2p.keychain }

And I'm still not sure why libp2p is created in two different ways to be honest. It would be good to get these cleaned up (in some followup). I'd also happily pair on this as lack of uncertainty here really bothered me.

Gozala · 2020-12-29T06:20:10Z

packages/ipfs-core/src/components/storage.js

@@ -43,6 +43,8 @@ class Storage {

    const { peerId, keychain, isNew } = await loadRepo(repo, options)

+    // TODO: throw error?


Seems like throw is not a great option here, if we keychain does not exists we should probably create it ? Otherwise I don't see how user is supposed to use this without getting an error.

As far as I understand, Keychain will always exist at this point as this is only used after start. The problem is that the types are not sure about it.

Gozala · 2020-12-29T06:21:30Z

packages/ipfs-core/src/components/swarm/disconnect.js

@@ -16,7 +16,7 @@ module.exports = ({ network }) => {
   */
  async function disconnect (addr, options) {
    const { libp2p } = await network.use(options)
-    return libp2p.hangUp(addr, options)
+    return libp2p.hangUp(addr)


Just to do the same thing you did with dial

Suggested change

return libp2p.hangUp(addr)

await libp2p.hangUp(addr)

hugomrdias · 2021-01-05T14:49:38Z

packages/interface-ipfs-core/src/utils/ipfs-options-websockets-filter-all.js

+const filters = require('libp2p-websockets/src/filters')
+const transportKey = WS.prototype[Symbol.toStringTag]
+
+module.exports = () => ({


this could be just an object since it doesn't take any input, it would reduce LoC in the others files

The main reason I created a function here is to enable the options to be customized from the caller without modifying the actual content thanks to the new reference. We had issues in the past in libp2p tests, where its base configuration used in tests was modified in some tests, which caused other tests to have different configurations than expected required.
We currently only use it directly and I can turn that into an object. What do you think?

sounds good maybe freeze it to be safe if it makes sense.

vasco-santos mentioned this pull request Dec 3, 2020

Extra websocket calls during .cat(cid) #2938

Closed

vasco-santos force-pushed the chore/update-libp2p-to-0.30 branch from f2bc0e0 to 4fa279c Compare December 10, 2020 15:14

vasco-santos mentioned this pull request Dec 10, 2020

fix: types from ipfs integration libp2p/js-libp2p#832

Merged

vasco-santos force-pushed the chore/update-libp2p-to-0.30 branch 5 times, most recently from ad978f0 to a45a814 Compare December 10, 2020 20:50

vasco-santos commented Dec 10, 2020

View reviewed changes

vasco-santos requested review from Gozala and achingbrain December 10, 2020 20:57

achingbrain mentioned this pull request Dec 11, 2020

vue-ipfs Can't resolve 'dns' in '/home/xxx/dev/group/node_modules/multiaddr/src/resolvers' #3441

Closed

vasco-santos requested a review from hugomrdias December 11, 2020 15:50

hugomrdias suggested changes Dec 11, 2020

View reviewed changes

vasco-santos force-pushed the chore/update-libp2p-to-0.30 branch 14 times, most recently from 40a6ac5 to 95499e7 Compare December 16, 2020 12:03

vasco-santos added 2 commits December 16, 2020 13:05

chore: update libp2p to 0.30

c6a5856

chore: use 0.30 rc

12721b9

vasco-santos and others added 10 commits December 16, 2020 13:05

chore: apply suggestions from code review

4d42fd1

Co-authored-by: Hugo Dias <[email protected]>

chore: use rc1

3be8cda

chore: use ws filter all on local address tests

c8802cf

chore: use interop branch

faf1df7

chore: use libp2p rc in libp2p daemon

f863a33

chore: use multicoedc with fixed version in libp2p

d8f9430

chore: use updated floodsub and gossipsub

7b50f2d

chore: use new gossipsub

6871dda

chore: update libp2p dep

2c5a6da

chore: add websocket all filter to circuit example

898a757

vasco-santos force-pushed the chore/update-libp2p-to-0.30 branch 2 times, most recently from 2fc5b0c to c487184 Compare December 19, 2020 18:42

chore: use final pubsub releases

2bafc88

vasco-santos force-pushed the chore/update-libp2p-to-0.30 branch from c487184 to 2bafc88 Compare December 19, 2020 18:48

vasco-santos marked this pull request as ready for review December 19, 2020 18:52

vasco-santos requested review from hugomrdias and achingbrain and removed request for achingbrain December 19, 2020 19:46

Gozala approved these changes Dec 29, 2020

View reviewed changes

chore: await hangup

ca38f19

hugomrdias suggested changes Jan 5, 2021

View reviewed changes

hugomrdias approved these changes Jan 6, 2021

View reviewed changes

achingbrain approved these changes Jan 15, 2021

View reviewed changes

achingbrain merged commit a39e6fb into master Jan 15, 2021

achingbrain deleted the chore/update-libp2p-to-0.30 branch January 15, 2021 08:42

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

chore: update libp2p to 0.30 #3427

chore: update libp2p to 0.30 #3427

vasco-santos commented Nov 27, 2020 •

edited

Loading

vasco-santos Dec 10, 2020

vasco-santos Dec 10, 2020

vasco-santos Dec 10, 2020

vasco-santos Dec 10, 2020

vasco-santos Dec 10, 2020

Gozala Dec 29, 2020

vasco-santos Dec 29, 2020

Gozala left a comment

Gozala Dec 29, 2020

Gozala Dec 29, 2020

vasco-santos Dec 29, 2020

Gozala Dec 29, 2020

hugomrdias Jan 5, 2021

vasco-santos Jan 6, 2021

hugomrdias Jan 6, 2021

		@@ -43,6 +43,8 @@ class Storage {

		const { peerId, keychain, isNew } = await loadRepo(repo, options)

		// TODO: throw error?

	const libp2p = createLibP2P({
	options: undefined,
	multiaddrs: undefined,
	peerId,
	repo,
	config,
	keychainConfig: {
	pass: options.pass
	}
	})

	if (libp2p.keychain && libp2p.keychain.opts) {
	await libp2p.loadKeychain()

	await repo.config.set('Keychain', {
	dek: libp2p.keychain.opts.dek
	})
	}

	return { peerId, keychain: libp2p.keychain }

chore: update libp2p to 0.30 #3427

chore: update libp2p to 0.30 #3427

Conversation

vasco-santos commented Nov 27, 2020 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Gozala left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

vasco-santos commented Nov 27, 2020 •

edited

Loading