Error: No content routers available

[klueq]

DHT doesn't seem to work because of this. Built the latest js-ipfs + the DHT PR + the latest js-libp2p:

https://klueq.github.io/?https://ipfs.io/ipfs/QmdFgDWRTB4L3Ks87615kSc6yQStXxRTfp1hmYUfDGTeBe/index.js

Then tried to use DHT:

• ipfs.dht.findprovs('QmXXtHtDwXFbCTzMDjacZxid6CCNY4m5oM6MgBhyPVT3Tr')
• ipfs._libp2pNode.contentRouting.findProviders('QmXXtHtDwXFbCTzMDjacZxid6CCNY4m5oM6MgBhyPVT3Tr', 5000, (...args) => console.log('done:', ...args));

Same result: the "No content routers available" error.

Dump of the console logs:

ipfs.dht.findprovs('QmXXtHtDwXFbCTzMDjacZxid6CCNY4m5oM6MgBhyPVT3Tr')
Promise {<rejected>: Error: No content routers available
    at Object.findProviders (https://ipfs.io/ipfs/QmdFgDWRTB4L3K…}
content-routing.js:28 Uncaught (in promise) Error: No content routers available
    at Object.findProviders (content-routing.js:28)
    at Function.findprovs.promisify (dht.js:95)
    at index.js:41
    at new Promise (<anonymous>)
    at Object.findprovs (index.js:35)
    at <anonymous>:1:10
findProviders @ content-routing.js:28
findprovs.promisify @ dht.js:95
(anonymous) @ index.js:41
(anonymous) @ index.js:35
(anonymous) @ VM1356:1
6crypto.js:80 Uncaught Error: dialed to the wrong peer, Ids do not match
    at PeerId.createFromPubKey (crypto.js:80)
    at pubKey.hash (index.js:201)
    at Multihashing.Multihashing.digest (index.js:33)
    at index.js:15
    at run (setImmediate.js:40)
    at runIfPresent (setImmediate.js:69)
    at onGlobalMessage (setImmediate.js:109)
PeerId.createFromPubKey @ crypto.js:80
pubKey.hash @ index.js:201
Multihashing.Multihashing.digest @ index.js:33
(anonymous) @ index.js:15
run @ setImmediate.js:40
runIfPresent @ setImmediate.js:69
onGlobalMessage @ setImmediate.js:109
postMessage (async)
registerImmediate @ setImmediate.js:120
setImmediate @ setImmediate.js:27
(anonymous) @ index.js:14
Promise.then (async)
nodeify @ index.js:13
(anonymous) @ crypto-sha1-2-browser.js:37
sha2256 @ crypto-sha1-2-browser.js:49
Ipfs../node_modules/peer-id/node_modules/multihashing-async/src/index.js.Multihashing.digest @ index.js:84
Multihashing @ index.js:28
hash @ rsa-class.js:43
Ipfs../node_modules/peer-id/src/index.js.exports.createFromPubKey @ index.js:196
Ipfs../node_modules/libp2p-secio/src/handshake/crypto.js.exports.identify @ crypto.js:72
waterfall @ propose.js:24
nextTask @ waterfall.js:16
next @ waterfall.js:23
(anonymous) @ onlyOnce.js:12
onComplete @ decode.js:49
reader.read @ decode.js:125
drain @ index.js:39
more @ index.js:55
(anonymous) @ index.js:66
(anonymous) @ index.js:20
(anonymous) @ map.js:19
(anonymous) @ index.js:114
(anonymous) @ index.js:114
(anonymous) @ map.js:19
(anonymous) @ source.js:31
ipfs._libp2pNode.contentRouting.findProviders('QmXXtHtDwXFbCTzMDjacZxid6CCNY4m5oM6MgBhyPVT3Tr', 5000, () => {});
undefined
2crypto.js:80 Uncaught Error: dialed to the wrong peer, Ids do not match
    at PeerId.createFromPubKey (crypto.js:80)
    at pubKey.hash (index.js:201)
    at Multihashing.Multihashing.digest (index.js:33)
    at index.js:15
    at run (setImmediate.js:40)
    at runIfPresent (setImmediate.js:69)
    at onGlobalMessage (setImmediate.js:109)
PeerId.createFromPubKey @ crypto.js:80
pubKey.hash @ index.js:201
Multihashing.Multihashing.digest @ index.js:33
(anonymous) @ index.js:15
run @ setImmediate.js:40
runIfPresent @ setImmediate.js:69
onGlobalMessage @ setImmediate.js:109
postMessage (async)
registerImmediate @ setImmediate.js:120
setImmediate @ setImmediate.js:27
(anonymous) @ index.js:14
Promise.then (async)
nodeify @ index.js:13
(anonymous) @ crypto-sha1-2-browser.js:37
sha2256 @ crypto-sha1-2-browser.js:49
Ipfs../node_modules/peer-id/node_modules/multihashing-async/src/index.js.Multihashing.digest @ index.js:84
Multihashing @ index.js:28
hash @ rsa-class.js:43
Ipfs../node_modules/peer-id/src/index.js.exports.createFromPubKey @ index.js:196
Ipfs../node_modules/libp2p-secio/src/handshake/crypto.js.exports.identify @ crypto.js:72
waterfall @ propose.js:24
nextTask @ waterfall.js:16
next @ waterfall.js:23
(anonymous) @ onlyOnce.js:12
onComplete @ decode.js:49
reader.read @ decode.js:125
drain @ index.js:39
more @ index.js:55
(anonymous) @ index.js:66
(anonymous) @ index.js:20
(anonymous) @ map.js:19
(anonymous) @ index.js:114
(anonymous) @ index.js:114
(anonymous) @ map.js:19
(anonymous) @ source.js:31
ipfs._libp2pNode.contentRouting.findProviders('QmXXtHtDwXFbCTzMDjacZxid6CCNY4m5oM6MgBhyPVT3Tr', 5000, (...args) => console.log('done:', ...args));
VM1532:1 done: Error: No content routers available
    at Object.findProviders (content-routing.js:28)
    at <anonymous>:1:33

[jacobheun]

In the latest release candidate No content routers available indicates that you have not supplied any custom content routers and that the DHT is not enabled. The index.js file you linked looks to be the whole build file so I wasn't able to fund your specific code.

Can you check that you've turned the dht on via the js-ipfs options?

[klueq]

What are the js-ipfs options? The ones that go to the IPFS ctor? I thought DHT is enabled by default.

What I did is I pulled the latest js-ipfs, added there changes from ipfs/js-ipfs/pull/856, which pulled the latest libp2p, and that's it. The config I use is this:

https://github.com/klueq/klueq.github.io/blob/master/index.js#L3

const config = {
  relay: {enabled: true, hop: {enabled: true}},
  Addresses: {Swarm: ['/dns4/ws-star.discovery.libp2p.io/tcp/443/wss/p2p-websocket-star']},
};

QmdFgDWRTB4L3Ks87615kSc6yQStXxRTfp1hmYUfDGTeBe/index.js is a copy of the npm run build output.

[jacobheun]

That PR still hasn't enabled DHT by default. The config for libp2p there is still setting EXPERIMENTAL.dht to false, which will disable it in libp2p.

[klueq]

How can I enable it? The experimental flag didn't make any difference:

klueq/klueq.github.io@52242e1

[jacobheun]

The problem is your IPFS configuration. You currently have

const ipfsconfig = {
  config: {
    relay: {enabled: true, hop: {enabled: true}},
    Addresses: {Swarm: ['/dns4/ws-star.discovery.libp2p.io/tcp/443/wss/p2p-websocket-star']},
    EXPERIMENTAL: {dht: true},
  }
};

which isn't correct. If you look at the constructor docs, https://github.com/ipfs/js-ipfs/tree/v0.33.0#ipfs-constructor, relay and EXPERIMENTAL need to be first level properties. So you should have:

const ipfsconfig = {
  config: {
    Addresses: {Swarm: ['/dns4/ws-star.discovery.libp2p.io/tcp/443/wss/p2p-websocket-star']}    
  },
  relay: {enabled: true, hop: {enabled: true}},
  EXPERIMENTAL: {dht: true}
};

[klueq]

Thanks! It's doing better now: trying to do dht lookups, etc. However it finds only itself for a popular file (go-ipfs finds a lot of peers) and makes a 50-100% CPU load, probably mining bitcoins in background :) Repro:

https://klueq.github.io/?ipfs=https://ipfs.io/ipfs/QmdFgDWRTB4L3Ks87615kSc6yQStXxRTfp1hmYUfDGTeBe/index.js

The dht lookup is made for this file:

QmQ2r6iMNpky5f1m4cnm3Yqw8VSvjuKpTcK1X7dBR1LkJF

The result (after a long delay) is only the local peer id.

Regarding the CPU load. The fact that the tab cannot even be closed means that it does something like a tight-for-loop: no async stuff, etc.

[jacobheun]

If a lot of dialing is going on that could cause the cpu to drain. You might try setting some connection manager settings, https://github.com/ipfs/js-ipfs#optionsconnectionmanager, to try and reduce the number of peers your connecting to. We do have some backlog items to improve that performance in the future.

The DHT can also be pretty resource intensive. The upcoming work on delegated routing support should prevent the need to run a dht in browser in the future.

cc'ing @vasco-santos in regards to the dht not finding a provider. This should be fully functional once ipfs/js-ipfs#856 is complete

[klueq]

Why is it so resource intensive, though?

What is delegated routing? Not needing to run DHT seems to contradict the effort to implement it.

[jacobheun]

Why is it so resource intensive, though?
The DHT is constantly working on behalf of the network, not just your node. You're providing information about the network to other peers, which can include creating new connections. Our goal is to make this as efficient as possible. Nodes in restricted environments such as mobile browsers or IoT devices, probably shouldn't run a DHT.

Delegated routing gives us the ability to use custom peer and content routers, that know how to talk to other DHTs via their exposed http apis. For IPFS we've created two such routers: https://github.com/libp2p/js-libp2p-delegated-content-routing and https://github.com/libp2p/js-libp2p-delegated-peer-routing. Once the go-ipfs gateway nodes release their api support, these can query the apis directly. They will also fetch your content from you if you register as a provider. This can save limited nodes a lot of resources.

Not needing to run DHT seems to contradict the effort to implement it.

Not needing a DHT and not being able to run a DHT are different. This becomes valuable when we start looking at self healing / dynamic networks. You may not run a DHT when you don't need to, but could potentially turn it on if the state of the network warrants turning that on.

[hapsody]

I have a similar question for you guys.
I tried to use dht on js-ipfs node(browser) with <script src="https://unpkg.com/ipfs/dist/index.min.js"></script>

But I got same error like below
even I set the browser node's config as you said

dht is never able to use on browser?

  const node = new Ipfs({

  repo:'fixedIPFS-storage',
  config: {
    Addresses: {
      Swarm: [
        '/dns4/ws-star.discovery.libp2p.io/tcp/443/wss/p2p-websocket-star'
      ]
    },
    Discovery: {
      MDNS: {Enabled:true, Interval:10}             
    }
  },
  relay: {enabled: true, hop: {enabled: true}},
  EXPERIMENTAL: {dht: true}
})

Uncaught (in promise) Error: No peer routers available
    at Object.findPeer (peer-routing.js:31)
    at ping-pull-stream.js:66
    at t.pingPullStream (ping-pull-stream.js:22)
    at Function.<anonymous> (ping.js:14)
    at index.js:41
    at new Promise (<anonymous>)
    at t.ping (index.js:36)
    at <anonymous>:1:6

[jacobheun]

@hapsody the latest version of js-ipfs has a different configuration for DHT now, as it has moved out of the experimental configuration block, so you dont actually have it enabled yet. Check out the libp2p configuration for the latest ipfs release, https://github.com/ipfs/js-ipfs/tree/v0.36.4#optionslibp2p.

[ohager]

I'm suffering with No content routers available too... what is the minimum configuration to make it work?
my current config is:

const ipfs = new Ipfs({
    EXPERIMENTAL: {
        pubsub: true,
        dht: true,
    },
})

(running jsipfs on Ubuntu 18.06, node 10.16 - ipfs 0.36.4) -- tried new configuration, but gives config validation error

const ipfs = new Ipfs({
    silent: true,
    EXPERIMENTAL: {
        pubsub: true,
    },
    dht: {
        enabled: true,
    },
})

gives me: TypeError: Expected a value of type undefinedfordhtbut received{"enabled":true}. at Function.Struct.validate.value [as validate]

[jacobheun]

The dht is currently enabled via the libp2p config option. The following should give you the minimum needed, with a basic example of finding one of the ipfs bootstrap nodes after a 5s delay

const IPFS = require('ipfs')
const node = new IPFS({
  libp2p: {
    config: {
      dht: {
        enabled: true
      }
    }
  }
})

node.on('ready', () => {
  setTimeout(() => {
    node.dht.findPeer('QmSoLnSGccFuZQJzRadHn95W2CrSFmZuTdDWP8HXaHca9z', console.log)
  }, 5e3)
})

[ohager]

@jacobheun Thanks for response... after checking the config validation scheme, I figured that out too. Am I right, that I need to configure timeouts/timings for findProvs? I "played" around a bit, but my app hangs (and has heavy CPU load, which I expected). My idea is to add bootstrap nodes using ipfs.bootstrap.add programmatically, and then run findProvs to fetch pinned files of added nodes. Would this work?

[jacobheun]

@ohager we are working on making the DHT more performant for when it launches enabled by default js-ipfs, you can track that at ipfs/js-ipfs#1994. I'd check that PR out for some potential configurations. Changing your High and Low Watermarks for connection manager could help, you might have too many connections open for your machine which could account for the cpu usage.

As long as all those nodes are running a DHT and connected to the same network you should be able to find them. You can add timeouts to the call, but I would recommend playing with the config to try and get your cpu down.

---

Since we've gotten a bit off topic from this original issue and the original question has been resolved, I am going to close this issue. If you'd like to talk more about the DHT and setup please jump over to https://discuss.libp2p.io. Happy to continue conversations over there and more people might benefit from the discussion there :)