[WIP] Add secio & move to pull-streams #67

dignifiedquire · 2016-05-23T20:47:10Z

Not finished yet

Closes #63

dignifiedquire · 2016-07-04T14:47:25Z

I'm pretty close to passing tests, there is just one issue left to fix. When initiating the secio protocol multistream-select seems to loose one message, which is very unfortunate.

-->  /multistream/1.0.0
<--  /multistream/1.0.0
<--  /secio/1.0.0
listener ready
handle: QmVg942ZGNuGw7AFpiQAPsdRxzCK4J9e15NZGEBaixowQV
  libp2p:secio 1. propose - start +0ms
  libp2p:secio 1. propse - writing proposal +1ms
writing:  <Buffer f3 02 0a 10 98 a4 3a fb f0 07 e4 74 09 8c 9c 14 28 43 b9 6a 12 ab 02 08 00 12 a6 02 30 82 01 22 30 0d 06 09 2a 86 48 86 f7 0d 01 01 01 05 00 03 82 01 ... >
  libp2p:secio 1. propose - proposal size 373 +8ms
I shall read: 373
-->  /secio/1.0.0
dial: QmRRTGZEykibGGAEpSDh3fK2UnL746x5zHeQzNzERc74uM
  libp2p:secio 1. propose - start +4ms
  libp2p:secio 1. propse - writing proposal +0ms
writing:  <Buffer f3 02 0a 10 1b c0 15 68 22 cd c6 e9 0d 99 7e ae 4e 64 d8 f3 12 ab 02 08 00 12 a6 02 30 82 01 22 30 0d 06 09 2a 86 48 86 f7 0d 01 01 01 05 00 03 82 01 ... >
  libp2p:secio 1. propose - proposal size 373 +6ms
I shall read: 373
reading <Buffer 0d 2f 73 65 63 69 6f 2f 31 2e 30 2e 30 0a f3 02 0a 10 98 a4 3a fb f0 07 e4 74 09 8c 9c 14 28 43 b9 6a 12 ab 02 08 00 12 a6 02 30 82 01 22 30 0d 06 09 ... >
reading msg <Buffer f3 02 0a 10 1b c0 15 68 22 cd c6 e9 0d 99 7e ae 4e 64 d8 f3 12 ab 02 08 00 12 a6 02 30 82 01 22 30 0d 06 09 2a 86 48 86 f7 0d 01 01 01 05 00 03 82 01 ... >
  libp2p:secio 1. propse - reading proposal +4ms <Buffer 0a 10 1b c0 15 68 22 cd c6 e9 0d 99 7e ae 4e 64 d8 f3 12 ab 02 08 00 12 a6 02 30 82 01 22 30 0d 06 09 2a 86 48 86 f7 0d 01 01 01 05 00 03 82 01 0f 00 ... >
  libp2p:secio 1.1 identify +1ms
  libp2p:secio 1.1 identify - QmVg942ZGNuGw7AFpiQAPsdRxzCK4J9e15NZGEBaixowQV - identified remote peer as QmRRTGZEykibGGAEpSDh3fK2UnL746x5zHeQzNzERc74uM +6ms
  libp2p:secio 1.2 selection +0ms
  libp2p:secio 1. propose - finish +1ms
  libp2p:secio 2. exchange - start +1ms
  libp2p:secio 2. exchange - writing exchange +0ms
writing:  <Buffer c6 02 0a 41 04 87 54 67 c3 15 ba f1 b6 ff b9 c3 0c 6f 8a e4 16 fc b6 7a 54 51 20 e6 ee 5d 63 c1 bd 89 f2 b0 59 15 4b dd 1b 4e 50 fc c2 81 24 42 79 96 ... >
I shall read: 328

# hangs..

The message that is lost is the first one with writing..

dignifiedquire · 2016-07-04T20:10:13Z

Adding the below snippet to https://github.com/diasdavid/js-multistream/blob/master/src/listener.js#L62

      if (handlers[msg]) {
        // Protocol supported, ACK back
        encode.write(new Buffer(msg + '\n'));

        if (conn.cork) {
          conn.cork()

          setTimeout(() => {
            conn.uncork()
          }, 100)
        }
        console.log('listener ready')
        return handlers[msg](conn);

fixes the issue.

dignifiedquire · 2016-08-09T12:00:01Z

Depends on

daviddias · 2016-08-18T13:43:29Z

@dignifiedquire just to confirm, this just depends on merges and releases, all the rest is 👌 ?

daviddias · 2016-08-18T13:44:25Z

What was the magic to not need node-forge anymore? libp2p-crypto? It doesn't require to be injected every single time anymore? @haadcode will be happy :)

daviddias · 2016-08-18T13:47:49Z

src/connection.js

@@ -35,7 +38,7 @@ module.exports = function connection (swarm) {
              ms.select(identify.multicodec, (err, conn) => {
                if (err) { return cb(err) }

-                identify.exec(conn, (err, peerInfo, observedAddrs) => {
+                identify.listen(conn, (err, peerInfo, observedAddrs) => {


note: This will change to 'dialer', once identify CR is applied

dignifiedquire · 2016-08-18T14:19:40Z

@dignifiedquire just to confirm, this just depends on merges and releases, all the rest is 👌 ?

Secio yes, but some of the other tests do not yet pass with all the new pull-stream things. Especially webrtc as that isn't migrated yet, and I'm having some issues with websockets, probably due to the halfOpen issue.

dignifiedquire · 2016-08-18T14:22:26Z

What was the magic to not need node-forge anymore? libp2p-crypto? It doesn't require to be injected every single time anymore? @haadcode will be happy :)

This: https://github.com/libp2p/js-libp2p-swarm/blob/56e558ea86c1eb5b5355ade8cda9652d1589c408/.aegir.js#L9-L12

daviddias · 2016-08-18T14:36:31Z

This: https://github.com/libp2p/js-libp2p-swarm/blob/56e558ea86c1eb5b5355ade8cda9652d1589c408/.aegir.js#L9-L12

Almost @haadcode, almost :) :D

dignifiedquire · 2016-08-24T15:13:05Z

All tests are now ported and mostly passing. Only some related to websockets are still failing:

dignifiedquire · 2016-08-30T19:20:42Z

Started doing some fixes for the websocket problems in

daviddias · 2016-09-02T15:24:39Z

~~@dignifiedquire having an hard time getting the stream muxing tests to pass:~~ nvm, got it to pass, it was the circular npm linking

    stream muxing with spdy (on TCP)
      ✓ add
      3) handle + dial on protocol
      4) dial to warm conn
      5) dial on protocol, reuse warmed conn
      6) enable identify to reuse incomming muxed conn
      7) with Identify, do getPeerInfo
      - make sure it does not blow up when the socket is closed
      - blow up a socket, with WebSockets
      ✓ close one end, make sure the other does not blow (1006ms)

daviddias · 2016-09-02T15:50:26Z

Last things before merge

point to lib instead of source
add documentation about pull-streams (same as libp2p-tcp)
make the encrypted channel be a option set by a func just like connection.reuse - [WIP] Add secio & move to pull-streams #67 (comment)
With [WIP] move to pull-streams js-libp2p-identify#2, listener and dialer need to be switched (as they should be)

daviddias · 2016-09-02T22:31:14Z

src/dial.js

+
+              const id = swarm._peerInfo.id
+              if (id.privKey == null || swarm.encrypt === false) {
+                return ms.select(tags.plaintex, cb)


this typo is what was causing my issue in libp2p-ipfs

dignifiedquire · 2016-09-07T10:23:05Z

make the encrypted channel be a option set by a func just like connection.reuse

This is done, needs the new secio api: libp2p/js-libp2p-secio#17

dignifiedquire · 2016-09-07T11:20:43Z

All tests pass locally when using the above mentioned secio branch plus libp2p/js-libp2p-webrtc-star#19

coveralls · 2016-09-07T13:54:26Z

Changes Unknown when pulling 6091e94 on secio into * on master*.

coveralls · 2016-09-07T14:00:11Z

Changes Unknown when pulling 6091e94 on secio into * on master*.

daviddias · 2016-09-07T14:12:11Z

@dignifiedquire why do we have coveralls badges all over again?

coveralls · 2016-09-07T14:12:16Z

Changes Unknown when pulling 6091e94 on secio into * on master*.

dignifiedquire · 2016-09-07T14:14:09Z

@dignifiedquire why do we have coveralls badges all over again?

trying to disable them one at a time, but they are enabled by default :(

dignifiedquire force-pushed the secio branch 3 times, most recently from 6d53c61 to bf1574d Compare May 27, 2016 12:56

dignifiedquire mentioned this pull request May 30, 2016

Sprint: May 16 → May 30th ipfs/team-mgmt#109

Closed

daviddias mentioned this pull request Jun 14, 2016

🚣 captain.log - IPFS JavaScript implementation 🌟 ipfs/js-ipfs#30

Closed

dignifiedquire force-pushed the secio branch from c33804e to ddab26f Compare July 1, 2016 14:23

dignifiedquire added 2 commits August 9, 2016 13:06

start integrating secio

d1e337e

first passing secio tests

5ab7db2

dignifiedquire force-pushed the secio branch from 10c73da to 5ab7db2 Compare August 9, 2016 11:57

dignifiedquire changed the title ~~[WIP] Add secio~~ [WIP] Add secio & move to pull-streams Aug 9, 2016

dignifiedquire mentioned this pull request Aug 9, 2016

refactor: ⎆ Migration to pull-streams (from Node.js Streams) ipfs/js-ipfs#403

Closed

28 tasks

all non ws tests passing

56e558e

daviddias reviewed Aug 18, 2016
View reviewed changes

fix more tests

ea9c93c

daviddias mentioned this pull request Aug 30, 2016

Sprint: Aug 29 → Sept 4 ipfs/team-mgmt#156

Closed

feat(deps): downgrade aegir to 6.0.1

784934d

daviddias reviewed Sep 2, 2016
View reviewed changes

fix(dial): fix typo causing plaintext mode to fail

381d3c2

update to latest dependencies and add new crypto api

c28b845

chore(deps): libp2p-secio is a devDependency now

42fafd8

daviddias force-pushed the secio branch 2 times, most recently from bfbe84f to c6bd83b Compare September 7, 2016 13:44

fix(dial): fix typo causing plaintext mode to fail

6091e94

daviddias force-pushed the secio branch from c6bd83b to 6091e94 Compare September 7, 2016 13:50

daviddias merged commit 546c9bc into master Sep 7, 2016

daviddias deleted the secio branch September 7, 2016 14:12

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[WIP] Add secio & move to pull-streams #67

[WIP] Add secio & move to pull-streams #67

dignifiedquire commented May 23, 2016

dignifiedquire commented Jul 4, 2016 •

edited

Loading

dignifiedquire commented Jul 4, 2016

dignifiedquire commented Aug 9, 2016

daviddias commented Aug 18, 2016

daviddias commented Aug 18, 2016 •

edited

Loading

daviddias Aug 18, 2016

dignifiedquire commented Aug 18, 2016

dignifiedquire commented Aug 18, 2016

daviddias commented Aug 18, 2016

dignifiedquire commented Aug 24, 2016

dignifiedquire commented Aug 30, 2016

daviddias commented Sep 2, 2016 •

edited

Loading

daviddias commented Sep 2, 2016 •

edited by dignifiedquire

Loading

daviddias Sep 2, 2016

dignifiedquire commented Sep 7, 2016

dignifiedquire commented Sep 7, 2016

coveralls commented Sep 7, 2016 •

edited

Loading

coveralls commented Sep 7, 2016 •

edited

Loading

daviddias commented Sep 7, 2016

coveralls commented Sep 7, 2016 •

edited

Loading

dignifiedquire commented Sep 7, 2016

[WIP] Add secio & move to pull-streams #67

[WIP] Add secio & move to pull-streams #67

Conversation

dignifiedquire commented May 23, 2016

dignifiedquire commented Jul 4, 2016 • edited Loading

dignifiedquire commented Jul 4, 2016

dignifiedquire commented Aug 9, 2016

daviddias commented Aug 18, 2016

daviddias commented Aug 18, 2016 • edited Loading

daviddias Aug 18, 2016

Choose a reason for hiding this comment

dignifiedquire commented Aug 18, 2016

dignifiedquire commented Aug 18, 2016

daviddias commented Aug 18, 2016

dignifiedquire commented Aug 24, 2016

dignifiedquire commented Aug 30, 2016

daviddias commented Sep 2, 2016 • edited Loading

daviddias commented Sep 2, 2016 • edited by dignifiedquire Loading

daviddias Sep 2, 2016

Choose a reason for hiding this comment

dignifiedquire commented Sep 7, 2016

dignifiedquire commented Sep 7, 2016

coveralls commented Sep 7, 2016 • edited Loading

coveralls commented Sep 7, 2016 • edited Loading

daviddias commented Sep 7, 2016

coveralls commented Sep 7, 2016 • edited Loading

dignifiedquire commented Sep 7, 2016

dignifiedquire commented Jul 4, 2016 •

edited

Loading

daviddias commented Aug 18, 2016 •

edited

Loading

daviddias commented Sep 2, 2016 •

edited

Loading

daviddias commented Sep 2, 2016 •

edited by dignifiedquire

Loading

coveralls commented Sep 7, 2016 •

edited

Loading

coveralls commented Sep 7, 2016 •

edited

Loading

coveralls commented Sep 7, 2016 •

edited

Loading