bubble the spec up #7
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -1,84 +1,168 @@ | ||
| multistream | ||
| =========== | ||
|
|
||
| [](http://ipn.io) | ||
| [](http://ipfs.io/) | ||
| [](http://webchat.freenode.net/?channels=%23ipfs) | ||
|
|
||
| > Friendly protocol multiplexing. It enables a multicodec to be negotiated between two entities. | ||
|
|
||
| ## Motivation | ||
|
|
||
| Some protocols have sub-protocols or protocol-suites. Often, these sub protocols are optional extensions. Selecting which protocol to use -- or even knowing what is available to chose from -- is not simple. | ||
|
|
||
| What if there was a protocol that allowed mounting or nesting other protocols, and made it easy to select which protocol to use. (This is sort of like ports, but managed at the protocol level -- not the OS -- and human readable). | ||
|
|
||
| ### Protocol | ||
|
|
||
| The actual protocol is very simple. It is a multistream protocol itself, it has a multicodec header. And it has a set of other protocols available to be used by the remote side. The remote side must enter: | ||
|
|
||
| ``` | ||
| > <multicodec-header-of-multistream> | ||
| > <multicodec-header-for-whatever-protocol-that-we-want-to-speak> | ||
| ``` | ||
|
|
||
| for example: | ||
|
|
||
| ``` | ||
| > /ipfs/QmdRKVhvzyATs3L6dosSb6w8hKuqfZK2SyPVqcYJ5VLYa2/multistream-select/0.3.0 | ||
| > /ipfs/QmVXZiejj3sXEmxuQxF2RjmFbEiE9w7T82xDn3uYNuhbFb/ipfs-dht/0.2.3 | ||
| ``` | ||
|
|
||
| - The `<multicodec-header-of-multistream>` ensures a protocol selection is happening. | ||
| - The `<multistream-header-for-whatever-protocol-is-then-selected>` hopefully describes a valid protocol listed. Otherwise we return a `na`("not available") error: | ||
|
|
||
| ``` | ||
| na\n | ||
|
|
||
| # in hex (note the varint prefix = 3) | ||
| # 0x036e610a | ||
| ``` | ||
|
|
||
| for example: | ||
|
|
||
| ```sh | ||
| # open connection + send multicodec headers, inc for a protocol not available | ||
| > /ipfs/QmdRKVhvzyATs3L6dosSb6w8hKuqfZK2SyPVqcYJ5VLYa2/multistream-select/0.3.0 | ||
| > /ipfs/QmVXZiejj3sXEmxuQxF2RjmFbEiE9w7T82xDn3uYNuhbFb/some-protocol-that-is-not-available | ||
|
|
||
| # open connection + signal protocol not available. | ||
| < /ipfs/QmdRKVhvzyATs3L6dosSb6w8hKuqfZK2SyPVqcYJ5VLYa2/multistream-select/0.3.0 | ||
| < na | ||
|
|
||
| # send a selection of a valid protocol + upgrade the conn and send traffic | ||
| > /ipfs/QmVXZiejj3sXEmxuQxF2RjmFbEiE9w7T82xDn3uYNuhbFb/ipfs-dht/0.2.3 | ||
| > <dht-traffic> | ||
| > ... | ||
|
|
||
| # receive a selection of the protocol + sent traffic | ||
| < /ipfs/QmVXZiejj3sXEmxuQxF2RjmFbEiE9w7T82xDn3uYNuhbFb/ipfs-dht/0.2.3 | ||
| < <dht-traffic> | ||
| < ... | ||
| ``` | ||
|
|
||
| **Note 1:** Every multistream message is a "length-prefixed-message", which means that every message is preprended by a varint that describes the size of the message. | ||
|
|
||
| **Node 2:** Every multistream message is appended by a `\n` character, this character is included in the byte count that is accounted by the prepended varint. | ||
|
|
||
| #### Listing | ||
|
|
||
| It is also possible to "list" the available protocols. A list message is simply: | ||
|
|
||
| ``` | ||
| ls\n | ||
|
|
||
| # in hex (note the varint prefix = 3) | ||
| 0x036c730a | ||
| ``` | ||
|
|
||
| So a remote side asking for a protocol listing would look like this: | ||
|
|
||
| ```sh | ||
| # request | ||
| <multistream-header-for-multistream-select> | ||
| ls\n | ||
|
|
||
| # response | ||
| <varint-total-response-size-in-bytes><varint-number-of-protocols> | ||
| <multicodec-of-available-protocol> | ||
| <multicodec-of-available-protocol> | ||
| <multicodec-of-available-protocol> | ||
| ... | ||
| ``` | ||
|
|
||
| For example | ||
|
|
||
| ```sh | ||
| # send request | ||
| > /ipfs/QmdRKVhvzyATs3L6dosSb6w8hKuqfZK2SyPVqcYJ5VLYa2/multistream-select/0.3.0 | ||
| > ls | ||
|
|
||
| # get response | ||
| < /ipfs/QmdRKVhvzyATs3L6dosSb6w8hKuqfZK2SyPVqcYJ5VLYa2/multistream-select/0.3.0 | ||
| < /ipfs/QmVXZiejj3sXEmxuQxF2RjmFbEiE9w7T82xDn3uYNuhbFb/ipfs-dht/0.2.3 | ||
| < /ipfs/QmVXZiejj3sXEmxuQxF2RjmFbEiE9w7T82xDn3uYNuhbFb/ipfs-dht/1.0.0 | ||
| < /ipfs/QmVXZiejj3sXEmxuQxF2RjmFbEiE9w7T82xDn3uYNuhbFb/ipfs-bitswap/0.4.3 | ||
| < /ipfs/QmVXZiejj3sXEmxuQxF2RjmFbEiE9w7T82xDn3uYNuhbFb/ipfs-bitswap/1.0.0 | ||
|
|
||
| # send selection, upgrade connection, and start protocol traffic | ||
| > /ipfs/QmVXZiejj3sXEmxuQxF2RjmFbEiE9w7T82xDn3uYNuhbFb/ipfs-dht/0.2.3 | ||
| > <ipfs-dht-request-0> | ||
| > <ipfs-dht-request-1> | ||
| > ... | ||
|
|
||
| # receive selection, and upgraded protocol traffic. | ||
| < /ipfs/QmVXZiejj3sXEmxuQxF2RjmFbEiE9w7T82xDn3uYNuhbFb/ipfs-dht/0.2.3 | ||
| < <ipfs-dht-response-0> | ||
| < <ipfs-dht-response-1> | ||
| < ... | ||
| ``` | ||
|
|
||
| ### Example | ||
|
|
||
| ``` | ||
| # greeting | ||
| > /http/multiproto.io/multistream-select/1.0 | ||
| < /http/multiproto.io/multistream-select/1.0 | ||
|
|
||
| # list available protocols | ||
| > /http/multiproto.io/multistream-select/1.0 | ||
| > ls | ||
| < /http/google.com/spdy/3 | ||
| < /http/w3c.org/http/1.1 | ||
| < /http/w3c.org/http/2 | ||
| < /http/bittorrent.org/1.2 | ||
| < /http/git-scm.org/1.2 | ||
| < /http/ipfs.io/exchange/bitswap/1 | ||
| < /http/ipfs.io/routing/dht/2.0.2 | ||
| < /http/ipfs.io/network/relay/0.5.2 | ||
|
|
||
| # select protocol | ||
| > /http/multiproto.io/multistream-select/1.0 | ||
| > ls | ||
| > /http/w3id.org/http/1.1 | ||
| > GET / HTTP/1.1 | ||
| > | ||
| < /http/w3id.org/http/1.1 | ||
| < HTTP/1.1 200 OK | ||
| < Content-Type: text/html; charset=UTF-8 | ||
| < Content-Length: 12 | ||
| < | ||
| < Hello World | ||
| ``` | ||
|
|
||
| ### ls example in detail | ||
|
|
||
| ``` | ||
| > <varint-length>ls<newline> | ||
| < <varint-total-response-size-in-bytes><varint-number-of-protocols><newline> | ||
| < <varint-protocol-length-plus-new-line><protocol><newline> | ||
| # ... | ||
| < <varint-protocol-length-plus-new-line><protocol><newline> | ||
| ``` | ||
|
There was a problem hiding this comment. I've added this example in detail to solve the ambiguity of the spec. @jbenet @whyrusleeping please confirm |
||
|
|
||
| # Implementations | ||
|
|
||
| - [js-multistream](https://github.com/diasdavid/js-multistream) - JavaScript Implementation | ||
| - [go-multistream](https://github.com/whyrusleeping/go-multistream) - Go Implementation | ||
| - [mss-nc](https://github.com/whyrusleeping/mss-nc) - multistream netcat written in Go | ||
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
MOAR varints!!
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@diasdavid we settled this to be:
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
For record,
MOAR varintstranslates to another varint that should be added on the beginning of line 158 that tells the byte size of the following two varints + newline