Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Protocol Spec Update #18

Closed
wants to merge 19 commits into from
Closed

Protocol Spec Update #18

Show file tree
Hide file tree
Changes from 15 commits
Commits
Show all changes
19 commits
Select commit Hold shift + click to select a range
9d2e6e6
add initial DHT spec with some questions
daviddias Jun 26, 2015
1feb2cf
add initial DHT spec with some questions
daviddias Jun 26, 2015
2135736
add ping
daviddias Jun 26, 2015
751c906
provinding segment
daviddias Jun 26, 2015
0e260f7
put question
daviddias Jun 26, 2015
dad0bb7
separate DHT, discovery and routing specs
daviddias Jun 30, 2015
091817b
landing ideas
daviddias Jun 30, 2015
aae56f2
layers
daviddias Jul 6, 2015
2546f85
make swarm part of the network layer and put NAT traversal, connectio…
daviddias Jul 6, 2015
a601ddc
some notes on swarm
daviddias Jul 6, 2015
61efc28
Merge branch 'master' of github.com:ipfs/specs into protocol-spec
daviddias Jul 17, 2015
61bfe39
add chapters numbers to sections on the readme so it is more clear ho…
daviddias Jul 18, 2015
c67ca71
moved the network layer part to its own spec/readme
daviddias Jul 18, 2015
6c508c6
update network spec to latest
daviddias Jul 18, 2015
1c17e08
remove layers.md
daviddias Jul 18, 2015
5a108f6
remove outdated protocol overview
daviddias Jul 18, 2015
b503949
add myself as an author, normalize how authors are written
daviddias Jul 19, 2015
fea3fde
merge wire in, first steps on the libp2p full spec
daviddias Jul 24, 2015
7117a9b
moar stuff
daviddias Jul 24, 2015
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
56 changes: 31 additions & 25 deletions protocol/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,22 +1,31 @@
# IPFS Protocol Spec (WIP!)
IPFS Protocol Spec
==================

Authors: [@jbenet](http://github.com/jbenet)
> **This spec is an Work In Progress (WIP)**

Authors:

- [@jbenet](http://github.com/jbenet)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

maybe add yourself ?


Reviewers:

* * *

This [spec](../) document defines the IPFS protocol stack, the subsystems, the
This spec document defines the IPFS protocol stack, the subsystems, the
interfaces, and how it all fits together. It delegates non-interface details
to other specs as much as possible. This is meant as a top-level view of the
protocol and how the system fits together.


Note, this document is not meant to be an introduction of the concepts in IPFS
and is not recommended as a first pass to understanding how IPFS works. For
that, please refer to the [IPFS paper](http://static.benet.ai/t/ipfs.pdf).

## IPFS and the Merkle DAG
# Index

- []()
- []()

## 1. IPFS and the Merkle DAG

At the heart of IPFS is the MerkleDAG, a directed acyclic graph whose links
are hashes. This gives all objects in IPFS useful properties:
Expand All @@ -41,7 +50,7 @@ publish, distribute, serve, and download merkledags. It is the authenticated,
decentralized, permanent web.


## Nodes and Network Model
## 2. Nodes and Network Model

The IPFS network uses PKI based identity. An "ipfs node" is a program that
can find, publish, and replicate merkledag objects. Its identity is defined
Expand All @@ -54,7 +63,7 @@ nodeID := multihash(publicKey)

TODO: constraints on keygen.

### multihash and upgradeable hashing
### 2.1 multihash and upgradeable hashing

All hashes in ipfs are encoded with
[multihash](https://github.com/jbenet/multihash/), a self-describing hash
Expand All @@ -75,7 +84,7 @@ sha3
```


## The Stack
## 3. The Stack

IPFS has a stack of modular protocols. Each layer may have multiple
implementations, all in different modules. This spec will only address the
Expand All @@ -94,7 +103,7 @@ IPFS has five layers:

These are briefly described bottom-up.

### Network -- connecting to peers
### [3.1 Network](network)

The **network** provides point-to-point transports (reliable and unreliable)
between any two IPFS nodes in the network. It handles:
Expand All @@ -105,7 +114,7 @@ between any two IPFS nodes in the network. It handles:

See more in the [network spec](network).

### Routing -- finding peers and data
### [3.2 Routing -- finding peers and data](routing)

The IPFS **Routing** layer serves two important purposes:
- **peer routing** -- to find other nodes
Expand All @@ -122,9 +131,9 @@ of implementations. For example:
to one of a set of supernodes. This is roughly like federated routing.
- **dns:** ipfs routing could even happen over dns.

See more in the routing spec (TODO).
See more in the [routing spec](https://github.com/ipfs/specs/tree/master/protocol/routing).

### Block Exchange -- transfering content-addressed data
### [3.3 Block Exchange -- transfering content-addressed data](exchange)

The IPFS **Block Exchange** takes care of negotiating bulk data transfers.
Once nodes know each other -- and are connected -- the exchange protocols
Expand All @@ -137,7 +146,7 @@ of implementations. For example:
of BitTorrent to work with arbitrary (and not known apriori) DAGs.
- **HTTP:** a simple exchange can be implemented with HTTP clients and servers.

### Merkledag -- making sense of data
### [3.4. Merkledag -- making sense of data](../merkledag)

[As discussed above](#IPFS-and-the-Merkle-DAG), the IPFS **merkledag** is the
datastructure at the heart of IPFS. It is an
Expand Down Expand Up @@ -170,7 +179,7 @@ on top of the merkledag, such as:

See more in the merkledag spec (TODO).

### Merkledag Paths
### [3.4.1 Merkledag Paths](../merkledag)

The merkledag is enough to resolve paths:

Expand All @@ -186,7 +195,7 @@ See more in the path resolution spec (TODO).

![](../media/ipfs-resolve/ipfs-resolve.gif)

### Naming -- PKI namespace and mutable pointers
### [3.5 Naming -- PKI namespace and mutable pointers]()

IPFS is mostly concerned with content-addressed data, which by nature
is immutable: changing an object would change its hash -- and thus its
Expand All @@ -209,7 +218,7 @@ See more in the namin spec (TODO).



## Applications and Datastructures -- on top of IPFS
## [4. Applications and Datastructures -- on top of IPFS]()

The stack described so far is enough to represent arbitrary datastructures
and replicate them accross the internet. It is also enough to build and
Expand All @@ -222,7 +231,7 @@ them to the rest of the world using any of the tools that understand IPFS.
See more in the datastructures and applications specs (TODO).


### unixfs -- representing traditional files
### [4.1 unixfs -- representing traditional files]()

The unix filesystem abstractions -- files and directories -- are the main way
people conceive of files in the internet. In IPFS, `unixfs` is a datastructure
Expand All @@ -235,7 +244,7 @@ to carry over information like:
See more in the unixfs spec (TODO).


## Lifetime of fetching an object.
## [5 Lifetime of fetching an object.]()

Suppose we ask an IPFS node to retrieve

Expand All @@ -253,11 +262,7 @@ Then, the IPFS node resolves the components.
The first component in an `/ipfs/...` path is always a multihash.
The rest are names of links, to be resolved into multihashes.





## IPFS User Interfaces
## [6 IPFS User Interfaces]()

IPFS is not just a protocol. It is also a toolset. IPFS implementations
include various tools for working with the merkledag, how to publish
Expand All @@ -271,9 +276,10 @@ design and implementation. Examples:
- The IPFS libs - implementations in various languages
- The IPFS gateways - nodes in the internet that serve HTTP over IPFS

## ~~WIP~~
* * *

### WIP Stack Dump:

WIP Stack Dump:
- How the layers fit together
- How they call on each other
- Mention all the ports
Expand Down
Binary file added protocol/figs/overview.monopic
View file
Open in desktop
Binary file not shown.
21 changes: 21 additions & 0 deletions protocol/figs/overview.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,21 @@
┌──────────────────────────────────────────────┐
│ MerkleDAG │
└──────────────────────────────────────────────┘
▷ ┌──────────────────────────────────────────────┐
│ │ bitswap │
exchange│ └──────────────────────────────────────────────┘
│ ┌──────────────────────────────────────────────┐
│ │ host │
▷ └──────────────────────────────────────────────┘
▷ ┌──────────────────────────────────────────────┐
│ │ routing │
routing│ └──────────────────────────────────────────────┘
│ ┌────────────────────┐ ▲ ┌ ─ ─ ─ ─ ─ ─ ─ ─
│ │ swarm │ │ ┌───────────────┐
│ │ │◀─┴─────┤│ discovery │
│ │ │ └───────────────┘
▷ └────────────────────┘ └ ─ ─ ─ ─ ─ ─ ─ ─
▷ ┌──────────┐┌────────┐
│ │connection││protocol│
network│ │ ││muxing │
▷ └──────────┘└────────┘
Loading