collaborative pinning

[eminence]

Someone should think about a protocol that X cooperating people/nodes to pin a large dataset (of size S) in a collaborative manner, so that each node is pinning something of size S/X.

This is something that would be very useful when hosting something that is generally too large for just 1 node to host.

A robust solution would allow for a configurable level of redundancy, and a way for data to be pinned by new nodes that join the cooperative group.

See also #49 and some of the discussions in the archives repo.

[eminence]

From IRC:

@jbenet : we need to make "ipfs-cluster" -- a tool to manage pin-sets larger than one
          node's local storage. (or really, representing one node with multiple)
@jbenet : but before that, just shard the dataset. if it's hierarchical at all, super easy.
          if not, just split the data for now.

[jbenet]

See ipfs-cluser at #58