[2021 Theme Proposal] Composability of core implementations

[dchoi27]

Note, this is part of the 2021 IPFS project planning process - feel free to add other potential 2021 themes for the IPFS project by opening a new issue or discuss this proposed theme in the comments, especially other example workstreams that could fit under this theme for 2021. Please also review others’ proposed themes and leave feedback here!

Theme description

Strategically refactor large parts out of the core of IPFS and demarcate the different modules while creating a seamless experience working across the InterPlanetary Stack.

Hypothesis

It is currently difficult to hack on the InterPlanetary Stack, either as a contributor or someone building tools, due to the lack of composability and modularity of the Stack. This limits contributions to the ecosystem to those who are already very deep on the history and context.

Vision statement

The composability of the InterPlanetary Stack unlocks huge value - upgrade pathways of existing parts of the stack are much clearer, the stack is extensible and the community builds their own implementations of pieces of the stack, and it enables a rich tooling ecosystem which makes the experience of developing end user applications much easier. There is a clear spec for the InterPlanetary Stack, increasing alignment across the project and making it easier for new developers to understand how to contribute.

Why focus this year

Improving composability can be a huge point of leverage, improving already active development efforts on IPFS, bringing new contributors into the ecosystem, and clearing up misconceptions (e.g., what is an IPFS, IPFS vs. Filecoin). Doing this before more technical debt builds up can potentially make things much easier in the long run.

Example workstreams

Separating spec from implementations, strategy around what lives in core vs. modules of IPFS implementations (like in node), refactoring to pull big parts of the stack out of the core, clarifying what an IPFS is vs. implementations, making docs clear to reflect this and implementation options for modules, upgrade paths for these various modules (e.g., CLI, HTTP API, DHT, UnixFS, CID, Bitswap/GraphSync, IPNS, DNSLink, multistream), seamless integration of IPLD and IPFS workflows

Other content

[aschmahmann]

Adding my 2c here that I think having upgrade paths for modules is really important to enable protocol improvements and to encourage experimentation.

It also seems like some components of this theme (e.g. clarifying the difference between IPFS and an IPFS implementation) seem required in order to complete IPFS v1.0. Others may not be strictly required but could be quite helpful (e.g. strategy around core vs modules) since it may enable us to leave certain things out of IPFS v1.0 but still provide mechanisms for people to build or include their own versions of those modules in their IPFS implementations.

[carsonfarmer]

I'd like to add a +1 here from @textileio's perspective. In particular, we are interested in composability of IPLD codecs across the IPFS stack. So an enthusiastic +1 for better custom IPLD - IPFS workflows would be great. In particular, a near-term move towards supporting go-ipld-prime would be ideal. Something like the first half of 2021 would be reasonable. Our team is ready to start writing custom codecs, and moving some of our custom application logic and bit further "down" the stack to be shared with other application developers as soon as possible. See also @sanderpick's comments on this theme: #65 (comment)

[github-actions]

This issue is stale because it has been open 30 days with no activity. Remove stale label or comment or this will be closed in 5 days.