Miniature solarnet lab: resource constraints, testings, and discoveries

[rht]

On building several raspis/c.h.i.ps scattered within wifi range (~$20xn) for the purpose of:

1. Pushing the resource constraints for low-powered devices. Has been around for a while (Resource Constraints + Limits kubo#1482, introduce low memory flag kubo#2012, bitswap consumption Smarter bitswap kubo#1750)
2. Testing (test + benchmark adding various workloads kubo#2055, add long running network tests that test different workloads kubo#2056, add iptb based tests to various network parts of the system kubo#2057)
3. Proving the O(torrent) speedup in local network (the first example in https://www.youtube.com/watch?v=HUVmypx9HGI#t=14m15s (but why?), Improve transfer speed kubo#2111, local transfer takes forever kubo#872).
4. Offline/high-latency functionalities (ref: https://www.youtube.com/watch?v=HUVmypx9HGI "I want to live in a world where the applications that I use will continue working whether or not I'm in the best place in the network" (and whether or not having adequate tools)).
5. Optimizing ipfs as a lan/local unixfs (fixing issues like FUSE mount is very slow kubo#2166) / immutable fs / for /nix/store. I don't know of the constraints, but what does it take to go O(zfs) here? imo, making the files api fast is a very time-consuming rewrite, instead of a reuse. I think, for now, git-annex could be a more suitable interface. Is there any relevant ipfs/notes to link from? I only know of IPLD Selector notes#12.
6. Mini-cluster (and pin coop) prototypes (ipfs-cluster - tool to coordinate between nodes notes#58)
7. There is more, but unspecified.

Not sure if this is a silly thing to do, whether it is better off to test everything on vm/iptb/network simulator.
Yet, this could uncover ux bugs / some unexpected use cases, while maintaining a consistent unixweb interface that already works cross-geographically (e.g. the ipfs.io solarnet).

I have questions on the choice of 1. programming system and 2. protocols.
For 1, if the language of the web is js, what would be the language of connected devices (other than safe c/cpp)? Is using golang (channels/goroutine) particularly advantageous? Or micropython or rust?
For 2, how would ipfs+ipld fit in the 'web of things'[1] ecosystem? I just discovered of CoAP (for "REST", rfc7252) and MQTT (for pub-sub), can they be used on cjdns, or require a rewrite?

Estimate of figures:

• file sizes: few KB to few distributed GB (measurement logs)
• file lifetime: hours to infty
• network bandwidth/latency: (no need to deal with dmca/isp)

[1] known to be highly fragmented, as stated in https://www.w3.org/community/wot/.

[davidar]

I think, for now, git-annex could be a more suitable interface

I've also thought about that. Cc: @cryptix

[ghost]

This work has started in https://github.com/ipfs/test-lab