major stress tests to run prior to releases

[jbenet]

(this summarizes some prior discussions)

Concretely, we need large workload tests:

• that exercise an ipfs node to a major extent
• that interleave sequences of operations and full integrity checks (pin set, content is correct, through a gc)
• that involve adding lots of content -- from all possible entry points, {add, files, object}
• that add and remove pins, and run gc
• that should stress with many GB of content (i'd shoot for a test that can scale with a parameter in GB),
• that involve millions of objects, thousands of pins, and thousands of ops (maybe these should all be parameters to the test too)
• that issue concurrent operations through the api (ipfs should be able to handle these just fine, whether they apply concurrently or serially, the api should work with requests made all at once)

[jbenet]

we need:

• operation specification
• a thing that generates a sequence of N ops randomly based on an input randomness seed
• a thing that -- given the sequence -- issues all ops to ipfs (using ipfs cli or api endpoint)
• a thing that -- given the sequence -- can then check the ipfs node state (using the api)
• a thing that -- given the sequence -- can then check the ipfs repo state (directly in the fs)
• a runner that puts it all together
• being able to bisect out problems

[dignifiedquire]

Rough prototype in JS using the api: https://github.com/dignifiedquire/randor

[Jorropo]

We now do this with https://github.com/ipfs-shipyard/thunderdome, it mirrors ipfs.io's traffic.