nodetool repair: try repairing all keyspaces even if one fails?

[kbr-]

When the user runs nodetool repair (without specifying a keyspace), nodetool will try to repair all keyspaces, iterating over them ony be one.

However, if repair fails at one of these keyspaces, nodetool will stop there. In particular, if one of the replicas for a keyspace is down, nodetool repair fails after that keyspace. But that may be undesirable. Consider the case where we want to remove this node. According to the procedure described here, the admin should run full cluster repair before removing a node. Unfortunately, because of this behavior, the admin cannot run a full cluster repair by simply executing nodetool repair, because it will fail on the first keyspace that uses the down node as one of the replicas. Thus the admin would need to go over all keyspaces manually.

A better behavior would be if nodetool iterated over all keyspaces even if one of them fails - so every keyspace gets partially repaired (i.e. all living nodes synchronize their data). Perhaps there should be a configuration option --continue-on-failure or something like that.

[kbr-]

cc @asias