Incrementally purge oldest revisions in SQLite databases

[snej]

The SQLite storage engine only prunes documents — deletes metadata of old revisions that are deeper down than the maxRevTreeDepth — when the database's Compact method is called by the app. This means that a doc that's frequently updated will accumulate more and more entries in the revs table.

We have evidence from the field that this eventually slows down database operations on that document to an unacceptable degree (albeit in an extreme scenario where a doc is updated several times a minute for hours, while multiple replications are running.)

We can fix this by pruning the revision tree periodically (the way the ForestDB storage does) as new revisions are added. -[CBLSQLiteStorage forceInsert:...] (called by pull replications) looks like it could be extended to prune old revisions, since it already loads the metadata of every revision of the document. The -addDocID:... method (called by local puts) is a bit harder to fix since it doesn’t have the whole document history available. It might be sufficient to check if the new revision’s generation # is a multiple of the maxRevTreeDepth, and if so trigger a prune operation on that document.

[snej]

Merged to dev branch; closing.