Proposal: streaming API for big UASTs

[bzz]

Right now, after #216 both bbfshd and driver gRCP servers have a hard-coded limit of 100MiB for the UAST.

This works, but is not a generic solution - UAST size is O(1) O(n log n) of the file size n but the constant can be big.

A generic solution proposed though initial discussion and enabled by v2 wire serialization format is: add a new API call that would chunk and stream the response to the client.

TODOs:

• verify initial idea with clients (gitbase, ML, Lookout)
• propose API draft/signature
• implement in bblfshd<=>driver
• expose in bblfshd/clinet-*

[creachadair]

As a side note, it's worse than O(1) in the file size—it's more like O(n log n) in the file size n, where the log base constant falls out of the grammar size (upper bounded by the branching factor, but probably not that bad on average). The bound isn't tight, since some files have flat parses, but it's definitely not in O(1).