Support on-the-fly tensorflow inference

[kodonnell]

Currently, I believe the inferences runs entirely first, before any annotation can begin. This can be pretty slow, especially on slow hardware and large videos etc. I'm requesting that the model runs 'on demand'. To prevent a visible slow-down to the user, we should buffer (e.g. when I start annotating frame N, inferences starts on frame N + 3, so that it's complete by the time I get to it). This seems effective as I can start annotating nearly immediately, and there's no noticeable slow-down while annotating (as the time taken annotating a frame - which things are currently idea - should hopefully exceed model evaluation time).

Note - we'd actually have to consider how to add in support for e.g. evaluating the model only every Nth frame (e.g. when I'm annotating I generally only annotate every 10th frame, and use interpolation for the rest - so it makes sense to only run tensorflow on every 10th frame too).

This seems reasonably feasible - is there likely to be interest?

[nmanovic]

@kodonnell ,

It is definitely interesting and a good idea.
Let's add the following features:

• Annotate data with a step, from start to end frames.
• Add a button to annotate a specific frame (during task annotation). Thus when you annotate a task you can click the button, client will send request to server and wait a reply. After a couple of seconds ther server will send response with annotation results.

If you can help to implement the future it will be awesome.

[kodonnell]

@nmanovic - I assume by the first point you're meaning "on the fly" as I described it? (I.e. not just running from start to end by step with the current batch processing.) If not, we might need to add my 'buffered' on-the-fly annotation (which could be enabled/disabled via the GUI). I'm unsure of your second option - clicking and having to wait is not great from a UI perspective, and is much less efficient than the 'buffered' approach.

Thinking more - we should probably cache the tensorflow annotations for any of these on-the-fly approaches. E.g. if you go back to a frame, there's no point recomputing the annotations. That should be pretty straightforward though.

Re a PR - I think it makes sense to wait for the PR from @savan77 (hopefully soon?). Does that seem reasonable?