Threading

main branch runs everything on the main thread:

Main thread     [read][demux|theora|vorbis][play|draw]  [read][demux|theora|vorbis][play|draw]

decoder-worker branch attempts to move the entire decoder to a worker thread -- this works, but there's not a lot of chance for parallel processing right now:

Main thread     [read]                     [play|draw]  [read]                     [play|draw]
Decoder thread        [demux|theora|vorbis]                   [demux|theora|vorbis]

On slower mobile devices at lower resolutions (160p/15fps) the audio handling ends up being almost as expensive as the video -- it may be advantageous to separate out the demuxer layer from the video and audio codecs, then we can decode the video and audio in parallel.

Main thread     [read][demux]        [play|draw]  [read][demux]        [play|draw]
Theora thread                [theora]                          [theora]
Vorbis thread                [vorbis]                          [vorbis]

This will require having separate emscripten programs for the demuxer, the theora decoder, and the vorbis decoder, and data will have to be copied out of the heaps. This may or may not be a win overall, but could be interesting to try.

It may also be possible to parallelize the YUV conversion by splitting the picture in half and giving half to each of two worker threads. Again, not sure if this would be worth the overhead, but it could help. However this means copying data outside of the CanvasPixelArrays on IE and could get messy... avoid.