Video stream issues: Jumpy MJPEG stream / isolating H264 frames from the stream

[wilpulling]

Hi,

I'm trying to do image processing on the video stream (from a Bebop 2) outside of node (for example, in a Python process). My goal is to use a named pipe to send each frame to the other process. I've managed to achieve this with getMjpegStream, but that stream is very inconsistent (even without my sending images over the pipe) and gives me about 1 fps. When I run the example, I get the following error repeatedly:

Error: Error loading file at Error (native) at Timeout._repeat (/home/pi/roboTC/controller_files/node-bebop/examples/mjpeg-stream.js:25:8) at Timeout.wrapper [as _onTimeout] (timers.js:417:11) at tryOnTimeout (timers.js:224:11) at Timer.listOnTimeout (timers.js:198:5)

And occasionally, I get the following error:
Corrupt JPEG data: premature end of data segment

I'd either like to either figure out a way to improve the quality of the MJPEG stream, or figure out how to use the H264 stream. The example provided for using the H264 stream pipes the feed to a file, but I need to process in real time.

Can anyone recommend a fix for the MJPEG stream, or give me guidance / an example of how I can isolate H264 frames so I can convert them to Base64 and pipe them out?

Relevant previous threads:
#9 (Live stream to browser) / #14 (newbie question. how can i get videostream off the bebop)

Both of these are focused on using Broadway to display the feed in a browser. Since I'm keeping the feed local it seems like this doesn't apply to my situation, and I haven't been able to get anything working after playing with Broadway.

Hardware and OS:
Raspberry Pi 3 running Raspbian Jessie.

Thanks everyone, appreciate any help you can provide!

[wilpulling]

Some additional information:
The "error loading file" I'm getting from mjpeg-stream.js is occuring every time the mjpg.on("data") listener receives a chunk which is less than ~40k bytes in length. These 'short' chunks are typically ~14k bytes in length. This makes me think the stream is emitting a partial frame.

Wanted to add this detail in case it gives anyone an idea of what's going on. Would still appreciate tips on using the h264 stream if that could be faster! Thanks!

[deadprogram]

Hi @wilpulling sorry about the greatly delayed response.

Please checkout the latest changes to the https://github.com/hybridgroup/node-bebop/tree/feature/firmware-4.0 branch I've added an example that uses RTP video:

https://github.com/hybridgroup/node-bebop/blob/feature/firmware-4.0/examples/rtp_video.js

Basically, you just turn on the video streaming on the drone during the initial discovery process, then some other code can connect to the RTP stream and control ports. This will work great from other language implementations in the same fashion, I've already added the same to Gobot: hybridgroup/gobot@e828f56

Hope that helps!

[tomrunia]

Hi @wilpulling, did you managed to read the video stream in Python? I am trying to use OpenCV (3.0 Python) to read the video stream, but so far I have no luck.