Building a HLS Flask based server with Picamera2

[BAChapin]

I'm not sure this is the place this should go, but I need help and I haven't been able to find anything anywhere else.

I am currently building a Flask server to serve HLS based content using Picamera2. If I run this code:

import time
from picamera2 import Picamera2
from picamera2.encoders import H264Encoder
from picamera2.outputs import FfmpegOutput

picam2 = Picamera2()
video_config = picam2.create_video_configuration(main={"size": (640, 480)})
picam2.configure(video_config)

encoder = H264Encoder(bitrate=16000, qp=30)

output = FfmpegOutput("-f hls -hls_time 5 -hls_list_size 10 -hls_flags delete_segments -hls_allow_cache 5 stream.m3u8")

picam2.start_recording(encoder, output)
time.sleep(30)
picam2.stop_recording()

It works fine, it produces 6 stream#.ts files, and a stream.m3u8 file. I do get an odd warning log, but it runs fine.

However, modifying the code to look like this:

import time
from flask import Flask, Response
from picamera2 import Picamera2
from picamera2.encoders import H264Encoder
from picamera2.outputs import FfmpegOutput

app = Flask(__name__)

picam2 = Picamera2()
video_config = picam2.create_video_configuration(main={"size": (640, 480)})
picam2.configure(video_config)

encoder = H264Encoder(bitrate=16000, qp=30)

output = FfmpegOutput("-f hls -hls_time 5 -hls_list_size 10 -hls_flags delete_segments -hls_allow_cache 5 stream.m3u8")

@app.route('/video')
def video():
    m3u8 = f'#EXTM3U\n#EXT-X-VERSION:3\n#EXT-X-TARGETDURATION:5\n#EXT-X-MEDIA-SEQUENCE:0\n#EXTINF:5,\n/video/0\n'
    return Response(m3u8, content_type='Application/vnd.apple.mpegurl')

@app.route('/video/<segment>')
def video_segment(segment):
    segment_number = int(segment)
    with open(f'stream{segment_number}.ts', 'rb') as segment_file:
        data = segment_file.read()
    yield data

if __name__ == '__main__':
    picam2.start_recording(encoder, output)
    app.run(host='0.0.0.0', port=5000, debug=True)

I get the following errors and doing searches on them haven't proven to be very fruitful:

[1:52:26.852115670] [6428]  INFO Camera camera_manager.cpp:284 libcamera v0.1.0+52-a858d20b
[1:52:27.047210476] [6431]  WARN RPiSdn sdn.cpp:39 Using legacy SDN tuning - please consider moving SDN inside rpi.denoise
[1:52:27.052830497] [6431]  INFO RPI vc4.cpp:387 Registered camera /base/soc/i2c0mux/i2c@1/imx708@1a to Unicam device /dev/media3 and ISP device /dev/media0
{'format': 'SRGGB10_CSI2P', 'size': (1536, 864)} 486.66666666666663
{'format': 'SRGGB10_CSI2P', 'size': (2304, 1296)} 786.6666666666666
{'format': 'SRGGB10_CSI2P', 'size': (4608, 2592)} 1686.6666666666667
[1:52:27.071655132] [6428]  INFO Camera camera.cpp:1213 configuring streams: (0) 640x480-XBGR8888 (1) 1536x864-SBGGR10_CSI2P
[1:52:27.072672210] [6431]  INFO RPI vc4.cpp:549 Sensor: /base/soc/i2c0mux/i2c@1/imx708@1a - Selected sensor format: 1536x864-SBGGR10_1X10 - Selected unicam format: 1536x864-pBAA
 * Serving Flask app 'live_streamer'
 * Debug mode: on
WARNING: This is a development server. Do not use it in a production deployment. Use a production WSGI server instead.
 * Running on all addresses (0.0.0.0)
 * Running on http://127.0.0.1:5000
 * Running on http://192.168.86.45:5000
Press CTRL+C to quit
 * Restarting with stat
[1:52:30.638341579] [6439]  INFO Camera camera_manager.cpp:284 libcamera v0.1.0+52-a858d20b
[1:52:30.723986886] [6442] ERROR V4L2 v4l2_device.cpp:353 'imx708': Unable to set controls: Device or resource busy
[1:52:30.753187557] [6442]  WARN RPiSdn sdn.cpp:39 Using legacy SDN tuning - please consider moving SDN inside rpi.denoise
[1:52:30.758983254] [6442]  INFO RPI vc4.cpp:387 Registered camera /base/soc/i2c0mux/i2c@1/imx708@1a to Unicam device /dev/media3 and ISP device /dev/media0
[1:52:30.762870212] [6439]  INFO Camera camera.cpp:1020 Pipeline handler in use by another process
Camera __init__ sequence did not complete.
Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/picamera2/picamera2.py", line 247, in __init__
    self._open_camera()
  File "/usr/lib/python3/dist-packages/picamera2/picamera2.py", line 455, in _open_camera
    self.camera.acquire()
RuntimeError: Failed to acquire camera: Device or resource busy

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/bachapin/Documents/Video_Streaming_Server/live_streamer.py", line 9, in <module>
    picam2 = Picamera2()
             ^^^^^^^^^^^
  File "/usr/lib/python3/dist-packages/picamera2/picamera2.py", line 259, in __init__
    raise RuntimeError("Camera __init__ sequence did not complete.")
RuntimeError: Camera __init__ sequence did not complete.

Admittedly, this is my first time building a server application that serves up media, and its my first experience using picamera2. I'm not entirely sure what is wrong with my code, from what I can tell the logic makes sense and I can't understand why I'm getting the specific errors I'm getting. Can anyone give me a hand? I would really appreciate it.

[Manuel-Angel-Es]

Hi, @BAChapin,

Most likely, the problem is that you have executed the second python script after executing the first and without having freed the camera resources.

To free the resources the following instructions usually work:
picam2.stop()
picam2.close()

Good luck

[BAChapin]

Thanks for the suggestion, this is the error that printed trying that:

[88:53:47.855774752] [6302]  INFO Camera camera_manager.cpp:284 libcamera v0.1.0+52-a858d20b
[88:53:48.045180469] [6307]  WARN RPiSdn sdn.cpp:39 Using legacy SDN tuning - please consider moving SDN inside rpi.denoise
[88:53:48.050833656] [6307]  INFO RPI vc4.cpp:387 Registered camera /base/soc/i2c0mux/i2c@1/imx708@1a to Unicam device /dev/media3 and ISP device /dev/media1
{'format': 'SRGGB10_CSI2P', 'size': (1536, 864)} 486.66666666666663
{'format': 'SRGGB10_CSI2P', 'size': (2304, 1296)} 786.6666666666666
{'format': 'SRGGB10_CSI2P', 'size': (4608, 2592)} 1686.6666666666667
[88:53:48.071452512] [6302]  INFO Camera camera.cpp:1213 configuring streams: (0) 640x480-XBGR8888 (1) 1536x864-SBGGR10_CSI2P
[88:53:48.072424901] [6307]  INFO RPI vc4.cpp:549 Sensor: /base/soc/i2c0mux/i2c@1/imx708@1a - Selected sensor format: 1536x864-SBGGR10_1X10 - Selected unicam format: 1536x864-pBAA
Traceback (most recent call last):
  File "/home/bachapin/Documents/Video_Streaming_Server/live_streamer.py", line 32, in <module>
    picam2.start_recording(encoder, output)
  File "/usr/lib/python3/dist-packages/picamera2/picamera2.py", line 1672, in start_recording
    self.configure(config)
  File "/usr/lib/python3/dist-packages/picamera2/picamera2.py", line 1088, in configure
    self.configure_(camera_config)
  File "/usr/lib/python3/dist-packages/picamera2/picamera2.py", line 1003, in configure_
    camera_config = self.create_preview_configuration()
                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3/dist-packages/picamera2/picamera2.py", line 662, in create_preview_configuration
    raise RuntimeError("Camera not opened")
RuntimeError: Camera not opened

I put the method calls you suggested right before I try to start the camera for recording. Is there a better place to try placing them?

[Manuel-Angel-Es]

The best place to free the camera resources is at the end of the script where the camera was used.
For example, using your own first script:

import time
from picamera2 import Picamera2
from picamera2.encoders import H264Encoder
from picamera2.outputs import FfmpegOutput

picam2 = Picamera2()
video_config = picam2.create_video_configuration(main={"size": (640, 480)})
picam2.configure(video_config)

encoder = H264Encoder(bitrate=16000, qp=30)

output = FfmpegOutput("-f hls -hls_time 5 -hls_list_size 10 -hls_flags delete_segments -hls_allow_cache 5 stream.m3u8")

picam2.start_recording(encoder, output)
time.sleep(30)
picam2.stop_recording()
picam2.close()

The error that now appears, RuntimeError: Camera not opened, is due to trying to use the camera after having closed it.

[BAChapin]

Thats a fair point. However, I only have the one program I'm trying to run. I may have been unclear, the first code snippet I posted above was converted over to the newer one posted after it. There really shouldn't be any lingering camera usage going on prior to the running of the program and I have posted the code in its entirety so as you can see there is no multiple camera objects attempting to run simultaneously.

I want to create this to be a live stream, and my concern with moving the code into one of the routes is that if multiple devices try to connect, it will try to create multiple instances of the camera object. Obviously based upon my current struggles, I can only imagine how much worse it would be if each time a device connects to a route it tries to create a new camera instance.

[Manuel-Angel-Es]

Good, but in any case, the error "RuntimeError: Failed to acquire camera: Device or resource busy", is due to the attempt to start an already started camera.

If you are sure that the camera is free, run your script, but it is always advisable when we are no longer going to need the camera, at the end of the script release the camera.

Sorry for not being more explicit, but I'm not familiar with using the camera to capture video. I normally use it to capture still images on a scanner from old Super8 films.