gstreamer sample code won't stream to rtsp

[ben-directai]

Which version are you using?

v2.1.0

Which operating system are you using?

• Linux amd64 standard
• Linux amd64 Docker
• Linux arm64 standard
• Linux arm64 Docker
• Linux arm7 standard
• Linux arm7 Docker
• Linux arm6 standard
• Linux arm6 Docker
• Windows amd64 standard
• Windows amd64 Docker (WSL backend)
• macOS amd64 standard
• macOS amd64 Docker
• Other (please describe)

Describe the issue

When running the sample gstreamer code (provided in the README) to stream frames to the bluenviron/mediamtx latest docker container, the stream never reaches the bluenviron/mediamtxcontainer (and isn't viewable via VLC). I've successfully been able to stream frames to the server via stdin & ffmpeg but gstreamer has yet to work. There is no confirmation log from the rtsp server that a stream has started.

Describe how to replicate the issue

I'm using docker compose to manage two containers. One is hosting the rtsp server, the other is initiating the stream. I'm running docker-compose build && docker-compose up from the root directory on an Ubuntu machine.

docker-compose.yml:

version: '3'
services:
  rtsp_server:
    image: bluenviron/mediamtx:latest
    ports:
      - "8554:8554"
  rtsp_broadcast:
    build: rtsp_broadcast/
    volumes:
      - /home/azureuser/images:/app/images
    ports:
      - "554:554"
    depends_on:
      - rtsp_server
    extra_hosts:
      - "host.docker.internal:host-gateway"

rtsp_broadcast/Dockerfile:

# Use an official Python runtime as a parent image
FROM python:3.8-slim-buster

# Set the working directory in the container
WORKDIR /app

# Install OpenCV and GStreamer
RUN apt-get update && apt-get install -y gstreamer1.0-python3-plugin-loader unzip cmake wget git pkg-config
RUN apt-get install -y libgstreamer1.0-dev libgstreamer-plugins-base1.0-dev 
RUN apt-get install -y ffmpeg libavcodec-dev libavformat-dev libavutil-dev libswscale-dev
RUN apt-get install -y ffmpeg libgtk2.0-dev libgtk-3-dev


COPY requirements.txt /app/requirements.txt
RUN pip install -r requirements.txt

RUN git clone https://github.com/opencv/opencv.git
WORKDIR /app/opencv
RUN git checkout 4.1.0

RUN mkdir build
WORKDIR /app/opencv/build
RUN cmake -D CMAKE_BUILD_TYPE=RELEASE \
-D INSTALL_PYTHON_EXAMPLES=ON \
-D INSTALL_C_EXAMPLES=OFF \
-D PYTHON_EXECUTABLE=$(which python3) \
-D BUILD_opencv_python2=OFF \
-D CMAKE_INSTALL_PREFIX=$(python3 -c "import sys; print(sys.prefix)") \
-D PYTHON3_EXECUTABLE=$(which python3) \
-D PYTHON3_INCLUDE_DIR=$(python3 -c "from distutils.sysconfig import get_python_inc; print(get_python_inc())") \
-D PYTHON3_PACKAGES_PATH=$(python3 -c "from distutils.sysconfig import get_python_lib; print(get_python_lib())") \
-D WITH_GSTREAMER=ON \
-D WITH_FFMPEG=ON \
-D WITH_QT=OFF \
-D WITH_GTK=ON \
-D WITH_GTK_2_X=ON \
-D BUILD_EXAMPLES=ON ..

RUN make -j$(nproc)
RUN make install
RUN ldconfig

# Copy the current directory contents into the container at /usr/src/app
WORKDIR /app

COPY .env /app/.env

RUN mkdir /app/raw_images
COPY annotate_stream.py /app/annotate_stream.py

# Run app.py when the container launches
CMD ["python", "annotate_stream.py"]

rtsp_broadcast/annotate_stream.py:

from datetime import datetime
from time import sleep, time

import cv2
import numpy as np

if __name__ == '__main__':

    fps = 15
    width = 800
    height = 600
    colors = [
        (0, 0, 255),
        (255, 0, 0),
        (0, 255, 0),
    ]

    out = cv2.VideoWriter('appsrc ! videoconvert' + \
        ' ! video/x-raw,format=I420' + \
        ' ! x264enc speed-preset=ultrafast bitrate=600 key-int-max=' + str(fps * 2) + \
        ' ! video/x-h264,profile=baseline' + \
        ' ! rtspclientsink location=rtsp://localhost:8554/mystream',
        cv2.CAP_GSTREAMER, 0, fps, (width, height), True)
    if not out.isOpened():
        raise Exception("can't open video writer")

    curcolor = 0
    start = time()

    while True:
        frame = np.zeros((height, width, 3), np.uint8)

        # create a rectangle
        color = colors[curcolor]
        curcolor += 1
        curcolor %= len(colors)
        for y in range(0, int(frame.shape[0] / 2)):
            for x in range(0, int(frame.shape[1] / 2)):
                frame[y][x] = color

        out.write(frame)
        print("%s frame written to the server" % datetime.now())

        now = time()
        diff = (1 / fps) - now - start
        if diff > 0:
            sleep(diff)
        start = now

Did you attach the server logs?

rtsp_server_1     | 2023/11/03 17:46:54 INF MediaMTX v1.2.1
rtsp_server_1     | 2023/11/03 17:46:54 INF configuration loaded from /mediamtx.yml
rtsp_server_1     | 2023/11/03 17:46:54 INF [RTSP] listener opened on :8554 (TCP), :8000 (UDP/RTP), :8001 (UDP/RTCP)
rtsp_server_1     | 2023/11/03 17:46:54 INF [RTMP] listener opened on :1935
rtsp_server_1     | 2023/11/03 17:46:54 INF [HLS] listener opened on :8888
rtsp_server_1     | 2023/11/03 17:46:54 INF [WebRTC] listener opened on :8889 (HTTP)
rtsp_server_1     | 2023/11/03 17:46:54 INF [SRT] listener opened on :8890 (UDP)
rtsp_broadcast_1  |
rtsp_broadcast_1  | General configuration for OpenCV 4.1.0 =====================================
rtsp_broadcast_1  |   Version control:               4.1.0
rtsp_broadcast_1  |
rtsp_broadcast_1  |   Platform:
rtsp_broadcast_1  |     Timestamp:                   2023-11-01T20:18:49Z
rtsp_broadcast_1  |     Host:                        Linux 6.2.0-32-generic x86_64
rtsp_broadcast_1  |     CMake:                       3.13.4
rtsp_broadcast_1  |     CMake generator:             Unix Makefiles
rtsp_broadcast_1  |     CMake build tool:            /usr/bin/make
rtsp_broadcast_1  |     Configuration:               RELEASE
rtsp_broadcast_1  |
rtsp_broadcast_1  |   CPU/HW features:
rtsp_broadcast_1  |     Baseline:                    SSE SSE2 SSE3
rtsp_broadcast_1  |       requested:                 SSE3
rtsp_broadcast_1  |     Dispatched code generation:  SSE4_1 SSE4_2 FP16 AVX AVX2 AVX512_SKX
rtsp_broadcast_1  |       requested:                 SSE4_1 SSE4_2 AVX FP16 AVX2 AVX512_SKX
rtsp_broadcast_1  |       SSE4_1 (15 files):         + SSSE3 SSE4_1
rtsp_broadcast_1  |       SSE4_2 (2 files):          + SSSE3 SSE4_1 POPCNT SSE4_2
rtsp_broadcast_1  |       FP16 (1 files):            + SSSE3 SSE4_1 POPCNT SSE4_2 FP16 AVX
rtsp_broadcast_1  |       AVX (5 files):             + SSSE3 SSE4_1 POPCNT SSE4_2 AVX
rtsp_broadcast_1  |       AVX2 (29 files):           + SSSE3 SSE4_1 POPCNT SSE4_2 FP16 FMA3 AVX AVX2
rtsp_broadcast_1  |       AVX512_SKX (2 files):      + SSSE3 SSE4_1 POPCNT SSE4_2 FP16 FMA3 AVX AVX2 AVX_512F AVX512_SKX
rtsp_broadcast_1  |
rtsp_broadcast_1  |   C/C++:
rtsp_broadcast_1  |     Built as dynamic libs?:      YES
rtsp_broadcast_1  |     C++ Compiler:                /usr/bin/c++  (ver 8.3.0)
rtsp_broadcast_1  |     C++ flags (Release):         -fsigned-char -W -Wall -Werror=return-type -Werror=non-virtual-dtor -Werror=address -Werror=sequence-point -Wformat -Werror=format-security -Wmissing-declarations -Wundef -Winit-self -Wpointer-arith -Wshadow -Wsign-promo -Wuninitialized -Winit-self -Wno-delete-non-virtual-dtor -Wno-comment -Wimplicit-fallthrough=3 -Wno-strict-overflow -fdiagnostics-show-option -Wno-long-long -pthread -fomit-frame-pointer -ffunction-sections -fdata-sections  -msse -msse2 -msse3 -fvisibility=hidden -fvisibility-inlines-hidden -O3 -DNDEBUG  -DNDEBUG
rtsp_broadcast_1  |     C++ flags (Debug):           -fsigned-char -W -Wall -Werror=return-type -Werror=non-virtual-dtor -Werror=address -Werror=sequence-point -Wformat -Werror=format-security -Wmissing-declarations -Wundef -Winit-self -Wpointer-arith -Wshadow -Wsign-promo -Wuninitialized -Winit-self -Wno-delete-non-virtual-dtor -Wno-comment -Wimplicit-fallthrough=3 -Wno-strict-overflow -fdiagnostics-show-option -Wno-long-long -pthread -fomit-frame-pointer -ffunction-sections -fdata-sections  -msse -msse2 -msse3 -fvisibility=hidden -fvisibility-inlines-hidden -g  -O0 -DDEBUG -D_DEBUG
rtsp_broadcast_1  |     C Compiler:                  /usr/bin/cc
rtsp_broadcast_1  |     C flags (Release):           -fsigned-char -W -Wall -Werror=return-type -Werror=non-virtual-dtor -Werror=address -Werror=sequence-point -Wformat -Werror=format-security -Wmissing-declarations -Wmissing-prototypes -Wstrict-prototypes -Wundef -Winit-self -Wpointer-arith -Wshadow -Wuninitialized -Winit-self -Wno-comment -Wimplicit-fallthrough=3 -Wno-strict-overflow -fdiagnostics-show-option -Wno-long-long -pthread -fomit-frame-pointer -ffunction-sections -fdata-sections  -msse -msse2 -msse3 -fvisibility=hidden -O3 -DNDEBUG  -DNDEBUG
rtsp_broadcast_1  |     C flags (Debug):             -fsigned-char -W -Wall -Werror=return-type -Werror=non-virtual-dtor -Werror=address -Werror=sequence-point -Wformat -Werror=format-security -Wmissing-declarations -Wmissing-prototypes -Wstrict-prototypes -Wundef -Winit-self -Wpointer-arith -Wshadow -Wuninitialized -Winit-self -Wno-comment -Wimplicit-fallthrough=3 -Wno-strict-overflow -fdiagnostics-show-option -Wno-long-long -pthread -fomit-frame-pointer -ffunction-sections -fdata-sections  -msse -msse2 -msse3 -fvisibility=hidden -g  -O0 -DDEBUG -D_DEBUG
rtsp_broadcast_1  |     Linker flags (Release):      -Wl,--gc-sections
rtsp_broadcast_1  |     Linker flags (Debug):        -Wl,--gc-sections
rtsp_broadcast_1  |     ccache:                      NO
rtsp_broadcast_1  |     Precompiled headers:         YES
rtsp_broadcast_1  |     Extra dependencies:          dl m pthread rt
rtsp_broadcast_1  |     3rdparty dependencies:
rtsp_broadcast_1  |
rtsp_broadcast_1  |   OpenCV modules:
rtsp_broadcast_1  |     To be built:                 calib3d core dnn features2d flann gapi highgui imgcodecs imgproc ml objdetect photo python3 stitching ts video videoio
rtsp_broadcast_1  |     Disabled:                    world
rtsp_broadcast_1  |     Disabled by dependency:      -
rtsp_broadcast_1  |     Unavailable:                 java js python2
rtsp_broadcast_1  |     Applications:                tests perf_tests examples apps
rtsp_broadcast_1  |     Documentation:               NO
rtsp_broadcast_1  |     Non-free algorithms:         NO
rtsp_broadcast_1  |
rtsp_broadcast_1  |   GUI:
rtsp_broadcast_1  |     GTK+:                        YES (ver 2.24.32)
rtsp_broadcast_1  |       GThread :                  YES (ver 2.58.3)
rtsp_broadcast_1  |       GtkGlExt:                  NO
rtsp_broadcast_1  |     VTK support:                 NO
rtsp_broadcast_1  |
rtsp_broadcast_1  |   Media I/O:
rtsp_broadcast_1  |     ZLib:                        /usr/lib/x86_64-linux-gnu/libz.so (ver 1.2.11)
rtsp_broadcast_1  |     JPEG:                        libjpeg-turbo (ver 2.0.2-62)
rtsp_broadcast_1  |     WEBP:                        build (ver encoder: 0x020e)
rtsp_broadcast_1  |     PNG:                         /usr/lib/x86_64-linux-gnu/libpng.so (ver 1.6.36)
rtsp_broadcast_1  |     TIFF:                        build (ver 42 - 4.0.10)
rtsp_broadcast_1  |     JPEG 2000:                   build (ver 1.900.1)
rtsp_broadcast_1  |     OpenEXR:                     build (ver 1.7.1)
rtsp_broadcast_1  |     HDR:                         YES
rtsp_broadcast_1  |     SUNRASTER:                   YES
rtsp_broadcast_1  |     PXM:                         YES
rtsp_broadcast_1  |     PFM:                         YES
rtsp_broadcast_1  |
rtsp_broadcast_1  |   Video I/O:
rtsp_broadcast_1  |     DC1394:                      NO
rtsp_broadcast_1  |     FFMPEG:                      YES
rtsp_broadcast_1  |       avcodec:                   YES (58.35.100)
rtsp_broadcast_1  |       avformat:                  YES (58.20.100)
rtsp_broadcast_1  |       avutil:                    YES (56.22.100)
rtsp_broadcast_1  |       swscale:                   YES (5.3.100)
rtsp_broadcast_1  |       avresample:                NO
rtsp_broadcast_1  |     GStreamer:                   YES (1.14.4)
rtsp_broadcast_1  |     v4l/v4l2:                    YES (linux/videodev2.h)
rtsp_broadcast_1  |
rtsp_broadcast_1  |   Parallel framework:            pthreads
rtsp_broadcast_1  |
rtsp_broadcast_1  |   Trace:                         YES (with Intel ITT)
rtsp_broadcast_1  |
rtsp_broadcast_1  |   Other third-party libraries:
rtsp_broadcast_1  |     Intel IPP:                   2019.0.0 Gold [2019.0.0]
rtsp_broadcast_1  |            at:                   /app/opencv/build/3rdparty/ippicv/ippicv_lnx/icv
rtsp_broadcast_1  |     Intel IPP IW:                sources (2019.0.0)
rtsp_broadcast_1  |               at:                /app/opencv/build/3rdparty/ippicv/ippicv_lnx/iw
rtsp_broadcast_1  |     Lapack:                      NO
rtsp_broadcast_1  |     Eigen:                       NO
rtsp_broadcast_1  |     Custom HAL:                  NO
rtsp_broadcast_1  |     Protobuf:                    build (3.5.1)
rtsp_broadcast_1  |
rtsp_broadcast_1  |   OpenCL:                        YES (no extra features)
rtsp_broadcast_1  |     Include path:                /app/opencv/3rdparty/include/opencl/1.2
rtsp_broadcast_1  |     Link libraries:              Dynamic load
rtsp_broadcast_1  |
rtsp_broadcast_1  |   Python 3:
rtsp_broadcast_1  |     Interpreter:                 /usr/local/bin/python3 (ver 3.8.17)
rtsp_broadcast_1  |     Libraries:                   /usr/local/lib/libpython3.8.so (ver 3.8.17)
rtsp_broadcast_1  |     numpy:                       /usr/local/lib/python3.8/site-packages/numpy/core/include (ver 1.24.4)
rtsp_broadcast_1  |     install path:                /usr/local/lib/python3.8/site-packages/cv2/python-3.8
rtsp_broadcast_1  |
rtsp_broadcast_1  |   Python (for build):            /usr/bin/python2.7
rtsp_broadcast_1  |
rtsp_broadcast_1  |   Java:
rtsp_broadcast_1  |     ant:                         NO
rtsp_broadcast_1  |     JNI:                         NO
rtsp_broadcast_1  |     Java wrappers:               NO
rtsp_broadcast_1  |     Java tests:                  NO
rtsp_broadcast_1  |
rtsp_broadcast_1  |   Install to:                    /usr/local
rtsp_broadcast_1  | -----------------------------------------------------------------
rtsp_broadcast_1  |
rtsp_broadcast_1  |
rtsp_broadcast_1  | 2023-11-03 17:46:55.269134 frame written to the server
rtsp_broadcast_1  | 2023-11-03 17:46:55.312674 frame written to the server
rtsp_broadcast_1  | 2023-11-03 17:46:55.355267 frame written to the server
rtsp_broadcast_1  | 2023-11-03 17:46:55.397129 frame written to the server
rtsp_broadcast_1  | 2023-11-03 17:46:55.439153 frame written to the server
rtsp_broadcast_1  | 2023-11-03 17:46:55.481872 frame written to the server
rtsp_broadcast_1  | 2023-11-03 17:46:55.523878 frame written to the server
rtsp_broadcast_1  | 2023-11-03 17:46:55.565610 frame written to the server
rtsp_broadcast_1  | 2023-11-03 17:46:55.607438 frame written to the server

yes

Did you attach a network dump?

no

[aler9]

Hello, from the logs you posted this seems quite trivial. In annotate_scripts.py, replace rtsp://localhost with rtsp://rtsp_server:

    out = cv2.VideoWriter('appsrc ! videoconvert' + \
        ' ! video/x-raw,format=I420' + \
        ' ! x264enc speed-preset=ultrafast bitrate=600 key-int-max=' + str(fps * 2) + \
        ' ! video/x-h264,profile=baseline' + \
        ' ! rtspclientsink location=rtsp://rtsp_server:8554/mystream',
        cv2.CAP_GSTREAMER, 0, fps, (width, height), True)

[ben-directai]

thanks for your help, @aler9! should I open a new bug if the results are the same given the change from localhost to rtsp_server?

[github-actions]

This issue is being locked automatically because it has been closed for more than 6 months.
Please open a new issue in case you encounter a similar problem.