Support http-mp4 file stream, fmp4 stream. #174

winlinvip · 2015-05-11T13:28:24Z

https://github.com/winlinvip/simple-rtmp-server/issues/174

TRANS_BY_GPT3

The text was updated successfully, but these errors were encountered:

winlinvip · 2017-01-11T05:30:01Z

MP4 itself is a box within a box, not a streaming mode. Later, someone improved the box structure of MP4, similar to turning the box into an FLV tag or stacking the boxes side by side. If the client supports parsing and playing the boxes simultaneously, it can achieve similar functionality to HTTP-FLV.

It seems that Silverlight, the deceased Silverlight, and the MP4 stream directly with IIS are examples of this approach.

TRANS_BY_GPT3

winlinvip · 2017-01-13T06:52:36Z

WeChat Mini Program supports playing HLS live streaming and on-demand streaming, as well as HTTP-MP4 on-demand files. Let's see if it also supports HTTP MP4 live streaming and how much the latency can be reduced.

Microsoft's Live Smooth Streaming uses fragmented MP4: https://technet.microsoft.com/en-us/library/7d5f47fa-886b-4321-8367-60b1a126d4dd

This article specifically discusses fragmented MP4: http://blog.csdn.net/yu_yuan_1314/article/details/9289827

mp4box can generate this type of file.

MP4Box -dash 500000 -frag 50000 -rap file_name.mp4

GPAC (mp4box) official website: https://gpac.wp.mines-telecom.fr/mp4box/

GPAC on GitHub: https://github.com/gpac/gpac

There is an article discussing the replacement of HTTP with RTMP: http://blog.streamingmedia.com/2016/09/flash-rtmp-streaming-dying-out.html. It does not propose a fundamental solution to support RTMP-level latency.

Here is someone who tried fmp4 and said that H5 does not support it: https://my.oschina.net/langhuihui/blog/519901.

You can convert the SRS instance avatar.flv to mp4.

ffmpeg -i doc/source.200kbps.768x320.flv -c copy -y avatar.mp4

Use mp4box to view information.

winlin:mp4 winlin$ mp4box -info avatar.mp4 
* Movie Info *
	Timescale 1000 - 2 tracks
	Computed Duration 00:03:30.652 - Indicated Duration 00:03:30.652
	Fragmented File: no
	File Brand isom - version 512
	Created: UNKNOWN DATE	Modified: UNKNOWN DATE
File has no MPEG4 IOD/OD

iTunes Info:
	Encoder Software: Lavf55.19.104
1 UDTA types: meta (1) 

Track # 1 Info - TrackID 1 - TimeScale 16000
Media Duration 00:03:30.560 - Indicated Duration 00:03:30.560
Track has 2 edit lists: track duration is 00:03:30.606
Media Info: Language "Undetermined (und)" - Type "vide:avc1" - 5264 samples
Visual Track layout: x=0 y=0 width=768 height=320
MPEG-4 Config: Visual Stream - ObjectTypeIndication 0x21
AVC/H264 Video - Visual Size 768 x 320
	AVC Info: 1 SPS - 1 PPS - Profile High @ Level 3.2
	NAL Unit length bits: 32
	Pixel Aspect Ratio 1:1 - Indicated track size 768 x 320
	Chroma format 0 - Luma bit depth 8 - chroma bit depth 8
	SPS#1 hash: 3031FBB2E6D922C701D45CCE28FFC6743FCA7D2A
	PPS#1 hash: 6BBE60BFBE383CE9144FA78B782534A704912778
Self-synchronized
	RFC6381 Codec Parameters: avc1.640020
	Average GOP length: 48 samples

Track # 2 Info - TrackID 2 - TimeScale 44100
Media Duration 00:03:30.651 - Indicated Duration 00:03:30.651
Track has 1 edit lists: track duration is 00:03:30.652
Media Info: Language "Undetermined (und)" - Type "soun:mp4a" - 9072 samples
MPEG-4 Config: Audio Stream - ObjectTypeIndication 0x40
MPEG-4 Audio AAC LC - 2 Channel(s) - SampleRate 44100
Synchronized on stream 1
	RFC6381 Codec Parameters: mp4a.40.2
Alternate Group ID 1
	All samples are sync

mp4box parameters:

winlin:mp4 winlin$ mp4box -h all
General Options:
 -frag time_in_ms     fragments file (track fragments of time_in_ms)
                       * Note: Always disables interleaving
 -out filename        specifies output file name
                       * Note: By default input (MP4,3GP) file is overwritten

DASH Options:
 -mpd m3u8            converts HLS manifest (local or remote http) to MPD 
                       Note: not compatible with other DASH options (except -out and -tmp) and does not convert associated segments
 -dash dur            enables DASH-ing of the file(s) with a segment duration of DUR ms
                       Note: the duration of a fragment (subsegment) is set
	                            using the -frag switch.
                       Note: for onDemand profile, sets duration of a subsegment

Convert regular mp4 to fmp4:

mp4box -frag 3000 avatar.mp4 -out avatar_f.mp4

You can use mp4info to see a small mp4 fragment every 3 seconds:

However, with WeChat mini program, it can only play the first 3 seconds, which means it does not have complete support for fragment mp4.

The loading speed and the speed of starting playback are both fast, but unfortunately it does not continue to parse the remaining moof box...

The two MP4 links are as follows. Regular MP4:
http://ossrs.net/weixin/avatar.mp4
Fragmented MP4, with a duration of 3 seconds per fragment:
http://ossrs.net/weixin/avatar_f.mp4

For other live streaming methods on WeChat, please refer to:
http://blog.csdn.net/win_lin/article/details/54410876

TRANS_BY_GPT3

winlinvip · 2017-01-13T09:02:33Z

In addition, the browsers on PC for fmp4 are as follows:

Chrome cannot play fmp4.
Safari can play fmp4, but it is very slow.
Firefox has relatively good support, continuously downloading the following fragments, and the duration keeps increasing.

Consider putting this unreliable protocol into consideration for SRS4+.

TRANS_BY_GPT3

jswh · 2017-11-07T05:07:05Z

You can try to support MPEG-TS and then use jsmpeg to achieve playback on regular HTML5 pages and within WeChat. If you use Canvas, it will also work fine in mini programs, although it may not be as efficient as WebGL.

It seems that it would be better to directly raise an issue for this.

TRANS_BY_GPT3

wony-zheng · 2018-04-27T13:31:23Z

flv.js is used in the frontend to convert flv to fmp4, why is it no longer supported?

TRANS_BY_GPT3

winlinvip · 2018-08-04T03:43:44Z

@wony-zheng MSE is supported, but playing MPEG4 streams directly in the browser is a different framework from MSE.

TRANS_BY_GPT3

winlinvip · 2018-08-04T03:44:02Z

The conclusion is that MP4 streaming is unreliable.

TRANS_BY_GPT3

jackyyf · 2022-11-18T18:49:31Z

Is it possible to reconsider this decision? Currently, a fragmented MP4 has been generated based on the method described in Comment#3.

$ ffmpeg -i source.200kbps.768x320.flv -c copy -y avatar.mp4
$ MP4Box -frag 1000 avatar.mp4 -out fmp4.mp4
$ ffprobe fmp4.mp4 
ffprobe version 5.1.2 Copyright (c) 2007-2022 the FFmpeg developers
<snip>
Input #0, mov,mp4,m4a,3gp,3g2,mj2, from 'fmp4.mp4':
  Metadata:
    major_brand     : isom
    minor_version   : 1
    compatible_brands: isomiso2avc1mp41
    creation_time   : 2022-11-18T18:29:54.000000Z
    encoder         : GPAC-2.0-rev
  Duration: 00:03:30.65, start: 0.000000, bitrate: 246 kb/s
  Stream #0:0[0x1](und): Video: h264 (High) (avc1 / 0x31637661), yuv420p(progressive), 768x320 [SAR 1:1 DAR 12:5], 212 kb/s, 25 fps, 25 tbr, 16k tbn (default)
    Metadata:
      handler_name    : VideoHandler
      vendor_id       : [0][0][0][0]
  Stream #0:1[0x2](und): Audio: aac (LC) (mp4a / 0x6134706D), 44100 Hz, stereo, fltp, 30 kb/s (default)
    Metadata:
      handler_name    : SoundHandler
      vendor_id       : [0][0][0][0]

$ MP4Box -info fmp4.mp4 
# Movie Info - 2 tracks - TimeScale 1000
Duration 00:00:00.000
Fragmented: yes - duration 00:03:30.652
211 fragments - 0 SegmentIndexes
Progressive (moov before mdat)
Major Brand isom - version 1 - compatible brands: isom iso2 avc1 mp41
Created: GMT Fri Nov 18 18:29:54 2022

File has root IOD (9 bytes)
Scene PL 0xff - Graphics PL 0xff - OD PL 0xff
Visual PL: AVC/H264 Profile (0x7f)
Audio PL: AAC Profile @ Level 2 (0x29)
No streams included in root OD

iTunes Info:
	tool: GPAC-2.0-rev

# Track 1 Info - ID 1 - TimeScale 16000
Media Duration 00:00:00.000 
Track has 1 edits: track duration is 00:00:00.000
Track flags: Enabled In Movie
Media Info: Language "Undetermined (und)" - Type "vide:avc1" - 0 samples
Fragmented track: 5264 samples - Media Duration 00:03:30.560
Fragment sample defaults: duration 640 size 0 stsd 1 sync 0 padding 0 degradation_priority 0
Visual Sample Entry Info: width=768 height=320 (depth=24 bits)
Visual Track layout: x=0 y=0 width=768 height=320
AVC/H264 Video - Visual Size 768 x 320
	AVC Info: 1 SPS - 1 PPS - Profile High @ Level 3.2
	NAL Unit length bits: 32
	Pixel Aspect Ratio 1:1 - Indicated track size 768 x 320
	Chroma format YUV 4:2:0 - Luma bit depth 8 - chroma bit depth 8
	SPS#1 hash: 3031FBB2E6D922C701D45CCE28FFC6743FCA7D2A
	PPS#1 hash: 6BBE60BFBE383CE9144FA78B782534A704912778
	RFC6381 Codec Parameters: avc1.640020
	All samples are sync
	Max sample duration: 0 / 16000

# Track 2 Info - ID 2 - TimeScale 44100
Media Duration 00:00:00.000 
Track flags: Enabled In Movie
Media Info: Language "Undetermined (und)" - Type "soun:mp4a" - 0 samples
Fragmented track: 9072 samples - Media Duration 00:03:30.651
Fragment sample defaults: duration 1024 size 0 stsd 1 sync 1 padding 0 degradation_priority 0
Alternate Group ID 1
MPEG-4 Audio AAC LC (AOT=2 implicit) - 2 Channel(s) - SampleRate 44100
	RFC6381 Codec Parameters: mp4a.40.2
	All samples are sync
	Max sample duration: 0 / 44100

Then I set up a local file server and created a very simple webpage.
<video src="fmp4.mp4" controls="true" />

After accessing, it can play normally.

Chrome version:

Google Chrome | 107.0.5304.110 (Official Build) (64-bit)
Revision | 2a558545ab7e6fb8177002bf44d4fc1717cb2998-refs/branch-heads/5304@{#1202}
OS | Linux
JavaScript | V8 10.7.193.22
User Agent | Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/107.0.0.0 Safari/537.36

When playing, there are a few drawbacks:

Although controls=true, it is not possible to fast forward or rewind, only adjust play/pause, volume, and fullscreen.
As shown in the screenshot, during playback, the total length of the file is not known (the file is loaded in segments).

The above two drawbacks have little impact in live streaming scenarios. Considering that Adobe has already made it clear that FLV will not be given Codec IDs for H265/VP9/AV1, fmp4 is currently the best alternative for FLV on webpages. It can be played without introducing JavaScript and using MSE, unlike FLV.

I hope you can reconsider this decision. fMP4 could be a very good solution for low-latency live streaming :)

TRANS_BY_GPT3

winlinvip added this to the srs 3.0 release milestone May 11, 2015

winlinvip added the Feature It's a new feature. label May 11, 2015

winlinvip mentioned this issue Jan 11, 2017

DVR: Support DVR as MP4 file. #738

Closed

winlinvip modified the milestones: srs 4.0 release, srs 3.0 release Jan 13, 2017

mort1k mentioned this issue Jan 24, 2017

APPLICATION: Support timeshift over DVR. #753

Closed

winlinvip closed this as completed Aug 4, 2018

winlinvip changed the title ~~支持http-mp4文件流~~ 支持http-mp4文件流，fmp4流 Aug 26, 2021

winlinvip self-assigned this Aug 26, 2021

winlinvip added the Won't fix We won't fix it. label Aug 26, 2021

winlinvip changed the title ~~支持http-mp4文件流，fmp4流~~ Support http-mp4 file stream, fmp4 stream. Jul 28, 2023

winlinvip added the TransByAI Translated by AI/GPT. label Jul 28, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Support http-mp4 file stream, fmp4 stream. #174

Support http-mp4 file stream, fmp4 stream. #174

winlinvip commented May 11, 2015 •

edited

Loading

winlinvip commented Jan 11, 2017 •

edited

Loading

winlinvip commented Jan 13, 2017 •

edited

Loading

winlinvip commented Jan 13, 2017 •

edited

Loading

jswh commented Nov 7, 2017 •

edited by winlinvip

Loading

wony-zheng commented Apr 27, 2018 •

edited by winlinvip

Loading

winlinvip commented Aug 4, 2018 •

edited

Loading

winlinvip commented Aug 4, 2018 •

edited

Loading

jackyyf commented Nov 18, 2022 •

edited by winlinvip

Loading

Support http-mp4 file stream, fmp4 stream. #174

Support http-mp4 file stream, fmp4 stream. #174

Comments

winlinvip commented May 11, 2015 • edited Loading

winlinvip commented Jan 11, 2017 • edited Loading

winlinvip commented Jan 13, 2017 • edited Loading

winlinvip commented Jan 13, 2017 • edited Loading

jswh commented Nov 7, 2017 • edited by winlinvip Loading

wony-zheng commented Apr 27, 2018 • edited by winlinvip Loading

winlinvip commented Aug 4, 2018 • edited Loading

winlinvip commented Aug 4, 2018 • edited Loading

jackyyf commented Nov 18, 2022 • edited by winlinvip Loading

winlinvip commented May 11, 2015 •

edited

Loading

winlinvip commented Jan 11, 2017 •

edited

Loading

winlinvip commented Jan 13, 2017 •

edited

Loading

winlinvip commented Jan 13, 2017 •

edited

Loading

jswh commented Nov 7, 2017 •

edited by winlinvip

Loading

wony-zheng commented Apr 27, 2018 •

edited by winlinvip

Loading

winlinvip commented Aug 4, 2018 •

edited

Loading

winlinvip commented Aug 4, 2018 •

edited

Loading

jackyyf commented Nov 18, 2022 •

edited by winlinvip

Loading