issue with process.read

[mylselgan]

Describe the bug
The function process.read is not working as expected.

To Reproduce

url =  process.read("/usr/local/bin/yt-dlp --no-warnings -f 91/worstvideo -g 'https://www.youtube.com/watch?v=gCNeDWCI0vo'")
s = input.ffmpeg(url)

output()

Expected behavior
the process.read should return a m3u8 URL because the command /usr/local/bin/yt-dlp --no-warnings -f 91/worstvideo -g 'https://www.youtube.com/watch?v=gCNeDWCI0vo' returns the m3u8 URL in the terminal

Version details
Virtualization: kvm
Operating System: Debian GNU/Linux 10 (buster) (same issue with Ubuntu 20.04LTS also.)
Kernel: Linux 4.19.0-5-amd64
Architecture: x86-64

• Version 2.0.7 and 2.1.4

Install method
Installed via opam
LOG

2023/09/23 20:27:21 >>> LOG START
2023/09/23 20:27:17 [main:3] Liquidsoap 2.1.4
2023/09/23 20:27:17 [main:3] Using: bytes=[distributed with OCaml 4.02 or above] pcre=7.5.0 sedlex=3.2 menhirLib=20230608 curl=0.9.2 uri=4.2.0 dtools=0.4.5 duppy=0.9.2 cry=0.6.7 mm=0.8.4 ogg=0.7.4 ogg.decoder=0.7.4 vorbis=0.8.0 vorbis.decoder=0.8.0 mad=0.5.2 dynlink=[distributed with Ocaml] lame=0.3.7 ffmpeg=1.1.7 samplerate=0.1.6 taglib=0.3.10 camomile=1.0.2 faad=0.5.1
2023/09/23 20:27:17 [dynamic.loader:3] Could not find dynamic module for fdkaac encoder.
2023/09/23 20:27:17 [clock:3] Using builtin (low-precision) implementation for latency control
2023/09/23 20:27:18 [sandbox:5] Command: /usr/local/bin/yt-dlp --no-warnings -f 91/worstvideo -g 'https://www.youtube.com/watch?v=qvQs9pBxxBI'
2023/09/23 20:27:21 [frame:4] frame.audio.samplerate set to: 44100
2023/09/23 20:27:21 [frame:4] frame.video.framerate set to: 25
2023/09/23 20:27:21 [frame:4] frame.audio.channels set to: 2
2023/09/23 20:27:21 [frame:4] frame.video.default set to: false
2023/09/23 20:27:21 [frame:4] frame.midi.channels set to: 0
2023/09/23 20:27:21 [frame:4] frame.video.width set to: 1280
2023/09/23 20:27:21 [frame:4] frame.video.height set to: 720
2023/09/23 20:27:21 [frame:4] frame.audio.samplerate set to: 44100
2023/09/23 20:27:21 [frame:3] Using 44100Hz audio, 25Hz video, 44100Hz main.
2023/09/23 20:27:21 [frame:3] Video frame size set to: 1280x720
2023/09/23 20:27:21 [frame:3] Frame size must be a multiple of 1764 ticks = 1764 audio samples = 1 video samples.
2023/09/23 20:27:21 [frame:3] Targeting 'frame.duration': 0.04s = 1764 audio samples = 1764 ticks.
2023/09/23 20:27:21 [frame:3] Frames last 0.04s = 1764 audio samples = 1 video samples = 1764 ticks.
2023/09/23 20:27:21 [sandbox:3] Sandboxing disabled
2023/09/23 20:27:21 [video.converter:3] Using preferred video converter: ffmpeg.
2023/09/23 20:27:21 [audio.converter:3] Using samplerate converter: libsamplerate.
2023/09/23 20:27:21 [lifecycle:5] At stage: "Liquidsoap application start"
2023/09/23 20:27:21 [clock:4] Currently 1 clock(s) allocated.
2023/09/23 20:27:21 [clock.main:4] Starting 2 source(s)...
2023/09/23 20:27:21 [source:4] Source input.ffmpeg_0 gets up with content kind: {audio=pcm(stereo),video=internal,midi=none}.
2023/09/23 20:27:21 [input.ffmpeg_0:5] Activations changed: static=[input.ffmpeg_0], dynamic=[].
2023/09/23 20:27:21 [input.ffmpeg_0:5] Enabling caching mode: active source.
2023/09/23 20:27:21 [source:4] Source output.icecast_0 gets up with content kind: {audio=pcm(stereo),video=none,midi=none}.
2023/09/23 20:27:21 [/v1temmgbwnhvv/source:5] Clock is main[].
2023/09/23 20:27:21 [/v1temmgbwnhvv/source:5] Content kind: {audio=pcm(stereo),video=none,midi=none}, content type: {audio=pcm(stereo),video=none,midi=none}
2023/09/23 20:27:21 [/v1temmgbwnhvv/source:4] Content type is {audio=pcm(stereo),video=none,midi=none}.
2023/09/23 20:27:21 [source:4] Source amplify_0 gets up with content kind: {audio=pcm(stereo),video=none,midi=none}.
2023/09/23 20:27:21 [amplify_0:5] Clock is main[].
2023/09/23 20:27:21 [amplify_0:5] Content kind: {audio=pcm(stereo),video=none,midi=none}, content type: {audio=pcm(stereo),video=none,midi=none}
2023/09/23 20:27:21 [amplify_0:4] Content type is {audio=pcm(stereo),video=none,midi=none}.
2023/09/23 20:27:21 [source:4] Source switch_1 gets up with content kind: {audio=pcm(stereo),video=none,midi=none}.
2023/09/23 20:27:21 [source:4] Source drop_video_0 gets up with content kind: {audio=pcm(stereo),video=none,midi=none}.
2023/09/23 20:27:21 [drop_video_0:5] Clock is main[].
2023/09/23 20:27:21 [drop_video_0:5] Content kind: {audio=pcm(stereo),video=none,midi=none}, content type: {audio=pcm(stereo),video=none,midi=none}
2023/09/23 20:27:21 [drop_video_0:4] Content type is {audio=pcm(stereo),video=none,midi=none}.
2023/09/23 20:27:21 [source:4] Source mksafe gets up with content kind: {audio=pcm(stereo),video=internal,midi=none}.
2023/09/23 20:27:21 [input.ffmpeg_0:5] Activations changed: static=[input.ffmpeg_0], dynamic=[mksafe:drop_video_0:switch_1:amplify_0:/v1temmgbwnhvv/source:/v1temmgbwnhvv/source].
2023/09/23 20:27:21 [source:4] Source safe_blank gets up with content kind: {audio=pcm(stereo),video=internal,midi=none}.
2023/09/23 20:27:21 [safe_blank:5] Clock is main[].
2023/09/23 20:27:21 [safe_blank:5] Content kind: {audio=pcm(stereo),video=internal,midi=none}, content type: {audio=pcm(stereo),video=none,midi=none}
2023/09/23 20:27:21 [safe_blank:4] Content type is {audio=pcm(stereo),video=none,midi=none}.
2023/09/23 20:27:21 [safe_blank:5] Activations changed: static=[], dynamic=[mksafe:drop_video_0:switch_1:amplify_0:/v1temmgbwnhvv/source:/v1temmgbwnhvv/source].
2023/09/23 20:27:21 [mksafe:5] Activations changed: static=[drop_video_0:switch_1:amplify_0:/v1temmgbwnhvv/source:/v1temmgbwnhvv/source], dynamic=[].
2023/09/23 20:27:21 [drop_video_0:5] Activations changed: static=[], dynamic=[switch_1:amplify_0:/v1temmgbwnhvv/source:/v1temmgbwnhvv/source].
2023/09/23 20:27:21 [switch_1:5] Activations changed: static=[amplify_0:/v1temmgbwnhvv/source:/v1temmgbwnhvv/source], dynamic=[].
2023/09/23 20:27:21 [amplify_0:5] Activations changed: static=[/v1temmgbwnhvv/source:/v1temmgbwnhvv/source], dynamic=[].
2023/09/23 20:27:21 [/v1temmgbwnhvv/source:5] Activations changed: static=[/v1temmgbwnhvv/source], dynamic=[].
2023/09/23 20:27:21 [/v1temmgbwnhvv/source:5] Enabling caching mode: active source.
2023/09/23 20:27:21 [threads:4] Created thread "clock_main" (1 total).
2023/09/23 20:27:21 [clock:4] Main phase starts.
2023/09/23 20:27:21 [threads:4] Created thread "generic queue #1" (1 total).
2023/09/23 20:27:21 [threads:4] Created thread "generic queue #2" (2 total).
2023/09/23 20:27:21 [threads:4] Created thread "non-blocking queue #1" (3 total).
2023/09/23 20:27:21 [threads:4] Created thread "non-blocking queue #2" (4 total).
2023/09/23 20:27:21 [clock.main:3] Streaming loop starts in auto-sync mode
2023/09/23 20:27:21 [clock.main:3] Delegating synchronisation to CPU clock
2023/09/23 20:27:21 [video.text:3] Using native implementation
2023/09/23 20:27:21 [switch_1:3] Switch to drop_video_0.
2023/09/23 20:27:21 [drop_video_0:5] Activations changed: static=[switch_1:amplify_0:/v1temmgbwnhvv/source:/v1temmgbwnhvv/source], dynamic=[switch_1:amplify_0:/v1temmgbwnhvv/source:/v1temmgbwnhvv/source].
2023/09/23 20:27:21 [mksafe:5] Content kind: {audio=pcm(stereo),video=internal,midi=none}, content type: {audio=pcm(stereo),video=none,midi=none}
2023/09/23 20:27:21 [mksafe:3] Switch to safe_blank.
2023/09/23 20:27:21 [safe_blank:5] Activations changed: static=[mksafe:drop_video_0:switch_1:amplify_0:/v1temmgbwnhvv/source:/v1temmgbwnhvv/source], dynamic=[mksafe:drop_video_0:switch_1:amplify_0:/v1temmgbwnhvv/source:/v1temmgbwnhvv/source].
2023/09/23 20:27:21 [/v1temmgbwnhvv/source:3] Connecting mount /v1temmgbwnhvv/source for [email protected]...
2023/09/23 20:27:21 [stderr:3] [https @ 0x7fb54c005d00] HTTP error 404 Not Found
2023/09/23 20:27:21 [input.ffmpeg_0:4] Connection failed: Avutil.Error(Server returned 404 Not Found)
2023/09/23 20:27:21 [/v1temmgbwnhvv/source:3] Connection setup was successful.
2023/09/23 20:27:21 [switch_1:5] Content kind: {audio=pcm(stereo),video=none,midi=none}, content type: {audio=pcm(stereo),video=none,midi=none}
2023/09/23 20:27:21 [input.ffmpeg_0:5] Content kind: {audio=pcm(stereo),video=internal,midi=none}, content type: {audio=pcm(stereo),video=none,midi=none}
2023/09/23 20:27:23 [stderr:3] [https @ 0x7fb544005d80] HTTP error 404 Not Found
2023/09/23 20:27:23 [input.ffmpeg_0:4] Connection failed: Avutil.Error(Server returned 404 Not Found)
2023/09/23 20:27:25 [stderr:3] [https @ 0x7fb54c18e980] HTTP error 404 Not Found
2023/09/23 20:27:25 [input.ffmpeg_0:4] Connection failed: Avutil.Error(Server returned 404 Not Found)
2023/09/23 20:27:28 [stderr:3] [https @ 0x7fb54c259000] HTTP error 404 Not Found
2023/09/23 20:27:28 [input.ffmpeg_0:4] Connection failed: Avutil.Error(Server returned 404 Not Found)

[smimram]

It's working here. What happens if you add

print("**** the url is '#{url}'")

to show the url?

[mylselgan]

yes it prints

**** the url is 'https://manifest.googlevideo.com/api/manifest/hls_playlist/expire/1695505000/ei/CAYPZcqcHKuM_9EPg8aAmAU/ip/38.45.64.217/id/qvQs9pBxxBI.5/itag/91/source/yt_live_broadcast/requiressl/yes/ratebypass/yes/live/1/sgoap/gir%3Dyes%3Bitag%3D139/sgovp/gir%3Dyes%3Bitag%3D160/hls_chunk_host/rr2---sn-vgqsknzy.googlevideo.com/playlist_duration/30/manifest_duration/30/spc/UWF9f4j-g0WHlWEOFnwdt62fOEhjfQ0/vprv/1/playlist_type/DVR/mh/v6/mm/44/mn/sn-vgqsknzy/ms/lva/mv/u/mvi/2/pl/25/dover/11/pacing/0/keepalive/yes/fexp/24007246/beids/24350018/mt/1695481891/sparams/expire,ei,ip,id,itag,source,requiressl,ratebypass,live,sgoap,sgovp,playlist_duration,manifest_duration,spc,vprv,playlist_type/sig/AOq0QJ8wRAIgOO8--T7iRqgLt1cbhqVItsmkcClP8X9-cWiVxzdIbYYCIG9k82K_A5xdLBseoYNuWZ1CiR2Zjkk8EStnJ7MX1uxD/lsparams/hls_chunk_host,mh,mm,mn,ms,mv,mvi,pl/lsig/AG3C_xAwRQIhANTB2WGd3oAmCsmCsYgD5FkqLZW1nVN0uPozoZrceIQ7AiBW2qLi4JsIbk8ZKai78gmNS0AzCU5pLrhXe5_EupMffw%3D%3D/playlist/index.m3u8

but

url = "https://manifest.googlevideo.com/api/manifest/hls_playlist/expire/1695505000/ei/CAYPZcqcHKuM_9EPg8aAmAU/ip/38.45.64.217/id/qvQs9pBxxBI.5/itag/91/source/yt_live_broadcast/requiressl/yes/ratebypass/yes/live/1/sgoap/gir%3Dyes%3Bitag%3D139/sgovp/gir%3Dyes%3Bitag%3D160/hls_chunk_host/rr2---sn-vgqsknzy.googlevideo.com/playlist_duration/30/manifest_duration/30/spc/UWF9f4j-g0WHlWEOFnwdt62fOEhjfQ0/vprv/1/playlist_type/DVR/mh/v6/mm/44/mn/sn-vgqsknzy/ms/lva/mv/u/mvi/2/pl/25/dover/11/pacing/0/keepalive/yes/fexp/24007246/beids/24350018/mt/1695481891/sparams/expire,ei,ip,id,itag,source,requiressl,ratebypass,live,sgoap,sgovp,playlist_duration,manifest_duration,spc,vprv,playlist_type/sig/AOq0QJ8wRAIgOO8--T7iRqgLt1cbhqVItsmkcClP8X9-cWiVxzdIbYYCIG9k82K_A5xdLBseoYNuWZ1CiR2Zjkk8EStnJ7MX1uxD/lsparams/hls_chunk_host,mh,mm,mn,ms,mv,mvi,pl/lsig/AG3C_xAwRQIhANTB2WGd3oAmCsmCsYgD5FkqLZW1nVN0uPozoZrceIQ7AiBW2qLi4JsIbk8ZKai78gmNS0AzCU5pLrhXe5_EupMffw%3D%3D/playlist/index.m3u8"
s = input.ffmpeg(url)

output()

successfully outputs audio to the icecast but not the first example with process.read

[vitoyucepi]

Hi @mylselgan,

Try input.ffmpeg(string.trim(url)). I think the culprit is a newline symbol at the end of the URL.
By the way, 2.1.4 and 2.2.1 work well without trimming the spaces.

Offtopic

Did you try streamlink?

[mylselgan]

Thanks, Works fine with input.ffmpeg(string.trim(url))

What is streamlink? I never tried before. I use liquidsoap to convert YouTube live link to audio only radio service. Will streamlink useful for me in this case?

Hi @mylselgan,

Try input.ffmpeg(string.trim(url)). I think the culprit is a newline symbol at the end of the URL. By the way, 2.1.4 and 2.2.1 work well without trimming the spaces.

Offtopic
Did you try streamlink?

[vitoyucepi]

The streamlink is a piece of software designed to pull live streaming sources.
Mostly, I use it to watch and record streams or dump VODs from various platforms.

[vitoyucepi]

@smimram, I think the version of ffmpeg from debian:10 and ubuntu:20.04 can't handle the newline symbol in the link or something. Is it necessary to trim the string on the liquidsoap side?
Has the problem been resolved in version ffmpeg=1.1.8? If so, we can close this issue.