[ARTE.tv] Unable to download JSON metadata #30919
Comments
|
Although the log is not exactly the same, see #30878: in particular #30878 (comment). |
dirkf
added
broken-IE
|
@zombo schrieb am 05.06.22 um 19:08 Uhr:
Do they have any English stuff? I would be willing to look at this,
but it needs to be something I might actually watch
Sure they do, so much wonderful stuff:
https://www.arte.tv/en/
They even have full concerts, e.g. Rolling stones – A bigger bang
https://www.arte.tv/de/videos/105608-000-A/rolling-stones-a-bigger-bang/
I'd really apprechiate you looking into this. :D
Greetz and thank you
laryllian
|
|
I've found a very easy workaround for this arte.tv problem with their API. No need to do any updates, download new PY files, or switch to a different code/app. After hours and hours of googling, I found a casual remark that just said, "API 2 simply means that you need to find the m3u8 URL." So basically, all one has to do is google how to find find the URL for a blob video in your specific browser (the routines are a little different in Firefox and Chrome, respectively). Once you've found it, you only need to fetch the URL for the m3u8 file and paste it into youtube-dl, done. |
|
on firefox, open Developer Tools, go to Media tab, search for mp4 file, the copy the url |
|
... Backporting from current https://www.arte.tv/en/videos/067103-000-A/mariupol/ |
|
I just figured out that https://github.com/yt-dlp/yt-dlp appears to work, just in case somebody needs a solution urgently. I hope, though, that the support can be added to youtube-dl proper. |
... The sample [debug] Command-line config: ['--ffmpeg-location', '..\\FFmpeg', '--downloader-args', 'ffmpeg:-v 8 -stats', '-v', '-F', 'https://www.arte.tv/en/videos/067103-000-A/mariupol/', '--ie', 'ArteTV']
[debug] Encodings: locale cp1253, fs utf-8, pref cp1253, out utf-8 (No VT), error utf-8 (No VT), screen utf-8 (No VT)
[debug] yt-dlp version 2023.02.14 [8b37c58f8] (win_x86_exe)
[debug] Python 3.8.16 (CPython x86 32bit) - Windows-Vista-6.0.6003-SP2 (OpenSSL 1.1.1s 1 Nov 2022)
[debug] exe versions: ffmpeg n5.2-dev-2245-N-109649-gab8cde6 (setts), ffprobe n5.2-dev-2245-N-109649-gab8cde6, phantomjs 2.1.1, rtmpdump 2.4
[debug] Optional libraries: Cryptodome-3.17, brotli-1.0.9, certifi-2022.12.07, mutagen-1.46.0, sqlite3-2.6.0, websockets-10.4
[debug] Proxy map: {}
[debug] Extractor Plugins: AGB+NSIG (YoutubeIE)
[debug] Plugin directories: ['<redacted>\\yt-dlp-plugins\\YTAgeGateBypass.zip\\yt_dlp_plugins', '<redacted>\\yt-dlp-plugins\\YTNSigProxy.zip\\yt_dlp_plugins']
[debug] Loaded 1 extractors
[ArteTV] Extracting URL: https://www.arte.tv/en/videos/067103-000-A/mariupol/
[ArteTV] 067103-000-A: Downloading JSON metadata
[ArteTV] 067103-000-A: Downloading m3u8 information
[ArteTV] 067103-000-A: Downloading m3u8 information
[ArteTV] 067103-000-A: Downloading m3u8 information
[ArteTV] 067103-000-A: Downloading m3u8 information
[ArteTV] 067103-000-A: Downloading m3u8 information
[ArteTV] 067103-000-A: Downloading m3u8 information
[debug] Formats sorted by: hasvid, ie_pref, lang, quality, res, fps, hdr:12(7), vcodec:vp9.2(10), channels, acodec, filesize, fs_approx, tbr, vbr, abr, asr, proto, vext, aext, hasaud, source, id
[info] Available formats for 067103-000-A:
ID EXT RESOLUTION FPS | FILESIZE TBR PROTO | VCODEC VBR ACODEC MORE INFO
------------------------------------------------------------------------------------------------------------------------------------
VO-STF-program_audio_0-VO mp4 audio only | m3u8 | audio only unknown [ru] French (Subtitles) [subFR]
VO-STF-426 mp4 384x216 25 | ~248.75MiB 426k m3u8 | avc1.42e00d 426k video only French (Subtitles) [subFR]
VO-STF-927 mp4 640x360 25 | ~541.19MiB 927k m3u8 | avc1.4d401e 927k video only French (Subtitles) [subFR]
VO-STF-1617 mp4 768x432 25 | ~943.97MiB 1617k m3u8 | avc1.4d401e 1617k video only French (Subtitles) [subFR]
VO-STF-2313 mp4 1280x720 25 | ~ 1.32GiB 2313k m3u8 | avc1.4d401f 2313k video only French (Subtitles) [subFR]... And a "word" of caution: yt-dlp "https://www.arte.tv/en/videos/067103-000-A/mariupol/" --list-subs =>
[ArteTV] Extracting URL: https://www.arte.tv/en/videos/067103-000-A/mariupol/
[ArteTV] 067103-000-A: Downloading JSON metadata
[ArteTV] 067103-000-A: Downloading m3u8 information
[ArteTV] 067103-000-A: Downloading m3u8 information
[ArteTV] 067103-000-A: Downloading m3u8 information
[ArteTV] 067103-000-A: Downloading m3u8 information
[ArteTV] 067103-000-A: Downloading m3u8 information
[ArteTV] 067103-000-A: Downloading m3u8 information
[info] Available subtitles for 067103-000-A:
Language Formats
en vtt
fr vtt
de vtt
es vtt
pl vtt
it vttIf you prefer your subs (like myself 😉) in the old but trusted yt-dlp "https://www.arte.tv/en/videos/067103-000-A/mariupol/" --skip-download --write-subs --sub-langs en --convert-subs srt =>
[ArteTV] Extracting URL: https://www.arte.tv/en/videos/067103-000-A/mariupol/
[ArteTV] 067103-000-A: Downloading JSON metadata
[ArteTV] 067103-000-A: Downloading m3u8 information
[ArteTV] 067103-000-A: Downloading m3u8 information
[ArteTV] 067103-000-A: Downloading m3u8 information
[ArteTV] 067103-000-A: Downloading m3u8 information
[ArteTV] 067103-000-A: Downloading m3u8 information
[ArteTV] 067103-000-A: Downloading m3u8 information
[info] 067103-000-A: Downloading subtitles: en
[info] 067103-000-A: Downloading 1 format(s): VO-STF-2313+VO-STF-program_audio_0-VO
[info] Writing video subtitles to: Mariupol [067103-000-A].en.vtt
[hlsnative] Downloading m3u8 manifest
[hlsnative] Total fragments: 1
[download] Destination: Mariupol [067103-000-A].en.vtt
[download] 100% of 39.89KiB in 00:00:02 at 19.65KiB/s
[SubtitlesConvertor] Converting subtitles
Deleting original file Mariupol [067103-000-A].en.vtt (pass -k to keep)... will leave you with an empty Simply fetch the original and use another tool to convert them to |
|
As the PR for the yt-dlp extractor has been merged, the only reason not to update the yt-dl extractor is that yt-dl doesn't have the subtitle extraction API used by yt-dlp, but I see that is problematic, pending a fix for the format conversion. So: $ python -m youtube_dl -v -F 'https://www.arte.tv/en/videos/067103-000-A/mariupol/'
[debug] System config: [u'--prefer-ffmpeg']
[debug] User config: []
[debug] Custom config: []
[debug] Command-line args: [u'-v', u'-F', u'https://www.arte.tv/en/videos/067103-000-A/mariupol/']
[debug] Encodings: locale UTF-8, fs UTF-8, out UTF-8, pref UTF-8
[debug] youtube-dl version 2021.12.17
[debug] Git HEAD: dd9aa74be
[debug] Python version 2.7.18 (CPython) - Linux-4.4.0-210-generic-i686-with-Ubuntu-16.04-xenial
[debug] exe versions: avconv 4.3, avprobe 4.3, ffmpeg 4.3, ffprobe 4.3
[debug] Proxy map: {}
[ArteTV] 067103-000-A: Downloading JSON metadata
[ArteTV] 067103-000-A: Downloading m3u8 information
[ArteTV] 067103-000-A: Downloading m3u8 information
[ArteTV] 067103-000-A: Downloading m3u8 information
[ArteTV] 067103-000-A: Downloading m3u8 information
[ArteTV] 067103-000-A: Downloading m3u8 information
[ArteTV] 067103-000-A: Downloading m3u8 information
[info] Available formats for 067103-000-A:
format code extension resolution note
VO-STE_ANG_-program_audio_0-VO mp4 audio only [ru] English (Subtitles) [?]
VO-STE_ANG_-1617 mp4 768x432 English (Subtitles) [?] 1617k , avc1.4d401e, 25.0fps, video only
VO-STE_ANG_-2313 mp4 1280x720 English (Subtitles) [?] 2313k , avc1.4d401f, 25.0fps, video only
VO-STE_ANG_-927 mp4 640x360 English (Subtitles) [?] 927k , avc1.4d401e, 25.0fps, video only
VO-STE_ANG_-426 mp4 384x216 English (Subtitles) [?] 426k , avc1.42e00d, 25.0fps, video only (best)
$If anyone has a replacement test video for G doesn't admit to having indexed any pages with URL Otherwise this extractor will be marked as not And see updated PR #29653. |
|
The following video https://www.arte.tv/de/videos/060738-167-A/blow-up-rainer-werner-fassbinder-in-bildern/ (ID: 060738-167-A) is claimed to be available until 01.08.2114 (though it might change without notice, which renders such availability declarations ad absurdum). |
|
... Apparently, a "fix" for (default) format sorting (according to increasing resolution) will also be needed: (redacted for brevity)
[info] Available formats for 067103-000-A:
format code extension resolution note
VO-STE_ANG_-program_audio_0-VO mp4 audio only [ru] English (Subtitles) [?]
VO-STE_ANG_-1617 mp4 768x432 English (Subtitles) [?] 1617k , avc1.4d401e, 25.0fps, video only
VO-STE_ANG_-2313 mp4 1280x720 English (Subtitles) [?] 2313k , avc1.4d401f, 25.0fps, video only
VO-STE_ANG_-927 mp4 640x360 English (Subtitles) [?] 927k , avc1.4d401e, 25.0fps, video only
VO-STE_ANG_-426 mp4 384x216 English (Subtitles) [?] 426k , avc1.42e00d, 25.0fps, video only (best)Notice how |
At its latest commit, cdcb4af, works adequately well, without addressing the format sorting issue I reported above: yt-dl -vF "https://www.arte.tv/en/videos/067103-000-A/mariupol/" =>
[debug] System config: []
[debug] User config: []
[debug] Custom config: []
[debug] Command-line args: ['--ffmpeg-location', '.\\FFmpeg', '--external-downloader-args', '-v 8 -stats', '-vF', 'https://www.arte.tv/en/videos/067103-000-A/mariupol/']
[debug] Encodings: locale cp1253, fs mbcs, out cp737, pref cp1253
[debug] youtube-dl version 2023.02.14.1919
[debug] Python version 3.4.4 (CPython) - Windows-Vista-6.0.6003-SP2
[debug] exe versions: ffmpeg n5.2-dev-2245-N-109649-gab8cde6, ffprobe n5.2-dev-2245-N-109649-gab8cde6, phantomjs 2.1.1, rtmpdump 2.4
[debug] Proxy map: {}
[ArteTV] 067103-000-A: Downloading JSON metadata
[ArteTV] 067103-000-A: Downloading m3u8 information
[ArteTV] 067103-000-A: Downloading m3u8 information
[ArteTV] 067103-000-A: Downloading m3u8 information
[ArteTV] 067103-000-A: Downloading m3u8 information
[ArteTV] 067103-000-A: Downloading m3u8 information
[ArteTV] 067103-000-A: Downloading m3u8 information
[info] Available formats for 067103-000-A:
format code extension resolution note
VO-STF-program_audio_0-VO mp4 audio only [ru] French (Subtitles) [subFR]
VO-STF-1617 mp4 768x432 French (Subtitles) [subFR] 1617k , avc1.4d401e, 25.0fps, video only
VO-STF-2313 mp4 1280x720 French (Subtitles) [subFR] 2313k , avc1.4d401f, 25.0fps, video only
VO-STF-927 mp4 640x360 French (Subtitles) [subFR] 927k , avc1.4d401e, 25.0fps, video only
VO-STF-426 mp4 384x216 French (Subtitles) [subFR] 426k , avc1.42e00d, 25.0fps, video only (best) |
How true. You can imagine the back-porting process:
The PR was updated from the same code as used above and will shortly be brought into line. Given the change in (1), a test for
Thanks, but that's just a normal |
Following works in a browser tab: |
|
Thanks, both variants of that are working with the PR code. Actually I realised that you can invent these URLs from any non-embedded URL, so I used this: |
Checklist
Verbose log
Description
I was trying to download a video from website https://arte.tv/. Before,
youtube-dlwas working well with this site, but not anymore. I tried to re-download previously (successfully) downloaded videos, but I still receive the same error. I guess the page format changed.Thanks for your job.
The text was updated successfully, but these errors were encountered: