"Video covers/thumbnails no longer appear"

[holta]

Thanks @EMG70 for the bug report.

Let's start by investigating confirming / reproducing / debugging on Ubuntu 24.10 VM's.

[holta]

Confirmed on 10.8.0.10 here: (attempting to download https://youtu.be/TE2mXDqWHyo to a freshly installed 24.10 VM with the latest IIAB Calibre-Web)

root@box:/opt/iiab/iiab# yt-dlp --version
2024.05.26
root@box:/opt/iiab/iiab# lb --version
2.8.046
root@box:/opt/iiab/iiab# cd /usr/local/calibre-web-py3; sudo git log --graph --oneline --decorate | head -50
*   86b6fdbd (HEAD -> master, origin/master, origin/HEAD) Merge remote-tracking branch 'upstream/master'
|\
| *   ab11919c Merge remote-tracking branch 'Synctoken/fix/kobo-sync-token'
| |\
| | * b1d7bade fix: change b64-encoded token to unicode string
| * |   58c26988 Merge remote-tracking branch 'barnesnnoble/db-barnesnoble'
| |\ \
| | * | ad05534e Add support for barnesnoble identifier
| * | |   6760d697 Fix for #2983 (visible OPDS categories reflect settings in web UI) Merge branch 'feature/OPDS-access'
| |\ \ \
| | |/ /
| |/| |
| | * | ee451fb2 Testrun
| | * |   ab13fcf6 Merge remote-tracking branch 'origin/feature/OPDS-access' into feature/OPDS-access
| | |\ \
| | | * | 2b1efdb5 Bugfixes opds feed
| | * | |   e99be72f Merge branch 'Develop' into feature/OPDS-access
| | |\ \ \
| | | |/ /
| | |/| |
| | | * |   f1ceff2b Merge branch 'master' into Develop (goodreads)
| | | |\ \
| | * | | | 49118431 Bugfix for flask-ldap 2.0.0
| | * | | | 2c375465 Code cosmetics
| | * | | | 60ed1904 testrun
| | * | | | cb62d36e Bugfix for visibility opds feed
| | * | | | 737d7583 Return 404 if current element is not visible
| * | | | | 6f60ec7b Change order of imports for goodreads to make import error message clear agan
| * | | | | 894fd9d3 get rid of apscheduler timezone warning
| * | | | | fc9a9cb9 Fix for #3016 (Parsing lubimyczytac: Tags instead of categories are taken, translator is appended to description)
| | |_|/ /
| |/| | |
| * | | | 7e85894b Bugfix for goodreads (html formated info for authors now visible)
| * | | | 5c49c8cd Fix for Flask-SimpleLDAP 2.0.0
| * | | |   c8c3b3cb Fix for goodreads not working anymore (due to blocked requests calls by goodreads.com)
| |\ \ \ \
| | * | | | 25a875b6 Fix for goodreads blocking "requests"
| | | |/ /
| | |/| |
| | * | |   506f0a33 Merge branch 'master' into Develop (apply fix for #3050)
| | |\ \ \
| | |/ / /
| |/| / /
| | |/ /
| * | | 921caf67 Fix for #3050 (metadata extraction for cb7 files not working)
| | * |   8e27912f Merge branch 'master' into Develop Bugfix for too new lxml
| | |\ \
| | |/ /
| |/| |
* | | | c78b55ec Merge branch 'janeczku:master' into master

[holta]

@EMG70 noticed that this used to work, e.g. on LRN2.ORG here:

root@lrn2:/usr/local/calibre-web-py3# yt-dlp --version
2024.03.10
root@lrn2:/usr/local/calibre-web-py3# lb --version
2.5.023
root@lrn2:/usr/local/calibre-web-py3# cd /usr/local/calibre-web-py3; sudo git log --graph --oneline --decorate | head -50
*   273b7370 (HEAD -> master, origin/master, origin/HEAD) Merge branch 'janeczku:master' into master
|\
| * 80a2d070 fix embed metadata, split library and download file / convert files
* |   859f4477 Merge pull request #131 from deldesir/deldesir-patch-13
|\ \
| * | f53ff013 metadata_extract.py: Tighten regex (remove paren group)
| * | feffabb1 metadata_extract.py: Explain re "lookahead assertion"
| * | 86cb6418 Use regex to safely sanitize original_url
| * | 978ba24d Replace '/media' by '/meta'
| * |   67c93d53 Merge branch 'iiab:master' into deldesir-patch-13
| |\ \
| |/ /
|/| |
* | |   b11a1e9a Merge pull request #133 from deldesir/deldesir-patch-14
|\ \ \
| * | | e62bc1c2 Increase task cleanup trigger to 101
* | | |   b2a7d55a Merge pull request #130 from deldesir/deldesir-patch-11
|\ \ \ \
| |/ / /
|/| | |
| * | | 23a22140 Improve task message
| | * | a2922352 Fix #106 405 Method not allowed
| |/ /
| * | f82002c6 Handle failed URLs
| * |   87b4f2c7 Merge branch 'iiab:master' into deldesir-patch-11
| |\ \
| |/ /
|/| |
* | |   9a1a8a01 Merge branch 'janeczku:master' into master
|\ \ \
| | |/
| |/|
| * | d6594301 Bugfix view None ratings caused error 500 in certain cases
| * | 859dac46 Update Teststatus
| | * c4ad3336 Display total duration in metadata task
| |/
|/|
* | ba4fabaa Merge branch 'janeczku:master' into master
|\|
| * 2bea4dbd Show List with no file formats available Bugfix save order of format list view
* |   d1038869 Merge pull request #125 from deldesir/deldesir-patch-10
|\ \
| * | 5d1cac47 cps/uploader.py: Link to Dedup Design discussion
| * | 9be939a7 cps/download.py: Link to Dedup Design discussion
| * | 7441e5b3 Update lb-wrapper
| * | afa7ed95 Use path to select the right row
| * |   30ad7ebf Merge branch 'iiab:master' into deldesir-patch-10

[holta]

@deldesir explains:

The cover issue [appears to be] caused by a regression in xklb. --write-thumbnail argument is no longer recognized as a valid one.

[deldesir]

I take back what I've said. --write-thumbnail does work on xklb. I wrongly debugged the issue by interpreting an error I received because I omitted --video option in lb dl command, sorry.
https://github.com/chapmanjacobd/library/blob/4b316e34253068016baee3d1e95373dcdd92392f/xklb/mediadb/download.py#L87-L89:

if unk and not args.profile in (DBType.video, DBType.audio):
        parser.error(f"unrecognized arguments: {' '.join(unk)}")
    args.unk = unk

    if not args.profile and not args.print:
        log.error("Download profile must be specified. Use one of: --video OR --audio OR --image OR --filesystem")
        raise SystemExit(1)

[holta]

Thanks @deldesir for explaining 💯

[deldesir]

I notice the thumbnail file is downloaded outside of the video directory created upon download (Youtube/<video_directory>) and it's named after the video_id. It may be an xklb or yt-dlp regression. I am still investigating.

[holta]

thumbnail file is downloaded outside of the video directory created upon download (Youtube/<video_directory>)

Any chance this new behavior is... intentional?

[holta]

@deldesir please close this issue within the coming week, if all looks solid!

• PR Fix missing cover #162

[chapmanjacobd]

Ah yeah this is because the file is moved after:

• Add option to pass/filter filenames through an arbitrary external program yt-dlp/yt-dlp#6521 (comment)
• chapmanjacobd/library@9f5a8be

Since you've already made a fix I think I'll leave this as-is but I might need to change it in the future if something else is broken by this, in which case just take out one of the os.path.dirnames in PR162.

In any case, I'd recommend using --embed-thumbnail or https://dearrow.ajay.app/ and avif is a bit better than webp

[holta]

Since you've already made a fix

We can refine PR #162, no worries!

e.g. if thumbnails and sub-title files [ETC!] can always appear in the same directory with the original video, if that turns out to be an option?

(Anyway that would be my 1st choice, if @chapmanjacobd thinks that's a sensible long-term solution with yt-dlp and xklb?!)

[holta]

Thanks @chapmanjacobd for clarifying the underlying root cause:

• «File name too long» should not error ytdl-org/youtube-dl#29912

(It sounds like he &/or @deldesir will try to restore the original behavior sometime soon — such that cover/thumbnail/poster once again appears in the very same directory as the video — so long as any excessively long path/filename is avoided with a very high degree of certainty!)

[holta]

Appears to be solved, with PR #162 refined by:

• PR Find thumbnail [restore scan for .webp, .jpg, .png or .gif to support Vimeo etc — not just .webp thumbnails as provided by yt-dlp from YouTube!] #163