Czkawka hung on scanning duplicate videos.

[vi]

Running https://github.com/qarmin/czkawka/releases/download/4.1.0/linux_czkawka_gui.AppImage, scanned the filesystem for duplicate videos. It was keeping CPU busy for some time, but stopped IO and CPU activities, at "Hashing of 2930/2932 video".

I see one FFmpeg running and writing to a pipe that Czkawka reads (or rather supposed to read).

# ps p 10046 | cat
  PID TTY      STAT   TIME COMMAND
10046 pts/59   SLl+   0:00 ffmpeg -hide_banner -loglevel warning -nostats -i /path/to/video.webm -vf fps=1/3 -vframes 10 -pix_fmt rgb24 -c:v rawvideo -f image2pipe - 

# strace -yy -p  10046
strace: Process 10046 attached
write(1<pipe:[24580051]>, ")\"6+$6+$6+$7,%9.'7,%9.'9.':/(<1*"..., 4096^C

# ls -l /proc/*/fd/* | rg 'pipe:\[24580051\]'
l-wx------ 1 root        root        64 Jul 13 19:44 /proc/10046/fd/1 -> pipe:[24580051]
lr-x------ 1 root        root        64 Jul 13 19:44 /proc/3922/fd/28 -> pipe:[24580051]

# ps p 3922 | cat
  PID TTY      STAT   TIME COMMAND
 3922 pts/59   SNl+  28:13 /tmp/.mount_czkawkihYjcK/AppRun.wrapped

# strace -P 'pipe:[24580051]' -fp 3922
strace: Process 3922 attached with 18 threads
^C ...

# strace -fcp 3922
% time     seconds  usecs/call     calls    errors syscall
------ ----------- ----------- --------- --------- ----------------
  0.00    0.000000           0        29           read
  0.00    0.000000           0        29           write
  0.00    0.000000           0        89           poll
  0.00    0.000000           0        30           writev
  0.00    0.000000           0        67        65 recvmsg
  0.00    0.000000           0        14         4 futex
  0.00    0.000000           0         2           restart_syscall
  0.00    0.000000           0        28           clock_nanosleep
------ ----------- ----------- --------- --------- ----------------
100.00    0.000000           0       288        69 total

# rust-gdb -p 3922
...
(gdb) thread apply all bt
...
Thread 17 (Thread 0x7fe2fbfff700 (LWP 6926) "AppRun.wrapped"):
...
#2  __pthread_cond_wait (cond=0x7fe2f470f950, mutex=0x7fe2f754efa0) at pthread_cond_wait.c:638
#3  0x00005579d90a5db6 in rayon_core::latch::LockLatch::wait_and_reset ()
#4  0x00005579d8c80087 in std::thread::local::LocalKey<T>::with ()
#5  0x00005579d8c9a050 in rayon_core::registry::in_worker ()
#6  0x00005579d8c953d8 in rayon::iter::plumbing::bridge_producer_consumer::helper ()
#7  0x00005579d8c0ad5a in <rayon::vec::IntoIter<T> as rayon::iter::IndexedParallelIterator>::with_producer ()
#8  0x00005579d8cdb5c9 in rayon::iter::extend::<impl rayon::iter::ParallelExtend<T> for alloc::vec::Vec<T>>::par_extend ()
#9  0x00005579d8ccb034 in czkawka_core::similar_videos::SimilarVideos::find_similar_videos ()
...

Thread 15 (Thread 0x7fe350f35700 (LWP 4186) "AppRun.wrapped"):
#0  0x00007fe35bfb8207 in __GI___wait4 (pid=10046, stat_loc=0x7fe350f32734, options=0, usage=0x0) at ../sysdeps/unix/sysv/linux/wait4.c:27
#1  0x00005579d90ea0ab in std::sys::unix::process::process_inner::{impl#1}::wait::{closure#0} () at library/std/src/sys/unix/process/process_unix.rs:596
#2  std::sys::unix::cvt_r<i32, std::sys::unix::process::process_inner::{impl#1}::wait::{closure_env#0}> () at library/std/src/sys/unix/mod.rs:212
#3  std::sys::unix::process::process_inner::Process::wait () at library/std/src/sys/unix/process/process_unix.rs:596
#4  0x00005579d90dbb81 in std::process::Child::wait () at library/std/src/process.rs:1781
#5  0x00005579d8f14a79 in <ffmpeg_cmdline_utils::ffmpeg_ops::FfmpegFrames as core::iter::traits::iterator::Iterator>::next ()
#6  0x00005579d8ce4739 in <alloc::vec::Vec<T> as alloc::vec::spec_from_iter::SpecFromIter<T,I>>::from_iter ()
#7  0x00005579d8be5cc7 in vid_dup_finder_lib::video_hashing::video_hash::VideoHash::from_path ()
#8  0x00005579d8cd7eb5 in core::ops::function::impls::<impl core::ops::function::FnOnce<A> for &mut F>::call_once ()
#9  0x00005579d8ce699c in <alloc::vec::Vec<T,A> as alloc::vec::spec_extend::SpecExtend<T,I>>::spec_extend ()
#10 0x00005579d8c952c7 in rayon::iter::plumbing::bridge_producer_consumer::helper ()
#11 0x00005579d8cf9de1 in std::panicking::try ()
#12 0x00005579d8c9a0c7 in rayon_core::registry::in_worker ()
#13 0x00005579d8c953d8 in rayon::iter::plumbing::bridge_producer_consumer::helper ()
#14 0x00005579d8c4c989 in rayon_core::job::StackJob<L,F,R>::run_inline ()
#15 0x00005579d8c9a1b4 in rayon_core::registry::in_worker ()
#16 0x00005579d8c953d8 in rayon::iter::plumbing::bridge_producer_consumer::helper ()
#17 0x00005579d8cf9de1 in std::panicking::try ()
#18 0x00005579d8c9a0c7 in rayon_core::registry::in_worker ()
#19 0x00005579d8c953d8 in rayon::iter::plumbing::bridge_producer_consumer::helper ()
#20 0x00005579d8c4c989 in rayon_core::job::StackJob<L,F,R>::run_inline ()
#21 0x00005579d8c9a1b4 in rayon_core::registry::in_worker ()
#22 0x00005579d8c953d8 in rayon::iter::plumbing::bridge_producer_consumer::helper ()
#23 0x00005579d8cf9de1 in std::panicking::try ()
#24 0x00005579d8c9a0c7 in rayon_core::registry::in_worker ()
#25 0x00005579d8c953d8 in rayon::iter::plumbing::bridge_producer_consumer::helper ()
#26 0x00005579d8cf8b44 in std::panicking::try ()
#27 0x00005579d8c4f698 in _$LT$rayon_core..job..StackJob$LT$L$C$F$C$R$GT$$u20$as$u20$rayon_core..job..Job$GT$::execute::h463311d92465f9c3 ()
#28 0x00005579d8adfb58 in rayon_core::registry::WorkerThread::wait_until_cold ()
#29 0x00005579d90a2989 in rayon_core::registry::ThreadBuilder::run ()
...

(no more interesting threads)

(gdb) generate-core-file
Saved corefile core.3922

(exited and detached)

# ffmpeg --version
ffmpeg version 4.3.4-0+deb11u1 Copyright (c) 2000-2021 the FFmpeg developers

Pressing "Stop" button did not cause any noticeable effect. GUI does not show my the videos that are already scanned. GUI does not let me to navigate to "Duplicate files" section where I have previous scan results (not fully acted on).

---

I think hangs are unavoidable when scanning filesystem, especially if FUSE/NFS is involved.
Czkawka should be able to handle and work around problematic file paths (or subprocesses) and be able to force close the child processes.
Also "Stop" button should return control to user immediately, even when underlying process is in progress of being stopped.
Obviously, FFmpeg deadlock is also to be fixed, but that may be even of lower priority than the points above.

[rickalex21]

Mine hangs on linux tried both appImage and yay AUR. Both of them hang on a small directory.

[qarmin]

I used this only on small subset of videos, but with ffmpeg 4.4.2 everything worked fine.
Can you provide example file which cause this freeze?