tail overhaul (--follow=name, etc.)

[jhscheer]

I finally consider this ready for review!

TLDR:

• Implement missing flags: --follow=name, --retry, -F, ---disable-inotify
• Add 44 new test cases
• Make 19 more GNU test suite tests pass that were previously failing

---

Long version:

• use the notify crate for cross-platform filesystem notifications in order to implement -F i.e. --follow=name --retry with backend support from: inotify (Linux), kqueue (macOS/BSD) and ReadDirectoryChanges (Windows)
  Note: this uses a release candidate of the notify crate v5 because v4 is very old and doesn't have all the required features.
• add support for polling in case a notification backend is not available or the resources are exhausted (e.g inotify.max_user_instances limit is reached), it can be forced with --use-polling
• add support for stdin redirects, e.g. $ tail -f < file, $ tail < DIR, $ tail -F - < file, etc.
• verify that -[nc]0 without -f, exit without reading
• fix handling of files if permission is denied
• fix handling of files if the file type is unsupported
• add support for character devices and fifos

Ensure that:

• --follow=name does not imply --retry
• --follow={descriptor,name} (without --retry) does not wait for the files to appear
• --retry without --follow results in a warning
• tail --retry --follow={name,descriptor} waits for the file to appear
• truncation is detected
• tail --follow=descriptor --retry exits when the file appears untailable
• --follow=descriptor (without --retry) does not try to open a file after an initial fail, even when there are other tailable files
• tail -F retries when the file is initially untailable
• inotify will switch to polling mode if directory of the watched file was initially missing and later created
• inotify will switch to polling mode if directory of the watched file was removed and recreated
• tail -f works when renaming the tailed files
• the headers are correct for --verbose
• tail --follow=name handles remove, move, truncation, rename, create events correctly
• truncating a file with the same content it already has does not trigger a truncation event

Summary

Ensure that the following GNU test-suite tests pass:

• F-headers.sh
• F-vs-missing.sh
• F-vs-rename.sh #
• append-only.sh # skipped test: must be run as root
• assert-2.sh
• assert.sh
• descriptor-vs-rename.sh
• flush-initial.sh
• follow-name.sh #
• inotify-dir-recreate.sh
• inotify-hash-abuse.sh
• inotify-hash-abuse2.sh
• inotify-rotate.sh #
• overlay-headers.sh #
• pid.sh
• pipe-f2.sh
• retry.sh #
• tail-n0f.sh
• truncate.sh
  Note: 6 of these do not show up in the CI as passes becauseappend-only needs root privileges, and the other 5 tests marked with # pass on my local machine only. I have no clue why and it's incredible cumbersome to debug the CI so I will refrain from that.

Note: there are further details in tail/README.md

Edit:
Now theoverlay-headers.sh test passes in the CI, although the tests rerun without changes. I suspect scheduling/load on the test VM is the reason.

[sylvestre]

The coverage job on linux fails with:

---- test_tail::test_follow_name_move_create2 stdout ----
current_directory_resolved: 
touch: /tmp/.tmprXSPHB/1
touch: /tmp/.tmprXSPHB/2
touch: /tmp/.tmprXSPHB/3
touch: /tmp/.tmprXSPHB/4
touch: /tmp/.tmprXSPHB/5
touch: /tmp/.tmprXSPHB/6
touch: /tmp/.tmprXSPHB/7
touch: /tmp/.tmprXSPHB/8
touch: /tmp/.tmprXSPHB/9
run: /home/runner/work/coreutils/coreutils/target/debug/coreutils tail -s.1 --max-unchanged-stats=1 -q -F 1 2 3 4 5 6 7 8 9
write(truncate): /tmp/.tmprXSPHB/9
rename: "/tmp/.tmprXSPHB/1" "/tmp/.tmprXSPHB/f"
write(truncate): /tmp/.tmprXSPHB/1
remove: "/tmp/.tmprXSPHB/f"
run: /home/runner/work/coreutils/coreutils/target/debug/coreutils tail -s.1 --max-unchanged-stats=1 -q -F 1 2 3 4 5 6 7 8 9 ---disable-inotify
write(truncate): /tmp/.tmprXSPHB/9
rename: "/tmp/.tmprXSPHB/1" "/tmp/.tmprXSPHB/f"
write(truncate): /tmp/.tmprXSPHB/1
---- test_tail::test_follow_name_move_create2 stderr ----
thread 'main' panicked at 'assertion failed: `(left == right)`
  left: `"tail: 9: file truncated\ntail: '1' has become inaccessible: No such file or directory\ntail: '1' has appeared;  following new file\n"`,
 right: `"tail: '1' has become inaccessible: No such file or directory\ntail: '1' has appeared;  following new file\n"`', /home/runner/work/coreutils/coreutils/tests/by-util/test_tail.rs:1891:9
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace

[tertsdiepraam]

This is excellent! Bravo for sticking with this, the complexity of this feature is wild. I have left some comments for further cleanup, but it looks very good.

[sylvestre]

And the CI is now green, bravo :)

[jhscheer]

@tertsdiepraam and @sylvestre thank you very much for your review and comments. I addressed all your concerns.

[tertsdiepraam]

Looks good to me! Thanks for addressing those comments and, again, this is a great PR!

[sylvestre]

LGTM, bravo!

[sylvestre]

I see an intermittent here:
https://github.com/uutils/coreutils/runs/6808598985?check_suite_focus=true

---- test_tail::test_follow_name_move_create2 stdout ----
current_directory_resolved: 
touch: /tmp/.tmpVYRqaL/1
touch: /tmp/.tmpVYRqaL/2
touch: /tmp/.tmpVYRqaL/3
touch: /tmp/.tmpVYRqaL/4
touch: /tmp/.tmpVYRqaL/5
touch: /tmp/.tmpVYRqaL/6
touch: /tmp/.tmpVYRqaL/7
touch: /tmp/.tmpVYRqaL/8
touch: /tmp/.tmpVYRqaL/9
run: /target/x86_64-unknown-linux-gnu/debug/coreutils tail -s.1 --max-unchanged-stats=1 -q -F 1 2 3 4 5 6 7 8 9
write(truncate): /tmp/.tmpVYRqaL/9
rename: "/tmp/.tmpVYRqaL/1" "/tmp/.tmpVYRqaL/f"
write(truncate): /tmp/.tmpVYRqaL/1
remove: "/tmp/.tmpVYRqaL/f"
run: /target/x86_64-unknown-linux-gnu/debug/coreutils tail -s.1 --max-unchanged-stats=1 -q -F 1 2 3 4 5 6 7 8 9 ---disable-inotify
write(truncate): /tmp/.tmpVYRqaL/9
rename: "/tmp/.tmpVYRqaL/1" "/tmp/.tmpVYRqaL/f"
write(truncate): /tmp/.tmpVYRqaL/1
thread 'test_tail::test_follow_name_move_create2' panicked at 'assertion failed: `(left == right)`
  left: `"tail: 9: file truncated\ntail: '1' has become inaccessible: No such file or directory\ntail: '1' has appeared;  following new file\n"`,
 right: `"tail: '1' has become inaccessible: No such file or directory\ntail: '1' has appeared;  following new file\n"`', /project/tests/by-util/test_tail.rs:1898:9