Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

dd: support the [io]flag=nocache option #4432

Merged
merged 4 commits into from
May 12, 2023
Merged

dd: support the [io]flag=nocache option #4432

merged 4 commits into from
May 12, 2023

Conversation

jfinkels
Copy link
Collaborator

Add support for the iflag=nocache and oflag=nocache to make dd
discard the filesystem cache for the processed portion of the input or
output file.

Fixes GNU test file tests/dd/nocache.sh.

@github-actions
Copy link

GNU testsuite comparison:

Congrats! The gnu test tests/dd/nocache is no longer failing!
GNU test failed: tests/dd/nocache_eof. tests/dd/nocache_eof is passing on 'main'. Maybe you have to rebase?

tertsdiepraam
tertsdiepraam reviewed Feb 25, 2023
View reviewed changes
src/uu/dd/src/dd.rs Outdated Show resolved Hide resolved
@github-actions
Copy link

github-actions bot commented Mar 5, 2023

GNU testsuite comparison:

Congrats! The gnu test tests/dd/nocache is no longer failing!
GNU test failed: tests/dd/nocache_eof. tests/dd/nocache_eof is passing on 'main'. Maybe you have to rebase?
GNU test failed: tests/misc/timeout. tests/misc/timeout is passing on 'main'. Maybe you have to rebase?

@github-actions
Copy link

GNU testsuite comparison:

Congrats! The gnu test tests/dd/nocache is no longer failing!
GNU test failed: tests/dd/nocache_eof. tests/dd/nocache_eof is passing on 'main'. Maybe you have to rebase?

@github-actions
Copy link

GNU testsuite comparison:

Congrats! The gnu test tests/dd/nocache is no longer failing!
GNU test failed: tests/dd/nocache_eof. tests/dd/nocache_eof is passing on 'main'. Maybe you have to rebase?

@sylvestre
Copy link
Contributor

clippy found a bunch of new warnings here :)

@jfinkels
Copy link
Collaborator Author

clippy found a bunch of new warnings here :)

Yeah, I'm having trouble getting the right combination of #[cfg] lines for all the different platforms :|. Maybe it's a good time to start thinking about adding a platform abstraction layer here like in some of the other utils? In the meantime, I'll try again to get this right.

@github-actions
Copy link

GNU testsuite comparison:

Congrats! The gnu test tests/dd/nocache is no longer failing!
GNU test failed: tests/dd/nocache_eof. tests/dd/nocache_eof is passing on 'main'. Maybe you have to rebase?

@github-actions
Copy link

GNU testsuite comparison:

Congrats! The gnu test tests/dd/nocache is no longer failing!
GNU test failed: tests/dd/nocache_eof. tests/dd/nocache_eof is passing on 'main'. Maybe you have to rebase?
GNU test failed: tests/tail-2/inotify-dir-recreate. tests/tail-2/inotify-dir-recreate is passing on 'main'. Maybe you have to rebase?

tertsdiepraam
tertsdiepraam reviewed Mar 20, 2023
View reviewed changes
src/uu/dd/src/dd.rs Outdated Show resolved Hide resolved
@github-actions
Copy link

GNU testsuite comparison:

Congrats! The gnu test tests/dd/nocache is no longer failing!
GNU test failed: tests/dd/nocache_eof. tests/dd/nocache_eof is passing on 'main'. Maybe you have to rebase?

@sylvestre
Copy link
Contributor

Linux failed with:

error[E0308]: arguments to this function are incorrect
   --> src/uu/dd/src/dd.rs:188:17
    |
188 |                 posix_fadvise(f.as_raw_fd(), offset, len, advice)
    |                 ^^^^^^^^^^^^^                ------  --- expected `i32`, found `i64`
    |                                              |
    |                                              expected `i32`, found `i64`
    |
note: function defined here
   --> /cargo/registry/src/github.com-1ecc6299db9ec823/nix-0.26.2/src/fcntl.rs:848:12

@github-actions
Copy link

github-actions bot commented Apr 4, 2023

GNU testsuite comparison:

Congrats! The gnu test tests/dd/nocache is no longer failing!
GNU test failed: tests/dd/nocache_eof. tests/dd/nocache_eof is passing on 'main'. Maybe you have to rebase?
GNU test failed: tests/du/threshold. tests/du/threshold is passing on 'main'. Maybe you have to rebase?
GNU test failed: tests/misc/wc-total. tests/misc/wc-total is passing on 'main'. Maybe you have to rebase?

@jfinkels
Copy link
Collaborator Author

jfinkels commented May 7, 2023

Linux failed with:

error[E0308]: arguments to this function are incorrect
   --> src/uu/dd/src/dd.rs:188:17
    |
188 |                 posix_fadvise(f.as_raw_fd(), offset, len, advice)
    |                 ^^^^^^^^^^^^^                ------  --- expected `i32`, found `i64`
    |                                              |
    |                                              expected `i32`, found `i64`
    |
note: function defined here
   --> /cargo/registry/src/github.com-1ecc6299db9ec823/nix-0.26.2/src/fcntl.rs:848:12

Hmm, on my machine posix_fadvise() takes i64 but I guess the number of bits depends on the libc::off_t type. I'll try to adjust the code to match that.

https://docs.rs/nix/latest/nix/fcntl/fn.posix_fadvise.html

jfinkels added 3 commits May 7, 2023 11:43
@jfinkels
dd: add bytes_total to ReadStat
12e4aca 
Add the `bytes_total` field to the `ReadStat` struct. This lets the
main loop of `dd` keep track of the total number of bytes read.
@jfinkels
dd: add discard_cache() funcs for Input, Output
cc2f97b 
Add the `Input::discard_cache()` and `Output::discard_cache()`
functions. These allow discarding the filesystem cache when `dd` no
longer needs to access a specified portion of the input or output
file, respectively.
@jfinkels
dd: support the [io]flag=nocache option
bd18a2a 
Add support for the `iflag=nocache` and `oflag=nocache` to make `dd`
discard the filesystem cache for the processed portion of the input or
output file.
@github-actions
Copy link

github-actions bot commented May 7, 2023

GNU testsuite comparison:

Congrats! The gnu test tests/dd/nocache is no longer failing!
GNU test failed: tests/dd/nocache_eof. tests/dd/nocache_eof is passing on 'main'. Maybe you have to rebase?

@github-actions
Copy link

github-actions bot commented May 7, 2023

GNU testsuite comparison:

Congrats! The gnu test tests/dd/nocache is no longer failing!
Congrats! The gnu test tests/tail-2/inotify-dir-recreate is no longer failing!
GNU test failed: tests/dd/nocache_eof. tests/dd/nocache_eof is passing on 'main'. Maybe you have to rebase?

@jfinkels
Copy link
Collaborator Author

jfinkels commented May 7, 2023

Okay please take another look. The tests/dd/nocache_eof GNU test is marked as newly FAILing because before this change it was being SKIPped.

@sylvestre sylvestre merged commit 64c49de into uutils:main May 12, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@tertsdiepraam tertsdiepraam tertsdiepraam left review comments

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@jfinkels @sylvestre @tertsdiepraam