dff – duplicate file finder

dff is a tool for locating duplicate files in a directory. The tool is modelled after Duff without any advanced options.

Building

cargo build --release

Installing

cargo install

Usage

In any mode of operation:

• Symlinks are ignored
• Zero-size files are ignored
• The specified directories are searched recursively
• Hard links are not reported as duplicates

Normal Mode

dff <dir1> [dir2 [dir3...]]

In normal mode of operation; dff searches through the specified directories searching for duplicates using the following algorithm:

• Only compare files if they are of equal sizes.
• Compare the hashes of the beginning 1024 bytes to check whether they are equal.

The two checks above quickly pin-point files which might be same.

Thorough Mode

dff -t <dir1> [dir2 [dir3...]]

In thorough mode, instead of comparing the hash of first 1024 bytes; hash of the complete file contents are compared. This greatly reduces the probability of false positives.

Exhaustive Mode

dff -e <dir1> [dir2 [dir3...]]

In exhaustive mode, the complete file contents are compared and should not have any false positives.

JSON output

dff -j [-t|-e] <dir1> [dir2 [dir3...]]

In order to produce JSON output, use the -j flag. This is useful for programatically parsing the data.