Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add exclude_by_attribute preference to exclude files based on xattr #498

Merged
merged 2 commits into from
Sep 26, 2020

Conversation

plasticrake
Copy link
Contributor

Add preference exclude_by_attribute (boolean). If true:

  • macOS: will exclude all files that have xattr com.apple.metadata:com_apple_backup_excludeItem that contain the value com.apple.backupd.
  • Linux: will excluded all files that have xattr dulicacy_exclude.

Tests Included, tested on macOS afps and Debian ext4

Fixes #478

I named the preference "exclude_by_attribute" and not "exclude_attribute" to make it clear this doesn't affect the storage of attributes on the files.
Alternatively this could be named something like "file_exclude_attribute"

Discussion:

  • Reading Attributes
    • I see that attributes are not always read based on backups size and DUPLICACY_ATTRIBUTE_THRESHOLD. I didn't change the existing code to force a read of the attributes when this occurs. However I think that if exclude_by_attribute is true that it should read the attributes and discard them at least (or call xattr.Getxattr directly on the file if discardAttributes is true). Based on input I can update the pull request.
  • Configurable Attributes
    • I started building out other preferences to make this configurable and possibly support multiple attributes. However I was hesitant to add event more new preferences and adding more parameters to ListEntries(). I can update the pull request with input on design if requested. I think the best design would be to change the bool into a map[string]string like the Keys preference. Based on input I can update the pull request.

@gilbertchen
Copy link
Owner

Thank you for this work, but I'll leave this PR open without merging it, for two reasons:

  • I plan to rework the directory scanning code and this may affect how xattrs are read and stored in the snapshot file
  • I don't like hardcoding the attribute names. Maybe a better way is to specify these names in the filters file to allow for some flexibility.

@plasticrake
Copy link
Contributor Author

@gilbertchen Is it still possible to have this feature added? I can rework this PR if needed (has the directory scanning code you referenced above been updated yet?). I've been running an old version of duplicacy using this PR and I'd like to be able to upgrade to the newest version. Thanks!

@gilbertchen
Copy link
Owner

This pull request has been mentioned on Duplicacy Forum. There might be relevant details there:

https://forum.duplicacy.com/t/honoring-com-apple-backup-excludeitem-on-macos/979/5

@gilbertchen
Copy link
Owner

Thank you for your contribution. I'm now merging this as it has been requested by other users too: https://forum.duplicacy.com/t/honoring-com-apple-backup-excludeitem-on-macos/979

@gilbertchen gilbertchen merged commit 04a858b into gilbertchen:master Sep 26, 2020
@gilbertchen
Copy link
Owner

This pull request has been mentioned on Duplicacy Forum. There might be relevant details there:

https://forum.duplicacy.com/t/cli-2-7-0-is-now-available/4260/1

@shirazd
Copy link

shirazd commented Nov 28, 2020

Hi Gilbert,

I'm very happy to read that this has been merged in. My question is, do we need to do anything for this to take? eg. do we have to add a line to the filters file? Or is it automatic?

Thanks!
Shiraz

ps. I have cross-posted this question to github and the duplicacy forum

@shirazd
Copy link

shirazd commented Nov 28, 2020

My question was answered on the forum:

duplicacy set -exclude-by-attribute=true

Thanks all!

@gilbertchen
Copy link
Owner

This pull request has been mentioned on Duplicacy Forum. There might be relevant details there:

https://forum.duplicacy.com/t/kopia-vs-duplicacy/6625/3

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Feature Request: Honor com_apple_backup_excludeItem on MacOS
3 participants