Better iCloud sync integration

[ACTCD]

Currently, we tell users in the README:

Tip: for optimal cross platform experience it's a good idea to use an iCloud folder for syncing between macOS and iOS

But the current iCloud sync has some doubts for users:

• We don't really support iCloud in the app, so it won't appear in the list of Apps using iCloud.
• Not out of the box, users need to manually set and select the same path in iCloud Drive in different devices.
• The folder is not indicated as belonging to the App in iCloud (there is a folder icon containing the App Icon)
• Syncing process of script files is delayed, uncertain and uncontrolled
• For devices with limited space, local files may be evictioned due to iCloud optimization
  • Could be resolved by the Keep Downloaded feature introduced in macOS 15 / iOS 18
• May be briefly inaccessible due to synching, downloading, etc.; remove bookmark and restore default path

We may need to officially support iCloud features, and the synchronization mechanism and process will be managed by the App.

References:
https://developer.apple.com/library/archive/documentation/DataManagement/Conceptual/CloudKitQuickStart/Introduction/Introduction.html
https://developer.apple.com/documentation/cloudkit/
https://developer.apple.com/documentation/technotes/tn3162-understanding-cloudkit-throttles

[biuuu]

Will the data saved with GM.setValue also be synced to iCloud?

[ACTCD]

@biuuu No, not at the moment.

[2xsaiko]

For devices with limited space, local files may be evictioned due to iCloud optimization

I just hit this and was confused as to why my scripts suddenly weren't working anymore. Seems like Userscripts doesn't pick up the files when they get evicted until manually downloaded again through Finder. Is this known?

[ACTCD]

@2xsaiko As I wrote. Since the extension does not currently use the iCloud related APIs, we won't know when the system decides to evict a file, and we won't be able to read it again. Thanks for your feedback.

[2xsaiko]

Sure, but usually, files will download on access, and non-downloaded files are visible in directories. You can see this by opening iCloud Drive in your terminal and e.g. running ls, cat or cp on a couple files. Everything is there, just accessing it might block a few seconds while it downloads, and these commands don't use the iCloud API either. Userscripts doesn't even pick the files up however.

[ACTCD]

@2xsaiko Check out our code, this is how it currently works, if you'd like to debug it, or know of a simple way to improve it, please let us know or submit a PR and we'll be happy to improve the experience.

userscripts/xcode/Ext-Safari/Functions.swift

Lines 695 to 717 in 7cb1857

	// get all file urls within save location
	guard let urls = try? fm.contentsOfDirectory(at: saveLocation, includingPropertiesForKeys: []) else {
	logger?.error("\(#function, privacy: .public) - failed at (2)")
	return nil
	}
	for url in urls {
	var fileData = [String: Any]()
	// only read contents for css & js files
	let filename = url.lastPathComponent
	if (!filename.hasSuffix(".css") && !filename.hasSuffix(".js")) {
	continue
	}
	// file will be skipped if metablock is missing
	guard
	let content = try? String(contentsOf: url, encoding: .utf8),
	let dateMod = try? fm.attributesOfItem(atPath: url.path)[.modificationDate] as? Date,
	let parsed = parse(content),
	let metadata = parsed["metadata"] as? [String: [String]],
	let type = filename.split(separator: ".").last
	else {
	logger?.info("\(#function, privacy: .public) - ignoring \(filename, privacy: .public), file missing or metadata missing from file contents")
	continue
	}

[2xsaiko]

Ah, that's the NSFileManager API. That might behave differently than the POSIX readdir, not sure. I'm not familiar with native macOS development at all apart from writing a handful lines of Objective-C. Might mess around with it.

[ACTCD]

@2xsaiko The same is true for us, learning native/swift from scratch, building and refactoring the extension.

I personally do not currently use icloud sync, and there's a lot more main refactoring to do. Therefore, this issue is not currently on the foreseeable task list, unless someone is willing to investigate and contribute. I will add a label to this issue.

[2xsaiko]

Well, a simple test program lists the files and automatically downloads the contents, so something else is going on here.

How do you get started with this? I tried just opening the project and running the "Mac" configuration in Xcode (not even sure if that would work for a Safari extension) but it said "No such file or directory" for Resources/_locales, /dist and /images.

[ACTCD]

@2xsaiko Please check: https://github.com/quoid/userscripts/blob/main/docs/dev.md

Those missing parts need to be built by executing the npm command.

[2xsaiko]

There we go: #629

[ACTCD]

Interestingly, one issue should be officially resolved in the upcoming macOS 15 / iOS 18, with an additional menu item "Keep Downloaded" in the iCloud Drive of the new version of the system, which allows you to mark a specific folder not to be automatically evicted by the system.

We would like to thank our users (@JOJOforshaun) for providing us with the above information.