-
Notifications
You must be signed in to change notification settings - Fork 37
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. Weβll occasionally send you account related emails.
Already on GitHub? Sign in to your account
migrate to using FFI and consolidate under the Guard/Listen banner #71
Comments
special ping to @thibaudgg as I would like your feedback to that comment specifically before proceeding. EDIT: I need your input on this before I can do anything at all. Like I mentioned in the referenced comment, I come and go from this project in an unpredictable manner. This makes it less than wise for me to have the final say in long-term vision. The comment outlines my best advice but you need to have the final say. @ioquatix appears to be overqualified to have an opinion here if you'd rather defer to him instead. |
I'm also a mac developer with C/C++ experience so let me know if I can help. Also, I think initially, FFI was only a suggestion, not a hard requirement. In the end - it's really what makes it easier and better - the internal design is up to you. If it makes sense (and less breaking) to just keep using the binary approach, I'm okay with that. Is it possible to use FFI to access the appropriate APIs without needing a compiled library? |
There is also a bug where sometimes, for no sane reason, the The communication format between parent and child process isn't nearly as well thought out as I thought it was at first. I had a plan at one point to move to otnetstring (an optimized variant of tagged netstrings that makes much more sense to me and is very easy to work with), but then I got distracted by other life events. Anyways, there is also a bug where if your file path looks sufficiently similar to the syntax I use for passing messages, then those events never make it to the library user. All of the above bugs go away if I'm not using a subprocess. The biggest concern for me is the horrible horrible things we have to do to work around an HFS+ caching bug in 10.7-10.11 and having that occur in the main ruby process rather than isolated in the subprocess. It is not, however, possible to use FFI to access the appropriate APIs without needing a compiled library. It is also not possible to perform the necessary workarounds without a compiled library. I don't see why this is an issue because I've been compiling (and signing with my personal Developer ID certificate) |
Before offering assistance, you should look at the depth of the cthulu insanity level BS happening here: https://github.com/thibaudgg/rb-fsevent/blob/master/ext/fsevent_watch/FSEventsFix.c ;) Ignore the initial comment block, it's from an earlier attempt and not in any way accurate. The custom realpath uses CoreFoundation. ...actually, you might want to read that code anyways just out of perverse curiosity. |
Uh, is this for real? Like, how does apple even use their own API if it's that broken? |
It's a VERY tricky bug, and sometimes just looking at it the wrong way makes it disappear. Describing it as an HFS+ caching bug is actually a bit simplistic as it's a series of bugs at multiple levels that have to happen at the same time for the issue to occur. I wouldn't go so far as to say it results in filesystem corruption... but it does result in completely faulty metadata in the filesystem so it's pretty close. The bug does not exist in 10.6, and was fixed in 10.12 after years of back and forth with Apple. It took that long to devise a way to reliably reproduce it and test for it... and the tech was absolutely horrified to discover that the problem existed somewhere on the filesystem of almost every machine they tested. |
lol @thibaudgg... I need more than a thumbs up emoji man. π |
Another thing to think about, regarding that wiki page... There are details we were very politely asked not to disclose. There's no gag order or NDA, but you most certainly don't bite the hand that feeds you. I believe that describing it as a metadata issue is probably sufficient... with a footnote that the kernel, |
I'm also thinking |
So you are saying that Time Machine is basically broken for 10.6 .. 10.11? |
Calling it broken is a bit of an exaggeration considering that the
filesystem is case preserving but case insensitive. No, instead the backup
would preserve an incorrect form of the path... and restoring from backup
wouldn't cause any problems because, again, the filesystem is case
insensitive. Nothing would stop working or work differently outside of
filesystem monitoring. It was, essentially, an invisible bug that broke
nothing... another reason why nobody ever noticed.
The issue existed between 10.7 and 10.11. 10.6 and 10.12 function as
intended.
The bug also doesn't exist in any way whatsoever if you're using the
(non-default) case sensitive version of HFS+. This is why I use case
sensitive HFS+ even now... (though I have to have Steam, Photoshop, and
other braindead software installed to a case insensitive disk image since
they break completely if your filesystem is case sensitive... it'll be
hilarious to see how they react to APFS, which is ONLY case sensitive right
now, with plans to potentially add case insensitivity later)
|
I use case sensitive HFS+ on all my machines :D I refuse to install crap which doesn't work on case sensitive machines :D |
Yeah, sure, sorry about that, yesterday was a bit packed for me. I completely agree with your comment about leaving If On the technical decisions (using FFI or not, ...) around |
@thibaudgg ok. I'll start out without hacks, at least... especially since the latest xcode appears to only include the 10.12 SDK anyways!
...so unless I'm missing something, I'm currently unable to target older versions of MacOS if I go the pre-compiled route for the dylib files (even though the "deployment" option still exists and goes back to 10.6, this is misleading without an older SDK available). 10.12 only it is I guess. |
@ttilley Replying specifically to your comment about SDK versions. I ran into the same problem when compiling things using Teapot. I settled on using the "MacOSX.sdk" rather than a versioned one. That SDK supports older versions of Mac OS X, you just need to run |
...which gives me even more reason to maintain I'll get properly started on friday evening or over the weekend and build from scratch to get rid of the cruft not necessary on 10.12. It'll be liberating not to care about compatibility since 10.12 works beautifully and the API has been stable for more than one major OS release at this point. Improvements have also been made to event coalescing as well so as to avoid overloading an app using file level events (initially, just renaming a file with file level events could fire as many as 10 or more separate events). I have already stated my reasons for wanting to defer to @thibaudgg on this decision and stick by them no matter what he says about my contribution to the project. π I do have a lady friend I'm hoping to have over friday afternoon for my famous homemade calzones with my own personal potato dough recipe, glorious bacon, and a mixture of bell and banana peppers fried in the bacon grease. So absolutely nothing before late evening... I hope. Wish me luck! π |
@ioquatix it'll ensure you don't use parts of the SDK not available to you, but if you pre-compile (which has been my strategy thus far) it still won't run on older releases. ...unless something has changed since I last cared enough to pay attention. |
I was under the impression if you specify that flag, it will run on older releases of mac os x. But, perhaps there is another flag you need to specify. I'd need to look it up. In any case, we can test this. |
@ioquatix - I'm thankful to have you on the job, and truly look forward to working together. #brofist |
So cheesy π§ |
-_- |
I don't mean #brofist is cheesy - the lyrics of the video are cheesy - I'm totally down with #brofists :D |
I've sent some casual thoughts to @ioquatix. This ticket will be updated if they become more than just fun ideas. |
...is there a sane way for FFI to handle C blocks as callbacks rather than function pointers? |
@ttilley no idea I never used FFI, sorry that I cannot help you more. |
If you decide to install 10.13 to play with APFS properly... don't. Older machines can't boot from an APFS root even though there's an EFI driver for it embedded into the filesystem (slightly overkill?) and if you repartition to have an HFS+ root and APFS /Users and etc (you can have multiple volumes on the same filesystem) it completely breaks both migration assistant and time machine. Oh yeah, and 10.13 doesn't wake up from sleep. |
@ttilley thanks for the info, I'll wait a little bit longer then. π |
I have no idea what I'm doing and providing a C callback via FFI to swift code is painful as fuck. Even if it was ObjC or C code, it's still kinda painful. SO I'm going to procrastinate further and install the 10.13 beta2 to play with because that's how I roll. At least I fixed most of the existing rb-fsevent bugs and released @ioquatix - with the 10.12 SDK what I thought of previously regarding compatibility is completely false. You CAN build against the 10.12 SDK and have it loosely link or use new APIs. This wasn't the case originally. I don't know exactly when the binary compatibility change was made. It HAD to be after garbage collection support was dropped completely though, not just when it was deprecated. π€· |
aside: it might be easier to rewrite and extend the existing |
Hi all, thanks!!! for all the fantastic work here. I found my way over here from codekitchen/dinghy#269, since And since we have a candidate fix already for Thanks again! |
this is a very old thread but regarding the discussion around dropping historic support on older (read: broken) versions of macOS, it seems like the M1 transition is a very good/easy place to make the jump. All M1/arm versions of the code require at least version 11, so that seems like a fairly easy safe way to make the drastic changes and not worry about historic compatibility since you will be sure anyone using an M1 won't be an older version. even if you decide to leave legacy Intel support in, you could make a fairly drastic change to the binary on the M1 version and just |
I merged #88 in (thanks @kevintom!), I'm still open to move this gem under the Guard organisation alongside the listen gem if that helps to centralize community effort as I don't have much time anymore to work on it. /cc @rymai |
Yes please do. |
@ioquatix done, the gem is now hosted under the Guard organisation: https://github.com/guard/rb-fsevent |
Thanks! |
It's somewhat awkward to completely outline a plan for your own library in a comment under somebody else's, but apparently that's how I roll. π€·ββοΈ
My stream of consciousness for this is outlined in guard/rb-inotify#66 at guard/rb-inotify#66 (comment)
The text was updated successfully, but these errors were encountered: