Problems with File.real_path on macOS catalina

[jaydorsey]

I have a little toy crystal app I've been building that just reads a list of files/folders from a path and spits out the files & folders with some terminal codes to colorize stuff.

I recently upgraded to macOS catalina and it appears that File.real_path on macOS Catalina is broken

I can reproduce the error with the steps below.

• mkdir -p foo/bar && cd foo
• echo "puts File.real_path(\"bar\")" > foo.cr
• crystal build foo.cr
• ./foo

An error message occurs:

Unhandled exception: Error resolving real path of 'bar': No such file or directory (Errno)
  from /Users/jay/.asdf/installs/crystal/0.31.1/src/crystal/system/unix/file.cr:92:5 in 'real_path'
  from /Users/jay/.asdf/installs/crystal/0.31.1/src/file.cr:554:5 in 'real_path'
  from foo.cr:1:1 in '__crystal_main'
  from /Users/jay/.asdf/installs/crystal/0.31.1/src/crystal/main.cr:97:5 in 'main_user_code'
  from /Users/jay/.asdf/installs/crystal/0.31.1/src/crystal/main.cr:86:7 in 'main'
  from /Users/jay/.asdf/installs/crystal/0.31.1/src/crystal/main.cr:106:3 in 'main'

The executable will work OK, intermittently, without making any changes to anything.

What I've tried to resolve:

• In Security & Privacy > Privacy > Full Disk Access, grant the foo executable full disk access
  • This doesn't actually seem to matter; I also disabled access and could run the executable a half dozen times and it would run once or twice
• Tried other Crystal versions (0.30.0, 0.31.0, 0.31.1)

I made zero code changes, didn't regenerate the binary, or make system preference changes in the example below, and note that it jus randomly worked once (and failed again after that):

I got the idea to run it a couple times from here; Julia users are seeing similar behavior with their implementation (not a Julia user, found that link via a Google search after I didn't see any results in the Github Issues here)

Tested on macOS Catalina (10.15) with a couple Crystal versions, but I'm trying to get it working in latest (using asdf for version management to download the latest compiled binaries)

Crystal 0.31.1 (2019-09-30)

LLVM: 6.0.1
Default target: x86_64-apple-macosx

[jaydorsey]

Here's another example: I ran the command one time, made zero changes to code, used the same binary, made zero changes to any settings, configuration, etc. anywhere and it just died

I can run the executable a dozen times and it runs fine twice, fails the other 10 with a mixture of failing on different file paths. It could 100% be my poorly written code (https://github.com/jaydorsey/colorls-cr) but it worked fine until the Catalina upgrade

[jaydorsey]

So, I found a workaround for this issue, but I think this might still be a bug?

In my code, I'm capturing both the relative path to the file/folder and the absolute path. I'm calling File.real_path on the relative path, and that's what's breaking intermittently.

Calling it against the absolute path every time seems to work OK

This may or may not even be a Catalina-related bug, but I know this wasn't an issue on 0.30.0 before I upgraded, and now it doesn't work on 0.30.0. I don't have another macOS machine to test this out on.

[rdp]

Works OK in /tmp ? Something doesn't feel right here...

[RX14]

If it's an OS bug there's not much we can do...

[jhass]

Well if it only affects relative paths we could consider running File.expand_path first...

[jaydorsey]

If it's an OS bug there's not much we can do...

I haven't tested in anything but macOS High Sierra & Catalina. Seems like it could be an OS bug but there's other factors that could come into play so I was hoping others could jump in as well.

[vtjnash]

If related, I've discovered over on the linked Julia issue that the name of the DARWIN_BETTER_REALPATH function is actually "realpath$DARWIN_EXTSN". This renaming is done automatically when parsing C header files, but I suspect that crystal (like julia) just embeds the name at

crystal/src/lib_c/x86_64-macosx-darwin/c/stdlib.cr

Line 20 in 0004179

fun realpath(x0 : Char*, x1 : Char*) : Char*

and avoids trying to parse the header files to define it. (also, fwiw, there also appears to be a "realpath$UNIX2003" variant).

[jaydorsey]

I'm closing this for now; there's a sufficient workaround as I noted above in addition to the workaround noted in the linked comment issue about the FFI function behavior.

With the workaround, I've had no issues

[waj]

@vtjnash apparently using that symbol is all we needed to fix this issue!