Fix bad interaction between promotion and incremental builds on OSX

[ghost]

Previously we were only using mtime but it's not precise enough on OSX.

[rgrinberg]

CI still fails:

        diff (internal) (exit 1)
(cd _build/default && /usr/bin/diff -u test/blackbox-tests/test-cases/promote/run.t test/blackbox-tests/test-cases/promote/run.t.corrected)
--- test/blackbox-tests/test-cases/promote/run.t	2018-01-29 16:53:36.000000000 +0000
+++ test/blackbox-tests/test-cases/promote/run.t.corrected	2018-01-29 16:53:58.000000000 +0000
@@ -17,11 +17,8 @@
 
   $ echo titi > x
   $ $JBUILDER build --root . -j1 --diff-command false @blah --auto-promote 2>&1 | sed 's/.*false.*/DIFF/'
-            sh (internal) (exit 1)
-  DIFF
-  Promoting _build/default/x.gen to x.
   $ cat x
-  toto
+  titi
   $ $JBUILDER build --root . -j1 --diff-command false @blah
   $ cat x
-  toto
+  titi

With a different messages now though. Here's the other one:

        diff (internal) (exit 1)
(cd _build/default && /usr/bin/diff -u test/blackbox-tests/test-cases/promote/run.t test/blackbox-tests/test-cases/promote/run.t.corrected)
--- test/blackbox-tests/test-cases/promote/run.t	2018-01-29 16:56:33.000000000 +0000
+++ test/blackbox-tests/test-cases/promote/run.t.corrected	2018-01-29 16:56:43.000000000 +0000
@@ -12,6 +12,9 @@
   toto
 
   $ $JBUILDER build --root . -j1 --diff-command false @blah
+            sh (internal) (exit 1)
+  (cd _build/default && /bin/sh -c 'false x x.gen')
+  [1]
   $ cat x
   toto
 
@@ -23,5 +26,8 @@
   $ cat x
   toto
   $ $JBUILDER build --root . -j1 --diff-command false @blah
+            sh (internal) (exit 1)
+  (cd _build/default && /bin/sh -c 'false x x.gen')
+  [1]
   $ cat x
   toto

[ghost]

And obviously, I can't reproduce it on my macbook...

[ghost]

OSX is very frustrating:

$ echo toto > x; stat -s x |tr " " "\n" > s1; cat x > y; cat y > x; stat -s x |tr " " "\n" > s2; diff -u s1 s2
--- s1	2018-01-30 08:35:55.000000000 -0500
+++ s2	2018-01-30 08:35:55.000000000 -0500
@@ -6,7 +6,7 @@
 st_gid=1068632065
 st_rdev=0
 st_size=5
-st_atime=1517319354
+st_atime=1517319355
 st_mtime=1517319355
 st_ctime=1517319355
 st_birthtime=1517319048

[ghost]

Ok, I made sure that we delete the destination file before copying to force a new inode to be allocated. I also removed some old stuff about updated files, which dates from when we were using timestamps for incremental compilation.

[ghost]

This issue is driving me mad

[ghost]

Ok, so basically we just can't rely on stat on OSX. That means that we must re-hash files outside of _build on every run. I need to change the code so that we only do that on OSX. I'm also trying to see if rehashing everything is OK with a faster hash.

[ghost]

@rgrinberg, i've made the minimal necessary changes to improve the behaviour and get the tests to pass. It's ready for review.

I'll look later into the idea of systematically rehashing external files and using a faster hash implementation.

[rgrinberg]

Would you mind explaining this change? Why was setting it to false a problem before?

[ghost]

• it was useless given that we were doing that for the targets of a rule that we always delete before running the rule anyway
• it didn't effectively removed entries from the on-disk cache