Watch mode: git diff on macOS triggers rebuild

[OliverJAsh]

Bug Report

🔎 Search Terms

• watch

🕗 Version & Regression Information

• This is the behavior in every version I tried, and I reviewed the FAQ for entries about configuring watch

⏯ Playground Link

N/A

💻 Code

Steps to reproduce:

1. In a Git repository with the following files: main.ts and tsconfig.json:

   {
     "files": ["./main.ts"]
   }

2. Modify main.ts so Git has unstaged changes and then run:

   $ tsc --project tsconfig.json --watch --diagnostics

3. Finally, run git diff.

🙁 Actual behavior

tsc rebuilds because git diff bumps the access timestamp (see stat -x main.ts).

This is particularly problematic for me because I use lazygit which refreshes the Git diff every few seconds.

Screen.Recording.2023-02-20.at.20.07.44.2.mov

--

Output from tsc:

FileWatcher:: Triggered with /Users/oliverash/Development/temp/lazygit-refresh-tsc/main.ts 1:: WatchInfo: /Users/oliverash/Development/temp/lazygit-refresh-tsc/main.ts 250 undefined Source file
Scheduling update
Elapsed:: 0ms FileWatcher:: Triggered with /Users/oliverash/Development/temp/lazygit-refresh-tsc/main.ts 1:: WatchInfo: /Users/oliverash/Development/temp/lazygit-refresh-tsc/main.ts 250 undefined Source file
Synchronizing program

🙂 Expected behavior

tsc should not rebuild when the access timestamp changes.

I suspect this is due to the behaviour of fs.watch. In that case, is there anything else we can do to improve the default behaviour for macOS users? (Assuming this issue only occurs on macOS because I haven't been able to test on other platforms.)

Note I am able to workaround by setting any of the following variables in my shell RC file:

export TSC_WATCHFILE=DynamicPriorityPolling
export TSC_WATCHFILE=PriorityPollingInterval
export TSC_WATCHFILE=UseFsEvents
export TSC_WATCHFILE=UseFsEventsOnParentDirectory
export TSC_NONPOLLING_WATCHER=true

[jakebailey]

#52535 feels related, if Windows suffers from the same problem with the new watching (though that example is node_modules).

[ikesau]

I can confirm that git diff started triggering this for me (which was happening every time I switched to VSCode) and that export TSC_WATCHFILE=DynamicPriorityPolling fixed it. Thank you @OliverJAsh !

[FDIM]

There is something odd going on with watching after update to typescript 4.9.5 from 4.8.4. VScode became barely usable (due to tsserver being used in multiple extensions) and after some debugging I managed to track down that it is file system changes that causes tsc to recompile/update.

More specifically it is fs.watch emitting a change event for a file in node_modules/**/package.json when tsc runs. Why that happens I have no idea.

Coincidently I was working on a little related feature in the project, and just for fun added fsPromises.watch('./node_modules/htmlparser2/package.json') and got quite surprised that I started getting events when tsserver inside vscode extension was running!

Now solution was to add this in the config:

    "watchOptions": {
        "watchFile": "dynamicPriorityPolling",
        "excludeDirectories": ["**/node_modules", "**/.angular", "**/.git"],
    },

Let me know if I can debug something more specifically. I'm experiencing this issue on Windows 10 and Node 16/18.

Now I only have to figure why lint takes 18min instead of 4min.

EDIT: This is a little crazy but somehow related to watchers or I'm misunderstanding something, runtime in CI went down from 16min to 2min!
Before:

After:

Only change is this in app tsconfig and spec tsconfig:

...
    "watchOptions": {
        "excludeDirectories": ["**/*"],
    }

[Codex-]

I can confirm this behaviour too, unfortunately, thanks for raising this already

[llllvvuu]

tsc rebuilds because git diff bumps the access timestamp (see stat -x main.ts).

git diff doesn't bump the Modify timestamp so maybe that would be the right thing for tsc to use

There are 3 kind of "timestamps":
Access - the last time the file was read
Modify - the last time the file was modified (content has been modified)
Change - the last time meta data of the file was changed (e.g. permissions)

https://unix.stackexchange.com/a/2803

If this is being handled purely by fs.watch then maybe this issue should be re-raised at https://github.com/nodejs/node ?

EDIT: I have confirmed this is a Node.js issue by reproducing it without TypeScript. Just filed nodejs/node#49916

EDIT 2: Update from Node.js side - wontfix working as intended. So these extraneous events will remain unless TypeScript's fs.watch handler stats the files itself. We are stuck with using other watchFile methods for now. UseFsEventsOnParentDirectory is actually quite good as it takes out much fewer watches.

[evelant]

Thanks for raising this issue, it was driving me crazy and it's nice to finally know the cause. If I tried to use watch I'd get seemingly random rebuilds and ~4 rebuilds on every change.

This can get triggered by so many things. Even default vscode git integration refreshing its list of changed files causes the watcher to recompile. Or using the starship shell prompt that indicates git status, every command entered into a terminal will cause a recompile.

[sheetalkamat]

@OliverJAsh can you please try out #56403 (comment) to see if that helps

[OliverJAsh]

I just gave it a go and unfortunately I'm getting an out of memory error.

For reference I'm coming from v5.2.2.

[OliverJAsh]

I just tested this in my reduced test case and it does seem to fix the problem.