--watch should not open inotify watches for files and folders listed in watchPathIgnorePatterns.

[jhanschoo]

🐛 Bug Report

Jest in --watch mode opens watches on all files in the project, even those blacklisted by watchPathIgnorePatterns. This is a problem in Linux where there is a limit on the number of concurrent watches.

This was first raised in #3254; current solution is a workaround, increasing the limit itself, which requires sudo permissions. It is not a true solution as reasonable (increased) limits are nevertheless reached when ~3-4 moderately-sized projects are open due to the sheer number of files in node_modules.

To Reproduce

A like to a minimal repo needed to reproduce the bug is provided in the next section with reproduction instructions. I present the most relevant excerpts here.

package.json

{
  ...
  "dependencies": {
    "jest": "^23.6.0"
  },
  "jest": {
    "watchPathIgnorePatterns": ["<rootDir>/node_modules/"]
  }
}

1. npm i
2. npx jest --watch
3. zsh processes_with_watches.sh | awk -v RS= -v ORS='\n\n' '/node_modules\//' | tac

processes_with_watches.sh

#!/usr/bin/env zsh
# this script prints a series of paragraphs:
# - each paragraph corresponds to an inotify instance
# - the first line prints the process owning that inotify instance
# - each line in the rest of the paragraph corresponds to a file being
#   watched by the inotify instance

workspace="./node_modules/"

# create an associative array of inodes to their paths in the workspace
typeset -A inode_map
# list all the files in the workspace, then pass all the paths to ls to obtain
# their corresponding inodes, then sort these lines to uniquify the inodes in case
# duplicate hard links are present; then for each line
find "$workspace" -exec ls -di {} + | sort -nu -k1,1 | while read line
do
  # split it into words by the word-separator, then initialize an array with these words
  line_arr=(${=line})
  # the first word is the inode, so assign it as the key, whereas the other words form the path
  inode_map[$line_arr[1]]="$line_arr[2,-1]"
done

# for each inotify instance (found through iterating through each
# file descriptor opened by each process and seeing if it is an
# inotify file descriptor)
find /proc/*/fd/* -type l -lname 'anon_inode:inotify' 2> /dev/null |
while read line
do
  # print the process name for later printing
  process=$(sed -e 's@^/proc/\([[:digit:]]*\)/fd/[[:digit:]]*@\1@' <<< $line)
  cat "/proc/${process}/cmdline"
  echo

  # for each path to the file descriptor
  echo $line |
  # modify it to a path to the file descriptor info file
  sed -e 's@/fd/@/fdinfo/@' |
  # and print its contents
  xargs cat |
  # filter for the lines containing information about the files
  # watched in this instance (one line corresponds to one watch)
  grep inotify |
  # extract the hex-encoded inode of the file being watched
  sed -e 's/^.*ino:\([[:xdigit:]]*\) .*$/0x\1/' |
  while read line
  do
    # if the inode corresponds to a file in the workspace
    if [[ $inode_map[$((line))] ]]
    then
      # echo the file path
      echo "$inode_map[$((line))]"
    fi
  done
  echo
done

Observe from the piped script output that the jest process maintains watches in node_modules files despite
it being blacklisted in the watchPathIgnorePatterns configuration option in package.json.

Expected behavior

The script should echo (a) blank line(s), i.e. the jest process should not maintain watches at all. Note that strictly speaking, I think this defective behavior still satisfies its behavior as documented in that the watched files still do not trigger a re-run of tests when updated, though I did not test this behavior.

Link to repl or repo (highly encouraged)

https://github.com/jhanschoo/jest-watch-bug

Run npx envinfo --preset jest

  System:
    OS: Linux 4.18 Ubuntu 18.10 (Cosmic Cuttlefish)
    CPU: (4) x64 Intel(R) Core(TM) m3-6Y30 CPU @ 0.90GHz
  Binaries:
    Node: 11.2.0 - ~/.asdf/shims/node
    npm: 6.5.0 - ~/.asdf/shims/npm
  npmPackages:
    jest: ^23.6.0 => 23.6.0 

[thymikee]

Thanks for the excellent bug report @jhanschoo, the fix to this bug is being reviewed currently.

[github-actions]

This issue has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.
Please note this issue tracker is not a help forum. We recommend using StackOverflow or our discord channel for questions.