[Rush] Add .changeignore file support

[mmkal]

Fixes #2263

rush change will now look for a .changeignore file in each project with changed files. If it finds one, it will filter out files similarly to .gitignore, .npmignore, .eslintignore etc.

For example, to make sure test changes don't result in unwanted publishes, you could add **/*.test.ts to each project's .changeignore file.

[mmkal]

CC @iclanton - again I couldn't find where tests for this kind of thing live. I don't want to invent testing conventions so would be great to get some help/guidance re how you think this should be tested. It's pretty straightforward and the interesting parts are delegated to the ignore package, but still would be good to make sure it doesn't have some boolean error somewhere.

[dmichon-msft]

Suggested change

	return FileSystem.exists(changeIgnoreFilePath)
	try {
	return [...parentIgnoreStrings, FileSystem.readFile(changeIgnoreFilePath);
	} catch (error) {
	if (FileSystem.isNotExistsError(error)) {
	return parentIgnoreStrings;
	}
	throw error;
	}

Uses half as many file system operations, since FileSystem.exists() is internally just a try/catch around an fs.stat call.

[dmichon-msft]

I have performance concerns with the change as currently implemented. Path.isUnderOrEqual and Path.isUnder are currently hideously expensive, and this change causes the same (total changed files) * (number of projects) performance relationship that PackageChangeAnalyzer was experiencing until I refactored it to use the new findProjectByPosixRelativePath function to linearize the relationship.

It might be fine in a small repo, but it'll have very visible scaling influences for large PRs (e.g. bulk file moves) in a larger monorepo.

[dmichon-msft]

I recommend inverting the lookup:

const changedFilesByProject: Map<RushConfigurationProject, string[]> = new Map();

for (const file of changedFiles) {
  const relevantProject: RushConfigurationProject | undefined = this.rushConfiguration.findProjectByPosixRelativePath(file);
  if (!relevantProject) {
    continue;
  }
  let projectFiles: string[] | undefined = changedFilesByProject.get(relevantProject);
  if (!projectFiles) {
    if (!project.shouldPublish || project.versionPolicy && project.versionPolicy.exemptFromRushChange) {
      continue;
    }
    projectFiles = [];
    changedFilesByProject.set(relevantProject, projectFiles);
  }

  // Potentially test the file here
  projectFiles.push(file);
}

for (const [project, relevantFiles] of changedFilesByProject) {
  // Do your ignorer config processing and filtering here, and find out what is left.
  // Might consider recording which files prompted the change for posterity, so that we can have a --verbose option or similar for rush change that logs which files required the change file.
}

[dmichon-msft]

Be advised that, as currently implemented, this function is hideously expensive.

[dmichon-msft]

This will skip checking the entire tree for a deleted folder, which means that if said folder was supposed to be ignored for change detection, it will no longer be. I recommend skipping the check.

[dmichon-msft]

Aren't ignore patterns defined relative to the file that defines them?

[dmichon-msft]

It's almost certainly worth deduplicating intermediary values to improve on performance.

const foldersToConsider: Set<string> = new Set(relevantFiles.map(path.dirname));
for (const folder of foldersToConsider) {
  const parent: string = path.dirname(folder);
  if (/* is parent in project */) {
    foldersToConsider.add(parent);
  }

  // Can lookup the local ignore file and add the patterns here, including relevant prefixing.
}