dependencyOf - the inverse of dependsOn

[fxposter]

• I'd be willing to implement this feature (contributing guide)

Description

I'd like to add the dependencyOf field to the TargetConfiguration interface, which would allow users and plugins to add additional targets that would automatically add themselves to other targets' dependsOn lists.

Motivation

Main motivation is to programmatically (in plugins) add targets that would execute before other targets. This would allow to easily implement preBuild functionality without needed to modify the existing build target or even know if it exists.

Suggested Implementation

The idea is to add a dependencyOf field to the TargetConfiguration and transform such dependencies to the dependsOn ones on the ProjectGraph creation step, so that everything that works with the ProjectGraph itself would not care about the dependencyOf and continue to rely only on dependsOn. It can be done after building full projectGraph here: https://github.com/nrwl/nx/blob/master/packages/nx/src/project-graph/build-project-graph.ts#L196 by iterating over all projects like this:

for (const project in finalGraph.nodes) {
  const targets = finalGraph.nodes[project].targets;
  if (targets) {
    for (const dependencyName in targets) {
      if (targets[dependencyName].dependencyOf) {
        for (const name of targets[dependencyName].dependencyOf) {
          if (targets[name]) {
            targets[name].dependsOn || (targets[name].dependsOn = []);
            targets[name].dependsOn.push(dependencyName);
          }
        }
        delete(targets[dependencyName].dependencyOf)
      }
    }
  }
}

Alternate Implementations

...

[fxposter]

the mentioned implementation would transform these targets

{
  'a': {},
  'b': {
    dependsOn: ['a'],
    dependencyOf: ['c'],
  }
  'c': {
    dependsOn: ['d'],
  },
  'd': {
  }
}

to

{
  'a': {},
  'b': {
    dependsOn: ['a'],
  }
  'c': {
    dependsOn: ['d', 'b'],
  },
  'd': {
  }
}

maybe it would be a useful feature to copy original dependsOn from the c target to b as well, so that in the end we get this targets instead:

{
  'a': {},
  'b': {
    dependsOn: ['a', 'd'],
  }
  'c': {
    dependsOn: ['d', 'b'],
  },
  'd': {
  }
}

[fxposter]

@AgentEnder @FrozenPandaz I see that you both work on project graph. Can you comment on whether it might be a good idea and PR for it might be merged or it would definitely be rejected?

[AgentEnder]

Hey, I wouldn't recommend submitting a PR for this just yet. Its not a crazy feature or anything, but its something we will need to discuss internally to decide if its something that we want to support. Anything that could be described by dependencyOf, could after all be described by one or more dependendsOn targets too.

[fxposter]

Maybe I am missing something, but how people handle cases where they simply want to split a target into 2 targets to run them in parallel?

ie: if I have setup, build and test targets, and build depends on setup, test depends on build.
if I want to split build in (build-css+build-js) and split test into (test-unit+test-e2e), that all can be run in parallel - on each such change - I'd need to change dependsOn lists on a lot of tasks, for example adding separate test-integration - I need to make it depend on (build-css+build-js) now.

With dependencyOf feature it can be modeled like this:
originally I have

{
  "targets": {
    "setup": {},
    "build": {
      "dependsOn": ["setup"],
    },
    "test": {
      "dependsOn": ["build"],
    }
  }
}

then I extract "build-css" into separate task:

{
  "targets": {
    "setup": {},
    "build": { // do build without css
      "dependsOn": ["setup"],
    },
    "build-css": {
      "dependencyOf": [{ "target": "build", "copyDependencies": true }],
    },
    "test": {
      "dependsOn": ["build"],
    }
  }
}

then I make "build" a dummy target, that does nothing and extract everything into separate tasks as well:

{
  "targets": {
    "setup": {},
    "build": { // do nothing
      "dependsOn": ["setup"],
    },
    "build-css": {
      "dependencyOf": [{ "target": "build", "copyDependencies": true }],
    },
    "build-js": {
      "dependencyOf": [{ "target": "build", "copyDependencies": true }],
    },
    "build-fonts": {
      "dependencyOf": [{ "target": "build", "copyDependencies": true }],
    },
    "test": {
      "dependsOn": ["build"],
    }
  }
}

then I can do the same trick with tests:

{
  "targets": {
    "setup": {},
    "build": { // do nothing
      "dependsOn": ["setup"],
    },
    "build-css": {
      "dependencyOf": [{ "target": "build", "copyDependencies": true }],
    },
    "build-js": {
      "dependencyOf": [{ "target": "build", "copyDependencies": true }],
    },
    "build-fonts": {
      "dependencyOf": [{ "target": "build", "copyDependencies": true }],
    },
    "test": { // do nothing
      "dependsOn": ["build"],
    },
    "test-unit": {
      "dependencyOf": [{ "target": "test", "copyDependencies": true }],
    },
    "test-e2e": {
      "dependencyOf": [{ "target": "test", "copyDependencies": true }],
    },
}

with such configuration the last target graph would be translated into this one:

{
  "targets": {
    "setup": {},
    "build": { // do nothing
      "dependsOn": ["setup", "build-css", "build-js", "build-fonts"],
    },
    "build-css": {
      "dependsOn": ["setup"]
    },
    "build-js": {
      "dependsOn": ["setup"]
    },
    "build-fonts": {
      "dependsOn": ["setup"]
    },
    "test": { // do nothing
      "dependsOn": ["build", "test-unit", "test-e2e"],
    },
    "test-unit": {
      "dependsOn": ["build"]
    },
    "test-e2e": {
      "dependsOn": ["build"]
    },
}

[AgentEnder]

Yeah, so you just shown that the proposed dependencyOf syntax could easily be converted into dependsOn blocks. I don't see a ton of value in supporting both.

[fxposter]

it's all about what can be added by developers without needing to rewrite their existing targets

[fxposter]

currently if you have a chain of A target depends on B, B depends on C, if you need to insert something in the B layer (imagine the B is the "build" in my example) - you need to rewrite both B and C. with my proposal you can insert B1 without touching neither B, nor C.

[fxposter]

the alternative that potentially can allow doing something like this is to allow dependOn to have wildcards, so that if I define

{
  "targets": {
    "build": {
      "dependsOn": ["^build"]
    },
    "test": {
      "dependsOn": ["build*"]
    }
  }
}

then I can split "build" to "build-css"+"build-js" without changing that test target. And I can define such additional targets in different plugins, for example.

[fxposter]

@AgentEnder would wildcard/regex target dependencies have more chances to be accepted?

[fxposter]

We're trying to build a CI system with multiple layers of task executions and nx in this case is lacking ability for plugins to add targets dynamically between already defined targets, cause V2 extension points don't allow you to see the "already defined targets for a project". Since v2 plugins were introduced for a reason - I understand that "modifying an already built project graph" is not something nx is willing to support - hence I am trying to find an extension points that could be accepted within "the nx vision".

[fxposter]

I've made an example implementation for wildcards in target dependencies to make it clear what we would like to have.

[fxposter]

@AgentEnder any chance to get at least preliminary response of the PR, please?

[fxposter]

Closing in favor of #19611, but unfortunately it doesn't seem to be interested anyone from core team :(

[github-actions]

This issue has been closed for more than 30 days. If this issue is still occuring, please open a new issue with more recent context.