Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(core): expose the task graph in the executor context #17111

Merged
merged 1 commit into from
May 24, 2023
Merged

feat(core): expose the task graph in the executor context #17111

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
13 changes: 5 additions & 8 deletions docs/generated/devkit/nx_devkit.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2064,14 +2064,11 @@ Note that the return value is a promise of an iterator, so you need to await bef

#### Parameters

| Name | Type |
| :--------------------------------- | :-------------------------------------------------------------------- |
| `targetDescription` | `Object` |
| `targetDescription.configuration?` | `string` |
| `targetDescription.project` | `string` |
| `targetDescription.target` | `string` |
| `overrides` | `Object` |
| `context` | [`ExecutorContext`](../../devkit/documents/nx_devkit#executorcontext) |
| Name | Type |
| :------------------ | :-------------------------------------------------------------------- |
| `targetDescription` | [`Target`](../../devkit/documents/nx_devkit#target) |
| `overrides` | `Object` |
| `context` | [`ExecutorContext`](../../devkit/documents/nx_devkit#executorcontext) |

#### Returns

Expand Down
13 changes: 5 additions & 8 deletions docs/generated/packages/devkit/documents/nx_devkit.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2064,14 +2064,11 @@ Note that the return value is a promise of an iterator, so you need to await bef

#### Parameters

| Name | Type |
| :--------------------------------- | :-------------------------------------------------------------------- |
| `targetDescription` | `Object` |
| `targetDescription.configuration?` | `string` |
| `targetDescription.project` | `string` |
| `targetDescription.target` | `string` |
| `overrides` | `Object` |
| `context` | [`ExecutorContext`](../../devkit/documents/nx_devkit#executorcontext) |
| Name | Type |
| :------------------ | :-------------------------------------------------------------------- |
| `targetDescription` | [`Target`](../../devkit/documents/nx_devkit#target) |
| `overrides` | `Object` |
| `context` | [`ExecutorContext`](../../devkit/documents/nx_devkit#executorcontext) |

#### Returns

Expand Down
13 changes: 3 additions & 10 deletions packages/devkit/src/utils/convert-nx-executor.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,10 +1,8 @@
import type { Observable } from 'rxjs';
import type { Executor, ExecutorContext } from 'nx/src/config/misc-interfaces';
import type { ProjectGraph } from 'nx/src/config/project-graph';
import { requireNx } from '../../nx';

const { createProjectGraphAsync, readCachedProjectGraph, Workspaces } =
requireNx();
const { Workspaces } = requireNx();

/**
* Convert an Nx Executor into an Angular Devkit Builder
Expand All @@ -21,12 +19,6 @@ export function convertNxExecutor(executor: Executor) {
});

const promise = async () => {
let projectGraph: ProjectGraph;
try {
projectGraph = readCachedProjectGraph();
} catch {
projectGraph = await createProjectGraphAsync();
}
const context: ExecutorContext = {
root: builderContext.workspaceRoot,
projectName: builderContext.target.project,
Expand All @@ -36,7 +28,8 @@ export function convertNxExecutor(executor: Executor) {
projectsConfigurations,
nxJsonConfiguration,
cwd: process.cwd(),
projectGraph,
projectGraph: null,
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why are we not passing in the project graph anymore?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is in line with another change @vsavkin did on a separate PR (for some reason, we have convertNxExecutor in nx and in devkit). I just aligned the devkit implementation with the change done in the nx one. I asked Victor about that change, and the reasoning is that if you have a builder in an Angular CLI project, there is no meaningful project graph because you aren’t in an Nx workspace and our invariants won’t hold.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok, interesting. We had a todo comment about marking the project graph as required in the interface there, we should make sure docs around the compat layer mention that the field will be undefined for Ng CLI workspaces.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agreed. Overall, the docs around Nx and Angular need a good review and more details about the integration and caveats. I have that on my list of things to do. This is not the only thing that's not documented properly.

taskGraph: null,
isVerbose: false,
};
return executor(options, context);
Expand Down
54 changes: 28 additions & 26 deletions packages/nx/bin/run-executor.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,5 +1,6 @@
import { appendFileSync, openSync, writeFileSync } from 'fs';
import { run } from '../src/command-line/run/run';
import { Target, run } from '../src/command-line/run/run';
import { TaskGraph } from '../src/config/task-graph';

if (process.env.NX_TERMINAL_OUTPUT_PATH) {
setUpOutputWatching(
Expand All @@ -12,32 +13,8 @@ if (!process.env.NX_WORKSPACE_ROOT) {
console.error('Invalid Nx command invocation');
process.exit(1);
}
requireCli();

function requireCli() {
process.env.NX_CLI_SET = 'true';
try {
const args = JSON.parse(process.argv[2]);
run(
process.cwd(),
process.env.NX_WORKSPACE_ROOT,
args.targetDescription,
args.overrides,
args.isVerbose,
false
)
.then((statusCode) => {
process.exit(statusCode);
})
.catch((e) => {
console.error(`Unexpected error`);
console.error(e);
});
} catch (e) {
console.error(`Could not find 'nx' module in this workspace.`, e);
process.exit(1);
}
}
process.env.NX_CLI_SET = 'true';

/**
* We need to collect all stdout and stderr and store it, so the caching mechanism
Expand Down Expand Up @@ -93,3 +70,28 @@ function setUpOutputWatching(captureStderr: boolean, streamOutput: boolean) {
}
});
}

process.on(
'message',
async (message: {
targetDescription: Target;
overrides: Record<string, any>;
taskGraph: TaskGraph;
isVerbose: boolean;
}) => {
try {
const statusCode = await run(
process.cwd(),
process.env.NX_WORKSPACE_ROOT,
message.targetDescription,
message.overrides,
message.isVerbose,
message.taskGraph
);
process.exit(statusCode);
} catch (e) {
console.error(`Could not find 'nx' module in this workspace.`, e);
process.exit(1);
}
}
);
4 changes: 1 addition & 3 deletions packages/nx/src/command-line/run/run-one.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -57,9 +57,7 @@ export async function runOne(
process.env.NX_VERBOSE_LOGGING = 'true';
}
if (nxArgs.help) {
await (
await import('./run')
).run(cwd, workspaceRoot, opts, {}, false, true);
await (await import('./run')).printTargetRunHelp(opts, workspaceRoot);
process.exit(0);
}

Expand Down
122 changes: 78 additions & 44 deletions packages/nx/src/command-line/run/run.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -20,6 +20,7 @@ import {
ProjectsConfigurations,
} from '../../config/workspace-json-project-json';
import { Executor, ExecutorContext } from '../../config/misc-interfaces';
import { TaskGraph } from '../../config/task-graph';
import { serializeOverridesIntoCommandLine } from '../../utils/serialize-overrides-into-command-line';
import {
readCachedProjectGraph,
Expand Down Expand Up @@ -123,28 +124,13 @@ function createImplicitTargetConfig(
return buildTargetFromScript(targetName, nx);
}

async function runExecutorInternal<T extends { success: boolean }>(
{
project,
target,
configuration,
}: {
project: string;
target: string;
configuration?: string;
},
overrides: { [k: string]: any },
async function parseExecutorAndTarget(
ws: Workspaces,
{ project, target, configuration }: Target,
root: string,
cwd: string,
projectsConfigurations: ProjectsConfigurations,
nxJsonConfiguration: NxJsonConfiguration,
projectGraph: ProjectGraph,
isVerbose: boolean,
printHelp: boolean
): Promise<AsyncIterableIterator<T>> {
validateProject(projectsConfigurations, project);

const ws = new Workspaces(root);
nxJsonConfiguration: NxJsonConfiguration
) {
const proj = projectsConfigurations.projects[project];
const targetConfig =
proj.targets?.[target] ||
Expand All @@ -159,21 +145,60 @@ async function runExecutorInternal<T extends { success: boolean }>(
throw new Error(`Cannot find target '${target}' for project '${project}'`);
}

configuration = configuration ?? targetConfig.defaultConfiguration;

const [nodeModule, executor] = targetConfig.executor.split(':');
const { schema, implementationFactory } = ws.readExecutor(
nodeModule,
executor
);

if (printHelp) {
printRunHelp({ project, target }, schema, {
plugin: nodeModule,
entity: executor,
});
process.exit(0);
}
return { executor, implementationFactory, nodeModule, schema, targetConfig };
}

async function printTargetRunHelpInternal(
{ project, target, configuration }: Target,
root: string,
projectsConfigurations: ProjectsConfigurations,
nxJsonConfiguration: NxJsonConfiguration
) {
const ws = new Workspaces(root);
const { executor, nodeModule, schema } = await parseExecutorAndTarget(
ws,
{ project, target, configuration },
root,
projectsConfigurations,
nxJsonConfiguration
);

printRunHelp({ project, target }, schema, {
plugin: nodeModule,
entity: executor,
});
process.exit(0);
}

async function runExecutorInternal<T extends { success: boolean }>(
{ project, target, configuration }: Target,
overrides: { [k: string]: any },
root: string,
cwd: string,
projectsConfigurations: ProjectsConfigurations,
nxJsonConfiguration: NxJsonConfiguration,
projectGraph: ProjectGraph,
taskGraph: TaskGraph,
isVerbose: boolean
): Promise<AsyncIterableIterator<T>> {
validateProject(projectsConfigurations, project);

const ws = new Workspaces(root);
const { executor, implementationFactory, nodeModule, schema, targetConfig } =
await parseExecutorAndTarget(
ws,
{ project, target, configuration },
root,
projectsConfigurations,
nxJsonConfiguration
);
configuration ??= targetConfig.defaultConfiguration;

const combinedOptions = combineOptionsForExecutor(
overrides,
Expand All @@ -197,6 +222,7 @@ async function runExecutorInternal<T extends { success: boolean }>(
targetName: target,
configurationName: configuration,
projectGraph,
taskGraph,
cwd,
isVerbose,
}) as Promise<T> | AsyncIterableIterator<T>;
Expand Down Expand Up @@ -254,11 +280,7 @@ async function runExecutorInternal<T extends { success: boolean }>(
* Note that the return value is a promise of an iterator, so you need to await before iterating over it.
*/
export async function runExecutor<T extends { success: boolean }>(
targetDescription: {
project: string;
target: string;
configuration?: string;
},
targetDescription: Target,
overrides: { [k: string]: any },
context: ExecutorContext
): Promise<AsyncIterableIterator<T>> {
Expand All @@ -273,22 +295,34 @@ export async function runExecutor<T extends { success: boolean }>(
context.projectsConfigurations,
context.nxJsonConfiguration,
context.projectGraph,
context.isVerbose,
false
context.taskGraph,
context.isVerbose
);
}

export function printTargetRunHelp(targetDescription: Target, root: string) {
const projectGraph = readCachedProjectGraph();
return handleErrors(false, async () => {
const projectsConfigurations =
readProjectsConfigurationFromProjectGraph(projectGraph);
const nxJsonConfiguration = readNxJson();

printTargetRunHelpInternal(
targetDescription,
root,
projectsConfigurations,
nxJsonConfiguration
);
});
}

export function run(
cwd: string,
root: string,
targetDescription: {
project: string;
target: string;
configuration?: string;
},
targetDescription: Target,
overrides: { [k: string]: any },
isVerbose: boolean,
isHelp: boolean
taskGraph: TaskGraph
) {
const projectGraph = readCachedProjectGraph();
return handleErrors(isVerbose, async () => {
Expand All @@ -303,8 +337,8 @@ export function run(
projectsConfigurations,
readNxJson(),
projectGraph,
isVerbose,
isHelp
taskGraph,
isVerbose
)
);
});
Expand Down
6 changes: 6 additions & 0 deletions packages/nx/src/config/misc-interfaces.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -210,6 +210,12 @@ export interface ExecutorContext {
*/
projectGraph?: ProjectGraph;

/**
* A snapshot of the task graph as
* it existed when the Nx command was kicked off
*/
taskGraph?: TaskGraph;

/**
* Deprecated. Use projectsConfigurations or nxJsonConfiguration
* The full workspace configuration
Expand Down
6 changes: 1 addition & 5 deletions packages/nx/src/executors/utils/convert-nx-executor.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -5,11 +5,6 @@
import type { Observable } from 'rxjs';
import { Workspaces } from '../../config/workspaces';
import { Executor, ExecutorContext } from '../../config/misc-interfaces';
import {
createProjectGraphAsync,
readCachedProjectGraph,
} from '../../project-graph/project-graph';
import { ProjectGraph } from '../../config/project-graph';

/**
* Convert an Nx Executor into an Angular Devkit Builder
Expand All @@ -34,6 +29,7 @@ export function convertNxExecutor(executor: Executor) {
nxJsonConfiguration,
cwd: process.cwd(),
projectGraph: null,
taskGraph: null,
isVerbose: false,
};
return executor(options, context);
Expand Down
3 changes: 2 additions & 1 deletion packages/nx/src/tasks-runner/batch/batch-messages.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,8 @@ export enum BatchMessageType {
export interface BatchTasksMessage {
type: BatchMessageType.Tasks;
executorName: string;
taskGraph: TaskGraph;
batchTaskGraph: TaskGraph;
fullTaskGraph: TaskGraph;
}
/**
* Results of running the batch. Mapped from task id to results
Expand Down
Loading