fix(core): collect all logs from forked processes

[dgreif]

Current Behavior

We have a monorepo with ~450 packages and run tasks with 16x parallelism in CI. We've recently noticed that running tasks with static logging can lead to random tasks losing logs. Even if the task is a simple echo SOMETHING, logs get lost sporadically.

After digging into the task runner, I found that the forked task runner currently collects logs from stdout and stderr, then joins those when the process exit event is emitted. The problem appears to stem from the fact that the exit event is occasionally emitted before the stdout and stderr streams are closed, leading to lost logs.

Expected Behavior

All logs should be collected, even if they are emitted after the process exit event.

I've updated the task runner to wait for the stdout and stderr streams to end before collecting logs.

Note: I didn't see any tests specifically for this file, but it should still run all the new code if theres is a happy path case of forked tasks with static logging.

[vercel]

The latest updates on your projects. Learn more about Vercel for Git ↗︎

1 Skipped Deployment

Name	Status	Preview	Updated (UTC)
nx-dev	⬜️ Ignored (Inspect)	Visit Preview	Sep 5, 2024 0:18am

[nx-cloud]

☁️ Nx Cloud Report

CI is running/has finished running commands for commit b9e33ad. As they complete they will appear below. Click to see the status, the terminal output, and the build insights.

📂 See all runs for this CI Pipeline Execution

---

✅ Successfully ran 4 targets

• nx affected --targets=lint,test,build,e2e,e2e-ci --base=eb7ac2c57a0e9a607d2efa108731c4eaaaedb716 --head=b9e33ad1b281ccd110a1ade78e4ddafaf8b45da9 --parallel=3
• nx run-many -t check-imports check-commit check-lock-files check-codeowners documentation --parallel=1 --no-dte
• nx-cloud record -- nx format:check --base=eb7ac2c57a0e9a607d2efa108731c4eaaaedb716 --head=b9e33ad1b281ccd110a1ade78e4ddafaf8b45da9
• nx documentation --no-dte

---

Sent with 💌 from NxCloud.

[rluvaton]

This seems like fixing the same bug as my pr from 7 months ago

• fix(core): wait for target output to be flushed before exiting #21498

[dgreif]

@rluvaton great catch! I'm surprised your PR hasn't gotten any traction yet 😞. Which approach would you lean toward for fixing the issue? I see the merit in your approach, but I'd be worried that some new issue could easily pop up which starts keeping the child process alive again. With the fix I have here, we are less dependent on the child process exiting with specific timing. WDYT?

[github-actions]

This pull request has already been merged/closed. If you experience issues related to these changes, please open a new issue referencing this pull request.