[rush] Design discussion / proposal for rush watch mode to allow non-terminating tasks

[scamden]

The problem:

Rush watch mode works great for terminating build tasks, like incremental compiles or even tests. It does not have a way to handle starting up continuously running tasks such dev servers (like webpack) or other continuously watched processes.

An example use case:

I would like to make a rush task called, for example, dev. Ideally dev would be a rush watched task that when run on a library type package will simply run the build without a clean similar to the rush documentation on watching mode now. But, when run on an "endpoint" package such as a node server, a webpack dev server, or even potentially a jest --watch process (to get only changed tests to run), it would simply start the task and then not call it again on watch changes. Ideally it would then collate the output from that task into the normal rush output when it gets it (allowing for the output of the standard terminating watch tasks to be printed as well).

Running the task wouldn't require any change to the rush cli, it would simply look like rush dev --to my-web-server --to my-backend-server

A possible solution:

Add a flag to rush commands in command-line.json called allowNonTerminating (certainly open to suggestions on the name haha). If that flag is set, then rush would change it's behavior a few ways:

1. Don't re-call a task on watch changes if it hasn't exited
2. Allow rush to move on from a task that hasn't exited and aggregate its output as it gets it rather than on termination. (To be printed later as the other tasks conclude)

@octogonz gave a good description of how rush might handle this here: #1202 (comment)

Questions:

1. Would the flag require the watchForChanges flag to be true? I could see a use case where the command was just start and the intention was just to launch dev servers. So rush wouldn't need to watch anything but would need to stay active to keep collating the output.

2. What's the implication when ignoreDependencyOrder : false? Since the servers will recompile and do things whenever they see changes, there's an implicit limitation that only the terminating process in the watch will happen in order. The "endpoints" will do their thing whenever they notice changes. That's fine when they are responding to upstream dep changes which will be in order. And I think mostly fine when responding to their own code changes since they generally won't be consumed as dependencies... but possibly if it were a test:watch or something we could actually not be an "endpoint" (maybe incremental terminating test executions are a better way to handle test watching anyway tho).

3. Could this be a cli flag instead? So we would execute rush start --allow-non-terminating and rush would go into this mode for any given task.

4. Since rush won't know ahead of time whether a certain task is planning to terminate or not, it will have to move on from all tasks and print their output as it gets it. How would rush make that output neat for the terminating tasks?

Discuss!

[scamden]

related: #1151