[WIP] fix(testing): don't allow test files to call Deno.exit()

[bartlomieju]

Just to get the prototype going, current solution is wrong as it would
error out for all tests in denoland/std#1628

Closes #12888

[bartlomieju]

A bunch of cases to consider:

• test file calls Deno.exit() in top level - this should be an immediate failure
• test file calls Deno.exit() in "unload" event handler - this one is tricky because any non-zero code should be immediate failure, but 0 exit code should still pass unless there were test failures
• test file sets exit code using op_set_exit_code - this one is also tricky, I think it should behave like previous example

[bnoordhuis]

W.r.t. (2) and (3): how do you stop the script under test in the Deno.exit(0) case? isolate.terminate_execution()?

[bartlomieju]

W.r.t. (2) and (3): how do you stop the script under test in the Deno.exit(0) case? isolate.terminate_execution()?

Currently I don't but I'm open to suggestions.

[bnoordhuis]

Discussed briefly OOB: Deno.exit() should not return so isolate.terminate_execution() may be the way to go (or suspend the worker thread until program exit but that's very meh.)

[bartlomieju]

Another crux I found that needs to be figured out: the test runner can execute multiple test files in parallel (with --jobs flag). This is done on separate threads with the main thread being a "coordinator". This is problematic because "exit code" is effectively a global (stored in deno_runtime) so threads can race between each other and override exit code value.

[bartlomieju]

@lucacasonato suggested to use Arc<AtomicUsize> that will be shared between "main worker" and its descendent "web workers"; that way we can always grab it's value from "main" worker when process shut be shutting down. Sounds like a reasonable solution, I will look into implementing that.

[bartlomieju]

Blocked by #13034

[bartlomieju]

This is now unblocked, I'll try to wrap it up this weekend.

[bartlomieju]

Discussed with @lucacasonato offline; we believe that any call to Deno.exit() in top level code in test files should result in immediate termination of isolate without heuristics for exit code; resulting in failing tests. If your code calls Deno.exit() it's most likely an application (in contrast to library) and thus you should be integration testing it using Deno.run().

[stale]

This pull request has been automatically marked as stale because it has not had recent activity. It will be closed in 7 days if no further activity occurs. Thank you for your contributions.

[stale]

This pull request has been automatically marked as stale because it has not had recent activity. It will be closed in 7 days if no further activity occurs. Thank you for your contributions.