Unhandled promise rejection: undefined thrown in CI

[ggetz]

I'm seeing some intermittent CI failures in the release test task (but not coverage) which appears to be due to a race condition in the Cesium3DTileset specs:

  Scene/Cesium3DTileset
    ✗ verify statistics
	Unhandled promise rejection: undefined thrown

[jjspace]

I've noticed this one happening again more recently

#12007 (comment)
https://github.com/CesiumGS/cesium/actions/runs/9354720455/job/25748200946
https://github.com/CesiumGS/cesium/actions/runs/9352809979/job/25741821665

[jjspace]

These seem to be much more frequent than "intermittent" lately. I feel like I'm re-running a job almost daily because of this one "verify statics" test failure. any chance we can try and track this one down soon or exclude it completely until it can be fixed so we can rely on the little red X from CI actually being an error more?
https://github.com/CesiumGS/cesium/actions/runs/10684175472/job/29613965055

[ggetz]

Let's exclude it for the time being, but leave this issue open to track the problem.

[jjspace]

Been seeing more of the other "statistics" tests failing apart from the one I already excluded

https://github.com/CesiumGS/cesium/actions/runs/10903427776/job/30257483763

  Scene/Cesium3DTileset
    ✗ verify batched features statistics
	Unhandled promise rejection: undefined thrown

Edit: another one https://github.com/CesiumGS/cesium/actions/runs/10930040038/job/30342107153#step:6:1

[lukemckinstry]

This continues to pop up https://github.com/CesiumGS/cesium/actions/runs/11126864591/job/30917806637 this is from CI after merging to main

1) verify batched features statistics
     Scene/Cesium3DTileset
     Unhandled promise rejection: undefined thrown

[ggetz]

I'll add that these have started to bubble up in pretty much 50% or more of CI runs, making the process pretty clunky.

[ggetz]

Another potential clue: I think I am only able to reproduce locally when using the --webgl-stub option.

[jjspace]

I was also able to trigger the statistics undefined thrown error with --webgl-stub

[javagl]

I was also able to trigger the statistics undefined thrown error with --webgl-stub

Just to confirm: You have been able to explicitly cause this error (deterministically and in isolation) by visiting http://localhost:8080/Specs/SpecRunner.html?category=none&webglStub=undefined&spec=Scene%2FCesium3DTileset%20verify%20batched%20features%20statistics
?

(That WebGL-stub part is surprising, insofar that it should not really affect these specs - unless I'm overlooking something...)

[ggetz]

@javagl I believe most of the squad primarily uses command line specs, i.e., npm run test.

At least for me, I am not able to trigger the error when running the test in isolation, i.e., fit("verify batched features statistics", .... I believe this is because there is a race condition where one test is having an effect on others.

[jjspace]

I spent half an hour or so looking at this on friday too. No conclusions but some observations I had

• I have never been able to reproduce the error when running the specs in browser or with --debug on and running them in the browser regardless if I use --webgl-stub or not
• This error seems to be an instance of a different async spec failing that just happens to trigger during this "statistics" spec. if you xit the batched one that's failing now the next spec down starts to fail instead
  • Further to this point if you wrap the specs in a try...catch or add .catch() blocks to the promises for these couple tests they never seem to actually catch anything
  • Stuff like this has always been a problem with jasmine (and I'm sure other test runners) see Fail with error & stack trace in asynchronous code jasmine/jasmine#529
• Unhandled Promise Exception is just a generic thing at top level node that some promise somewhere wasn't caught
• the undefined thrown makes it really hard (or maybe impossible) to actually get a stack trace of where it's actually coming from. There's also a chance even if we do get a trace the way that karma works might make it really hard to track down, see Difficulty debugging errors, stack traces missing and/or poor formatting karma-runner/karma#3296
• Locally I've never been able to reproduce 100% of the time but it's usually around 40-50% I believe. This does still happen if I fdescribe the whole Cesium3DTileset suite so I'm pretty confident it's something in that file. If I focus fit only the couple tests that commonly fail they all pass with no issues which further supports that it's a separate async spec in that file