Continuous testing should be mostly green

[samreid]

As discussed at today's dev meeting, we would like to invest more resources into getting CT to show more green more of the time so there is less trouble and less noise. This will take coordination from many developers in fixing sim-specific code, common code, etc, so it is an epic-level issue. I opened this so it can be triaged and scheduled for 2022 Q2.

[samreid]

For instance, here are the red cells from the last 10 columns:

[kathy-phet]

I want check in on how I should interpret the above ... are the only sim-specific repos that currently need attention just the top 9 rows? Should ignore the 7/4 column? Or is 7/4 more accurate because it had more tests done and I should add those rows? (e.g. light versus dark shades).
buoyancy
center-and-variability
CCK AC
CCK DC
density
fractions-mided-numbers
gravity-and-orbits
number-play

[chrisklus]

It looks like the latest columns finished tests and has similar results to your list above. I think puppeteer is sometimes causing errors that are showing up repeatedly, like the 12:14 column from below. see the error for RIAW - it was the same for the those last 6.

[samreid]

I fixed 3 separate CT issues today.

I pushed a proposed fix for the acid-base-solutions CT error in phetsims/acid-base-solutions#174
I also fixed CT error phetsims/nitroglycerin#13
I also fixed CT error phetsims/circuit-construction-kit-ac-virtual-lab#6

[samreid]

I pushed a fix for phetsims/build-a-molecule#232
I also fixed phetsims/circuit-construction-kit-common#891
I added a workaround for phetsims/collision-lab#217
I reverted phetsims/simula-rasa#26 (comment)

[samreid]

After the fixes, my local CT has been been pretty green, I'll put on hold and check again in the coming week.

[samreid]

I also fixed phetsims/energy-forms-and-changes#416

[samreid]

Since last week, I have fixed or worked around 10 CT issues. I'll continue monitoring these as I have time.

[samreid]

I kept an eye on this again this week and addressed several issues.

[samreid]

Nov 17, 2022

• Center and Variability: @samreid volunteered himself
  CCK: @samreid too (will be addressed in CT Various circuit-construction-kit-common#897)
  fractions-mixed-numbers: property re-entrant @AgustinVallejo
  number-compare matching tenFrameNode not found!: @chrisklus
  number-play: @chrisklus

[samreid]

Here are the last 10 columns with any problems:

I'll look into the Circuit Construction Kit ones.

[chrisklus]

From 12/8/22 dev meeting:

Here is the current state:

[samreid]

Today's status:

[zepumph]

Today I made some changes that added some metrics to each column, now we can see how many tests are complete and how many are failing:

I hope this helps with this issue. There are ~40 problems on CT right now, but much of that is duplicated, and some of that is because of sparky weirdness that @jonathanolson and I are still sorting out over in https://github.com/phetsims/special-ops/issues/234.

[samreid]

CT was mostly green for a while, closing. (We agreed on this as a developer meeting team). Nice work everyone!