memory test for brand=phet (1.1 branch)

[pixelzoom]

This is one of the "standard GitHub issues" required by the code-review checklist.

[pixelzoom]

Test procedure:

1. Run the sim with secondsPerGeneration=0.25 (so this test can be done quickly)
2. Go to the Lab screen
3. In the "Add Mutations" panel, press the dominant buttons for Fur, Ears, Teeth (so that we create a lot of points on the Population graph)
4. In the "Environmental Factors" panel, check the "Limited Food" checkbox (so that the population stabilizes)
5. In the Population graph, zoom out to y-axis range 0-500 (so that we create the view for plots)
6. Take a heap snapshot at Generation 0
7. Press the "Add a Mate" button
8. Take a heap snapshot every ~100 generations, including generation 1000
9. Close the "Memory Limit" dialog that pops up at Generation 1000.
10. Press the "Reset All" button, then take another Generation 0 heap snapshot

[pixelzoom]

Test results with 1.1.0-dev.13 brand=phet, built using grunt dev --brands=phet,phet-io --minify.mangle=false

Test configuration: 2019 MacBook Pro (MacBookPro16,1), macOS 10.15.6, Chrome 84.0.4147.135

Generation	Heap Size (MB)
0	25.0
100	61.1
200	83.6
300	105
400	130
500	152
600	172
700	199
800	221
900	245
1000	266
0	253

It certainly looks like there's a memory leak here. I will investigate.

[pixelzoom]

Unfortunately labeling this as "blocks-sim-publication" for the prototype. This leak is bad enough that it's very likely to be encountered by the user -- especially on iPad -- and will crash the browser.

[pixelzoom]

The commonality that I see between all heap snapshot comparisons is that Tandem and DynamicTandem show large #New and #Deleted=0. For example:

ProportionsCounts here is expected to be #Deleted=0, because counts are retained for all generations until we Reset All or Start Over (or reach the 1000-generation limit).

[pixelzoom]

@samreid identified a memory leak in Tandem, see phetsims/tandem#203. He created a patch (see the issue) which has not been committed yet.

Snapshots for the first 400 generations BEFORE the patch:

Snapshots for the first 400 generations AFTER the patch:

And Tandem and DynamicTandem instances are no longer appearing as #Deleted=0:

I'm going to continue testing with the patch in my local copy, to see if the sim continues to leak, or if it stabilizes.

[pixelzoom]

Test results with local unbuilt, natural-selection sha e32a316, with the patch from phetsims/tandem#203 applied.

Test configuration: 2019 MacBook Pro (MacBookPro16,1), macOS 10.15.6, Chrome 84.0.4147.135

Generation	Heap Size (MB)
0	27.7
100	48.0
200	54.2
300	58.3
400	65.3
500	69.2
600	73.7
700	84.3
800	88.5
900	93.9
1000	98.1
0	83.4

Compared to #189 (comment), this is huge improvement. There may be additional leak(s), I'll need to investigate further. I did let the sim continue to run and took several more snapshots before and after Reset All, and heap size seems to stabilize in the range of 84-100MB.

[pixelzoom]

Looking at the heap snapshot taken after Reset All, I see hundreds (thousands?) of what look like phetioID strings for Genotype's abbreviationProperty. For example:

I would expect to see 1 such phetioID, for the lone bunny, and it would be "...bunny_0.genotype.abbreviationProperty".

[pixelzoom]

Inspecting the above arrays further (screenshot below), all of the strings in these arrays are phetioIDs for DerivedProperty instances. Some exist for the lifetime of the sim, but others are dynamic. I suspect that iO is not cleaning up phetioIDs for DerivedProperty.

In the snapshot following Reset All, these arrays account for 31% of the (retained) heap size.

[pixelzoom]

@samreid and I worked on this via Zoom and identified another big iO leak, see phetsims/axon#328.

Test results with local unbuilt, brand=phet, natural-selection sha e32a316, with the patches from phetsims/tandem#203 and phetsims/axon#328 applied.

Test configuration: 2019 MacBook Pro (MacBookPro16,1), macOS 10.15.6, Chrome 84.0.4147.135

Generation	Heap Size (MB)
0	27.7
100	44.6
200	44.6
300	44.9
400	44.7
500	44.7
600	44.6
700	44.7
800	45.1
900	45.5
1000	45.2
0	29.3

Two things make me happy about these results:

(1) We're testing with a stable population, and the memory footprint is immediately stable.
(2) The memory footprint for Generation 0 is consistent.

So... Work is done here. All leaks were in PhET-iO. On hold until phetsims/tandem#203 and phetsims/axon#328 are addressed. Then I will sanity check with one more test.

[pixelzoom]

After fixes for phetsims/tandem#203 and phetsims/axon#328 were pushed...

Testing results on 2019 MacBook Pro, macOS 10.15.6, Chrome 85.0.4183.83 (Incognito), with https://phet-dev.colorado.edu/html/natural-selection/1.1.0-dev.17/phet/natural-selection_en_phet.html?secondsPerGeneration=0.25

Generation	Heap Size (MB)
0	24.8
100	41.3
200	41.7
300	41.7
400	42.1
500	42.5
600	42.2
700	42.1
800	41.8
900	42.5
1000	42.5
0	26.8

This looks great. Stable memory with stable population, consistent memory for Generation 0.

Closing.