Remove QCheck2.TestResult.get_instances to address memory leak #288
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
QCheck.TestResult.t.instances
was originally added in dc53caf to address #51.With the addition of
QCheck2
it was moved there,QCheck.TestResult.t
was made abstract, and insteadwas exposed in the 0.18 interface.
Now https://sherlocode.com/?q=get_instances reveals that noone uses the binding, however #287 reveals that keeping all test inputs causes memory leaks
This PR therefore proposes to remove the list of instances from
TestResult
.With the PR, on the test case from @esope QCheck2 (and QCheck) regains constant memory usage across increasing test counts - a nice property IMO (beware slight variation below due to different seeds):
In addition, 12.6MB also seems more reasonable compared to the 300MB, 583MB, 881MB the above use to take!
I suspect performance will improve a bit, due to the decreased GC stress.
For the 2 cases reported by @Robotechnic the situation improves nicely too:
To recap, in a list of 3 tests this uses ~64MB down from ~1151MB
whereas 3 separate runs uses ~65 MB down from ~438MB (again ignoring different seeds)
Because of the way we process tests:
we cannot realistically obtain constant memory usage as test lists grow.
However, as the above numbers show, this seems much more acceptable in the light of the overall memory reduction this PR offers.
Edit - Addition:
I believe this will make an even bigger difference for QCheck2 test than for QCheck ones, as the former's generator produces larger lazy shrink trees.