Skip to content
This repository has been archived by the owner on Jan 3, 2024. It is now read-only.

Use random arrays for memory testing #82

Merged
merged 1 commit into from
Feb 28, 2023

Conversation

dstansby
Copy link
Member

It turns out if you increase the size of a dask array using da.repeat(...), dask is clever and doesn't make actual copies of the data. This is great, unless you're running memory benchmarks and want the memory useage to increase 🙈

This PR changes the sample data to be random data generated by dask, instead of repeating existing data. Although the cell detection results won't be meaningful on this data, it provides a easy way to run memory benchmarks.

To confirm this works, here's the memory usage of just detection, which shows a clear increase when the array size is increased:

Repeats = 1

1-rep

Repeats = 2

2-rep

@dstansby dstansby marked this pull request as ready for review February 28, 2023 10:34
@dstansby dstansby requested a review from a team February 28, 2023 10:34
@codecov
Copy link

codecov bot commented Feb 28, 2023

Codecov Report

Base: 99.13% // Head: 99.13% // No change to project coverage 👍

Coverage data is based on head (2148a24) compared to base (6ee7685).
Patch has no changes to coverable lines.

Additional details and impacted files
@@           Coverage Diff           @@
##             main      #82   +/-   ##
=======================================
  Coverage   99.13%   99.13%           
=======================================
  Files           9        9           
  Lines         231      231           
=======================================
  Hits          229      229           
  Misses          2        2           

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

☔ View full report at Codecov.
📢 Do you have feedback about the report comment? Let us know in this issue.

@adamltyson adamltyson merged commit b31ff82 into brainglobe:main Feb 28, 2023
@dstansby dstansby deleted the meaningful-benchmark branch February 28, 2023 16:01
willGraham01 pushed a commit that referenced this pull request Aug 24, 2023
* Add basic widget test

* Update comment on choosing a backend for tests

Co-authored-by: Alessandro Felder <[email protected]>

Co-authored-by: Alessandro Felder <[email protected]>
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants