Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

MemorySampler to show memory breakdown #6241

Draft
wants to merge 1 commit into
base: main
Choose a base branch
from

Conversation

crusaderky
Copy link
Collaborator

@crusaderky crusaderky commented Apr 28, 2022

WIP

DONE: most coding
TODO: tests
TODO: documentation
TODO: align colours in the plot with those in the GUI
TODO: enhance automatic plotting of multiple samples

import dask.array as da
import distributed
from distributed.diagnostics import MemorySampler

c = distributed.Client(n_workers=16, threads_per_worker=2, memory_limit="4 GiB")
a = da.random.random((28_000, 28_000), chunks=(4_000, 4_000))
b = (a @ a.T).mean()

ms = MemorySampler()
with ms.sample("demo"):
    b.compute()
ms.plot(figsize=(15, 10), grid=True)

download

@github-actions
Copy link
Contributor

github-actions bot commented Apr 29, 2022

Unit Test Results

       15 files  ±0         15 suites  ±0   6h 51m 33s ⏱️ +54s
  2 775 tests ±0    2 694 ✔️ ±  0    78 💤  - 1    3 +  1 
20 587 runs  ±0  19 661 ✔️  - 18  905 💤  - 1  21 +19 

For more details on these failures, see this check.

Results for commit 228acaf. ± Comparison against base commit 50d2911.

♻️ This comment has been updated with latest results.

@github-actions
Copy link
Contributor

github-actions bot commented Jun 27, 2022

Unit Test Results

See test report for an extended history of previous test failures. This is useful for diagnosing flaky tests.

       23 files   -        1         23 suites   - 1   10h 16m 48s ⏱️ - 38m 39s
  3 335 tests ±       0    3 226 ✔️  -        6     103 💤 ±  0    6 +  6 
38 021 runs   - 1 297  36 159 ✔️  - 1 290  1 825 💤  - 44  37 +37 

For more details on these failures, see this check.

Results for commit b452c1a. ± Comparison against base commit 70abff0.

♻️ This comment has been updated with latest results.

@crusaderky
Copy link
Collaborator Author

Closes #6864

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant