Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

pygmt.filter1d: Improve performance by storing output in virtual files #3085

Merged
merged 7 commits into from
Mar 13, 2024

Conversation

seisman
Copy link
Member

@seisman seisman commented Mar 4, 2024

@seisman seisman added documentation Improvements or additions to documentation needs review This PR has higher priority and needs review. labels Mar 4, 2024
@seisman seisman added this to the 0.12.0 milestone Mar 4, 2024
@seisman seisman marked this pull request as draft March 4, 2024 11:44
@seisman seisman removed the needs review This PR has higher priority and needs review. label Mar 4, 2024
@seisman seisman force-pushed the docstrings/output_type branch from a991908 to 8857ce6 Compare March 8, 2024 14:26
@seisman seisman changed the base branch from main to clib/return_table March 8, 2024 14:27
@seisman seisman changed the title Improve the docstrings of 'output_type' and 'outfile' pygmt.filter1d: Improve performance by getting rid of temporary files Mar 8, 2024
@seisman seisman added run/benchmark Trigger the benchmark workflow in PRs and removed documentation Improvements or additions to documentation labels Mar 8, 2024
Copy link

codspeed-hq bot commented Mar 8, 2024

CodSpeed Performance Report

Merging #3085 will not alter performance

Comparing docstrings/output_type (628c3f9) with main (2f598c5)

Summary

✅ 97 untouched benchmarks

🆕 1 new benchmarks
⁉️ 1 dropped benchmarks

⚠️ Please fix the performance issues or acknowledge them on CodSpeed.

Benchmarks breakdown

Benchmark main docstrings/output_type Change
🆕 test_filter1d N/A 173.6 ms N/A
⁉️ test_filter1d_format 199.7 ms N/A N/A

@seisman seisman force-pushed the docstrings/output_type branch from f39d2d4 to 92dc36a Compare March 11, 2024 14:51
@seisman
Copy link
Member Author

seisman commented Mar 12, 2024

Benchmarks:

>>> import numpy as np
>>> for n in [10, 100, 1000, 10000, 100000, 1000000]:
...     time = np.arange(0, n, 1)
...     data = np.random.random(n)
...     np.savetxt(fname=f"test-{n}.txt", X=np.array([time, data]).T, fmt="%.2f")
>>> import pygmt
>>> for n in [10, 100, 1000, 10000, 100000, 1000000]:
...     %timeit pygmt.filter1d(f"test-{n}.txt", filter_type="g5")

Main branch:

4.48 ms ± 16.4 µs per loop (mean ± std. dev. of 7 runs, 100 loops each)
4.63 ms ± 16.6 µs per loop (mean ± std. dev. of 7 runs, 100 loops each)
6 ms ± 9.02 µs per loop (mean ± std. dev. of 7 runs, 100 loops each)
23 ms ± 126 µs per loop (mean ± std. dev. of 7 runs, 10 loops each)
168 ms ± 798 µs per loop (mean ± std. dev. of 7 runs, 10 loops each)
1.68 s ± 5.18 ms per loop (mean ± std. dev. of 7 runs, 1 loop each)

This branch:

4.1 ms ± 14 µs per loop (mean ± std. dev. of 7 runs, 100 loops each)
4.15 ms ± 12.2 µs per loop (mean ± std. dev. of 7 runs, 100 loops each)
4.59 ms ± 35.1 µs per loop (mean ± std. dev. of 7 runs, 100 loops each)
8.91 ms ± 272 µs per loop (mean ± std. dev. of 7 runs, 100 loops each)
54.3 ms ± 118 µs per loop (mean ± std. dev. of 7 runs, 10 loops each)
504 ms ± 342 µs per loop (mean ± std. dev. of 7 runs, 1 loop each)

@seisman seisman marked this pull request as ready for review March 12, 2024 02:11
@seisman seisman changed the title pygmt.filter1d: Improve performance by getting rid of temporary files pygmt.filter1d: Improve performance using virtual files instead of temporary files for output Mar 12, 2024
@seisman seisman added maintenance Boring but important stuff for the core devs needs review This PR has higher priority and needs review. labels Mar 12, 2024
@michaelgrund michaelgrund added final review call This PR requires final review and approval from a second reviewer and removed needs review This PR has higher priority and needs review. labels Mar 12, 2024
@seisman seisman changed the title pygmt.filter1d: Improve performance using virtual files instead of temporary files for output pygmt.filter1d: Improve performance by storing output in virtual files Mar 13, 2024
@seisman seisman removed final review call This PR requires final review and approval from a second reviewer run/benchmark Trigger the benchmark workflow in PRs labels Mar 13, 2024
@seisman seisman merged commit bf7b9a1 into main Mar 13, 2024
18 of 20 checks passed
@seisman seisman deleted the docstrings/output_type branch March 13, 2024 07:20
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
maintenance Boring but important stuff for the core devs
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants