-
-
Notifications
You must be signed in to change notification settings - Fork 28
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
remove dependency matrix
from asv.conf.json
#113
Conversation
570a774
to
47b408c
Compare
req
key in asv.conf.json
matrix
matrix
from asv.conf.json
"packaging": [], | ||
"scipy": [], | ||
"matplotlib": [] | ||
}, |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
We need them to build the wheel in the benchmarks. I don't think we will be able to run the asv benchmark without the matrix.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@MridulS , are you happy with the PR now? Would this work with the PR benchmark you have set up at https://github.com/astropy/astropy/blob/main/.github/workflows/ci_benchmark.yml ? Hope you can advise. Thanks!
We can indeed think about adding pins to the dependencies to make the benchmarks even more precise but I'm not sure if we can just get rid of installing them. |
I removed astropy-benchmarks/asv.conf.json Lines 6 to 8 in 2c57d6e
I ran |
"build_command": [ | ||
"python -m build --wheel -o {build_cache_dir} {build_dir}" | ||
"install_command": [ | ||
"pip install . matplotlib scipy" |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
What if someone wants to run with different versions of these?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
they'd have to manually include those versions in the matrix.reqs
in their asv.conf.json
; in the case of the bench
repository, I added an input to the workflow dispatch that automatically inserts version specifications into the matrix, so in that case the user would specify something like "matplotlib==3.5.1" "matplotlib==3.5.0"
in the popup after clicking Run Workflow
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Should this be documented in the README?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
oh yes! Good catch, I'll add that
fda6cb2
to
debcc80
Compare
README.rst
Outdated
installed dependency or dependencies, add the ``matrix.req`` entry to | ||
``asv.conf.json``, including the versions you wish to run benchmarks against: | ||
|
||
.. code-block:: diff |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think there is a formatting error somewhere here. It is not rendering on "View File".
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
debcc80
to
6fd121b
Compare
6fd121b
to
a7ea752
Compare
README.rst
Outdated
installed dependency or dependencies, add the ``matrix.req`` entry to | ||
``asv.conf.json``, including the version(s) you wish to run benchmarks against: | ||
|
||
.. code-block:: diff |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I don't think diff
produces the correct rendering.
(suggestion retracted)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Oh wait, nvm, you are trying to show a diff ... But still it is all underlined in preview.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
README.rst
Outdated
}, | ||
... | ||
} | ||
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This works in Markdown but not sure about RST. 🤷♀️
--- a/asv.conf.json
+++ b/asv.conf.json
@@ -6,6 +6,17 @@
"install_command": [
"pip install . matplotlib scipy"
],
+ "matrix": {
+ "req": {
+ "matplotlib": [
+ "3.5.1"
+ ],
+ "numpy": [
+ "1.26.0",
+ "2.0.0rc1"
+ ]
+ }
+ },
"branches": ["main"],
"show_commit_url": "http://github.com/astropy/astropy/commit/",
"pythons": ["3.11"],
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think GitHub's rendering of RST is a bit wonky (maybe they want us to use markdown instead). We could change the filetype to markdown, possibly
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Would you mind doing that here? It is technically out of scope so you can say no. I just noticed that the Python formatting is also off. 🤷♀️
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
011808f
to
123d353
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Just minor comments from me. Thanks for the README updates. Formatting definitely is better in markdown. Would still like to hear back from Mridul first if possible.
"packaging": [], | ||
"scipy": [], | ||
"matplotlib": [] | ||
}, |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@MridulS , are you happy with the PR now? Would this work with the PR benchmark you have set up at https://github.com/astropy/astropy/blob/main/.github/workflows/ci_benchmark.yml ? Hope you can advise. Thanks!
Co-authored-by: P. L. Lim <[email protected]>
Co-authored-by: P. L. Lim <[email protected]>
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Given the doc clarification, I will approve but we should wait for Mridul to have another look?
Actually @zacharyburnett , would you mind opening an experimental PR at |
sure thing! Here: astropy/astropy#16296 |
Conclusion: astropy/astropy#16296 (comment) Should we merge? 😸 |
sounds good! Let's merge |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This looks good to me, now I don't remember why I wanted to build the wheel in the first place.
Thanks, all! |
why are these packages in the matrix in the first place? Would it be okay if we removed them?
I added a
set_asv_dependencies
script to thebench
repository so we can add dependencies on the fly when starting a new benchmark run; otherwise, as they are currently, this matrix just defaults to the latest versions of these dependencies.Additionally, the format of the configuration is deprecated; it should be under
matrix.req
, notmatrix