Paper: multinterp: A Unified Interface for Multivariate Interpolation in the Scientific Python Ecosystem #937
Conversation
|
Curvenote Preview
|
|
@alanlujan91 can you recheck the DOIs that fail checks? As long as they are valid DOIs we should be okay. you may be able to add citation keys you want to ignore if their DOIs don't exist in myst.yml under error_rules: |
|
@alanlujan91 are you skipping valid DOIs? If so, please refrain and try the checks again. 😅 |
|
@ameyxd I believe the ones I'm skipping don't have DOIs. I can try to look again |
|
Okay! please check "Paszke2019" in particular. lmk when complete and passing checks. |
|
@ameyxd Oh I remember what happened. I found https://dl.acm.org/doi/10.5555/3454287.3455008 |
|
Review reminders sent to @gcdeshpande and @aparoha |
There was a problem hiding this comment.
Thanks for raising your PR. The overall concepts look promising. I have provided in-line comments as well. Overall comments:-
- The text could benefit from clearer organization and structure. For instance, some sections seem to repeat concepts without introducing new information or advancing the narrative.
- The introduction of technical terms and concepts could be more gradual and systematic, ensuring the audience can follow the content smoothly.
|
|
||
| As we might imagine, interpolation on regular grids is much easier than interpolation on irregular grids as we are able to exploit the structure of the grid to make predictions about the function's behavior between known values. Irregular grid interpolation is much more difficult, and often requires *regularizing* and/or *regression* techniques to make predictions about the function's behavior between known values. `multinterp` aims to provide a comprehensive set of tools for both regular and irregular grid interpolation, and we will discuss some of these tools in the following sections. | ||
|
|
||
| ```{list-table} Grids and structures implemented in "multinterp". |
There was a problem hiding this comment.
Question: Do you have plans to support other interpolations e.g. spline, inverse distance weight, natural neighbor etc.?
papers/alan_lujan/main.md
Outdated
|
|
||
| Functions are powerful mappings between sets of inputs and outputs, indicating how one set of values is related to another. Functions, however, are also infinitely dimensional, in that the inputs can range over an infinite number of values each mapping 1-to-1 (typically) to an infinite number of outputs. This makes it difficult to represent non-analytic functions in a computational environment, as we can only store a finite number of values in memory. For this reason, interpolation is a powerful tool in scientific computing, as it allows us to represent functions with a finite number of values and to approximate the function's behavior between these values. | ||
|
|
||
| The set of input values on which we know the function's output values is called a **grid**. A grid (or input grid) of values can be represented in many ways, depending on its underlying structure. The broadest categories of grids are regular or structured grids, and irregular or unstructured grids. Regular grids are those where the input values are arranged in a regular pattern, such as a triangle or a quadrangle. Irregular grids are those where the input values are not arranged in a particularly structured way and can seem to be scattered randomly across the input space. |
There was a problem hiding this comment.
The introduction of technical terms and concepts could be more gradual and systematic, ensuring the audience can follow the content smoothly. The 2nd paragraph directly jumps to grid interpolation without setting the context before.
papers/alan_lujan/main.md
Outdated
| - Hardware Adaptability: Seamless support for CPU (NumPy, SciPy), parallel (Numba), and GPU (CuPy, PyTorch, JAX) backends, empowering users to optimize performance based on their computational resources. | ||
| - Broad Functionality: Tools for regular/rectilinear interpolation, multivalued interpolation, and derivative calculations, addressing a wide range of scientific problems. | ||
|
|
||
| The multinterp package (<https://github.com/alanlujan91/multinterp>) is currently in its beta stage. It offers a strong foundation but welcomes community contributions to reach its full potential. We invite collaboration to improve documentation, expand the test suite, and ensure the codebase aligns with the highest standards of Python package development. |
There was a problem hiding this comment.
While the text mentions documentation and community contributions, specific links or guidelines for how users can contribute or access detailed documentation are not provided. Adding clear references or links to supplementary materials would enhance the text's utility.
|
Hi @alanlujan91 - we talked about this today in our editorial team meeting and would love to showcase your article with computational capabilities and reproducibility (as per this note in the readme!). I think following the review comment by @aparoha "some sections seem to repeat concepts without introducing new information or advancing the narrative.", it looks as though you are including the full notebooks, rather than individual cells. There is a way in MyST to include individual cells or outputs that keep links to the original location of the supporting materials. The documentation is here. That could be a way to change how you are showcasing the information and better highlight and link to your supporting materials. If you are open to it I would love to work on you to change the use of MyST slightly to better highlight the computational aspects. Let me know if you have time this week or next week? |
|
@rowanc1 that would be great! I am at a conference this week, so I haven't had time to address the previous comments, but will be available next week to discuss how we can improve this work |
|
Fantastic - I will reach out to you separately and get a meeting scheduled soon. Very very excited to show off your content in the SciPy Proceedings this year, and elevate the use of Jupyter Notebooks and reproducibility. 🚀 |
|
Re: alanlujan91#1 Dear Alan @alanlujan91, File edit: main.md Sincerely, |
|
@JennEYoon thanks for your reviews, please leave your commit as comments in this thread. You may refer to the work by other reviewers. Let us know if you have issues! |
There was a problem hiding this comment.
Review comments moved from a PR on author's multinterp branch
( https://github.com/alanlujan91/scipy_proceedings/pulls )
to scipy_proceedings/pull/937 comments section.
Jennifer
Sunday August 4, 2024 update
| ## Conclusion | ||
|
|
||
| Multivariate interpolation is a cornerstone of scientific computing, yet the Python ecosystem (@Oliphant2007) presents a fragmented landscape of tools. While individually powerful, these packages often lack a unified interface. This fragmentation makes it difficult for researchers to experiment with different interpolation methods, optimize performance across diverse hardware, and handle varying data structures (regular, rectilinear, curvilinear, unstructured). | ||
|
|
There was a problem hiding this comment.
Is there a way to give a nod to PyTorch and TensorFlow for having call-backs, hooks, and function wrappers to allow a user to swap out an optimization function or module mid-stream? They do not cover all the use cases of the multinterp package, but some effort went into developing a layered API to cover varying use cases.
NumPy also has structured data type, that can be used for custom data type and hierarchial data structures. It can be seen as an attempt to provide a flexible (customizable) user interface, even though its aim and scope is different from 'multinterp.' (NumPy structured datatype's goal seems mostly for C code interface and optimized C module or C numerical recipe interface and explicit memory control or memory layout control.) The general reader may appreciate having some context. This package may be viewed as a further development of previous efforts at a flexible user interface for users of varying data types and data geometries.
(See structured arrays in https://numpy.org/doc/stable/user/basics.rec.html)
| :label: unstructured | ||
| :alt: Unstructured grids are irregular and often require a triangulation step which might be computationally expensive and time-consuming. | ||
| :align: center | ||
|
|
There was a problem hiding this comment.
Memo, the alt-text sections in code blocks do not wrap properly when viewed on an iPad/Tablet device. Curvenotes build version works properly. So this issue only comes up when viewed direcly from a GitHub repo. Not sure if this is an issue for proceedings.
(Also alt-text wrap issue in Rectilinear Interpolation and Curvilinear Interpolation sections.)
|
|
||
| ```{include} notebooks/Multivariate_Interpolation.ipynb | ||
| ``` | ||
|
|
There was a problem hiding this comment.
Multilinear_Interpolation.ipynb comment
def squared_coords(x, y):
return x**2 + y**2
It will be helpful to provide a short reason for choosing the squared coordinates function. Example: A squared x and y coordinates function is used to draw a figure whose grid geometry looks like a curved sheet (bowl) in 3D projection.
This closed-form solution function, for which all points along the curved surface are known, is used as the baseline model. From this known model, we can draw sample points to approximate similarly shaped unknown functions.
|
|
||
| ```{include} notebooks/Multivariate_Interpolation.ipynb | ||
| ``` | ||
|
|
There was a problem hiding this comment.
fig 2: interpolated figures
missing title, axes labels (x, y z)
suggest ways to show this figure is interpolated.
Possibly blow up a small area with enhanced pixelation.
|
|
||
| ```{include} notebooks/Multivariate_Interpolation_with_Derivatives.ipynb | ||
| ``` | ||
|
|
There was a problem hiding this comment.
Multivariate_Interpolation_with_Derivatives.ipynb comment
fig1, fig2, fig3, fig4
Axes need labels on all 4 figures (x, y, z) or (dz/dx, dz/dy, f(z)).
Difficult to see the relationship between first group of 2 figures and second group of 2 figures (partial derivatives). Perhaps some lines can be drawn to show an example of the original function and its partial derivatives. I leave this decision up to the author.
|
|
||
| ```{include} notebooks/Unstructured_Interpolation.ipynb | ||
| ``` | ||
|
|
There was a problem hiding this comment.
Unstructured_Interpolation.ipynb comment
For figures after the 1st one, (group 1: nearest, linear, cubic, radial basis) and (group 2: original, gaussian process regression), eye-balling the differences in the figures maybe easier with grid lines drawn in white or black ink. Also just a suggestion.
|
|
||
| ```{include} notebooks/Unstructured_Interpolation.ipynb | ||
| ``` | ||
|
|
papers/alan_lujan/main.md
Outdated
| - Hardware Adaptability: Seamless support for CPU (NumPy, SciPy), parallel (Numba), and GPU (CuPy, PyTorch, JAX) backends, empowering users to optimize performance based on their computational resources. | ||
| - Broad Functionality: Tools for regular/rectilinear interpolation, multivalued interpolation, and derivative calculations, addressing a wide range of scientific problems. | ||
|
|
||
| The multinterp package (<https://github.com/alanlujan91/multinterp>) is currently in its beta stage. It offers a strong foundation but welcomes community contributions to reach its full potential. We invite collaboration to improve documentation, expand the test suite, and ensure the codebase aligns with the highest standards of Python package development. |
There was a problem hiding this comment.
Curvenoted build version comment
This paper when viewed on-screen, right-bottom screen area shows a list of related files. If possible, files should be listed in its order of appearance.
Supporting Documents
(shown in the order of appearance)
Multivariate_Interpolation.ipynb
Multivariate_Interpolation_with_Derivatives.ipynb
Multivalued_Interpolation.ipynb
Curvilinear_Interpoliation.ipynb
Unstructured_Interpolation.ipynb
manim_notebook.ipynb (figure animation on Curvenotes build)
figures.ipynb (add?)
|
@JennEYoon, @aparoha - can you confirm the author's updates are satisfactory for your feedback and give me a thumbs up! 🙂 |
|
Looks good. All of my comments were up to the author's discretion. :-) |
|
Hi folks, it doesn't look like the performance comparison made it to the published paper? |
|
Hey @stefanv - thanks for noticing this. As far as I can tell, labels referenced in the If the notebooks with these figures are added, we can update the proceedings site and PDF! cc @alanlujan91 |
|
thanks for catching that! I will make a PR against |
If you are creating this PR in order to submit a draft of your paper, please name your PR with
Paper: <title>. An editor will then add apaperlabel and GitHub Actions will be run to check and build your paper.See the project readme for more information.
Editor: Amey Ambade @ameyxd
Reviewers: