[REVIEW]: OpenCMP: An Open-Source Computational Multiphysics Package

[whedon]

Submitting author: @nasserma (Nasser Mohieddin Abukhdeir)
Repository: https://github.com/uw-comphys/opencmp
Branch with paper.md (empty if default branch): publications
Version: v1.0.0
Editor: @lucydot
Reviewers: @bonh, @WilkAndy
Archive: 10.5281/zenodo.6515912

⚠️ JOSS reduced service mode ⚠️

Due to the challenges of the COVID-19 pandemic, JOSS is currently operating in a "reduced service mode". You can read more about what that means in our blog post.

Status

Status badge code:

HTML: <a href="https://joss.theoj.org/papers/efe19348de7efe519e1db433f5692960"><img src="https://joss.theoj.org/papers/efe19348de7efe519e1db433f5692960/status.svg"></a>
Markdown: [![status](https://joss.theoj.org/papers/efe19348de7efe519e1db433f5692960/status.svg)](https://joss.theoj.org/papers/efe19348de7efe519e1db433f5692960)

Reviewers and authors:

Please avoid lengthy details of difficulties in the review thread. Instead, please create a new issue in the target repository and link to those issues (especially acceptance-blockers) by leaving comments in the review thread below. (For completists: if the target issue tracker is also on GitHub, linking the review thread in the issue or vice versa will create corresponding breadcrumb trails in the link target.)

Reviewer instructions & questions

@bonh & @WilkAndy, please carry out your review in this issue by updating the checklist below. If you cannot edit the checklist please:

1. Make sure you're logged in to your GitHub account
2. Be sure to accept the invite at this URL: https://github.com/openjournals/joss-reviews/invitations

The reviewer guidelines are available here: https://joss.readthedocs.io/en/latest/reviewer_guidelines.html. Any questions/concerns please let @lucydot know.

✨ Please start on your review when you are able, and be sure to complete your review in the next six weeks, at the very latest ✨

Review checklist for @bonh

✨ Important: Please do not use the Convert to issue functionality when working through this checklist, instead, please open any new issues associated with your review in the software repository associated with the submission. ✨

Conflict of interest

• I confirm that I have read the JOSS conflict of interest (COI) policy and that: I have no COIs with reviewing this work or that any perceived COIs have been waived by JOSS for the purpose of this review.

Code of Conduct

• I confirm that I read and will adhere to the JOSS code of conduct.

General checks

• Repository: Is the source code for this software available at the repository url?
• License: Does the repository contain a plain-text LICENSE file with the contents of an OSI approved software license?
• Contribution and authorship: Has the submitting author (@nasserma) made major contributions to the software? Does the full list of paper authors seem appropriate and complete? #3803
• Substantial scholarly effort: Does this submission meet the scope eligibility described in the JOSS guidelines

Functionality

• Installation: Does installation proceed as outlined in the documentation?
• Functionality: Have the functional claims of the software been confirmed?
• Performance: If there are any performance claims of the software, have they been confirmed? (If there are no claims, please check off this item.)

Documentation

• A statement of need: Do the authors clearly state what problems the software is designed to solve and who the target audience is?
• Installation instructions: Is there a clearly-stated list of dependencies? Ideally these should be handled with an automated package management solution.
• Example usage: Do the authors include examples of how to use the software (ideally to solve real-world analysis problems).
• Functionality documentation: Is the core functionality of the software documented to a satisfactory level (e.g., API method documentation)?
• Automated tests: Are there automated tests or manual steps described so that the functionality of the software can be verified?
• Community guidelines: Are there clear guidelines for third parties wishing to 1) Contribute to the software 2) Report issues or problems with the software 3) Seek support

Software paper

• Summary: Has a clear description of the high-level functionality and purpose of the software for a diverse, non-specialist audience been provided?
• A statement of need: Does the paper have a section titled 'Statement of Need' that clearly states what problems the software is designed to solve and who the target audience is?
• State of the field: Do the authors describe how this software compares to other commonly-used packages?
• Quality of writing: Is the paper well written (i.e., it does not require editing for structure, language, or writing quality)?
• References: Is the list of references complete, and is everything cited appropriately that should be cited (e.g., papers, datasets, software)? Do references in the text use the proper citation syntax?

Review checklist for @WilkAndy

✨ Important: Please do not use the Convert to issue functionality when working through this checklist, instead, please open any new issues associated with your review in the software repository associated with the submission. ✨

Conflict of interest

• I confirm that I have read the JOSS conflict of interest (COI) policy and that: I have no COIs with reviewing this work or that any perceived COIs have been waived by JOSS for the purpose of this review.

Code of Conduct

• I confirm that I read and will adhere to the JOSS code of conduct.

General checks

• Repository: Is the source code for this software available at the repository url?
• License: Does the repository contain a plain-text LICENSE file with the contents of an OSI approved software license?
• Contribution and authorship: Has the submitting author (@nasserma) made major contributions to the software? Does the full list of paper authors seem appropriate and complete?
• Substantial scholarly effort: Does this submission meet the scope eligibility described in the JOSS guidelines

Functionality

• Installation: Does installation proceed as outlined in the documentation?
• Functionality: Have the functional claims of the software been confirmed?
• Performance: If there are any performance claims of the software, have they been confirmed? (If there are no claims, please check off this item.)

Documentation

• A statement of need: Do the authors clearly state what problems the software is designed to solve and who the target audience is?
• Installation instructions: Is there a clearly-stated list of dependencies? Ideally these should be handled with an automated package management solution.
• Example usage: Do the authors include examples of how to use the software (ideally to solve real-world analysis problems).
• Functionality documentation: Is the core functionality of the software documented to a satisfactory level (e.g., API method documentation)?
• Automated tests: Are there automated tests or manual steps described so that the functionality of the software can be verified?
• Community guidelines: Are there clear guidelines for third parties wishing to 1) Contribute to the software 2) Report issues or problems with the software 3) Seek support

Software paper

• Summary: Has a clear description of the high-level functionality and purpose of the software for a diverse, non-specialist audience been provided?
• A statement of need: Does the paper have a section titled 'Statement of Need' that clearly states what problems the software is designed to solve and who the target audience is?
• State of the field: Do the authors describe how this software compares to other commonly-used packages?
• Quality of writing: Is the paper well written (i.e., it does not require editing for structure, language, or writing quality)?
• References: Is the list of references complete, and is everything cited appropriately that should be cited (e.g., papers, datasets, software)? Do references in the text use the proper citation syntax?

[whedon]

Hello human, I'm @whedon, a robot that can help you with some common editorial tasks. @bonh, @WilkAndy it looks like you're currently assigned to review this paper 🎉.

⚠️ JOSS reduced service mode ⚠️

Due to the challenges of the COVID-19 pandemic, JOSS is currently operating in a "reduced service mode". You can read more about what that means in our blog post.

⭐ Important ⭐

If you haven't already, you should seriously consider unsubscribing from GitHub notifications for this (https://github.com/openjournals/joss-reviews) repository. As a reviewer, you're probably currently watching this repository which means for GitHub's default behaviour you will receive notifications (emails) for all reviews 😿

To fix this do the following two things:

1. Set yourself as 'Not watching' https://github.com/openjournals/joss-reviews:

2. You may also like to change your default settings for this watching repositories in your GitHub profile here: https://github.com/settings/notifications

For a list of things I can do to help you, just type:

@whedon commands

For example, to regenerate the paper pdf after making changes in the paper's md or bib files, type:

@whedon generate pdf

[whedon]

Software report (experimental):

github.com/AlDanial/cloc v 1.88  T=0.29 s (370.5 files/s, 58814.9 lines/s)
-------------------------------------------------------------------------------
Language                     files          blank        comment           code
-------------------------------------------------------------------------------
Python                          66           2737           5300           6037
reStructuredText                39            860            743           1407
Markdown                         2             24              0             19
make                             1              4              6              9
-------------------------------------------------------------------------------
SUM:                           108           3625           6049           7472
-------------------------------------------------------------------------------


Statistical information for the repository '15e27ad81d067e4382f138f9' was
gathered on 2021/09/21.
The following historical commit information, by author, was found:

Author                     Commits    Insertions      Deletions    % of changes
Alex Vasile                      1          1992           1031           12.42
elizabethmonte                   6         17212           4099           87.58

Below are the number of rows from each author that have survived and are still
intact in the current revision:

Author                     Rows      Stability          Age       % in comments
Alex Vasile                1992          100.0          0.0               24.60
elizabethmonte            12082           70.2          1.4               25.00

[whedon]

PDF failed to compile for issue #3742 with the following error:

 Can't find any papers to compile :-(

[lucydot]

@whedon generate pdf from branch publications

[whedon]

Attempting PDF compilation from custom branch publications. Reticulating splines etc...

[whedon]

👉📄 Download article proof 📄 View article proof on GitHub 📄 👈

[WilkAndy]

@nasserma , @elizabethmonte and @Alex-Vasile - i have a question about authorship. You can read the JOSS guidelines here: https://joss.readthedocs.io/en/latest/submitting.html#authorship . Given those guidelines, are you OK with James and Nasser being co-authors, even though they appear to have contributed zero code?

[WilkAndy]

@nasserma , since i am a heavy contributor to MOOSE i don't like your statement "Open-source packages ... such as MOOSE ... have very steep learning curves and are inaccessible to the general
scientific and engineering communities." MOOSE was designed to be easy to use. Perhaps it has changed into something more complicated over time. Anyway, your statement is just your opinion. Feel free to make no changes, but I guess i'd tone it down a little, by perhaps changing to "... can have steep learning curves." (without the "inaccessible" part).

[WilkAndy]

@nasserma , i have not yet read your paper in detail, nor downloaded the software, played with it, etc, but overall i think this paper should be significantly longer. You may have satisfied the minimum requirements for a JOSS paper, but i feel to excite readers to try your software, you need more details. I think you should mention:

• code coverage
• perhaps outline the tutorials
• a bit more details about the user interface
• a picture of an example, perhaps explaining why your code/approach is advantageous in terms of usability, accuracy, computational efficiency, etc, compared with other codes. Sometimes it is hard to not denigrate excellent alternative codes when you're doing this, but you needn't go over-the-top: you're just trying to show the reader your code is nice.

This is really summarizing work you've already done, so i hope it won't be a huge chore for you.

[WilkAndy]

@nasserma , i'm going on holiday for the next ~10 days, so won't do any more on this for a while, but hopefully you can address the above concerns while i'm away.

Oh, just one final point: your title "multiphysics package" gives the wrong idea to me. When i think of "multiphysics", i'm thinking of solid mechanics + fluid mechanics (of all kinds) + heat transfer + electromagnetics, etc. But your code, i think is just Navier Stokes. Can you comment here, or perhaps change the title?

[whedon]

👋 @WilkAndy, please update us on how your review is going (this is an automated reminder).

[whedon]

👋 @bonh, please update us on how your review is going (this is an automated reminder).

[nasserma]

@lucydot @WilkAndy This is our first time submitting to JOSS, I just want to confirm how we respond to reviewer comments. Should we be collating them and responding after they are complete or should we respond to each one individually? We have a response ready for @WilkAndy but it also involves revisions to the manuscript.

[bonh]

@nasserma I am interested in your response to the points raised by @WilkAndy too.

@lucydot I am assessing the scholarly effort right now and will require some time on that point alone.

[lucydot]

Hi @nasserma - The JOSS review process is a little different as it is more discussion based. It is very common for some issues/points to be raised, which are then responded to by the authors, then another issue/point is raised, with another response and so-on. Your response does not need to be as formal as for other more traditional journals. So please feel free to respond now to the issues raised already, and make any adjustments needed to the manuscript - this can be updated as needed as we progress with the review.

[lucydot]

Hello @bonh I can see you raised an issue in the JOSS reviews repo. Raising issues is a good way to progress the review (we want to avoid long discussions on this thread) but please create a new issue in the target repository (https://github.com/uw-comphys/opencmp) and link to those issues (especially acceptance-blockers) by leaving comments here. Thanks - Lucy.

[WilkAndy]

Just trying to install and use. Please add instructions for getting "NGSolve version 6.2.2101" to your "IO" docs

[nasserma]

Just trying to install and use. Please add instructions for getting "NGSolve version 6.2.2101" to your "IO" docs

I will update the website over the weekend, for now please refer to NGSolves download page,

https://ngsolve.org/downloads

for installation instructions. The newest version available 6.2.2105 should work, but the Windows, MacOS, and Ubuntu PPA all provide access to installers for older versions, including 6.2.2101.

[nasserma]

Hi @nasserma - The JOSS review process is a little different as it is more discussion based. It is very common for some issues/points to be raised, which are then responded to by the authors, then another issue/point is raised, with another response and so-on. Your response does not need to be as formal as for other more traditional journals. So please feel free to respond now to the issues raised already, and make any adjustments needed to the manuscript - this can be updated as needed as we progress with the review.

Thanks for the clarification, I'll post responses to all of the comments (except for those explicitly referring to other authors) today.

[nasserma]

@nasserma , since i am a heavy contributor to MOOSE i don't like your statement "Open-source packages ... such as MOOSE ... have very steep learning curves and are inaccessible to the general scientific and engineering communities." MOOSE was designed to be easy to use. Perhaps it has changed into something more complicated over time. Anyway, your statement is just your opinion. Feel free to make no changes, but I guess i'd tone it down a little, by perhaps changing to "... can have steep learning curves." (without the "inaccessible" part).

We have relaxed the language in the revised manuscript, as suggested. Our intention was never to diminish other software packages' utility, but rather differentiate OpenCMP's purpose. Our core mandate is to enable users who are familiar with the command-line interface to perform simulations without having to do any programming whatsoever, while simultaneously empowering developers to contribute new multiphysics models.

[nasserma]

@nasserma , i have not yet read your paper in detail, nor downloaded the software, played with it, etc, but overall i think this paper should be significantly longer. You may have satisfied the minimum requirements for a JOSS paper, but i feel to excite readers to try your software, you need more details. I think you should mention:

* code coverage

* perhaps outline the tutorials

* a bit more details about the user interface

* a picture of an example, perhaps explaining why your code/approach is advantageous in terms of usability, accuracy, computational efficiency, etc, compared with other codes.  Sometimes it is hard to not denigrate excellent alternative codes when you're doing this, but you needn't go over-the-top: you're just trying to show the reader your code is nice.

This is really summarizing work you've already done, so i hope it won't be a huge chore for you.

Based on the JOSS requirements for submissions, "the paper should be between 250-1000 words" and the submitted manuscript is at the upper limit. We could add more details into the paper, but it would be at the expense of other text, most of which has been added to meet the requirements of what the paper should include from the JOSS website,

https://joss.readthedocs.io/en/latest/submitting.html#submission-requirements

We have added references to the OpenCMP website in the revised manuscript. The website content addresses all of your comments except for code coverage, which we have added to the revised manuscript without exceeding the 1000 word limit.

[nasserma]

Oh, just one final point: your title "multiphysics package" gives the wrong idea to me. When i think of "multiphysics", i'm thinking of solid mechanics + fluid mechanics (of all kinds) + heat transfer + electromagnetics, etc. But your code, i think is just Navier Stokes. Can you comment here, or perhaps change the title?

The term "multiphysics" has a relatively broad definition of which yours is definitely included. From our perspective, coupling fluid flow, heat transfer, mass transfer, and chemical reaction is inclusive of the term "multiphysics". Please see Tutorial 9 from the website for an example of this. Additionally, we are in the process of development (dispersed) multiphase multicomponent flow models which will further support the use of the classification of the package as supporting "multiphysics".

We do not plan on contributing solid mechanics models but hope that future contributions will now that we have developed a website and a publicly accessible software repository.

[Alex-Vasile]

@nasserma , @elizabethmonte and @Alex-Vasile - i have a question about authorship. You can read the JOSS guidelines here: https://joss.readthedocs.io/en/latest/submitting.html#authorship . Given those guidelines, are you OK with James and Nasser being co-authors, even though they appear to have contributed zero code?

First, a note about code contributions to the project. The project began on a private internal repo, with commits being periodically being rolled up and copied into the GitHub repo. This results in actual code contributions being unintentionally obscured on the GitHub repo. Future commits will better reflect authorship as development branches are switched from internal repos to GitHub.

To address your question directly, yes I am okay with James and Nasser being co-authors given that they have been active participants. James has contributed code to the repository in the past and is actively implementing new models at the moment. Nasser, as well as James, has provided direction in terms of code structure, and help in understanding, implementing, debugging the numerical methods.

[elizabethmonte]

@nasserma , @elizabethmonte and @Alex-Vasile - i have a question about authorship. You can read the JOSS guidelines here: https://joss.readthedocs.io/en/latest/submitting.html#authorship . Given those guidelines, are you OK with James and Nasser being co-authors, even though they appear to have contributed zero code?

First, a note about code contributions to the project. The project began on a private internal repo, with commits being periodically being rolled up and copied into the GitHub repo. This results in actual code contributions being unintentionally obscured on the GitHub repo. Future commits will better reflect authorship as development branches are switched from internal repos to GitHub.

To address your question directly, yes I am okay with James and Nasser being co-authors given that they have been active participants. James has contributed code to the repository in the past and is actively implementing new models at the moment. Nasser, as well as James, has provided direction in terms of code structure, and help in understanding, implementing, debugging the numerical methods.

I agree with @Alex-Vasile, the paper’s current set of authors accurately reflects contributions to OpenCMP.

*edit to clarify meaning

[editorialbot]

👋 @openjournals/joss-eics, this paper is ready to be accepted and published.

Check final proof 👉 openjournals/joss-papers#3199

If the paper PDF and the deposit XML files look good in openjournals/joss-papers#3199, then you can now move forward with accepting the submission by compiling again with the command @editorialbot accept

[kyleniemeyer]

Hi @nasserma, I'm the AEIC on duty this week doing some final checks before publishing. I made a few small edits to your paper, can you merge those? uw-comphys/opencmp#25

[nasserma]

Hi @kyleniemeyer, thanks, I merged the corrections.

[kyleniemeyer]

@editorialbot generate pdf

[editorialbot]

👉📄 Download article proof 📄 View article proof on GitHub 📄 👈

[kyleniemeyer]

@editorialbot accept

[editorialbot]

Doing it live! Attempting automated processing of paper acceptance...

[editorialbot]

🐦🐦🐦 👉 Tweet for this paper 👈 🐦🐦🐦

[editorialbot]

🚨🚨🚨 THIS IS NOT A DRILL, YOU HAVE JUST ACCEPTED A PAPER INTO JOSS! 🚨🚨🚨

Here's what you must now do:

0. Check final PDF and Crossref metadata that was deposited 👉 Creating pull request for 10.21105.joss.03742 joss-papers#3202
1. Wait a couple of minutes, then verify that the paper DOI resolves https://doi.org/10.21105/joss.03742
2. If everything looks good, then close this review issue.
3. Party like you just published a paper! 🎉🌈🦄💃👻🤘

Any issues? Notify your editorial technical team...

[kyleniemeyer]

Congratulations @nasserma on your article's publication in JOSS!

Many thanks to @bonh and @WilkAndy for reviewing this, and @lucydot for editing.

[editorialbot]

🎉🎉🎉 Congratulations on your paper acceptance! 🎉🎉🎉

If you would like to include a link to your paper from your README use the following code snippets:

Markdown:
[![DOI](https://joss.theoj.org/papers/10.21105/joss.03742/status.svg)](https://doi.org/10.21105/joss.03742)

HTML:
<a style="border-width:0" href="https://doi.org/10.21105/joss.03742">
  <img src="https://joss.theoj.org/papers/10.21105/joss.03742/status.svg" alt="DOI badge" >
</a>

reStructuredText:
.. image:: https://joss.theoj.org/papers/10.21105/joss.03742/status.svg
   :target: https://doi.org/10.21105/joss.03742

This is how it will look in your documentation:

We need your help!

The Journal of Open Source Software is a community-run journal and relies upon volunteer effort. If you'd like to support us please consider doing either one (or both) of the the following:

• Volunteering to review for us sometime in the future. You can add your name to the reviewer list here: https://joss.theoj.org/reviewer-signup.html
• Making a small donation to support our running costs here: https://numfocus.org/donate-to-joss

[nasserma]

Congratulations @nasserma on your article's publication in JOSS!

Many thanks to @bonh and @WilkAndy for reviewing this, and @lucydot for editing.

Thanks to all!

[tarleb]

Hi all! We are working towards a more unified archive of all JOSS papers. In this process, I noted that the Markdown sources for this paper contain markup to include an image that does not exist, namely images/tutorial_8_geom.png, and is hence not included in the PDF. Judging from the surrounding text, the image may have been combined with another graphic, so this may not be a problem. Otherwise we may want to re-publish this paper with the correct image. @nasserma, could you take a look?

[nasserma]

Hi @tarleb , the situation is as you expected and line 106 of the current version of the Markdown source for the paper should be removed. I will push a revision of the file to the OpenCMP repository, but since this does not result in a change in the compiled PDF does anything else need to be done?

[tarleb]

Thanks for the feedback. I'm generating JATS from a copy of the paper sources, so I can fix locally. With the PDF looking as intended, there is nothing else we'd need to do here. The rest is on my side. 👍