pystiche: A Framework for Neural Style Transfer

[pmeier]

Submitting Author: Philip Meier (@pmeier)
All current maintainers: Philip Meier (@pmeier)
Package Name: pystiche
One-Line Description of Package: Framework for Neural Style Transfer (NST) built upon PyTorch
Repository Link: https://github.com/pystiche/pystiche
Version submitted: 0.5.0post0
Editor: @NickleDave
Reviewer 1: @edgarriba
Reviewer 2: @soumith
Archive:
JOSS DOI:
Version accepted: v 0.6.0
Date accepted (month/day/year): 10/08/2020

---

Description

pystiche is a framework for Neural Style Transfer (NST) algorithms based on PyTorch. NST is a neural-net-based technique to merge the content of one and the artistic style of another image. Similar to deep learning frameworks pystiche eases up the workflow for researchers in this field. Rather than implementing everything yourself, pystiche provides common building blocks of NST algorithms that can be conveniently combined. Thus, researchers can focus on implementing new ideas rather than implementing the periphery over and over again.

Scope

• Please indicate which category or categories this package falls under:

  • Data retrieval
  • Data extraction
  • Data munging
  • Data deposition
  • Reproducibility
  • Geospatial
  • Education
  • Data visualization*

• Explain how the and why the package falls under these categories (briefly, 1-2 sentences):

  pystiche can be used to reproduce NST papers while focusing on core aspects.

• Who is the target audience and what are scientific applications of this package?

  The primary intended audience are researchers as described above. Apart from them pystiche could also be interesting for recreational use by non-scientists.

• Are there other Python packages that accomplish the same thing? If so, how does yours differ?

  AFAIK there are no other packages provide a similar functionality. However, due to its popularity, there are many implementations, which are limited to a specific NST algorithm. An exception might be this which features the implementation of multiple papers.

• If you made a pre-submission enquiry, please paste the link to the corresponding issue, forum post, or other discussion, or @tag the editor you contacted:

  Presubmission Inquiry: pystiche #21

Technical checks

For details about the pyOpenSci packaging requirements, see our packaging guide. Confirm each of the following by checking the box. This package:

• does not violate the Terms of Service of any service it interacts with.
• has an OSI approved license
• contains a README with instructions for installing the development version.
• includes documentation with examples for all functions.
• contains a vignette with examples of its essential functions and uses.
• has a test suite.
• has continuous integration, such as Travis CI, AppVeyor, CircleCI, and/or others.

Publication options

• Do you wish to automatically submit to the Journal of Open Source Software? If so:

JOSS Checks

• The package has an obvious research application according to JOSS's definition in their submission requirements. Be aware that completing the pyOpenSci review process does not guarantee acceptance to JOSS. Be sure to read their submission requirements (linked above) if you are interested in submitting to JOSS.
• The package is not a "minor utility" as defined by JOSS's submission requirements: "Minor ‘utility’ packages, including ‘thin’ API clients, are not acceptable." pyOpenSci welcomes these packages under "Data Retrieval", but JOSS has slightly different criteria.
• The package contains a paper.md matching JOSS's requirements with a high-level description in the package root or in inst/.
• The package is deposited in a long-term repository with the DOI:

Note: Do not submit your package separately to JOSS

Are you OK with Reviewers Submitting Issues and/or pull requests to your Repo Directly?

This option will allow reviewers to open smaller issues that can then be linked to PR's rather than submitting a more dense text based review. It will also allow you to demonstrate addressing the issue via PR links.

• Yes I am OK with reviewers submitting requested changes as issues to my repo. Reviewers will then link to the issues in their submitted review.

Code of conduct

• I agree to abide by pyOpenSci's Code of Conduct during the review process and in maintaining my package should it be accepted.

P.S. Have feedback/comments about our review process? Leave a comment here

Editor and Review Templates

Editor and review templates can be found here

[pmeier]

Some addtional remarks:

• I'm currently battling with Zenodo since the webhook did not work when I created the v0.5.0 release. To make matters worse the manual upload is also not working for. I'll work this out soon. Edit: This was resolved in ad DOI badge to README pystiche/pystiche#343

• As discussed in Presubmission Inquiry: pystiche #21 (comment) (@lwasser, @NickleDave) I've found two reviewers for the technical aspects of pystiche:

  • @edgarriba: Edgar Riba is the creator of kornia, a CV library for PyTorch.
  • @soumith: Soumith Chintala is a member of the PyTorch core team.

  My apologies for the late submission. Due to CoViD I had to help out in other more pressing projects first. I hope your willingness to review still stands.

[NickleDave]

Hi @pmeier thank you for submitting and finding reviewers!
And thank you for your patience -- I should have replied sooner.

@edgarriba and @soumith welcome!
It seems like Philip has definitely brought people with the right expertise to review the package.

[NickleDave]

@lwasser here are my editor checks:

Editor checks:

• Fit: The package meets criteria for fit and overlap.
• Automated tests: Package has a testing suite and is tested via Travis-CI or another CI service.
• License: The package has an OSI accepted license
• Repository: The repository link resolves correctly
• Archive (JOSS only, may be post-review): The repository DOI resolves correctly
• Version (JOSS only, may be post-review): Does the release version given match the GitHub release (v1.0.0)?

[NickleDave]

Editor comments

@lwasser I will follow up this weekend with editor comments

---

Reviewers: @edgarriba @soumith
Due date: August 5th

@edgarriba @soumith typically we ask for 2-3 week turnaround for reviews.
Although we should see if we can get editors to meet that standard first (I'm sorry @pmeier -- thank you again for your patience!). And I know everything is extra crazy for everyone right now.

Can you both let me know how doable three weeks sounds to you?

• Our guidelines for reviewers are here: https://www.pyopensci.org/contributing-guide/open-source-software-submissions/reviewer-guide.html
• There's a template for reviews here:
  https://www.pyopensci.org/contributing-guide/appendices/templates.html
• And here's an example of reviewers filling out the template are on these previous reviews:
  Pandera: A flexible and expressive pandas data validation library. #12 (comment)

[edgarriba]

@NickleDave no prob from my side. Let's just put a deadline :)

[NickleDave]

Excellent, thank you @edgarriba. Makes sense
Setting the due date for Aug 5 -- cc @soumith @pmeier @lwasser

Please let me know whatever I can do assist with the review process. Happy to provide additional info / guidance on top of what's here: #25 (comment)

[lwasser]

thank you so much @NickleDave et all!!

[pmeier]

@NickleDave

Note that we probably need a third reviewer since, to my knowledge, both, @edgarriba and @soumith, have no domain knowledge about Neural Style Transfer (NST). Shall I assist the search?

[NickleDave]

Thank you for making that clear; I agree that it would be good to have someone with domain expertise.

It would be great if you can assist with the search. I can also search starting this weekend.

[pmeier]

@edgarriba, @soumith

I just saw that AppVeyor Badge for v0.5.0 is not working properly:

This is due to the fact that I switched to GitHub Actions for the current master, and AppVeyor kept failing since pystiche was still activated. After I deactivated it, the badge went to Project unknown, which was not better either. I tried to re-run for v0.5.0, but AFAIK that is not possible. Re-running from master always fails, since the needed scripts no longer exist. Thus, the badge is now stuck at cancelled.

I made no changes to the functionality or the test-suite of pystiche since v0.5.0. You can find the test results for Windows (and the other OSs) here.

[NickleDave]

@pmeier @lwasser just want to follow up about reviewers and editor comments

Editor comments:

For me, the main fit with pyopensci for this package is in terms of reproducibility + education.
You aim to provide a package that encapsulates key concepts from this area of research, making them easier to learn about, and making it possible to reproduce key results without boilerplate.

To that end, the ideal reviewer would be able to speak to whether you are achieving those goals.

I also think we should find at least one of the reviewers (and when I say we, that means: me) to make sure the process is fair. I'm sure you agree. I put out an ask on Twitter (thank you Leah for the retweet) and I have at least one person in mind that I am contacting now. Will update as soon as I hear back

[pmeier]

@NickleDave

For me, the main fit with pyopensci for this package is in terms of reproducibility + education.
You aim to provide a package that encapsulates key concepts from this area of research, making them easier to learn about, and making it possible to reproduce key results without boilerplate.

1. Its not only re-producibility, but also "producibility". While I certainly aim to reproduce already published papers, my hope is that researchers use pystiche in upcoming publications. Reproducibility was just the most fitting out the available categories.
2. I'm not sure of the education part. It is certainly a goal for the future, albeit a secondary one. Unless you mean easing the on-boarding in the field, pystiche currently targets other researchers.

I also think we should find at least one of the reviewers (and when I say we, that means: me) to make sure the process is fair. I'm sure you agree.

Agreed. I haven't heard back from anyone yet anyway (not that I'm hyper connected since this will be my first major contribution to the field). If someone does get back to me, I'll fill them in about this and refer them to you.

[NickleDave]

Thank you @pmeier I hear you, and I think we are on the same page.
My comments are specifically about fit with PyOpenSci criteria but I don't mean to limit the scope of your package.

By education I do mean "easing the on-boarding in the field". To me this is a central goal for scientific packages: providing abstractions that make the research easier to understand. The package should enable "coding at the speed of thought". Which lines up with your point 1 above.

I am still reaching out to reviewers.
@edgarriba @soumith as soon as a third reviewer joins us I will set the review due date as three weeks from that day

[pmeier]

@edgarriba, @soumith, @NickleDave

I've just released pystiche==0.5.0.post0 since yesterdays release of torch==1.6.0 broke compatibility.

[NickleDave]

thank you @pmeier
I am still reaching out to reviewers, will update as soon as I know more

[pmeier]

@NickleDave @lwasser

Could we get an update on this? Its been almost a month. If I can help in any way, please let me know.

My grant is running until the end of the year and I need this published (in JOSS) before that. I'm aiming to have this published in late November or early December since I have some other duties at the end of the year. This leaves us with approximately 3 months from now including the JOSS process. Is this still reasonable together with pyOpenSci? If not, I need to take this to JOSS directly although I really want the pyOpenSci review.

[NickleDave]

Hi @pmeier I am sorry I have not gotten back to you, totally my fault.

Unfortunately I have not heard back from multiple invited reviewers.

I think part of it might also be that the package targets a relatively new and developing research area, and it's hard to find someone that has the right mix of domain expertise and OSS development experience.

I really appreciate that you want the scientific rigor added by our review.
Let's go ahead with just @soumith and @edgarriba as reviewers.
I'm sure their valuable input will help speed up review at JOSS as well.

@soumith and @edgarriba can you commit to finishing review two weeks from now, by Saturday September 12th, so we can help Philip get this to JOSS?
I'm sure you are also both busy, and I don't mean to make it your problem that I have not been more on top of this review.
But I think we all have some idea of what it's like to be a PhD student whose grant is running out.

It would be great if you can do a quick review that mainly focuses on the abstractions the library provides for this research.
JOSS reviewers will be able to address questions of software engineering, so you do not need to spend as much time on those (even though I know you could). But you can give big picture feedback on the goals of the library and how it achieves those, given your knowledge of deep learning and computer vision.

Please @soumith @edgarriba let me know if that deadline of Saturday September 12th works for you, and if not, when you expect you will be able to complete review.
That will help @pmeier decide how to move forward.

[pmeier]

@NickleDave I appreciate the change of course. I don't want to interfere with the process, just wanted to say that we don't need to rush it that much. I think it should be enough if I get the reviews until the end of September.

[NickleDave]

ok, thank you for that additional information

if we start today (ignoring that I have made put this review very behind) then the standard three weeks deadline would put us at September 19th.

I am going to say September 19th and make sure I have that in my calendar so I stay on top of this, unless @edgarriba and @soumith absolutely cannot make that deadline

[lwasser]

hey all!! just a note that once the review is complete here - the JOSS process is fast as they fully accept our review. So the key will be getting the reviewers on board with the Sept deadline and then addressing any feedback promptly. Thank you for your time. Just a note i've been struggling a bit with keeping up given COVID, beginning of the semester and other issues happening in the world right now. it would be great to find funding for this project so someone can work on it full time! i just haven't had time to devote to it.

[soumith]

i'll send in my review by that deadline. thanks for the heads-up!

[NickleDave]

🙏 🙏 🙏 thank you @soumith !!!

@edgarriba can you please confirm whether you can make that deadline of September 19th, when you get a chance?

[edgarriba]

Sure, will send too 💪

[NickleDave]

🙌🙌🙌 thank you both!!!

[edgarriba]

Package Review

Please check off boxes as applicable, and elaborate in comments below. Your review is not limited to these topics, as described in the reviewer guide

• As the reviewer I confirm that there are no conflicts of interest for me to review this work (If you are unsure whether you are in conflict, please speak to your editor before starting your review).

Documentation

The package includes all the following forms of documentation:

• A statement of need clearly stating problems the software is designed to solve and its target audience in README
• Installation instructions: for the development version of package and any non-standard dependencies in README
  • this is missing for development mode.
• Vignette(s) demonstrating major functionality that runs successfully locally
  • not in the readme. Working examples just in the docs.
• Function Documentation: for all user-facing functions
• Examples for all user-facing functions
  • missing in docs usage example for some functions.
• Community guidelines including contribution guidelines in the README or CONTRIBUTING.
  • not linked in the README.
• Metadata including author(s), author e-mail(s), a url, and any other relevant metadata e.g., in a setup.py file or elsewhere.

Readme requirements
The package meets the readme requirements below:

• Package has a README.md file in the root directory.

The README should include, from top to bottom:

• The package name
• Badges for continuous integration and test coverage, a repostatus.org badge, and any other badges. If the README has many more badges, you might want to consider using a table for badges: see this example. Such a table should be more wide than high. (Note that the badge for pyOpenSci peer-review will be provided upon acceptance.)
• Short description of goals of package, with descriptive links to all vignettes (rendered, i.e. readable, cf the documentation website section) unless the package is small and there’s only one vignette repeating the README.
• Installation instructions
• Any additional setup required (authentication tokens, etc)
  • not sure if it's applicable here.
• Brief demonstration usage
  • missing
• Direction to more detailed documentation (e.g. your documentation files or website).
• If applicable, how the package compares to other similar packages and/or how it relates to other packages
• Citation information
  • missing

Usability

Reviewers are encouraged to submit suggestions (or pull requests) that will improve the usability of the package as a whole.
Package structure should follow general community best-practices. In general please consider:

• The documentation is easy to find and understand
• The need for the package is clear
• All functions have documentation and associated examples for use
  • missing usage examples in the docs

Functionality

• Installation: Installation succeeds as documented.
• Functionality: Any functional claims of the software been confirmed.
• Performance: Any performance claims of the software been confirmed.
• Automated tests: Tests cover essential functions of the package and a reasonable range of inputs and conditions. All tests pass on the local machine.
  • ERROR tests/integration/enc/models/test_vgg.py
• Continuous Integration: Has continuous integration, such as Travis CI, AppVeyor, CircleCI, and/or others.
• Packaging guidelines: The package conforms to the pyOpenSci packaging guidelines.
  • provided link for packaging guidelines doesn't work.

For packages co-submitting to JOSS

• The package has an obvious research application according to JOSS's definition in their submission requirements.

Note: Be sure to check this carefully, as JOSS's submission requirements and scope differ from pyOpenSci's in terms of what types of packages are accepted.

The package contains a paper.md matching JOSS's requirements with:

• A short summary describing the high-level functionality of the software
• Authors: A list of authors with their affiliations
• A statement of need clearly stating problems the software is designed to solve and its target audience.
• References: with DOIs for all those that have one (e.g. papers, datasets, software).

Final approval (post-review)

• The author has responded to my review and made changes to my satisfaction. I recommend approving this package.

Estimated hours spent reviewing: 3.5

---

Review Comments

The package looks good overall with a good organization and using updated tools to keep high quality standards in terms of software engineering. The repository seems to have a simple and proper structure for the specific goals of the project. The documentation seems quite complete including few but clear working examples in addition to a good and complete package reference.

The only weakness I see in the package is the development support. I list below few comments and suggestions on that:

• Apart from pip install I miss short git clone instructions in the README. Add short development instructions in the README about how to git clone the project since probably users landing to github are more development oriented. Might be the case that not everyone is familiar with git.
• I miss a link to the CONTRIBUTING doc in the README.
  • inside the contributing guides I suggest to include also some initial explanation about setting up the development environment and make a fresh check for the current branch.
  • Missing some sort of script to create a virtual environment for development to avoid installing the dependencies in the main system. What if you don't even have pip, or tox ? :)
  • Needed to install all the dependencies and tools by hand to get a success in tests.
• Docs are okay, just missing very few documentation (like in the image module). I would expect to see a small usage example in the most relevant classes.
• I see few usage for type or shape checking and raising informative errors in some of the functionalities.

[NickleDave]

Thank you @edgarriba ! I will let @pmeier reply but just want to say I appreciate your making time to review this

[NickleDave]

🎉 🎉 🎉 It's official! ✨ ✨ ✨

@pmeier I merged in your PR above adding pystiche to the website--I can see the link is live now.
I also merged in the PR that added you and @edgarriba as contributors (@soumith feel free to reply here if you would still like to be added, I can do that too).

Thank you again everyone for your contributions and for your patience with this process; this was my first time as editor and I'm still getting the hang of it. @pmeier really glad we could help you improve pystiche and I actually learned a lot from seeing how you develop.

We will for sure want to share with the world that pystiche has successfully gone through review at PyOpenSci, on Twitter and elsewhere, like our blog. Let's stay in touch about that as it finishes up review at JOSS.

Closing this issue for now

[lwasser]

yahoooo!!!!!

[pmeier]

@NickleDave @lwasser FYI: JOSS review was successful (openjournals/joss-reviews#2761) and the paper is already published.

[choldgraf]

wow now that's what I call a fast turnaround!

[NickleDave]

Excellent, thank you for letting us know @pmeier
Looks great. Tweeting about the paper now :)

[lwasser]

heck. yea!! thank you @pmeier and @NickleDave !! this is awesome. Dave, i should give you superpowers to also retweet from pyopensci if you are open to it!! @choldgraf JOSS and @arfon are super fast once the review goes through POS. it's fantastic!

[lwasser]

hi there @NickleDave and @pmeier i'm just popping back here to confirm what version of pystiche was accepted? i'm going through logs and noticed at the very top of this review editors and reviewers and version weren't filled in! minor but important for record keeping! what version did we accept? Many thanks all!!

[pmeier]

pystiche==0.6.2

[lwasser]

thank you @pmeier !! 🙏

[pmeier]

@lwasser I need to correct myself:

pystiche==0.6.0 was accepted by pyOpenSci whereas pystiche==0.6.2 was accepted and published by JOSS. I've edited the top post accordingly.

[lwasser]

Ahhh – thank you for the clarification!! That is perfect.

[lwasser]

hey 👋 @pmeier @NickleDave @soumith @edgarriba ! I hope that you are all well. I am reaching out here to all reviewers and maintainers about pyOpenSci now that i am working full time on the project (read more here). We have a survey that we'd like for you to fill out so we can:

🔗 HERE IS THE SURVEY LINK 🔗

1. invite you to our slack channel to participate in our community (if you wish to join - no worries if that is not how you prefer to communicate / participate).
2. Collect information from you about how we can improve our review process and also better serve maintainers.
   The survey should take about 10 minutes to complete depending upon how much you decide to write. This information will help us greatly as we make decisions about how pyOpenSci grows and serves the community. Thank you so much in advance for filling it out.

NOTE: this is different from the form designed for reviewers to sign up to review.
If there are other maintainers for this project, please ping them here and ask them to fill out the survey as well. It is important that we ensure packages are supported long term or sunsetted with sufficient communication to users. Thus we will check in with maintainers annually about maintenance.

Thank you in advance for doing this and supporting pyOpenSci.

---

@NickleDave you are going to see me ping you on several issues. we can figure out a way for you and ivan to fill this out once and collect information as needed about what packages you've reviewed vs served as an editor for!

[lwasser]

hey there @soumith @edgarriba 👋 Just a friendly reminder to take 5-10 minutes to fill out our survey . We really appreciate it. Thank you in advance for helping us by filling out the survey!! 🙌

✨ Phillip thank you so much for taking the time to fill it out 🙌

🔗 HERE IS THE SURVEY LINK 🔗