Update recipe_thermodyn_diagtool.yml: code improvements and more user options

[ValerioLembo]

Description

This is a new version of recipe thermodyn_diagtool.yml

Main changes:

• Input fields are selected via metadata. This allows the user to provide only the necessary fields, depending on the options that he used;
• Land-sea mask (sftlf) is now included in the recipe as an independent variable (cfr. Update recipes and diagnostics to use simplified fx file definition in recipe #1272 );
• thermodyn_diagnostics.py and mkthe.py have been reformatted in order to reduce duplicates and decrease the number of lines;

---

• Closes/Fixes: Recipe: recipe_thermodyn_diagtool.yml (TheDiaTo) #2233
• Link to documentation:

---

Before you get started

• ☝ Create an issue to discuss what you are going to do

Checklist

It is the responsibility of the author to make sure the PR is ready to review. The icons indicate whether the item will be subject to the 🛠 Technical or 🧪 Scientific review.

• 🛠 PR has a descriptive title for the changelog
• 🛠 Code follows the style guide
• 🛠 Documentation is available
• 🛠 YAML files pass pre-commit or yamllint checks
• 🛠 Circle/CI tests pass
• 🛠 Codacy code quality checks pass
• 🛠 Documentation builds successfully on readthedocs

New or updated recipe/diagnostic:

• 🛠 Provenance information has been added
• 🛠 New dependencies are added to the project requirements
• 🧪 Documentation for the recipe/diagnostic clearly describes what it calculates from a scientific point of view
• 🧪 Recipe runs successfully on @esmvalbot or some other machine without modification
• 🧪 Figure(s)/data look as expected
• 🧪 Code is well documented and scientifically sound

---

To help with the number pull requests:

• 🙏 We kindly ask you to review two other open pull requests in this repository

[valeriupredoi]

@ValerioLembo awesome, Ill have a look at the code, but most importantly - does it work with the new fx vars implementation? Have you used #1272 as parent branch? 🍺

[ValerioLembo]

@valeriupredoi well... I included the sftlf_fx in a different way (l. 408 in thermodyn_diagtool.py) but I changed the recipe according to #1272 . And yes, it works!

[mattiarighi]

It would make sense to merge this in #1272 and test everything there.
This is recipe is anyway the last to-do in #1272.

[bouweandela]

It would make sense to merge this in #1272 and test everything there.
This is recipe is anyway the last to-do in #1272.

It might be better to keep it separate, as this pull request contains many changes so it may take longer to get it done.

[ValerioLembo]

@bouweandela I am not actually planning to perform other changes to this pull request, so as far as I am concerned, it can be merged with version2_development.

[bouweandela]

It looks like there are a lot of merge conflicts since #1272 got merged. Could you fix those please?

[ValerioLembo]

Dear @Peter9192 @nielsdrost @valeriupredoi @bouweandela @zklaus

It's been a while since a scientific request has been asked for this long-awaited merging. Is there any chance we can get this done any time soon?

[valeriupredoi]

we should (finally) get this in for the release in 2 weeks @zklaus - I think this recipe gets the prize for the longest wait, longer than Penelope 🤣

[ValerioLembo]

@valeriupredoi is there a prize for that? ;)

[valeriupredoi]

@valeriupredoi is there a prize for that? ;)

yes, a merge in main 😆

[Peter9192]

Thanks for reminding us @ValerioLembo , I wholeheartedly agree! I'm just not sure who we can ask to do the scientific review. Giving it another try @ESMValGroup/atmosphere

[zklaus]

@axel-lauer, could you perhaps find a scientific reviewer for this?

[bouweandela]

I have asked the co-authors of the GMD paper describing this diagnostic if they would be willing to do the scientific review.

[katjaweigel]

I have started to look into it and can run both the new and the old version with the new core, but I can't promise how fast I could do a review (probably not today).

[ValerioLembo]

Thanks @katjaweigel!

[franklunkeit]

I was asked to do a scientific review for this request, and I'm happy to do so.
I have to admit that I'm a bit biased as co-author of the respective paper, but I find the TheDiaTo package a very useful tool to evaluate climate models, and I'm very happy that it will be part of ESMValTool.

In my view, it is well documented (see minor below) and contains the appropriate references. However, I guess a user needs some experience to run it, and, in particular, to interpret the results, as it is common for such comprehensive diagnostics.
As far as I can see, all results (figures) produced by Niels in his test run (thank you for running it) are reasonable.

Thus, from my side I can support and approve the pull request.

As I never did such a review, I just tried out. I'm pretty sure that I did something wrong (or missed important points). E.g., I was ot able to check a checkbox for the review above, and did not find my name as a reviewer below. Please let me know, how to do it correctly.

@bouweandela: sorry, for presumably not being able to follow your instructions correctly.

Minor
In the paragraph for Lorenz energy cycle the last sentence:

'In order to account for possible gaps in pressure levels, the daily
fields of 2D near-surface temperature and horizontal velocities.'

seems somehow incomplete to me. You may like to have a look on this.

[ValerioLembo]

Dear @franklunkeit, thank you for the review. While @bouweandela and the others decide on whether it is sufficient, I will change the documentation according to your suggestion...

[zklaus]

Thanks, @franklunkeit for the review. I don't think that your role as a co-author on the original paper is an obstacle here at all. After all, the paper was peer-reviewed, so the goal of the review here is not to establish the authority of the method, but rather to guarantee that the method as implemented in ESMValTool reflects the method as presented in the paper. Who could be better qualified to sign for that?

Thanks for addressing the documentation request swiftly, @ValerioLembo.

[bouweandela]

Thanks a lot for taking the time to review @franklunkeit! I guess being able to check the boxes takes a bit more familiarity with the project, but from your comments I can see that the review addressed all the important points we expect from a scientific review.

@ValerioLembo If you would be able to improve the documentation as requested by @franklunkeit before Monday, we might still be able to include these updates in the imminent v2.3 release of the tool.

[zklaus]

@ValerioLembo, seeing as this seems to be only a one-sentence change, could you address it today? I am afraid otherwise we might need to bump it again to the next release 2.4.0.

[ValerioLembo]

@zklaus Gosh! If this is the case, I will certainly do it this afternoon! It's definitely time for having this recipe out! 🤣

[zklaus]

@ValerioLembo, great. Looks like this is basically ready. All that's left is a bit of housekeeping. Could you please find the existing issue for this PR or create a new one to link to and double-check in the checklist above that all items are either ticked or struck out/removed?

[ValerioLembo]

@zklaus Not sure if this was the right way to go for, but I did what you asked. I created a new PR, which is now available here: #2233 . I went through the checklist as well, and ticked everything.

[zklaus]

Thanks, @ValerioLembo, what you did was exactly right. Just to clarify the terminology: You didn't open a new PR (pull request), but rather a new issue. Issues contain descriptions and discussions of problems, PRs contain descriptions and discussions of solutions and are linked to the implementation of this solution by the corresponding branch.

Anyway, I edited the description at the top here to include the issue number and I think all is well now.

[valeriupredoi]

well done everyone, happy to see this finally merged 👴 😁

[ValerioLembo]

Awesome! And so it is gone now! 1 year, 8 months, 26 days since the PR was initially open: that's a major achievement in my short scientific career... It almost makes me dropping some tears!

[Peter9192]

Thanks everyone for all the effort, and especially congrats to Valerio for pulling through! Happy to see this is PR finally merged. Hopefully the process will be smoother in the future.