NorESM user workshop 2021 - retrospective

[TomasTorsvik]

@YanchunHe @monsieuralok @matsbn @adagj @DirkOlivie @TomasTorsvik

Hi,
I thought it would be useful to have a discussion about the NorESM user workshop we had last week - what worked well and what could be improved. Maybe this github discussion page is not the best platform, but it has the advantage that it is linked to the NorESM repository, and I hope that it can be easy to find next time we have a similar event. If you think it's a bad idea to have this discussion on an open platform, I can remove this entry and we can move the discussion somewhere else.

My general impression is that the event went fairly well. We managed to get through the whole program, although we got a bit delayed on the first day and had to move the last hands-on session to Tuesday, and we didn't manage to go through all the prepared exercises.

1. We had technical issues regarding the queue reservation on Betzy, that we needed to solve on the first day of the workshop. Eventually this was solved, but it caused a bit of delays and confusion.

• It would be useful to test a queue reservation some time before the event. Perhaps Sigma2 does not want to keep a reservation open for much longer than the workshop period, but I think it should be possible to have a meeting with Sigma2 engineers some time in advance of the event to test a reservation and get the necessary instructions on how to use it.

2. The workshop was organized as a "hybrid" event, with most attendees in the room but some following by zoom. I think we managed to follow up questions from the room reasonably well, but some questions in the zoom chat was left unanswered for a long time.

• If we organize more hybrid events, perhaps we should have one assistant who mainly takes care of the on-line participants? This will probably depend on the number of participants, but it could be good to assign this role to someone in advance of the workshop.

3. There was quite a bit of overlap between information presented by Dirk, Ada and myself. This was partly intentional, we wanted the participants to quickly set up their first experiment, and then we went back and filled in more details on each of the steps in the setup and run process. I think this worked reasonably well, but we should try to get some feedback on this from the participants.

4. We had hands-on sessions for building and running the model, running NorESM from a container and on post-processing with the diagnostics toolbox. Most participants managed to get through most of the exercises, but I don't think anyone went through everything during the workshop (unless they worked on it afterwards). Still, I think the level of the hands-on exercises was adequate, taking into account that the participants had very different level of pre-knowledge.

• The green and red/pink post-it notes were helpful to notify about progress or problems during the hands-on sessions.
• I think the online participants did not receive as much attention as we would have liked.
• Maybe we should have left some time to discuss common problems after each hands-on session? We tried after the first one, but there was not much response. Still, it could be useful to have this feedback.

5. We plan to make a survey for the participants after the workshop to find out what they got out of it. Hopefully we can make such a survey this week.

• I think it would be better to have a survey ready during the workshop, and leave some time at the end to have the participants fill out a form on this (or send out the form during the workshop, so that it can be completed right after the event).

These are the points that I have thought about. Please add your own points, or include your comments on the ones that I have already listed.

Tomas

[YanchunHe]

Thanks @TomasTorsvik for initiating this discussion!

Just two quick comments:

The workshop was organized as a "hybrid" event,..., but some questions in the zoom chat was left unanswered for a long time.

As we announced to the participants by emails and also the workshop webpage, that "We will also open Zoom streaming for the presentations, but we will not support the online participation for the hands-on sessions.", so we only promise the zoom streaming of the presentations.

We did response to the online questions, but usually delayed. I think this is understandable by the participants.

We plan to make a survey for the participants after the workshop to find out what they got out of it. Hopefully we can make such a survey this week.

This is a good idea, and we should try to do this the next time.

However, the time is quite limited, as usual. For running NorESM and especially for the model output and diagnostic session. I have to go very quickly through part of the materials I prepared.

We should either have less to teach for running the NorESM and post-process the data, or we have longer time for the workshop.

[adagj]

Thanks @TomasTorsvik !
A very well organized start :)

I think the participants will benefit from (way) more hands-on sessions and less presentations. Maybe we can find a way to include more of the information presented into the exercises so we don't need to present that information in the presentations-

And as I mentioned in the email, I wonder if is it more useful for the participants to come to such a workshop with a more predefined "problem" which can be sent to us before the workshop starts. Then we can be more targeted in our teaching and help them to solve their specific needs, i.e. more of the "meeting-with-experts" kind of workshop and not so broad. I mean, if say 3/4 of the participants want to run a CMIP6 historical simulation with some parameter changes, we probably don't have to spend a lot of time on env_mach_pes settings and github stuff :-)

And I agree with @YanchunHe ; when we try to cover so many aspects in such a short time, it is probably challenging for them to learn anything at all. So either less of something or more narrowed and targeted.

[TomasTorsvik]

Hi @adagj , @YanchunHe ,
thanks for your feedback. I updated the workshop survey form with the two first suggestions from Ada, but I don't know how to formulate a sensible question regarding the predefined problem suggestion. I think this will change the format of the workshop quite a lot, perhaps make it more like a "hackathon"?

[adagj]

I didn't really mean it as a question, more a thought .
Regarding the format, I guess that depends on us. To have some information about the participants technical level, background and research interests before the workshop can be useful if we want to shorten the time spent on presentations, even if we don't make it a hackathon

[TomasTorsvik]

For future reference, here is the link to the report from the user survey after the workshop:

https://skjemaker.app.uib.no/report.php?key=11772420x19c3f0c681

Additional questions not in the report:

Q: Any overall feedback about logistics?
A1: Very good
A2: Very fancy

Q: Any overall feedback for the event?
A1: Very nice. Some times the hands on sessions went a bit too fast, especially when there was combined presentation/hands-on. If you had fallen off it wasn't always possible to get help to get back in.

Q: Other comments to the workshop structure?
A1: Presentations were adequate but needed more exercise/hands-on sessions.

Q: Any additional comments regarding the sessions or overall agenda?
A1: Overall, the workshop agenda was/is very helpful. The session on NorESM in containers would require more in-depth information on how to e.g. package containers to enable making use of the knowledge. Mentioning the option of using containers is certainly worth the information, but it was a bit unclear, what the session was aiming at (i.e. will it be the future default working environment for NorESM?).

Q: Was there any topic that you think was missing or inadequately covered in the workshop?
A1: From a developers perspective, it would be nice to have a slightly more in-depth overview of the configuration and cmake processes involved (i.e. where are the information stored, how to e.g. add and link new routines, etc.). Further, it could be helpful to get some simple post-processing scripts with subsequent plotting for the individual components (as a starting reference).

Q: What were your key take aways from this event?
A1: Hands-on experience with running NorESM and tips about where to find more information and documentation.
A2: Useful hands-ons and some interesting presentations.
A3: In summary, the NorESM user workshop gave a good overview to start using NorESM and to enable further in-depth investigation and understanding, how to setup experiments.