Feature #15 yaml

[JohnHalleyGotway]

Pull Request Testing

• Describe testing already performed for these changes:
  
  Testing described in this pull request comment below.

• Recommend testing for the reviewer(s) to perform, including the location of input datasets, and any additional instructions:
  
  Review the proposed changes. Confirm that this MET GHA run actually completes now rather than TC-Diag erroring out. Although differences in the output are expected and can be ignored.

• Do these changes include sufficient documentation updates, ensuring that no errors or warnings exist in the build of the documentation? [Yes or No]
  I made no documentation changes, but I did create the v3.1 milestone to include both Enhance METbaseimage to compile the ecKit and Atlas libraries #13 and Enhance METbaseimage to install the YAML Python package #15.

• Do these changes include sufficient testing updates? [Yes]
  No testing updates are need for METbaseimage.

• Will this PR result in changes to the test suite? [No]
  
  If yes, describe the new output and/or changes to the existing output:
  

• Please complete this pull request review by [Wed 11/8/23].
  

Pull Request Checklist

See the METplus Workflow for details.

• Review the source issue metadata (required labels, projects, and milestone).
• Complete the PR definition above.
• Ensure the PR title matches the feature or bugfix branch name.
• Define the PR metadata, as permissions allow.
  Select: Reviewer(s)
  Select: Organization level software support Project or Repository level development cycle Project
  Select: Milestone as the version that will include these changes
• After submitting the PR, select the ⚙️ icon in the Development section of the right hand sidebar. Search for the issue that this PR will close and select it, if it is not already selected.
• After the PR is approved, merge your changes. If permissions do not allow this, request that the reviewer do the merge.
• Close the linked issue and delete your feature or bugfix branch from GitHub.

[JohnHalleyGotway]

Note that my initial MET GHA run failed because I'd failed to update the METbaseimage version number in 2 spots. With this updated GHA compile job, I see that it really is compiling with dtcenter/met-base:v3.1 as expected:

    MET_BASE_REPO: met-base-unit-test
    MET_BASE_TAG: v3.1

[JohnHalleyGotway]

Here's a little more background about this:

The code for MET PR's is compiled inside Docker image. Rather than creating the entire compilation environment separately for each and every run (which takes 30 min or so), we provide a base image in which to compile the MET code.

This dtcenter/METbaseimage repo houses the Dockerfiles which build the compilation/runtime environment for MET.

MET issue dtcenter/MET#2550 (and PR dtcenter/MET#2728) adds a new requirement on the Python pyyaml package. So METbaseimage issue #15 updates the METbaseimage environment to provide that new dependency.

METbaseimage#13 is not directly related but is needed for Howard's UGRID work. I want to group both #13 and #15 into the same tagged v3.1 release.

[JohnHalleyGotway]

• After explicitly specifying the pyyaml version number with pyyaml==6.0.1, the image builds locally:

docker build -t dtcenter/met-base:v3.1 .
docker build --build-arg MET_BASE_TAG=v3.1 -t dtcenter/met-base-unit-test:v3.1 -f Dockerfile.unit_test_env .

• Logged into the container and compiled my MET feature branch:

docker run -it --rm dtcenter/met-base-unit-test:v3.1 /bin/bash
git clone https://github.com/dtcenter/MET MET-feature_2550_compute_diagnostics
cd MET-feature_2550_compute_diagnostics
git checkout feature_2550_compute_diagnostics
source internal/scripts/environment/development.docker
export MET_GIT_NAME=feature_2550_compute_diagnostics
internal/scripts/docker/build_met_docker.sh

• Ran a simple Python embedding example and it now runs without a segfault:

plot_data_plane PYTHON_NUMPY plot.ps 'name="scripts/python/examples/read_ascii_numpy.py data/python/fcst.txt fcst";'
echo $?
0

• Also ran the newly updated unit_tc_diag.xml tests and it actually ran without error!

root@f897ba1238c3:/met/MET-feature_2550_compute_diagnostics/internal/test_unit# perl/unit.pl xml/unit_tc_diag.xml 
TEST: tc_diag_IAN  - pass - 101.593 sec

[JohnHalleyGotway]

Removing @DanielAdriaansen and adding @jprestop as reviewer. @jprestop it all seems to be working now. Please review this PR when you can. Switching from pyyaml>=6.0 to pyyaml==6.0.1 would appear to have made a difference.

[jprestop]

Thanks, @JohnHalleyGotway, for adding the comments about the required environment variables. That is really helpful information. I reviewed the GHA run but there are still some errors, two were seg faults and one was due to not finding 'scipy', so unfortunately, things aren't quite right yet...

[jprestop]

Please excuse my last comment. It looks like the runs at dtcenter/MET#2728 did pass just fine and had the expected differences. I approve this request.