Replicate the creation of the gw-ci cycling test #1114

guillaumevernieres · 2024-05-17T12:30:02Z

It's useful to me so I suspect it could be useful to others. This PR adds 3 ctests that run the g-w create_experiment.py script for :

Aero/Snow JEDI DA
Atmos JEDI DA
WCDA (SOCA + GSI)

each ctest will create an exp directory with a COMROOT and EXPDIR, the user can then cycle the experiment with rocoto.

I stopped at the creation of the experiment but this could be pushed further if we wanted too ...

CoryMartin-NOAA

Looks good, thanks!

test/gw-ci/create_exp.sh

CoryMartin-NOAA

Instead of removing it, can we have the cmake define ICSDIR based on the machine it was built on?

guillaumevernieres · 2024-05-17T14:32:16Z

Instead of removing it, can we have the cmake define ICSDIR based on the machine it was built on?

this is done under the hood in the g-w

emcbot · 2024-05-17T18:33:26Z

Automated Global-Workflow GDASApp Testing Results:
Machine: orion

Start: Fri May 17 12:30:45 CDT 2024 on Orion-login-1.HPC.MsState.Edu
---------------------------------------------------
Build:                                 *SUCCESS*
Build: Completed at Fri May 17 13:15:31 CDT 2024
---------------------------------------------------
Tests:                                  *Failed*
Tests: Failed at Fri May 17 13:33:26 CDT 2024
Tests: 93% tests passed, 3 tests failed out of 45
	1781 - test_gdasapp_atm_jjob_ens_run (Failed)
	1782 - test_gdasapp_atm_jjob_ens_inc (Failed)
	1783 - test_gdasapp_atm_jjob_ens_final (Failed)
Tests: see output at /work2/noaa/stmp/cmartin/CI/GDASApp/workflow/PR/1114/global-workflow/sorc/gdas.cd/build/log.ctest

emcbot · 2024-05-17T19:42:58Z

Automated Global-Workflow GDASApp Testing Results:
Machine: hera

Start: Fri May 17 17:30:42 UTC 2024 on hfe04
---------------------------------------------------
Build:                                 *SUCCESS*
Build: Completed at Fri May 17 18:13:20 UTC 2024
---------------------------------------------------
Tests:                                  *Failed*
Tests: Failed at Fri May 17 19:42:57 UTC 2024
Tests: 80% tests passed, 9 tests failed out of 45
	1756 - test_gdasapp_soca_JGDAS_GLOBAL_OCEAN_ANALYSIS_BMAT (Failed)
	1757 - test_gdasapp_soca_JGDAS_GLOBAL_OCEAN_ANALYSIS_RUN (Failed)
	1760 - test_gdasapp_soca_JGDAS_GLOBAL_OCEAN_ANALYSIS_CHKPT (Failed)
	1761 - test_gdasapp_soca_socahybridweights (Failed)
	1762 - test_gdasapp_soca_incr_handler (Failed)
	1763 - test_gdasapp_soca_ens_handler (Failed)
	1780 - test_gdasapp_atm_jjob_ens_run (Failed)
	1781 - test_gdasapp_atm_jjob_ens_inc (Failed)
	1782 - test_gdasapp_atm_jjob_ens_final (Failed)
Tests: see output at /scratch1/NCEPDEV/da/Cory.R.Martin/CI/GDASApp/workflow/PR/1114/global-workflow/sorc/gdas.cd/build/log.ctest

guillaumevernieres · 2024-05-20T14:36:37Z

Instead of removing it, can we have the cmake define ICSDIR based on the machine it was built on?

this is done under the hood in the g-w

I thought I made sure I didn't have icsdir_root defined in the shell ... But apparently not, ICSDIR_ROOT needs to be defined.

emcbot · 2024-05-20T16:18:09Z

Automated Global-Workflow GDASApp Testing Results:
Machine: hera

Start: Mon May 20 15:12:53 UTC 2024 on hfe12
---------------------------------------------------
Build:                                 *SUCCESS*
Build: Completed at Mon May 20 15:56:31 UTC 2024
---------------------------------------------------
Tests:                                  *Failed*
Tests: Failed at Mon May 20 16:18:09 UTC 2024
Tests: 93% tests passed, 3 tests failed out of 45
	1780 - test_gdasapp_atm_jjob_ens_run (Failed)
	1781 - test_gdasapp_atm_jjob_ens_inc (Failed)
	1782 - test_gdasapp_atm_jjob_ens_final (Failed)
Tests: see output at /scratch1/NCEPDEV/da/Cory.R.Martin/CI/GDASApp/workflow/PR/1114/global-workflow/sorc/gdas.cd/build/log.ctest

AndrewEichmann-NOAA

looks reasonable

* origin/develop: Update wcoss2.intel.lua (#1142) Add yaml file for prepobsaero task (#1141) Diffusion parameter yaml template (#1140) Adjust absolute float tolerance for variational and ensemble DA jjob tests (#1136) Insitu temp and salt (#1135) Adds letkf.yaml(.j2) (#1134) Add optional testing block to 3dvar and lgetkf input YAMLs for jjobs (#1129) updates to build and run some ctests on WCOSS2 (#1122) Addin a GFSv17 ctest (#1130) Feature/rtofs in situ (#1110) update jcb-gdas hash to bring in satwnd yaml changes (#1128) Optionally run specified rocoto task as part of the ctests (#1121) Add build capability to Gaea-C5 (#1101) Python Converter and Json for the ADPUPA BUFR DUMP (#1115) Revert "Fix ice water FV3 increment variable name and add a few more … (#1125) Satwnd and variable name-convention updates for jcb implementation in GDASApp (#1119) Add inverse distance weighting option to superob function (#1116) Resume the Marine Vrfy Task on ctests (#1107) Replicate the creation of the gw-ci cycling test (#1114) Added YAML, JSON, python files for assimilating LEOGEO satwinds (#1099) Add an OOPS-based application to recenter snow analysis (#1102) Fix ice water FV3 increment variable name and add a few more (#1112) Update hera.intel.lua (#1109)

* develop: Update wcoss2.intel.lua (#1142) Add yaml file for prepobsaero task (#1141) Diffusion parameter yaml template (#1140) Adjust absolute float tolerance for variational and ensemble DA jjob tests (#1136) Insitu temp and salt (#1135) Adds letkf.yaml(.j2) (#1134) Add optional testing block to 3dvar and lgetkf input YAMLs for jjobs (#1129) updates to build and run some ctests on WCOSS2 (#1122) Addin a GFSv17 ctest (#1130) Feature/rtofs in situ (#1110) update jcb-gdas hash to bring in satwnd yaml changes (#1128) Optionally run specified rocoto task as part of the ctests (#1121) Add build capability to Gaea-C5 (#1101) Python Converter and Json for the ADPUPA BUFR DUMP (#1115) Revert "Fix ice water FV3 increment variable name and add a few more … (#1125) Satwnd and variable name-convention updates for jcb implementation in GDASApp (#1119) Add inverse distance weighting option to superob function (#1116) Resume the Marine Vrfy Task on ctests (#1107) Replicate the creation of the gw-ci cycling test (#1114) Added YAML, JSON, python files for assimilating LEOGEO satwinds (#1099) Add an OOPS-based application to recenter snow analysis (#1102) Fix ice water FV3 increment variable name and add a few more (#1112) Update hera.intel.lua (#1109)

into feature/rename_atm * 'feature/rename_atm' of https://github.com/NOAA-EMC/GDASApp: Update wcoss2.intel.lua (#1142) Add yaml file for prepobsaero task (#1141) Diffusion parameter yaml template (#1140) Adjust absolute float tolerance for variational and ensemble DA jjob tests (#1136) Insitu temp and salt (#1135) Adds letkf.yaml(.j2) (#1134) Add optional testing block to 3dvar and lgetkf input YAMLs for jjobs (#1129) updates to build and run some ctests on WCOSS2 (#1122) Addin a GFSv17 ctest (#1130) Feature/rtofs in situ (#1110) update jcb-gdas hash to bring in satwnd yaml changes (#1128) Optionally run specified rocoto task as part of the ctests (#1121) Add build capability to Gaea-C5 (#1101) Python Converter and Json for the ADPUPA BUFR DUMP (#1115) Revert "Fix ice water FV3 increment variable name and add a few more … (#1125) Satwnd and variable name-convention updates for jcb implementation in GDASApp (#1119) Add inverse distance weighting option to superob function (#1116) Resume the Marine Vrfy Task on ctests (#1107) Replicate the creation of the gw-ci cycling test (#1114) Added YAML, JSON, python files for assimilating LEOGEO satwinds (#1099) Add an OOPS-based application to recenter snow analysis (#1102) Fix ice water FV3 increment variable name and add a few more (#1112)

guillaumevernieres and others added 4 commits May 15, 2024 18:36

wip

8cf8e14

wip

e0dfc43

added 3 gw-ci test

d99c7f9

overwrite exp

6dc6139

guillaumevernieres requested review from CoryMartin-NOAA, jiaruidong2017, danholdaway and RussTreadon-NOAA May 17, 2024 12:30

Merge branch 'develop' into feature/gdas-cycle-ci

1660380

CoryMartin-NOAA previously approved these changes May 17, 2024

View reviewed changes

CoryMartin-NOAA reviewed May 17, 2024

View reviewed changes

test/gw-ci/create_exp.sh Outdated Show resolved Hide resolved

CoryMartin-NOAA reviewed May 17, 2024

View reviewed changes

test/gw-ci/create_exp.sh Show resolved Hide resolved

guillaumevernieres and others added 2 commits May 17, 2024 13:12

don't spec icsdir

7b551a5

don't spec. icsdir

e445f27

guillaumevernieres dismissed CoryMartin-NOAA’s stale review via e445f27 May 17, 2024 13:14

guillaumevernieres requested a review from CoryMartin-NOAA May 17, 2024 13:14

CoryMartin-NOAA reviewed May 17, 2024

View reviewed changes

CoryMartin-NOAA previously approved these changes May 17, 2024

View reviewed changes

danholdaway previously approved these changes May 17, 2024

View reviewed changes

guillaumevernieres marked this pull request as ready for review May 17, 2024 15:03

guillaumevernieres and others added 2 commits May 17, 2024 15:35

post is broken

a4063d3

I broke post ... waiting for gw2584

0ddcb80

guillaumevernieres dismissed stale reviews from danholdaway and CoryMartin-NOAA via 0ddcb80 May 17, 2024 15:42

guillaumevernieres added hera-GW-RT Queue for automated testing with global-workflow on Hera orion-GW-RT Queue for automated testing with global-workflow on Orion labels May 17, 2024

emcbot added hera-GW-RT-Running Automated testing with global-workflow running on Hera orion-GW-RT-Running Automated testing with global-workflow running on Orion and removed hera-GW-RT Queue for automated testing with global-workflow on Hera labels May 17, 2024

emcbot added orion-GW-RT-Failed Automated testing with global-workflow failed on Orion and removed orion-GW-RT-Running Automated testing with global-workflow running on Orion labels May 17, 2024

emcbot added hera-GW-RT-Failed Automated testing with global-workflow failed on Hera and removed hera-GW-RT-Running Automated testing with global-workflow running on Hera labels May 17, 2024

Merge branch 'develop' into feature/gdas-cycle-ci

1d8585d

guillaumevernieres mentioned this pull request May 20, 2024

soca gridgen is failing #1118

Closed

guillaumevernieres added 2 commits May 20, 2024 14:32

I lied ... ICSDIR_ROOT needs to be defined

ef25435

merged upstream

334e9e6

guillaumevernieres added hera-GW-RT Queue for automated testing with global-workflow on Hera and removed hera-GW-RT-Failed Automated testing with global-workflow failed on Hera labels May 20, 2024

guillaumevernieres requested review from AndrewEichmann-NOAA and apchoiCMD May 20, 2024 14:50

emcbot added hera-GW-RT-Running Automated testing with global-workflow running on Hera and removed hera-GW-RT Queue for automated testing with global-workflow on Hera labels May 20, 2024

emcbot added hera-GW-RT-Failed Automated testing with global-workflow failed on Hera and removed hera-GW-RT-Running Automated testing with global-workflow running on Hera labels May 20, 2024

AndrewEichmann-NOAA approved these changes May 20, 2024

View reviewed changes

guillaumevernieres merged commit 39cf2b2 into develop May 20, 2024
11 checks passed

guillaumevernieres deleted the feature/gdas-cycle-ci branch May 20, 2024 17:01

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Replicate the creation of the gw-ci cycling test #1114

Replicate the creation of the gw-ci cycling test #1114

guillaumevernieres commented May 17, 2024

CoryMartin-NOAA left a comment

CoryMartin-NOAA left a comment

guillaumevernieres commented May 17, 2024 •

edited

Loading

emcbot commented May 17, 2024

emcbot commented May 17, 2024

guillaumevernieres commented May 20, 2024

emcbot commented May 20, 2024

AndrewEichmann-NOAA left a comment

Replicate the creation of the gw-ci cycling test #1114

Replicate the creation of the gw-ci cycling test #1114

Conversation

guillaumevernieres commented May 17, 2024

CoryMartin-NOAA left a comment

Choose a reason for hiding this comment

CoryMartin-NOAA left a comment

Choose a reason for hiding this comment

guillaumevernieres commented May 17, 2024 • edited Loading

emcbot commented May 17, 2024

emcbot commented May 17, 2024

guillaumevernieres commented May 20, 2024

emcbot commented May 20, 2024

AndrewEichmann-NOAA left a comment

Choose a reason for hiding this comment

guillaumevernieres commented May 17, 2024 •

edited

Loading