Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Updates for new 20231027 orog fix file set #867

Closed
KateFriedman-NOAA opened this issue Nov 1, 2023 · 11 comments · Fixed by #881
Closed

Updates for new 20231027 orog fix file set #867

KateFriedman-NOAA opened this issue Nov 1, 2023 · 11 comments · Fixed by #881
Assignees
Labels
enhancement New feature or request maintenance Basic upkeep

Comments

@KateFriedman-NOAA
Copy link
Collaborator

A new orog fix file timestamp is being created for the GFS fix file developmental set. Some updates are needed in UFS_UTILS to accommodate changes in this new timestamp set.

  1. Update UFS_UTILS to use orog_ver=20231027 for the new GlobalLake* fix files that previously lived in the unversioned raw/orog subset. Any other related updates for this. Discussed in Update global lake data NOAA-EMC/global-workflow#1984.
    Files in new orog set:
[role.glopara@hfe02 20231027]$ pwd
/scratch1/NCEPDEV/global/glopara/fix/orog/20231027
[role.glopara@hfe02 20231027]$ ll | grep GlobalLake
-rw-r--r-- 1 role.glopara global 1866240000 May 31  2018 GlobalLakeDepth_GLDBv2release.dat
-rw-r--r-- 1 role.glopara global       2191 Oct 25 17:59 GlobalLakeDepth_GLDBv2release.txt
-rw-r--r-- 1 role.glopara global       2695 Oct 25 19:02 GlobalLakeDepth_GLDBv3release.txt
-rw-r--r-- 1 role.glopara global  933120000 May 31  2018 GlobalLakeStatus_GLDBv2release.dat
-rw-r--r-- 1 role.glopara global       2845 Oct 25 18:42 GlobalLakeStatus_GLDBv2release.txt
-rw-r--r-- 1 role.glopara global       4185 Oct 25 17:54 GlobalLakeStatus_GLDBv3release.txt
  1. Update the oro_data filenames used by gdas_init utility and anything else in UFS_UTILS (no longer using symlink in fix set). Change from oro_C###.mx###.tile#.nc to C###.mx###_oro_data.tile#.nc. Discussed in Update fix files for HR3/EP5 NOAA-EMC/global-workflow#1981. Example in C384:
[role.glopara@hfe02 20231027]$ ll C384.mx025/ | grep oro
-rw-r--r-- 1 role.glopara global  13000006 May  4  2022 C384.mx025_oro_data.tile1.nc
-rw-r--r-- 1 role.glopara global  13000006 May  4  2022 C384.mx025_oro_data.tile2.nc
-rw-r--r-- 1 role.glopara global  13000006 May  4  2022 C384.mx025_oro_data.tile3.nc
-rw-r--r-- 1 role.glopara global  13000006 May  4  2022 C384.mx025_oro_data.tile4.nc
-rw-r--r-- 1 role.glopara global  13000006 May  4  2022 C384.mx025_oro_data.tile5.nc
-rw-r--r-- 1 role.glopara global  13000006 May  4  2022 C384.mx025_oro_data.tile6.nc
lrwxrwxrwx 1 role.glopara global        28 Jan  5  2023 oro_C384.mx025.tile1.nc -> C384.mx025_oro_data.tile1.nc
lrwxrwxrwx 1 role.glopara global        28 Jan  5  2023 oro_C384.mx025.tile2.nc -> C384.mx025_oro_data.tile2.nc
lrwxrwxrwx 1 role.glopara global        28 Jan  5  2023 oro_C384.mx025.tile3.nc -> C384.mx025_oro_data.tile3.nc
lrwxrwxrwx 1 role.glopara global        28 Jan  5  2023 oro_C384.mx025.tile4.nc -> C384.mx025_oro_data.tile4.nc
lrwxrwxrwx 1 role.glopara global        28 Jan  5  2023 oro_C384.mx025.tile5.nc -> C384.mx025_oro_data.tile5.nc
lrwxrwxrwx 1 role.glopara global        28 Jan  5  2023 oro_C384.mx025.tile6.nc -> C384.mx025_oro_data.tile6.nc

Full list of affected files:

[role.glopara@hfe02 20230615]$ ll C*/oro_* | grep -v frac                                                                                                          
lrwxrwxrwx 1 role.glopara global 29 Jan  5  2023 C1152.mx025/oro_C1152.mx025.tile1.nc -> C1152.mx025_oro_data.tile1.nc
lrwxrwxrwx 1 role.glopara global 29 Jan  5  2023 C1152.mx025/oro_C1152.mx025.tile2.nc -> C1152.mx025_oro_data.tile2.nc
lrwxrwxrwx 1 role.glopara global 29 Jan  5  2023 C1152.mx025/oro_C1152.mx025.tile3.nc -> C1152.mx025_oro_data.tile3.nc
lrwxrwxrwx 1 role.glopara global 29 Jan  5  2023 C1152.mx025/oro_C1152.mx025.tile4.nc -> C1152.mx025_oro_data.tile4.nc
lrwxrwxrwx 1 role.glopara global 29 Jan  5  2023 C1152.mx025/oro_C1152.mx025.tile5.nc -> C1152.mx025_oro_data.tile5.nc
lrwxrwxrwx 1 role.glopara global 29 Jan  5  2023 C1152.mx025/oro_C1152.mx025.tile6.nc -> C1152.mx025_oro_data.tile6.nc
lrwxrwxrwx 1 role.glopara global 28 Jul 31 16:49 C192.mx025/oro_C192.mx025.tile1.nc -> C192.mx025_oro_data.tile1.nc
lrwxrwxrwx 1 role.glopara global 28 Jul 31 16:49 C192.mx025/oro_C192.mx025.tile2.nc -> C192.mx025_oro_data.tile2.nc
lrwxrwxrwx 1 role.glopara global 28 Jul 31 16:49 C192.mx025/oro_C192.mx025.tile3.nc -> C192.mx025_oro_data.tile3.nc
lrwxrwxrwx 1 role.glopara global 28 Jul 31 16:49 C192.mx025/oro_C192.mx025.tile4.nc -> C192.mx025_oro_data.tile4.nc
lrwxrwxrwx 1 role.glopara global 28 Jul 31 16:49 C192.mx025/oro_C192.mx025.tile5.nc -> C192.mx025_oro_data.tile5.nc
lrwxrwxrwx 1 role.glopara global 28 Jul 31 16:49 C192.mx025/oro_C192.mx025.tile6.nc -> C192.mx025_oro_data.tile6.nc
lrwxrwxrwx 1 role.glopara global 28 Jan  5  2023 C192.mx050/oro_C192.mx050.tile1.nc -> C192.mx050_oro_data.tile1.nc
lrwxrwxrwx 1 role.glopara global 28 Jan  5  2023 C192.mx050/oro_C192.mx050.tile2.nc -> C192.mx050_oro_data.tile2.nc
lrwxrwxrwx 1 role.glopara global 28 Jan  5  2023 C192.mx050/oro_C192.mx050.tile3.nc -> C192.mx050_oro_data.tile3.nc
lrwxrwxrwx 1 role.glopara global 28 Jan  5  2023 C192.mx050/oro_C192.mx050.tile4.nc -> C192.mx050_oro_data.tile4.nc
lrwxrwxrwx 1 role.glopara global 28 Jan  5  2023 C192.mx050/oro_C192.mx050.tile5.nc -> C192.mx050_oro_data.tile5.nc
lrwxrwxrwx 1 role.glopara global 28 Jan  5  2023 C192.mx050/oro_C192.mx050.tile6.nc -> C192.mx050_oro_data.tile6.nc
lrwxrwxrwx 1 role.glopara global 28 Jan  5  2023 C384.mx025/oro_C384.mx025.tile1.nc -> C384.mx025_oro_data.tile1.nc
lrwxrwxrwx 1 role.glopara global 28 Jan  5  2023 C384.mx025/oro_C384.mx025.tile2.nc -> C384.mx025_oro_data.tile2.nc
lrwxrwxrwx 1 role.glopara global 28 Jan  5  2023 C384.mx025/oro_C384.mx025.tile3.nc -> C384.mx025_oro_data.tile3.nc
lrwxrwxrwx 1 role.glopara global 28 Jan  5  2023 C384.mx025/oro_C384.mx025.tile4.nc -> C384.mx025_oro_data.tile4.nc
lrwxrwxrwx 1 role.glopara global 28 Jan  5  2023 C384.mx025/oro_C384.mx025.tile5.nc -> C384.mx025_oro_data.tile5.nc
lrwxrwxrwx 1 role.glopara global 28 Jan  5  2023 C384.mx025/oro_C384.mx025.tile6.nc -> C384.mx025_oro_data.tile6.nc
lrwxrwxrwx 1 role.glopara global 27 Jan  5  2023 C48.mx500/oro_C48.mx500.tile1.nc -> C48.mx500_oro_data.tile1.nc
lrwxrwxrwx 1 role.glopara global 27 Jan  5  2023 C48.mx500/oro_C48.mx500.tile2.nc -> C48.mx500_oro_data.tile2.nc
lrwxrwxrwx 1 role.glopara global 27 Jan  5  2023 C48.mx500/oro_C48.mx500.tile3.nc -> C48.mx500_oro_data.tile3.nc
lrwxrwxrwx 1 role.glopara global 27 Jan  5  2023 C48.mx500/oro_C48.mx500.tile4.nc -> C48.mx500_oro_data.tile4.nc
lrwxrwxrwx 1 role.glopara global 27 Jan  5  2023 C48.mx500/oro_C48.mx500.tile5.nc -> C48.mx500_oro_data.tile5.nc
lrwxrwxrwx 1 role.glopara global 27 Jan  5  2023 C48.mx500/oro_C48.mx500.tile6.nc -> C48.mx500_oro_data.tile6.nc
lrwxrwxrwx 1 role.glopara global 28 Jan  5  2023 C768.mx025/oro_C768.mx025.tile1.nc -> C768.mx025_oro_data.tile1.nc
lrwxrwxrwx 1 role.glopara global 28 Jan  5  2023 C768.mx025/oro_C768.mx025.tile2.nc -> C768.mx025_oro_data.tile2.nc
lrwxrwxrwx 1 role.glopara global 28 Jan  5  2023 C768.mx025/oro_C768.mx025.tile3.nc -> C768.mx025_oro_data.tile3.nc
lrwxrwxrwx 1 role.glopara global 28 Jan  5  2023 C768.mx025/oro_C768.mx025.tile4.nc -> C768.mx025_oro_data.tile4.nc
lrwxrwxrwx 1 role.glopara global 28 Jan  5  2023 C768.mx025/oro_C768.mx025.tile5.nc -> C768.mx025_oro_data.tile5.nc
lrwxrwxrwx 1 role.glopara global 28 Jan  5  2023 C768.mx025/oro_C768.mx025.tile6.nc -> C768.mx025_oro_data.tile6.nc
lrwxrwxrwx 1 role.glopara global 27 Jan  5  2023 C96.mx100/oro_C96.mx100.tile1.nc -> C96.mx100_oro_data.tile1.nc
lrwxrwxrwx 1 role.glopara global 27 Jan  5  2023 C96.mx100/oro_C96.mx100.tile2.nc -> C96.mx100_oro_data.tile2.nc
lrwxrwxrwx 1 role.glopara global 27 Jan  5  2023 C96.mx100/oro_C96.mx100.tile3.nc -> C96.mx100_oro_data.tile3.nc
lrwxrwxrwx 1 role.glopara global 27 Jan  5  2023 C96.mx100/oro_C96.mx100.tile4.nc -> C96.mx100_oro_data.tile4.nc
lrwxrwxrwx 1 role.glopara global 27 Jan  5  2023 C96.mx100/oro_C96.mx100.tile5.nc -> C96.mx100_oro_data.tile5.nc
lrwxrwxrwx 1 role.glopara global 27 Jan  5  2023 C96.mx100/oro_C96.mx100.tile6.nc -> C96.mx100_oro_data.tile6.nc
lrwxrwxrwx 1 role.glopara global 27 May  9 16:48 C96.mx500/oro_C96.mx500.tile1.nc -> C96.mx500_oro_data.tile1.nc
lrwxrwxrwx 1 role.glopara global 27 May  9 16:48 C96.mx500/oro_C96.mx500.tile2.nc -> C96.mx500_oro_data.tile2.nc
lrwxrwxrwx 1 role.glopara global 27 May  9 16:48 C96.mx500/oro_C96.mx500.tile3.nc -> C96.mx500_oro_data.tile3.nc
lrwxrwxrwx 1 role.glopara global 27 May  9 16:48 C96.mx500/oro_C96.mx500.tile4.nc -> C96.mx500_oro_data.tile4.nc
lrwxrwxrwx 1 role.glopara global 27 May  9 16:48 C96.mx500/oro_C96.mx500.tile5.nc -> C96.mx500_oro_data.tile5.nc
lrwxrwxrwx 1 role.glopara global 27 May  9 16:48 C96.mx500/oro_C96.mx500.tile6.nc -> C96.mx500_oro_data.tile6.nc
@CoryMartin-NOAA
Copy link
Contributor

any update on this? Are the older versions of the fix files affected by this too or just the latest version? @andytangborn is having some issues generating cold start ICs and I don't know if it's related or not to this.

@KateFriedman-NOAA
Copy link
Collaborator Author

For this issue, I'm still waiting for the fix files for NOAA-EMC/global-workflow#1981 to be ready and then we'll be wrapping up that issue and NOAA-EMC/global-workflow#1984 together and can do this.

These updates are all new so I wouldn't suspect they'd impact current functionality. Just need to move to a new hash of UFS_UTILS in global-workflow when we also move to the new orog fix file timestamp copy.

@CoryMartin-NOAA
Copy link
Contributor

Thanks @KateFriedman-NOAA, just as an FYI, it seems that while building UFS_UTILS standalone this works fine, but building it as part of the workflow (which uses this https://github.com/NOAA-EMC/global-workflow/blob/1fc8b5bf55581de18e786611c65bc3f84bb0c082/versions/fix.ver#L17) it does not work

@KateFriedman-NOAA
Copy link
Collaborator Author

I'm going to guess it's because UFS_UTILS is pointing to 20220805 orog version by default:
https://github.com/ufs-community/UFS_UTILS/blob/develop/fix/link_fixdirs.sh#L60C22-L60C30
...but when run from within global-workflow it uses 20230615 (as you linked to). This mismatch should go away when we move UFS_UTILS and global-workflow to the new 20231027 orog version.
Thanks for letting me know about this issue because of the mismatch!

@CoryMartin-NOAA
Copy link
Contributor

@KateFriedman-NOAA thanks yes that is what I guess too. Ok, well since it seems to still work using the old orog, I think @andytangborn and I can get by with the old orog until the new files are supported.

@GeorgeGayno-NOAA
Copy link
Collaborator

@KateFriedman-NOAA and @CoryMartin-NOAA - I will fix the gdas_init utility.

@GeorgeGayno-NOAA GeorgeGayno-NOAA self-assigned this Dec 7, 2023
GeorgeGayno-NOAA added a commit to GeorgeGayno-NOAA/UFS_UTILS that referenced this issue Dec 7, 2023
GeorgeGayno-NOAA added a commit to GeorgeGayno-NOAA/UFS_UTILS that referenced this issue Dec 7, 2023
climo fixed files - i.e., C96 or C96.mx050.

Fixes ufs-community#867.
GeorgeGayno-NOAA added a commit to GeorgeGayno-NOAA/UFS_UTILS that referenced this issue Dec 7, 2023
GeorgeGayno-NOAA added a commit to GeorgeGayno-NOAA/UFS_UTILS that referenced this issue Dec 7, 2023
name. Update chgres_cube regression tests accordingly.

Fixes ufs-community#867.
@GeorgeGayno-NOAA
Copy link
Collaborator

@AlexanderRichert-NOAA - any idea why my workflows are failing? The spack-based installs are failing.

@AlexanderRichert-NOAA
Copy link
Collaborator

Can you point me to a failed run?

@GeorgeGayno-NOAA
Copy link
Collaborator

@AlexanderRichert-NOAA
Copy link
Collaborator

When the 'precision' variant is set for ip, it expects the 'precision' variant from sp to be set, but it's only available with [email protected] onward. Would you be able to use a more recent sp?

@GeorgeGayno-NOAA
Copy link
Collaborator

When the 'precision' variant is set for ip, it expects the 'precision' variant from sp to be set, but it's only available with [email protected] onward. Would you be able to use a more recent sp?

Let's try 2.4.0.

GeorgeGayno-NOAA added a commit to GeorgeGayno-NOAA/UFS_UTILS that referenced this issue Dec 13, 2023
GeorgeGayno-NOAA added a commit to GeorgeGayno-NOAA/UFS_UTILS that referenced this issue Dec 13, 2023
GeorgeGayno-NOAA added a commit to GeorgeGayno-NOAA/UFS_UTILS that referenced this issue Dec 13, 2023
GeorgeGayno-NOAA added a commit to GeorgeGayno-NOAA/UFS_UTILS that referenced this issue Dec 13, 2023
GeorgeGayno-NOAA added a commit to GeorgeGayno-NOAA/UFS_UTILS that referenced this issue Dec 14, 2023
GeorgeGayno-NOAA added a commit to GeorgeGayno-NOAA/UFS_UTILS that referenced this issue Dec 14, 2023
complete. Fixing will require a separate PR.

Fixes ufs-community#867.
GeorgeGayno-NOAA added a commit to GeorgeGayno-NOAA/UFS_UTILS that referenced this issue Dec 14, 2023
GeorgeGayno-NOAA added a commit to GeorgeGayno-NOAA/UFS_UTILS that referenced this issue Dec 14, 2023
GeorgeGayno-NOAA added a commit to GeorgeGayno-NOAA/UFS_UTILS that referenced this issue Dec 14, 2023
GeorgeGayno-NOAA added a commit to GeorgeGayno-NOAA/UFS_UTILS that referenced this issue Dec 14, 2023
GeorgeGayno-NOAA added a commit to GeorgeGayno-NOAA/UFS_UTILS that referenced this issue Dec 14, 2023
combinations of model and ocean resolutions.

Fixes ufs-community#867.
GeorgeGayno-NOAA added a commit to GeorgeGayno-NOAA/UFS_UTILS that referenced this issue Dec 15, 2023
GeorgeGayno-NOAA added a commit to GeorgeGayno-NOAA/UFS_UTILS that referenced this issue Dec 20, 2023
GeorgeGayno-NOAA added a commit to GeorgeGayno-NOAA/UFS_UTILS that referenced this issue Dec 20, 2023
GeorgeGayno-NOAA added a commit to GeorgeGayno-NOAA/UFS_UTILS that referenced this issue Dec 21, 2023
GeorgeGayno-NOAA added a commit to GeorgeGayno-NOAA/UFS_UTILS that referenced this issue Dec 21, 2023
GeorgeGayno-NOAA added a commit that referenced this issue Dec 22, 2023
Update "./fix/link_fixdirs.sh" to point to the new 20231027 "orog" directory.

Update "global_cycle" regression tests to point to a local copy of the required
"orog" files. 

Update "global_cycle" scripts to use either form of the 'orog' files - i.e.,
C96 or C96mx100.

Update "chgres_cube" to read either form of the surface climatological files,
i.e., C96 or C96.mx100.

Update the "gdas_init" scripts to use the 20231027 'orog' files.

Update "./ush/chgres_cube.sh" and "./reg_tests/update_baseline.sh" for the 
new fixed surface directory name (is now ./sfc.  previously ./fix_sfc). Update 
chgres_cube regression tests accordingly.

Fixes #867.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request maintenance Basic upkeep
Projects
None yet
Development

Successfully merging a pull request may close this issue.

4 participants