Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Port ufs_utils for GEFSv12 to WCOSS2/Acorn #552

Closed
XianwuXue-NOAA opened this issue Jul 20, 2021 · 17 comments
Closed

Port ufs_utils for GEFSv12 to WCOSS2/Acorn #552

XianwuXue-NOAA opened this issue Jul 20, 2021 · 17 comments
Assignees
Labels
maintenance Basic upkeep

Comments

@XianwuXue-NOAA
Copy link

Port based on tag: ops-gefsv12.1

XianwuXue-NOAA added a commit to XianwuXue-NOAA/UFS_UTILS that referenced this issue Jul 20, 2021
to port to Acorn

 On branch feature/port2acorn_ops-gefsv12.1
	modified:   .gitmodules

Refs: ufs-community#552
@edwardhartnett
Copy link
Collaborator

Not sure what "port based on tag" means. Are there changes that will be merged to master?

Instead of a tag, we need to do a proper release.

@XianwuXue-NOAA
Copy link
Author

Yes, the tag is for GEFS v12.1. It is better to create a release branch (release/gefsv12) for the porting. Regarding merging into master, after I ported to acorn, you can decide whether it is merged to master or not. BTW, the lib name was changed in acorn. So if you want to merge into master, we should make it more general to be compatible with all machines.

@edwardhartnett
Copy link
Collaborator

We must do a release, and it must be done in a backwards-compatible way. If you wish to have an extra tag, that's fine, but it also needs to be a versioned release.

What library name was changed?

@XianwuXue-NOAA
Copy link
Author

I agree. Please create a release branch for us based on tag: ops-gefsv12.1

On acorn, some libs do not have libName_4, such as gfsio, sfcio, bacio, and sigio, maybe NCO will install it later. So I modified the CMakeLists.txt files to delete the 4. In CMakeModules, the "versioned_lib_name" is changed to ${name} instead the original "${name}${version}". However, since this is the first try, I may find other better method to make it work. Welcome all suggestions for this porting. I will push my changes tomorrow and then you will see my changes.

@edwardhartnett
Copy link
Collaborator

@GeorgeGayno-NOAA looks like this will be the 1.6.0 release. We need to point the tag and the release at the same hash in the repo. Best would be to do a release, then add the extra tag.

XianwuXue-NOAA added a commit to XianwuXue-NOAA/UFS_UTILS that referenced this issue Jul 22, 2021
Update the CMakeModules commit to the ported commit

 On branch feature/port2acorn_ops-gefsv12.1
	modified:   cmake

Refs: ufs-community#552
XianwuXue-NOAA added a commit to XianwuXue-NOAA/UFS_UTILS that referenced this issue Jul 22, 2021
 On branch feature/port2acorn_ops-gefsv12.1
	modified:   CMakeLists.txt
	new file:   modulefiles/build.acorn
	modified:   modulefiles/module-setup.sh.inc
	new file:   modulefiles/module_base.acorn
	modified:   sorc/machine-setup.sh
	modified:   ush/load_fv3gfs_modules.sh

Refs: ufs-community#552
XianwuXue-NOAA added a commit to XianwuXue-NOAA/UFS_UTILS that referenced this issue Jul 30, 2021
 On branch feature/port2acorn_ops-gefsv12.1
	modified:   modulefiles/build.acorn
	modified:   modulefiles/module-setup.sh.inc
	modified:   sorc/machine-setup.sh

Refs: ufs-community#552
XianwuXue-NOAA added a commit to XianwuXue-NOAA/UFS_UTILS that referenced this issue Aug 9, 2021
 On branch feature/port2acorn_ops-gefsv12.1
	modified:   cmake
	renamed:    modulefiles/build.acorn -> modulefiles/build.wcoss2
	modified:   modulefiles/module-setup.sh.inc
	renamed:    modulefiles/module_base.acorn -> modulefiles/module_base.wcoss2
	modified:   sorc/machine-setup.sh
	modified:   ush/load_fv3gfs_modules.sh

Refs: ufs-community#552
XianwuXue-NOAA added a commit to XianwuXue-NOAA/UFS_UTILS that referenced this issue Sep 9, 2021
 On branch feature/port2acorn_ops-gefsv12.1
 Changes to be committed:
	modified:   build_all.sh

Refs: ufs-community#552
XianwuXue-NOAA added a commit to XianwuXue-NOAA/UFS_UTILS that referenced this issue Sep 9, 2021
 On branch feature/port2acorn_ops-gefsv12.1
 Changes to be committed:
	modified:   modulefiles/build.wcoss2

Refs: ufs-community#552
XianwuXue-NOAA added a commit to XianwuXue-NOAA/UFS_UTILS that referenced this issue Sep 16, 2021
 On branch feature/port2acorn_ops-gefsv12.1
 Changes to be committed:
	modified:   modulefiles/build.wcoss2

Refs: ufs-community#552
XianwuXue-NOAA added a commit to XianwuXue-NOAA/UFS_UTILS that referenced this issue Sep 23, 2021
 On branch feature/port2acorn_ops-gefsv12.1
 Changes to be committed:
	modified:   modulefiles/build.wcoss2
Refs: ufs-community#552
@GeorgeGayno-NOAA GeorgeGayno-NOAA added the maintenance Basic upkeep label Oct 8, 2021
XianwuXue-NOAA added a commit to XianwuXue-NOAA/UFS_UTILS that referenced this issue Oct 15, 2021
 On branch feature/port2acorn_ops-gefsv12.1
 Changes to be committed:
	modified:   cmake

Refs: ufs-community#552
XianwuXue-NOAA added a commit to XianwuXue-NOAA/UFS_UTILS that referenced this issue Oct 15, 2021
 On branch feature/port2acorn_ops-gefsv12.1
 Changes to be committed:
	modified:   sorc/CMakeLists.txt

Refs: ufs-community#552
XianwuXue-NOAA added a commit to XianwuXue-NOAA/UFS_UTILS that referenced this issue Oct 16, 2021
1) Update some modules using new versions
2) Update cmake to use the updated Find[sigio,sfcio]
3) Remove _4 in sigio[,sfcio]_4

 On branch feature/port2acorn_ops-gefsv12.1
 Changes to be committed:
	modified:   CMakeLists.txt
	modified:   cmake
	modified:   modulefiles/build.wcoss2
	modified:   sorc/chgres_cube.fd/CMakeLists.txt
	modified:   sorc/global_chgres.fd/CMakeLists.txt

Refs: ufs-community#552
XianwuXue-NOAA added a commit to XianwuXue-NOAA/UFS_UTILS that referenced this issue Oct 20, 2021
GEFS needs to use wgrib2/2.0.7 and grb2_UNDEFINED does not exit
  in wgrib2/2.0.7 since it is added since wgrib2/2.0.8.
The input format for chgres_cube in GEFS is gaussian_netcdf,
  not wgrib2, so it will not affect the results

 On branch feature/port2acorn_ops-gefsv12.1
 Changes to be committed:
	modified:   sorc/chgres_cube.fd/input_data.F90

Refs: ufs-community#552
XianwuXue-NOAA added a commit to XianwuXue-NOAA/UFS_UTILS that referenced this issue Oct 20, 2021
Update build modules to remove hardwired version numbers
 On branch feature/port2acorn_ops-gefsv12.1
 Changes to be committed:
	modified:   modulefiles/build.wcoss2

Refs: ufs-community#552
XianwuXue-NOAA added a commit to XianwuXue-NOAA/UFS_UTILS that referenced this issue Oct 26, 2021
1) gfsio is not needed
2) nemsiogfs is only needed in global_chgres
	but we don't use global_chgres now,
	so I just commented it instead of delete
	nemsiogfs statements

 On branch feature/port2acorn_ops-gefsv12.1
 Changes to be committed:
	modified:   CMakeLists.txt
	modified:   modulefiles/build.wcoss2
	modified:   sorc/global_chgres.fd/CMakeLists.txt

Refs: ufs-community#552
@XianwuXue-NOAA
Copy link
Author

@GeorgeGayno-NOAA 1) Based on your the issue "Can we remove wgrib2 from the dependencies? #591", so I should remove wgrib2 and use g2, however, this will make some code changes from your repo. Should I do this? 2) I found the latest ufs_utils does not use submodule "CMakeModules" and just have a copy in the branch, can I also remove the CMakeModules submodule and add the copy of CMakeModules as "cmake" folder?

@GeorgeGayno-NOAA
Copy link
Collaborator

@GeorgeGayno-NOAA 1) Based on your the issue "Can we remove wgrib2 from the dependencies? #591", so I should remove wgrib2 and use g2, however, this will make some code changes from your repo. Should I do this? 2) I found the latest ufs_utils does not use submodule "CMakeModules" and just have a copy in the branch, can I also remove the CMakeModules submodule and add the copy of CMakeModules as "cmake" folder?

To answer (1), I am currently working to convert chgres_cube to use g2. It is a lot of work. So I would not worry about it.

@GeorgeGayno-NOAA
Copy link
Collaborator

@GeorgeGayno-NOAA 1) Based on your the issue "Can we remove wgrib2 from the dependencies? #591", so I should remove wgrib2 and use g2, however, this will make some code changes from your repo. Should I do this? 2) I found the latest ufs_utils does not use submodule "CMakeModules" and just have a copy in the branch, can I also remove the CMakeModules submodule and add the copy of CMakeModules as "cmake" folder?

To answer (2), you are welcome to change the build if it makes the port easier. Will this WOCSS2 'tag' be used on other machines? If so, you will need to test the updated build on all required machines.

@XianwuXue-NOAA
Copy link
Author

@GeorgeGayno-NOAA Thanks. OK. That means I don't need to change g2 for this time. I think the WCOSS2 tag only for WCOSS2 machine. I didn't change the confirmation for other machines. However, some changes for FindXX maybe affect other machines. Since Kyle closed my issue NOAA-EMC/CMakeModules#53 and think my branch used the out-of-date version of CMakeModules, so I'd like to keep the copy of CMakeModules as cmake folder in the repo.

XianwuXue-NOAA added a commit to XianwuXue-NOAA/UFS_UTILS that referenced this issue Oct 27, 2021
 On branch feature/port2acorn_ops-gefsv12.1_cmake
 Changes to be committed:
	deleted:    .gitmodules

Refs: ufs-community#552
XianwuXue-NOAA added a commit to XianwuXue-NOAA/UFS_UTILS that referenced this issue Oct 27, 2021
 On branch feature/port2acorn_ops-gefsv12.1_cmake2
 Changes to be committed:
	modified:   .gitmodules
	deleted:    cmake

Refs: ufs-community#552
XianwuXue-NOAA added a commit to XianwuXue-NOAA/UFS_UTILS that referenced this issue Oct 27, 2021
 On branch feature/port2acorn_ops-gefsv12.1_cmake2
 Changes to be committed:
New folder: cmake

Refs: ufs-community#552
XianwuXue-NOAA added a commit to XianwuXue-NOAA/UFS_UTILS that referenced this issue Oct 29, 2021
 On branch feature/port2acorn_ops-gefsv12.1
 Changes to be committed:
	modified:   cmake/Modules/FindESMF.cmake
	modified:   modulefiles/build.wcoss2
	modified:   modulefiles/module_base.wcoss2

Refs: ufs-community#552
@XianwuXue-NOAA
Copy link
Author

@edwardhartnett @GeorgeGayno-NOAA Could you create a branch release/gefs_v12 based on tag: ops-gefsv12.1? I'd like to submit PR to this branch. Thanks a lot in advance.

@GeorgeGayno-NOAA
Copy link
Collaborator

@edwardhartnett @GeorgeGayno-NOAA Could you create a branch release/gefs_v12 based on tag: ops-gefsv12.1? I'd like to submit PR to this branch. Thanks a lot in advance.

Thanks. Let me discuss with @edwardhartnett about how to best do this.

@GeorgeGayno-NOAA
Copy link
Collaborator

Not sure what "port based on tag" means. Are there changes that will be merged to master?

Instead of a tag, we need to do a proper release.

@edwardhartnett Agreed. The original tag was created from an old version of 'develop' and used our old naming convention, which included the name of the process it supported. In this case, the GEFS. Once it is ported to WCOSS2, we can create a new release. How should we name this releases. Something like ops-gefsv12.2?

@GeorgeGayno-NOAA
Copy link
Collaborator

@XianwuXue-NOAA I see two branches in your fork - feature/port2acorn_ops-gesfv12.1 and feature/port2acorn_ops-gesfv12.1_cmake2. What do you want tagged?

@XianwuXue-NOAA
Copy link
Author

@GeorgeGayno-NOAA could you create a new branch as "release/ops-gefsv12.2" based on tag "ops-gefsv12.1" (commit id: "1b44601" on "Tue Oct 6 20:21:33 2020 +0000"). Then I will submit a PR to merge my branch "feature/port2acorn_ops-gesfv12.1" into the new created branch "release/ops-gefsv12.2". After the PR approved, then create a new tag "ops-gefsv12.2" based on the merged branch "release/ops-gefsv12.2". Thanks.

@GeorgeGayno-NOAA
Copy link
Collaborator

@GeorgeGayno-NOAA could you create a new branch as "release/ops-gefsv12.2" based on tag "ops-gefsv12.1" (commit id: "1b44601" on "Tue Oct 6 20:21:33 2020 +0000"). Then I will submit a PR to merge my branch "feature/port2acorn_ops-gesfv12.1" into the new created branch "release/ops-gefsv12.2". After the PR approved, then create a new tag "ops-gefsv12.2" based on the merged branch "release/ops-gefsv12.2". Thanks.

I created the release branch.

@XianwuXue-NOAA
Copy link
Author

Awesome. Thanks. @GeorgeGayno-NOAA

@GeorgeGayno-NOAA
Copy link
Collaborator

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
maintenance Basic upkeep
Projects
None yet
Development

No branches or pull requests

3 participants