Combine g2tmpl and g2 (and maybe g2c as well?)

[edwardhartnett]

Should g2tmpl, g2, g2c, and grib_util be combined? @BoiVuong-NOAA what do you think?

[BoiVuong-NOAA]

Hi Edward,
No, We should not combine g2tmpl, g2, g2c, and grib_util. We can combine g2 and g2tmpl because they are all FORTRAN code. The g2c is C code. The g2c lib is not use in our production. However, The g2c library uses many places outside of NCEP (Such as: GrDAS, some companies ( i don know ?). The grib_util consists of many GRIB2 and GRIB1 utilities. They are: copygb, copygb2, tocgrib2, tocgrib, ...ect.
Thanks,
Boi

[DusanJovic-NOAA]

There's a huge code duplication between g2 and g2c. I think g2 should be implemented as small Fortran wrapper around g2c functions. g2c should stay as a standalone grib2 encoding/decoding library (repo).

[edwardhartnett]

OK, great discussion already!

Some points:

• We can combine C and Fortran libraries in one package.
• Agree that C + Fortran wrappers is much better than a separate Fortran implementation.
• If g2c is not used in production, should be be taken out of NCEPLIBS (though kept as a separate project)?
• What is the difference between g2tmpl and g2? @BoiVuong-NOAA you believe these can be combined?
• Does grib_util use g2, g2c, or g2tmpl?

We would like to combine packages when we can to reduce overall NCEPLIBS work and costs.

[BoiVuong-NOAA]

Hi Edward,
Yes, I agree. We can remove g2c out of NCEPLIBS.

• What is the difference between g2tmpl and g2 ?
  g2tmpl lib uses to support users to call directly to grib2 tables and mnemonic. g2 lib uses grib2 tables as pdt 4, gdt3 and octets for parameters.
• Does grib_util use g2, g2c, or g2tmpl.
  grib_util needs to compile with g2 lib, w3emc, w3nco, ip, sp, bacio,.. not g2c and g2tmpl. However, some models in NCEP need to compile with libraries: g2 and g2tmpl not g2c lib.

[edwardhartnett]

@BoiVuong-NOAA what do you think about @DusanJovic-NOAA point about g2c and g2? Should g2 be refactored as fortran wrappers in g2c?

We need g2 right? So if we made g2 a set of wrappers around g2c, then we would also need g2c.

I will remove grib_util from this discussion and we will discuss that separately...

[Hang-Lei-NOAA]

So, g2, g2c and g2tmpl will be combined.
grib_util will stay.

[edwardhartnett]

I think we want to start by combining g2tmpl into g2. Does that sound like a good path forward?

Can we accomplish that, and then circle back around to the g2c library in a future release?

[Hang-Lei-NOAA]

I agree with you. Let see Boi's expert opinion. He is the code manager and developer for these libs.

…

On Tue, Oct 6, 2020 at 10:43 AM Edward Hartnett ***@***.***> wrote: I think we want to start by combining g2tmpl into g2. Does that sound like a good path forward? Can we accomplish that, and then circle back around to the g2c library in a future release? — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#127 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AKWSMFBFEDTPOWYOYDCMOPDSJMUI7ANCNFSM4R6I2M3A> .

[edwardhartnett]

@BoiVuong-NOAA has already agreed that combining g2 and g2tmpl is a good idea, noting that both are Fortran.

[BoiVuong-NOAA]

They are FORTRAN.

[edwardhartnett]

You mean F77 code? That's fine. The good news with F77 code is we can rely on F90 compilers, which help a bit. ;-)

[edwardhartnett]

We have decided to do this only after some testing is in place.

[BoiVuong-NOAA]

Yes, You are right. The testing will take some times to write some codes. Do you have some one to help ?

[edwardhartnett]

@BoiVuong-NOAA not yet. ;-)

[edwardhartnett]

This can be done after the next NCEPLIBS release.

[edwardhartnett]

OK, here's an update on this discussion.

• Unit testing is now in place for g2c, g2, and g2tmpl. grib_util still needs unit tests.
• To reduce code duplication and deal with a Jasper issue, g2 now depends on g2c. The png and jpeg compression calls in g2 are now wrappers around the g2c versions of these functions.
• I am now adding code to g2c to read the XML code definition files from the WMO. This will allow the g2c library to serve up all code info to any library that needs it. This will take some time to get working. Also we have our own mnemonics for the field names, and that will have to continue to be supported. So there is some work to be done here.
• The end-goal is to have all code information in the g2c library, and have the g2tmpl code be Fortran which calls the g2c library to get the code information. In this way the codes can be supported for g2tmpl, but also for any other applications using g2c, including potentially the pyhon library, which currently maintains its own set of duplicate codes (@EricEngle-NOAA this may be of interest to you.)
• The multiple different lists of codes (XML, in g2tmpl, and in the python library) can potentially be combined into g2c, and in doing so there will be opportunities to test whether there are any mistakes in the g2tmpl code tables.
• Once all this code stuff is sorted out, we can move the g2tmpl code into g2 and deprecate the g2tmpl project. g2 already depends on g2c, and g2c will have all the code information, so we might as well move the fortran functions that deal with codes into the g2 library.
• We will keep the g2c library separate from the g2 repo, but we may eventually end up combining grib_utils into g2, since grib_utils depends on g2. So we would end up with g2c, with all the C code, and g2, with the fortran wrappers and the grib_util utilities.