how to exclude a model from a relationship confrontation?; OR how to get proper gray values for the overall relationship score?

[mcguirepatr]

Hi ILAMB'ers:
For models that are missing data, they show gray values for the overall score for a (single-variable) confrontation. This is good.

But for a (two-variable) relationship confrontation, there are two overall scores given for this confrontation.
For example, in the summary plot for the confrontation of GrossPrimaryProductivity/FLUXCOM with Evapotranspiration/GLEAMv3.3a, if evapotranspiration is missing for a given model, but gpp is available for that model, then there will be a gray score in the relationship for that model for evaporation, but not for gpp.
I think it should be a gray score for both variables. Or how can I exclude just a single model from this relationship confrontation?

Here's the sample code for this confrontation:

[h2: Gross Primary Productivity]
variable       = "gpp"
cmap           = "Greens"
weight         = 5
mass_weighting = True
[FLUXCOM]
source        = "DATA/gpp/FLUXCOM/gpp.nc"
weight        = 15
table_unit    = "Pg yr-1"
plot_unit     = "g m-2 d-1"
space_mean    = False
skip_iav      = True
relationships = "Evapotranspiration/GLEAMv3.3a"

And here is the ILAMB homepage for the run I did:
https://gws-access.jasmin.ac.uk/public/ncas_climate/pmcguire/ILAMB_output_v4_24/GCP_280724c/

Can you help?
Patrick

[mcguirepatr]

I can open up a separate issue, if you'd like.
But I also notice that the relationship for LAI vs. Precipitation in that same ILAMB homepage has the axis min max wrong for precipitation. Maybe it's related to missing models? Can you advise?

Here's the code for the confrontation:

[h2: Leaf Area Index]
variable       = "lai"
cmap           = "Greens"
weight         = 3
mass_weighting = True

[AVH15C1]
source        = "DATA/lai/AVH15C1/lai.nc"
weight        = 15
relationships = "Precipitation/GPCPv2.3"

Thanks!
Patrick

[nocollier]

I think I understand the problem. It is

The relationships section of the main overview is not the same as the other sections. I build it up dynamically from relationships that are specified in the configure file. So in the main variable sections, when I synthesize a score for, say, LAI, it is grabbing the scalar Overall Score values and appropriately masks them out if any are nan's.

However, in the relationship section, the second level scores are actually components of the overall score. See this table:

https://gws-access.jasmin.ac.uk/public/ncas_climate/pmcguire/ILAMB_output_v4_24/GCP_280724c/EcosystemandCarbonCycle/LeafAreaIndex/AVH15C1/AVH15C1.html#Relationships

You will see that the precip score is blank (which means I computed a nan) but it gets synthesized into a 0, not a nan. I believe the fix is to create a check in ComputeOverallScore for if all components are nan, to return nan.

[nocollier]

However, this only partially fixes what appears to be another problem. The bins for the precip are very stretched out and thus the whole discrete distribution lies in a single bin. The culprits are EDv3 and JULES. Look at the precip dataset page, there must be a masking problem in these model files which aren't being properly handled:

https://gws-access.jasmin.ac.uk/public/ncas_climate/pmcguire/ILAMB_output_v4_24/GCP_280724c/Forcings/Precipitation/GPCPv2.3/GPCPv2.3.html

[nocollier]

@mcguirepatr Do you have these precip files somewhere I can grab one of them and check locally what is going on?

[mcguirepatr]

Here's one precip file for one model. I will try to upload the others.
https://gws-access.jasmin.ac.uk/public/ncas_climate/pmcguire/ILAMB_INPUT/TRENDYv12/PROCESSED_v4_24/S3/SDGVMv4_24/pr/SDGVM_S3_pr.nc

[mcguirepatr]

Here is a webpage that has all the models in it (18 of them?) for pr & lai:
https://gws-access.jasmin.ac.uk/public/ncas_climate/pmcguire/ILAMB_INPUT/TRENDYv12/PROCESSED_v4_24/S3/
And here is a tar.gz file with all the pr & lai files, with directory structure:
https://gws-access.jasmin.ac.uk/public/ncas_climate/pmcguire/ILAMB_INPUT/TRENDYv12/TRENDYv12_PROCESSED_v4_24_v1.tgz

[nocollier]

Thanks Patrick. It appears that the _FillValue was not encoded in some output files. So the different values that models use to flag invalid cells are being read as floats. This fixes these files:

ncatted -a _FillValue,pr,c,f,9.96920996838686904677855295210258432e36 SDGVM_S3_pr.nc
ncatted -a _FillValue,pr,c,f,-9999.0 JULES_S3_pr.nc

Then you will need to remove the ILAMB temporary netcdf files associated with pr and lai:

rm -rf _build/EcosystemandCarbonCycle/LeafAreaIndex/AVH15C1/*EDV3*.nc
rm -rf _build/Forcings/Precipitation/GPCPv2.3/*EDV3*.nc

and the same for JULES. Then when you re-run ILAMB it will only recompute these confrontations (but unfortunately replot everything).

I still owe you a fix for the 0.0 overall score when all components are nan. I think this should clean up the issues.

[mcguirepatr]

Hi Nathan:
Wow! Thanks for investigating this. I'm sorry about the bad data, and I will try to fix this somehow.

We have a model in the ensemble called EDv3, which seems to match the string EDV3 that you suggest I delete build files for. Or is that just a coincidental string match?
Are SDGVM and JULES the only two models that you found this error for?

I think it should be either a NAN or a 0.0 (not sure which), whenever ANY of the components are NAN. Not when ALL of the components are NAN. Or am I misunderstanding something?
Patrick

[mcguirepatr]

Hi again, Nathan:
I was wondering why I didn't notice any problem with the pr scores for single variables, instead of relationship 2-variable confrontations.

I guess I didn't look at the geographic plots for pr. I also see now that the pr (solo) plots also have the same problem as the lai-pr relationship plots.

Thanks again for figuring this out, and for doing it so quickly.
Patrick

[mcguirepatr]

Hi again2, Nathan:
I think the landmask/FillValue was only not set properly for the JULES pr & tas (precipitation & Surface Air Temperature) files. I don't think it was set wrong for the SDGVM or EDv3 files.

But we'll see. I am rerunning ILAMB now (after fixing JULES pr & tas), with your suggested shortcut. I am actually rerunning 3 different versions, with 3 different sets of land models.
Thanks,
Patrick McGuire

[mcguirepatr]

Hi Nathan:
I think the EDv3 model may be ok for the landmask/FillValue for precipitation, but its precipitation is a few orders of magnitude too high. There might be something wrong with tas for EDv3, as well. So I am just going to drop these variables from the ILAMB analysis for the EDv3 model.
Patrick

[mcguirepatr]

Hi Nathan:
Your suggested changes worked! Thank you very much. Here is the new copy of the web page. Note the changes to the relationships as well as the forcing variables (pr & tas):
https://gws-access.jasmin.ac.uk/public/ncas_climate/pmcguire/ILAMB_output_v4_24/GCP_280724c/

Here's the old version, for reference:
https://gws-access.jasmin.ac.uk/public/ncas_climate/pmcguire/ILAMB_output_v4_24_old/GCP_280724c/

If you can figure out the remaining issue "for the 0.0 overall score when all/any components are nan", that would be great.

Thanks a lot!
Patrick

[mcguirepatr]

Thanks, Nate! It's working now, thanks to your great help!
#103

[mcguirepatr]

Thanks again for fixing it, Nathan!
Here's a new version screenshot, with the fixed code, highlighting the overall scores of the relationships being the proper values. And the link to the TRENDYv12/GlobalCarbonProject2023) (last year's):
https://gws-access.jasmin.ac.uk/public/landsurf_rdg/pmcguire/ILAMB_output_TRENDYv12/GCP_310824c/