Aerosol buget metrics htmls got missing if aerosol_budget was listed before lat_lon set

[mahf708]

1. There are several zppy configs in this folder
2. Their corresponding outputs are here, here, and here
3. If you look at the aerosol budget component in the viewer, you won't find it (e.g., "ANN"). But it is clearly there if you look at the raw files outside he viewer. This applies to both mvm and model-vs-obs
4. Another interesting thing is that the viewer layout has changed a bit (see screenshot)

--

[chengzhuzhang]

This is on my todo list. And thanks for submitting this issue. It is strange that the metrics tables were not linked correctly. This problem was not shown in my recent stand alone e3sm_diags tests for e3sm_unified release. I'm looking at the zppy files to see if there are some config caused this problem.

[chengzhuzhang]

well, chrysalis and anvil are under maintenance. I will try reproduce after they come back.

[mahf708]

Same with Xue's runs btw, that's why I filed this issue. See here: https://web.lcrc.anl.gov/public/e3sm/diagnostic_output/ac.xzheng/E3SMv3_dev/20230924.v3alpha04_trigrid.piControl.chrysalis/e3sm_diags/atm_monthly_180x360_aave_mvm/

[chengzhuzhang]

thanks for providing more one data point. I tried to run stand alone e3sm_diags to reproduce on Perl-mutter with a testing dataset, with no success. It seems like the htmls that supposed to host those metrics .csv files were not generated in your and Xue's cases. I will try later on chrysalis with the zppy configs provided.

[chengzhuzhang]

@mahf708 No related to the issue mentioned, just to note that I found that in two of the zppy configuration files: 20230918.v3alpha04.F2010.PI.chrysalis.cfg and 20230918.v3alpha04.F2010.4K.chrysalis. The short_name was not consistent with the casename.

[mahf708]

I thought I fixed it, but also introduced another silly mistake:

[e3sm_diags]
active = True
years = "0002:0011:10", 
ts_num_years = 1
ref_start_yr = 1979
ref_final_yr = 2016

  [[ atm_monthly_180x360_aave ]]
  short_name = '84e5056-smb1.pd.chrys'
  grid = '180x360_aave'
  reference_data_path = '/lcrc/soft/climate/e3sm_diags_data/obs_for_e3sm_diags/climatology'
  obs_ts = '/lcrc/soft/climate/e3sm_diags_data/obs_for_e3sm_diags/time-series'
  dc_obs_climo = '/lcrc/group/e3sm/public_html/e3sm_diags_test_data/unit_test_complete_run/obs/climatology'
  climo_diurnal_subsection = "atm_monthly_diurnal_8xdaily_180x360_aave"
  climo_diurnal_frequency = "diurnal_8xdaily"
  output_format_subplot = "pdf",

  [[ atm_monthly_180x360_aave_mvm ]]
  climo_subsection = "atm_monthly_180x360_aave"
  grid = '180x360_aave'
  diff_title = "Difference"
  ref_final_yr = 0011
  ref_start_yr = 0002
  reference_data_path = '/lcrc/group/e3sm/ac.ngmahfouz/E3SMv3_dev/84e5056-smb1.pi.chrys/post/atm/180x360_aave/clim'
  ref_name = "84e5056-smb1.pi.chrys"
  ref_years = "0002-0011",
  run_type = "model_vs_model"
  years = "0002-0011", 
  short_ref_name = '84e5056-smb1.pd.chrys'       <-------- swapped
  short_name = '84e5056-smb1.pi.chrys'           <-------- swapped
  swap_test_ref = False

[chengzhuzhang]

I think I know what is going on. The aerosol_budget and lat_lon sets viewers have a conflict that if the folder generated by aerosol_budget, that containing these budget metrics htmls would be purged by the lat_lon set viewer, if lat_lon viewer is generated after aerosol_budget.

I will try to come up with a fix. But at the time being, the workaround is to always list lat_lon first.
lat_lon first vs aerosol_budget first.

Example:
sets = 'lat_lon', 'aerosol_aeronet', 'aerosol_budget', 'annual_cycle_zonal_mean', 'cosp_histogram', 'meridional_mean_2d', 'polar', 'zonal_mean_2d', 'zonal_mean_2d_stratosphere', 'zonal_mean_xy',

This will fix the viewer layout change as well.

I think there was a change in zppy that caused the default sets to order differently in the new version.

[mahf708]

Confirmed. Thank you! Would you like this issue closed or should we keep it open?

[chengzhuzhang]

Thank you for confirming! I will update the title and keep this issue open.

[mahf708]

Should be resolved now! Thanks everyone!