Solar radiation error handling

[jkshuman]

FATES has consistently been producing radiation warning messages that end up producing very large logs. (see these issues: #236 #191 #772) A PR was introduced to reduce unnecessary log messages here: (#792) In review of this PR792 it became apparent that one of the most consistent warnings for a "BalanceCheck on solar radiation" is triggered outside of FATES here in CTSM: https://github.com/ESCOMP/CTSM/blob/bb3d227887b551c8d4e71bc1473753231bbdeb54/src/biogeophys/BalanceCheckMod.F90#L924-L934

On the FATES side there is handling of solar radiation error that needs attention and discussion. This issue will act as a prompt, so that we address this part of the FATES code. tagging @ckoven @rosiealice @rgknox @jenniferholm @adrifoster @mpaiao (but all are welcome to comment!) The whole section within SurfaceAlbedoMod devoted to conservation check needs thorough discussion and review, but there is a section where the error appears to be added back onto the albedo (second linked section below here).

fates/biogeophys/EDSurfaceAlbedoMod.F90

Lines 916 to 925 in 2fa452a

	!==============================================================================!
	! Conservation check
	!==============================================================================!
	! Total radiation balance: absorbed = incoming - outgoing
	if (radtype == idirect)then
	error = abs(currentPatch%sabs_dir(ib) - (currentPatch%tr_soil_dir(ib) * &
	(1.0_r8-currentPatch%gnd_alb_dir(ib) ) + &
	currentPatch%tr_soil_dir_dif(ib) * (1.0_r8-currentPatch%gnd_alb_dif(ib) )))
	if ( abs(error) > 0.0001)then

Here is a link to that specific section of the code with comments on the error handling that was noted in PR792:

fates/biogeophys/EDSurfaceAlbedoMod.F90

Lines 968 to 988 in 2fa452a

	!here we are adding a within-ED radiation scheme tolerance, and then adding the diffrence onto the albedo
	!it is important that the lower boundary for this is ~1000 times smaller than the tolerance in surface albedo.
	if (abs(error) > 1.e-9_r8 .and. abs(error) < 0.15_r8)then
	albd_parb_out(ib) = albd_parb_out(ib) + error
	!this terms adds the error back on to the albedo. While this is partly inexcusable, it is
	! in the medium term a solution that
	! prevents the model from crashing with small and occasional energy balances issues.
	! These are extremely difficult to debug, many have been solved already, leading
	! to the complexity of this code, but where the system generates occasional errors, we
	! will deal with them for now.
	end if
	if (abs(error) > 0.15_r8)then
	write(fates_log(),*) 'Large Dir Radn consvn error',error ,ib
	write(fates_log(),*) 'diags', albd_parb_out(ib), ftdd_parb_out(ib), &
	ftid_parb_out(ib), fabd_parb_out(ib)
	write(fates_log(),*) 'lai_change',lai_change(currentpatch%ncl_p,1:numpft,1:diag_nlevleaf)
	write(fates_log(),*) 'elai',currentpatch%elai_profile(currentpatch%ncl_p,1:numpft,1:diag_nlevleaf)
	write(fates_log(),*) 'esai',currentpatch%esai_profile(currentpatch%ncl_p,1:numpft,1:diag_nlevleaf)
	write(fates_log(),*) 'ftweight',ftweight(1,1:numpft,1:diag_nlevleaf)
	write(fates_log(),*) 'cp',currentPatch%area, currentPatch%patchno
	write(fates_log(),*) 'ground albedo diffuse (ib)', currentPatch%gnd_alb_dir(ib)

[jenniferholm]

Thanks for starting this Jackie! These solar radiation errors are annoying.

[jkshuman]

This issue was referenced in a SE call today, and we discussed that the radiation code in general would benefit from an overall refactoring, potential update to variables names and a stand alone unit test.