Fix AGRI L1 C07 having a valid LUT value for its fill value

[djhoese]

Closes ssec/polar2grid#565

CC @simonrp84

This was first encountered in Geo2Grid development by @kathys. Some debugging showed that C07 files for AGRI L1 have a LUT with 65536 elements in it even though 65535 is a fill value. The end result is fill values being replaced by real BT values.

This PR implements one possible solution. It says "if the LUT already includes the fill value, force that LUT value to NaN". An alternative could be slicing the LUT array to the input data's valid_range and then getting NaN for anything beyond the LUT. The ending implementation ends up looking pretty similar I think.

Note: I haven't added tests for this yet because I wanted to give @simonrp84 time to review the solution as soon as possible. I also wanted to see if this would work without having to (likely) rewrite a ton of the tests to fit the situation...which is basically a bug in the input data.

• Closes #xxxx
• Tests added
• Fully documented
• Add your name to AUTHORS.md if not there already

[codecov]

Codecov Report

All modified and coverable lines are covered by tests ✅

Comparison is base (90a3131) 95.40% compared to head (20cb176) 95.40%.

Additional details and impacted files

@@           Coverage Diff           @@
##             main    #2726   +/-   ##
=======================================
  Coverage   95.40%   95.40%           
=======================================
  Files         371      371           
  Lines       52825    52825           
=======================================
  Hits        50399    50399           
  Misses       2426     2426           

Flag	Coverage Δ
behaviourtests	4.15% <0.00%> (ø)
unittests	96.02% <100.00%> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[coveralls]

Pull Request Test Coverage Report for Build 7657460878

• 0 of 0 changed or added relevant lines in 0 files are covered.
• No unchanged relevant lines lost coverage.
• Overall coverage remained the same at 95.968%

---

Totals
Change from base Build 7608592238:	0.0%
Covered Lines:	50525
Relevant Lines:	52648

---

💛 - Coveralls

[djhoese]

Ok this is ready for review @mraspaud and @simonrp84 if you have the time. The existing test code was...rough. The expected values and the coefficients used were based on the index of the channel in the file, not the actual channel number and it made it very confusing. Additionally there was a lot of "group all this stuff into a method to make linters happy about less complexity", but a lot of it didn't need to be that way. Or rather, it should have been done even further to make the code even simpler.

[mraspaud]

Looks good to me! Thanks for refactoring the tests on the way there.

[mraspaud]

Is this a reminder of the contents of band_names or a forgotten comment?

[djhoese]

Forgotten comment. I'll get rid of it.

[simonrp84]

Unless I'm doing somethign wrong, this doesn't seem to solve the issue. With the data linked to earlier in this issue I still get a value of 200K for deep space pixels.

[djhoese]

Created with:

In [1]: from satpy import Scene; from glob import glob
scn
In [2]: scn = Scene(reader="agri_fy4a_l1", filenames=glob("/data/satellite/agri_l1/20220120/FY4A-_AGRI--_N_DISK_1047E_L1-_*"))

In [3]: scn.load(["C07"])

In [4]: scn["C07"].plot.imshow().figure.show()

@simonrp84 possibly incorrect installation on your end?

[simonrp84]

Aah,I was using the FY4B reader for FY4A. Maybe that is the problem. Will test later.

[djhoese]

I'm going to consider this good to go. Merging...