Input data validation could be better

[fpavogt]

Is there already an issue for this?

• I found no existing issue for this bug.

Describe the bug

A code user reported the error in #119.

When looking at it, it was noticed that the DataFrame being fed to ampycloud contained something like:

#id, dt, alt, type, ceilo
…
177,-30.0,1020.0,1,ceiloX
178,-30.0,,0,ceiloX
179,-30.0,,0,ceiloX

All lines are correct on their own, but l.178 and l.179 (no cloud base detected at dt=-30) are directly inconsistent with l. 177 (1 cloud base detected at dt=-30). Lines 178 and 179 are also redundant.

Currently, the function check_data_consistency() in src/ampycloud/utils.py does not check for these elements (duplicated hits or inconsistent hits). This was not directly related to the error of #119, but it's not ideal nonetheless.

The solution is to enhance check_data_consistency() to cover these cases, and issue a meaningful error to inform the user.

Screenshot

No response

Steps to reproduce

No response

Relevant log output

No response

Environment

The problem was seen is both ampycloud v1.0.0 and v2.0.0.dev.

Code of Conduct

• I agree to follow the ampycloud Code of Conduct.