You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Currently, batch fails if any lightcurve passed to the analysis function fails. This all or nothing behavior creates some extreme user-unfriendliness at scale, where a single nan can crash a full computation at the very end. Dask error logging is very unsupportive for this as well, as it really doesn't give enough to go on to debug properly.
This motivates a potential debug function, that does batch but guarantees that the run will execute, but the result is all object and/or source rows that failed the computation. This would allow for users to easily key into the cause of the issue between their batch function and the lightcurves and filter/adjust as needed.
The text was updated successfully, but these errors were encountered:
Currently, batch fails if any lightcurve passed to the analysis function fails. This all or nothing behavior creates some extreme user-unfriendliness at scale, where a single nan can crash a full computation at the very end. Dask error logging is very unsupportive for this as well, as it really doesn't give enough to go on to debug properly.
This motivates a potential debug function, that does batch but guarantees that the run will execute, but the result is all object and/or source rows that failed the computation. This would allow for users to easily key into the cause of the issue between their batch function and the lightcurves and filter/adjust as needed.
The text was updated successfully, but these errors were encountered: