feat: add option to sigma-clip sub-bands homogeneously #927
Conversation
This adds several options to a new `sigma_clip()` function, including ``clip_type`` which specifies whether data should be directly clipped or accumulated over some axis before assessing the threshold. A placeholder for allowing the expected variance to be passed has also been added, but not piped through (i.e. you can't specify this on the CLI yet).
Codecov ReportAll modified and coverable lines are covered by tests ✅
Additional details and impacted files@@ Coverage Diff @@
## main #927 +/- ##
==========================================
- Coverage 97.16% 97.15% -0.01%
==========================================
Files 23 23
Lines 10638 10692 +54
==========================================
+ Hits 10336 10388 +52
- Misses 302 304 +2
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Sentry. |
scripts/lstbin_simple.py
Outdated
| @@ -60,6 +60,8 @@ | |||
|
|
|||
| kwargs['output_flagged'] = not kwargs.pop('no_flagged_mode') | |||
| kwargs['output_inpainted'] = kwargs.pop("do_inpaint_mode") | |||
| kwargs['sigma_clip_subbands'] = [int(b) for b in kwargs["sigma_clip_subbands"].split(",")] | |||
There was a problem hiding this comment.
Has this been added to the argparser?
Also, this raises an interesting question about the best way to specify subbands from the commandline and/or from a TOML.
I'm using this format, which is in MHz. It's similar to something Aaron EW did for H4C, though I think he used channels. It's a lot more human readable, but it's a bit clunky:
There was a problem hiding this comment.
Yes, it's been added to the argparser. The format I'm using is like:
--sigma-clip-subbands 0,150,1000
This assumes that every channel belongs to a subband, so you just specify the edges. I could make it more general, using the tilde notation, but I'm not perfectly sure it's necessary in this case?
There was a problem hiding this comment.
I'd like to use the tilde notation. It's more standard and it's not necessarily true that every channel belongs to a subband (e.g. FM).
There was a problem hiding this comment.
OK, I've done this now.
This adds several options to a new
sigma_clip()function, includingclip_typewhich specifies whether data should be directly clipped or accumulated over some axis before assessing the threshold. A placeholder for allowing the expected variance to be passed has also been added, but not piped through (i.e. you can't specify this on the CLI yet).The reason that the expected variance can't be piped through easily is that there's no obvious way of getting the auto spectra that works for both redundantly averaged and non-averaged data input.