Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Poor error reporting from AsymptoticLimits with bad fits #925

Open
andrzejnovak opened this issue Mar 26, 2024 · 0 comments
Open

Poor error reporting from AsymptoticLimits with bad fits #925

andrzejnovak opened this issue Mar 26, 2024 · 0 comments

Comments

@andrzejnovak
Copy link
Contributor

I have a workspace for which the expected limits fits are currently failing kind of silently by just not printing the expected results

╰─ff combine -M AsymptoticLimits -d model_combined.root
 <<< Combine >>>
>>> random number generator seed is 123456
>>> method used is AsymptoticLimits
SimNLL created with 15 channels, 0 generic constraints, 181 fast gaussian constraints, 0 fast poisson constraints, 0 fast group constraints,
SimNLL created with 15 channels, 0 generic constraints, 181 fast gaussian constraints, 0 fast poisson constraints, 0 fast group constraints,
SimNLL created with 15 channels, 0 generic constraints, 181 fast gaussian constraints, 0 fast poisson constraints, 0 fast group constraints,

 -- AsymptoticLimits ( CLs ) --
Observed Limit: r < 0.6503

Done in 16.02 min (cpu), 16.02 min (real)

In the meantime, while looking at available args to debug, and came across --picky Abort on fit failures which does not actually abort, but instead prints the garbage expected limit along with extra errors

╰─ff combine -M AsymptoticLimits -d model_combined.root --picky
 <<< Combine >>>
>>> random number generator seed is 123456
>>> method used is AsymptoticLimits
SimNLL created with 15 channels, 0 generic constraints, 181 fast gaussian constraints, 0 fast poisson constraints, 0 fast group constraints,
SimNLL created with 15 channels, 0 generic constraints, 181 fast gaussian constraints, 0 fast poisson constraints, 0 fast group constraints,
SimNLL created with 15 channels, 0 generic constraints, 181 fast gaussian constraints, 0 fast poisson constraints, 0 fast group constraints,
Minimization failed in an unrecoverable way
Minimization failed in an unrecoverable way

 -- AsymptoticLimits ( CLs ) --
Observed Limit: r < 1.2718
Expected  2.5%: r < 0.0078
Expected 16.0%: r < 0.0087
Expected 50.0%: r < 0.0625
Expected 84.0%: r < 0.0665
Expected 97.5%: r < 0.0670

Done in 18.35 min (cpu), 18.37 min (real)

This would ideally report more meaningful errors.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant