PolyChord convergence issue when using emulators

[ThomasTram]

In the following lines,

montepython_public/montepython/PolyChord.py

Lines 251 to 256 in d3e2c72

	if nslow:
	data.PC_arguments['grade_dims'].append(nslow)
	data.PC_arguments['grade_frac'].append(0.75)
	if nfast:
	data.PC_arguments['grade_dims'].append(nfast)
	data.PC_arguments['grade_frac'].append(0.25)

the PolyChord parameter grade_frac is being set to 0.75 for cosmo parameters and 0.25 for nuisance parameters. These numbers tell PolyChord to spend 75% of the (wallclock) time doing repetitions in the cosmological parameters and 25% of the time doing repetitions in the nuisance parameter space. A test run using OMP_NUM_THREADS=8 resulted in an oversampling of nuisance space of approximately 90 using Planck TTTEEE and Bicep-Keck likelihoods. However, using a neural network emulator from CONNECT, leads to an under-sampling of nuisance parameters of around 0.7. This leads to a large statistical uncertainty on the evidence. Note that grade_frac is not exposed as an input parameter, so it is not possible to change this behaviour as a user.

There are other situations where the current implementation is far from ideal:

• A slow model (or even ΛCDM with OMP_NUM_THREADS=1) would oversample nuisance space by an excessive factor.
• When using PlanckLite, 25% of the time will be spend exploring a single parameter, A_Planck, which is unnecessary and wasteful.
• The evidence obtained depends on the status and speed of the given system which makes it difficult to compare evidences between different runs, even for the same model.

Suggestion for fixes:

1. Avoid oversampling/speed grades completely, i.e. just delete the offending lines.
2. Use that actual oversampling numbers from the MontePython parameter file. The docstring for pyPolyChord suggests that grade_frac can accept a list of oversampling factors instead.

It would be great with some additional input from @williamjameshandley if he could find some time.

[brinckmann]

Hi Thomas,

Thanks for the detailed report! I definitely support at least having the option to manipulate this parameter rather than having it hardcoded, either directly as a flag, or somehow through the oversampling field (how does that work with multiple likelihoods with nuisance parameters, I guess the factor would need to be the same for all?). I don't have the knowledge of whether it can be omitted entirely, we'd need Will to chime in on that.

Best,
Thejs