Semi-automatic summary statistics and sample weighting

[yannikschaelte]

Breaking changes:

• API of the (Adaptive)PNormDistance was altered substantially to allow cutom definition of update indices.
• Internal weighting of samples (should not affect users).

Semi-automatic summary statistics:

• Implement (Adaptive)PNormDistance with the ability to learn summary statistics from simulations.
• Add sumstat submodule for generic mappings (id, trafos), and especially a PredictorSumstat summary statistic that can make use of Predictor objects.
• Add subsetting routines that allow restricting predictor model training samples.
• Add predictor submodule with generic Predictor class and concrete implementations including linear regression, Lasso, Gaussian Process, Neural Network.
• Add InfoWeightedPNormDistance that allows using predictor models to weight data not only by scale, but also by information content.

Changes to internal sample weighting:

• Do not normalize weights of in-memory particles by model; this allows to more easily use the sampling weights and the list of particles for adaptive components (e.g. distance functions)
• Normalization of population to 1 is applied on sample level in the sampler wrapper function
• In the database, normalization is still by sample to not break old db support; would be nicer to also there only normalize by total sum -- requires a db update though.

Changes to internal object instruction from samples:

• Pass sample instead of weighted_sum_stats to distance function. This is because thus the distance can choose on its own what it wants -- all or only accepted particles; distances; weights; parameters; summary statistics.

Visualization:

• Function to plot adaptive distance weights from log file.

[codecov-io]

Codecov Report

Merging #429 (95b3365) into develop (23c08bf) will decrease coverage by 37.57%.
The diff coverage is 36.61%.

@@             Coverage Diff              @@
##           develop     #429       +/-   ##
============================================
- Coverage    87.34%   49.76%   -37.58%     
============================================
  Files          103      107        +4     
  Lines         6098     6561      +463     
============================================
- Hits          5326     3265     -2061     
- Misses         772     3296     +2524     

Impacted Files	Coverage Δ
pyabc/distance/__init__.py	100.00% <ø> (ø)
pyabc/epsilon/base.py	78.57% <ø> (-17.86%)	⬇️
pyabc/epsilon/epsilon.py	72.50% <ø> (-18.75%)	⬇️
pyabc/distance/distance.py	27.34% <18.22%> (-58.28%)	⬇️
pyabc/sumstat/util.py	18.30% <18.30%> (ø)
pyabc/predictor.py	25.75% <25.75%> (ø)
pyabc/sampler/redis_eps/sampler.py	61.32% <34.78%> (-35.19%)	⬇️
pyabc/sumstat/learn.py	35.06% <35.06%> (ø)
pyabc/distance/util.py	40.00% <40.00%> (ø)
pyabc/sumstat/base.py	40.00% <40.00%> (ø)
... and 87 more

---

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 23c08bf...95b3365. Read the comment docs.