diff --git a/statistics/readme.md b/statistics/readme.md new file mode 100644 index 0000000..41a49da --- /dev/null +++ b/statistics/readme.md @@ -0,0 +1,23 @@ + +# Overview + +This is for Statistics related Papers including topics like + +- Bayesian Inference +- MCMC +- Variational Bayes + +# Bayesian Inference + +## Variational Bayes Monte Carlo + +- Original Paper [Variational Bayesian Monte Carlo](https://arxiv.org/abs/1810.05558?fbclid=IwAR2Irobgi5jW_RVJL4iRyv0QS_WfmgzKk-KmyoeXSZf3X26eFkz8gaTRxk4) published at [NIPS 2018](https://papers.nips.cc/paper/8043-variational-bayesian-monte-carlo) + +- [Summary on GitHub](vbmc_20190505_1832_1/) ([MathJax Plugin for GitHub](https://chrome.google.com/webstore/detail/mathjax-plugin-for-github/ioemnmodlmafdkllaclgeombjnmnbima) suggested to render Latex browser side) + +- [Summary on Kaggle](https://www.kaggle.com/nicolabernini/papersummary-variational-bayes-monte-carlo) + + + + +Work in progress diff --git a/statistics/vbmc_20190505_1832_1/VBMC_Algo1.PNG b/statistics/vbmc_20190505_1832_1/VBMC_Algo1.PNG new file mode 100644 index 0000000..b14b848 Binary files /dev/null and b/statistics/vbmc_20190505_1832_1/VBMC_Algo1.PNG differ diff --git a/statistics/vbmc_20190505_1832_1/VBMC_Performance1.PNG b/statistics/vbmc_20190505_1832_1/VBMC_Performance1.PNG new file mode 100644 index 0000000..549746f Binary files /dev/null and b/statistics/vbmc_20190505_1832_1/VBMC_Performance1.PNG differ diff --git a/statistics/vbmc_20190505_1832_1/readme.md b/statistics/vbmc_20190505_1832_1/readme.md new file mode 100644 index 0000000..34f860d --- /dev/null +++ b/statistics/vbmc_20190505_1832_1/readme.md @@ -0,0 +1,61 @@ +# Paper Summary - Variational Bayesian Monte Carlo + +Original Paper + +[Variational Bayesian Monte Carlo](https://arxiv.org/abs/1810.05558?fbclid=IwAR2Irobgi5jW_RVJL4iRyv0QS_WfmgzKk-KmyoeXSZf3X26eFkz8gaTRxk4) +- Published in [NIPS 2018](https://papers.nips.cc/paper/8043-variational-bayesian-monte-carlo) + +The **goal** is about performing Bayesian Inference hence computing the Model Posterior given a Dataset + +Sticking to paper notation we have + +- $\mathcal{D}$ : Dataset +- $x \in \mathcal{X}$ : Model Parametrization + +the goal is to compute + +1. Model Posterior + +$$ P(x | \mathcal{D}) = \frac{P(\mathcal{D} | x) P(X)}{P(\mathcal{D})} $$ + + +2. Marginal Likelihood or Model Evidence + +$$ P(\mathcal{D}) = \int_{\mathcal{X}} P(\mathcal{D} | x) P(x) dx $$ + + +Of course in practical cases the closed form computation is impossible because of intractable integral hence **approximation methods** need to be used + +## Approximation Methods + +The cost related to the approximation methods regards the **kind** and **amount of knowledge** that is needed: + +- **gradient** is typically a valuable information but not always accessible, like in the case of Black Box Functions +- **samples** are accessible also in the case of black box functions, however as they have an associated **evaluation cost** the **samples efficiency** of the estimation algorithm is an important aspect + +The Variational Inference Framework is based on the of approximating the True Posterior $P(x | \mathcal{D})$ with a simpler Parametric Function $q_{\theta}(x)$ and to fit its params $\theta$ so to make it as simple as possible to the original one + +This params fitting problem is defined as a Similarity Distance Minimization problem between the 2 PDFs hence using an appropriate PDF Similarity Function (e.g. KL Divergence) + + + +## Goal + +This paper proposes a method working with **Black Box Model Likelihood** (no gradient needed) to perform **Model Posterior Approximation** via **Variational Inference** using +- Active Sampling strategy, to make it a **data efficient** method +- Bayesian Quadrature Framework relying on Gaussian Process as a prior to approximate the Likelihood + + +# Algorithm + +![Algo1](https://raw.githubusercontent.com/NicolaBernini/PapersAnalysis/statistics_20190505_1832_1/statistics/vbmc_20190505_1832_1/VBMC_Algo1.PNG) + + +# Performance + +![Performance1](https://raw.githubusercontent.com/NicolaBernini/PapersAnalysis/statistics_20190505_1832_1/statistics/vbmc_20190505_1832_1/VBMC_Performance1.PNG) + + + +Work in progress +