Automatic marginalization in PG

[devmotion]

Some of my colleagues just published a paper in which they use Birch (another PPL) to automatically marginalize out the parameters from the state update in PG (with and without ancestor sampling). Maybe that could be done in Turing as well?

Reference

Anna Wigren, Riccardo Sven Risuleo, Lawrence Murray, Fredrik Lindsten: Parameter elimination in particle Gibbs sampling. https://arxiv.org/abs/1910.14145

[yebai]

Certainly - I think it is a good idea to improve the vanilla PG sampler with AS, looking-ahead proposal and marginalisation when possible.

Actually, we talked about separating the PG (and other particle samplers) into a standalone package called AdvancedPS.jl (like AdvancedHMC.jl, and AdvancedMH.jl), so that

1. sampling algorithms are decoupled from Turing's DSL
2. provide better support for algorithmic research on sampling algorithms.

[yebai]

Ps. I like Birch a lot, it would be great if we can make Turing and Birch compatible in the future, e.g. sharing some sampling algorithms, MCMC diagnostics, and/or create a common DSL.

[yebai]

cc @donhausk

[yebai]

Closing for similar reasons to #40