Differentiable Probabilistic Models
- Arcsine
- Asymmetric Laplace
- Bernoulli
- Beta
- Binomial
- Categorical
- Cauchy
- ChiSquare
- Conditional Model
- Uses a Neural Network to take inputs and create the parameters of a distribution.
- Sampling -> takes a value, runs the network to create the distribution, sample from conditional distribution.
- Log Prob -> Create distribution conditioned on X, take log_prob w.r.t. Z
- Convolution -Sum of component distributions, only allows sampling
- Data Distribution - Randomly sample from a given set of data.
- Dirac Delta
- Dirichlet
- Exponential
- Fisher-Snedecor (F-Distribution)
- Gamma
- Generator
- Uses a latent distribution to sample inputs to a neural network to generate a distribution. Train with the adversarial losses.
- Geometric
- Gumbel Softmax (Relaxed Categorical)
- Gumbel
- Half Cauchy
- Half Normal
- Hyperbolic Secant
- Kumaraswamy
- Langevin
- Adds Langevin Dynamics to sampling methods (see wikipedia)
- Laplace
- Log Cauchy
- Log Laplace
- Log Normal
- Logistic
- Logit Normal
- Negative Binomial
- Normal (Multivariate)
- Normal (Independent)
- Pareto
- Poisson
- Rayleigh
- Relaxed Bernoulli
- Student T
- Transform Distribution
- Composes a list of transforms on a distribution
- Example: Exp(Normal) ~ LogNormal
- Uniform
- Weibull
- Mixture Model
- Static weights to pick from sub-models using a categorical distribution.
- Gumbel Mixture Model
- Uses the Gumbel Softmax as a differentiable approximation to the categorical distribution, allowing mixture weights to be learned.
- Infinite Mixture Model
- Student T written as a Mixture Model.
- Affine
- Exp
- Expm1
- Gumbel
- Identity
- InverseTransform (Inverts a transform)
- Kumaraswamy
- Log
- Logit
- NICE
- Planar
- Power
- Radial
- Reciprocal
- Sigmoid
- SinhArcsinh
- Softplus
- Softsign
- Square
- Tanh
- Weibull
- Divergences
- Cross Entropy
- Perplexity
- Exponential
- Forward KL Divergence
- P Model -> Sampling (rsample)
- Q Model -> PDF Function (log_prob)
- Reverse KL Divergence
- P Model -> PDF Function (log_prob)
- Q Model -> Sampling + PDF Function
- Jensen-Shannon Divergence (JS)
- P Model -> PDF + Sampling
- Q Model -> PDF + Sampling
- Adversarial
- Variational
- ELBO
- Implements SVI with ELBO loss.
- Requires a Conditional Model to learn, in addition to P and Q models.
- ELBO
- Regression
- Linear Regression (Normal)
- L1 Regression (Laplace)
- Ridge Regression (Normal + Normal Prior on weights) (Bayesian Linear Regression)
- Lasso Regression (Normal + Laplace Prior on weights)
- Poisson Regression (Poisson)
- Negative Binomial Regression (Generalized Poisson)
- Classification
- Logistic Regression (Bernoulli)
- Bayesian Logistic Regression (Bernoulli + Normal Prior on weights)
- Softmax Regression (Categorical)
- Gaussian Naive Bayes
- Bernoulli Naive Bayes
- Multinomial Naive Bayes
- Linear Discriminant Analysis (Shared Covariance)
- Gaussian Discriminant Analysis (Multivariate Normal)
- Clustering
- Gaussian Mixture Model
- Decomposition
- Functional PCA
- Dynamic SVD Based (can update projection size)
- EM PPCA
- Variational PPCA
- Unconstrained Matrix Factorization
- Generative Adversarial Networks
- GAN
- MMGAN
- WGAN
- LSGAN
- Variational Auto-Encoder (VAE)
- Ordinal Models (For Ordinal Targets)
- OrdinalLayer (Layer to convert Real Value to Target Prob)
- OrdinalModel (Wraps a Predictor + Ordinal in one module)
- OrdinalLoss (Reminder to use NLLLoss b/c you don't want to softmax the probs)
- Functional CDFs
- exp_cdf -> Exponential Function
- erf_cdf -> Error Function as a CDF
- tanh_cdf -> Uses Hyperbolic Tangent to fake a CDF
- Distribution CDFs
- normal_cdf
- laplace_cdf
- cauchy_cdf
- Approximations (Integration, Expectation, Variance, etc.)
- Inverse Transform Sampling
- Rejection Sampling (and Mode Sampling)
- Metropolis
- Metropolis-Hastings
- Simulated Annealing
- Metropolis-Adjusted Langevin Algorithm (MALA)
- Hamiltonian Monte Carlo (HMC)