Discrete Energy Discrepancy

This repo contains PyTorch implementation of the paper "Energy-Based Modelling for Discrete and Mixed Data via Heat Equations on Structured Spaces"

by Tobias Schröder, Zijing Ou, Yingzhen Li and Andrew Duncan.

We propose discrete energy discrepancy, a loss function which only requires the evaluation of the energy function at data points and their perturbed counterparts, thus eliminating the need for Markov chain Monte Carlo. We introduce perturbations of the data distribution by simulating a diffusion process on the discrete state space endowed with a graph structure. This allows us to inform the choice of perturbation from the structure of the modelled discrete variable, while the continuous time parameter enables fine-grained control of the perturbation.

Experiments

We provide the code for different settings under different branches:

• The density_estimation branch contains the experiment of discrete density estimation at section 6.1.
• The tabular_synthetic branch contains the experiment of synthetic tabular data modelling at section 6.2.
• The tabular_modelling branch contains the experiment of real-world tabular data modelling at section 6.2.
• The image_modelling branch contains the experiment of image data modelling at section 6.3.

Please also check this repository for the code of continuous energy discrepancy!!!