This is the Code submission accompanying the JMLR Submission "Euler Lagrange Analysis of GANs"
This codebase consists of the TensorFlow2.0 implementation WGAN-FS and WAEFR, along with the baseline comparisons. Additionally, high-resolution counterparts of the figures presented in the paper are also included.
Dependencies can be installed via anaconda. The ELFGAN_GPU.yml
file list the dependencies to setup the GPU system based environment:
GPU accelerated TensorFlow2.0 Environment:
- pip=20.0.2
- python=3.6.10
- cudatoolkit=10.1.243
- cudnn=7.6.5
- pip:
- absl-py==0.9.0
- h5py==2.10.0
- ipython==7.15.0
- ipython-genutils==0.2.0
- matplotlib==3.1.3
- numpy==1.18.1
- scikit-learn==0.22.1
- scipy==1.4.1
- tensorflow-gpu==2.2.0
- tensorboard==2.2.0
- tensorflow-addons
- tensorflow-estimator==2.2.0
- tqdm==4.42.1
- gdown==3.12
If a GPU is unavailable, the CPU only environment can be built with ELFGAN_CPU.yml
. This setting is meant to run evaluation code. While the WGAN-FS codes could potentially be trained on the CPU, WAEFR training is not advisable.
CPU based TensorFlow2.0 Environment:
- pip=20.0.2
- python=3.6.10
- pip:
- absl-py==0.9.0
- h5py==2.10.0
- ipython==7.15.0
- ipython-genutils==0.2.0
- matplotlib==3.1.3
- numpy==1.18.1
- scikit-learn==0.22.1
- scipy==1.4.1
- tensorboard==2.0.2
- tensorflow-addons==0.6.0
- tensorflow-datasets==3.0.1
- tensorflow-estimator==2.0.1
- tensorflow==2.0.0
- tensorflow-probability==0.8.0
- tqdm==4.42.1
- gdown==3.12
Codes were tested locally on the following system configurations:
*SYSTEM 1: Ubuntu 18.04.4LTS
- GPU: 'NVIDIA GeForce GTX 1080'
- RAM: '32GB'
- CPU: 'Intel Core i7-7820HK @2.9GHz x 8'
- CUDA: '10.2'
- NVIDIA_drivers: '440.82'
*SYSTEM 2: macOS Catalina, Version 10.15.6
- GPU: -
- RAM: '16GB'
- CPU: '8-Core Intel Core i9 @2.3GHz'
- CUDA: -
- NVIDIA_drivers: -
The main difference between existing WGAN variants and out WGAN-FS, is the inclusion of a Fourier series solver in place of the gradient descent based optimization of the discriminator network. To highlight this fact, the snippets of code associated with the Fourier Solver (FS) are described below:
The Fourier series network, as described in Figure 6 of the main manuscript is implemented as follows. The first layer, consisting of static weights, is implemented as below:
inputs = tf.keras.Input(shape=(latent_dims,))
w0_nt_x = tf.keras.layers.Dense(L, activation=None, use_bias = False)(inputs)
w0_nt_x2 = tf.math.scalar_mul(2., w0_nt_x)
cos_terms = tf.keras.layers.Activation( activation = tf.math.cos)(w0_nt_x)
sin_terms = tf.keras.layers.Activation( activation = tf.math.sin)(w0_nt_x)
cos2_terms = tf.keras.layers.Activation( activation = tf.math.cos)(w0_nt_x2)
discriminator_model_A = tf.keras.Model(inputs=inputs, outputs= [inputs, cos_terms, sin_terms, cos2_terms])
The weight layers w0_nt_x
is the predetermined w_o M
matrix. Subsequently, the Fourier coefficients of p_g
and p_d
are evaluated by the sample estimates of the characteristic function:
_, a_real, a_imag, _ = discriminator_model_A(input_data, training = False)
a_real = tf.expand_dims(tf.reduce_mean( ar, axis = 0), axis = 1)
a_imag = tf.expand_dims(tf.reduce_mean( ai, axis = 0), axis = 1)
Next, the Fourier coefficients D(x)
are evaluated based on the solution to the Poisson's PDE, and are given by:
Gamma_imag = tf.math.divide(tf.constant(1/(w_o**2))*tf.subtract(alpha_imag, beta_imag), n_norm)
Gamma_real = tf.math.divide(tf.constant(1/(w_o**2))*tf.subtract(alpha_real, beta_real), n_norm)
Tau_imag = tf.math.divide(tf.constant(1/(2*(w_o**2)))*tf.square(tf.subtract(alpha_imag, beta_imag)), n_norm)
Tau_real = tf.math.divide(tf.constant(1/(2*(w_o**2)))*tf.square(tf.subtract(alpha_real, beta_real)), n_norm)
Tau_sum = tf.reduce_sum(tf.add(Tau_imag,Tau_real))
Finally, with these terms, the second layer of the discriminator can be defined:
inputs = tf.keras.Input(shape=(self.latent_dims,))
cos_terms = tf.keras.Input(shape=(self.L,))
sin_terms = tf.keras.Input(shape=(self.L,))
cos2_terms = tf.keras.Input(shape=(self.L,))
cos_sum = tf.keras.layers.Dense(1, activation=None, use_bias = True)(cos_terms) # Gamma_real weights
sin_sum = tf.keras.layers.Dense(1, activation=None, use_bias = False)(sin_terms) # Gamma_imag weights
cos2_c_sum = tf.keras.layers.Dense(1, activation=None, use_bias = False)(cos2_terms) # Tau_real weights
cos2_s_sum = tf.keras.layers.Dense(1, activation=None, use_bias = False)(cos2_terms) # Tau_imag weights
lambda_x_term = tf.keras.layers.Subtract()([cos2_s_sum, cos2_c_sum]) # (tau_imag - tau_real) term
if self.latent_dims == 1:
phi0_x = inputs # 1-D Homogeneous component
else:
phi0_x = tf.divide(tf.reduce_sum(inputs, axis=-1, keepdims=True), latent_dims) # n-D Homogeneous component
if homogeneous_component:
Out = tf.keras.layers.Add()([cos_sum, sin_sum, phi0_x])
else:
Out = tf.keras.layers.Add()([cos_sum, sin_sum])
discriminator_model_B = tf.keras.Model(inputs= [inputs, cos_terms, sin_terms, cos2_terms], outputs=[Out,lambda_x_term])
MNIST and CIFAR-10 are loaded from TensorFlow-Datasets. The CelebA dataset (1.2GB) can be downloaded by running the following code (requires wget
dependency):
python download_celeba.py
Alternatively you can manually download the img_align_celeba
folder from the official CelebA website. Additionally, the list_attr_celeba.csv
file can also be downloaded directly. Both must be saved at ELF_GANs/data/CelebA/
.
Similarly, the Ukiyo-E dataset (1.23 GB) can be downloaded by running the following script (requires the gdown
dependency. It can be installed manually, or is a part of the ELFGAN environment setup):
python download_ukiyoe.py
Alternatively, you can download the official tarball from the Ukiyo-E dataset website and extract the ukiyoe-1024
folder to ELF_GAN/data/UkiyoE/
.
Finally, the SVHN dataset's aligned and cropped 32x32
mat file is used for training. This can be downloaded by running the following script:
python download_svhn.py
Alternatively, you can directly download the train_32x32.mat file (182 MB), and if needed for evaluation, test_32x32.mat file (61.3 MB), from the SVHN website, and placed in ELF_GAN/data/SVHN/
.
The code provides training procedure on synthetic Gaussians for WGAN-FS, along with the baseline WGAN with weight clipping, WGAN-GP, WGAN-LP and WGAN-ALP are included. For image data, the proposed WAEFR, and the WAE-GAN baseline with JS, KL, WGAN-LP and WGAN-ALP cost penalties are provided
- Running
train_*.sh
bash files: The fastest was to train a model is by running the bash files inELF_GANs/bash_files/train/
. To train the Model for a given test case: Code to train eachFigure X Subfig (y)
is present in these files. Uncomment the desired command to train for the associated testcase. For example, to generate the discriminator function plot from Figure 1(a), the associated code can be uncommented intrain_WGANFS.sh
, and the file can be run from theELF_GAN
folder as
bash bash_files/train/train_WGANFS.sh
-
Manually running
gam_main.py
: Aternatively, you can train any model of your choice by runninggan_main.py
with custom flags and modifiers. The list of flags and their defauly values are are defined ingan_main.py
. -
Training on Colab: This code is capable of training models on Google Colab (although it is not optimized for this). For those familiar with the approach, this repository could be cloned to your google drive and steps (1) or (2) could be used for training. CelebA must be downloaded to you current instance on Colab as reading data from GoogleDrive currently causes a Timeout error. Setting the flags
--colab_data 1
,--latex_plot_flag 0
, and--pbar_flag 0
is advisable. Thecolab_data
flag modifies CelebA, Ukiyo-E and SVHN data-handling code to read data from the local folder, rather thanELF_GANs/data/CelebA/
. Additionally, you may need to manually download the SVHN mat file, or the Ukiyo-E tarball to the target GoogleDrive folder. Thelatex_plot_flag
flag removes code dependency on latex for plot labels, since the Colab isntance does not native include this. (Alternatively, you could install texlive_full in your colab instance). Lastly, turning off thepbar_flag
was found to prevent the browser from eating too much RAM when training the model. The .ipynb file for training on Colab will be included with the public release of the paper.
The license is committed to the repository in the project folder as LICENSE.txt
.
Please see the LICENSE.txt
file for full informations.
Siddarth Asokan
Robert Bosch Centre for Cyber Physical Systems
Indian Institute of Science
Bangalore, India
Email: [email protected]