README

#The code is largely based on [1], [2] and [3]

-Files:

---FS_example.py: Minimal example (20 lines of code) of how to create
                  a FS-LSTM and run one step of it, using the Tensorflow LSTM.
                  
---main.py:       Main file which should be run to replicate the PTB
                  and enwik8 experiments.
                  To run PTB experiment:     python3 main.py --model ptb
                  To run enwik8 experiment:  python3 main.py --model enwik8
                  We only include the PTB dataset due to size limitations.
                  To run the enwik8 experiments the dataset should be 
                  added to "data/enwik8" divided in three files: "train",
                  "valid" and "test"

---config.py:     Hyper-parameters are loaded from here according
                  to which experiment is being performed. All hyper-
                  parameters from Table 3 of the paper can be set
                  using this file.

---reader.py:     Reads training, validation and testing data from
                  the data folder.

---LNLSTM.py:     Layer normalized LSTM with Zoneout.

---aux.py:        File with auxiliary functions for LNLSTM.py.

---FSRNN.py:      Fast-Slow RNN class that can be used with any
                  standard Tensorflow RNN. 



[1] https://github.com/tensorflow/models/tree/master/tutorials/rnn/ptb
[2] https://github.com/pbhatia243/tf-layer-norm
[3] https://github.com/hardmaru/supercell