Skip to content

jacobwjs/QuantizedNeuralNetworks-Keras-Tensorflow

 
 

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

25 Commits
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

Training Quantized Neural Networks

Introduction

Train your own Quantized Neural Networks (QNN) - networks trained with quantized weights and activations - in Keras / Tensorflow. If you use this code, please cite "B.Moons et al. "Minimum Energy Quantized Neural Networks", Asilomar Conference on Signals, Systems and Computers, 2017". Take a look at our presentation or at the paper on arxiv.

This code is based on a lasagne/theano and a Keras/Tensorflow version of BinaryNet.

Preliminaries

Running this code requires:

  1. Tensorflow
  2. Keras 2.0
  3. pylearn2 + the correct PYLEARN2_DATA_PATH in ./personal_config/shell_source.sh
  4. A GPU with recent versions of CUDA and CUDNN
  5. Correct paths in ./personal_config/shell_source.sh

Make sure your backend='tensorflow' and image_data_format='channels_last' in the ~/.keras/keras.json file.

Training your own QNN

This repo includes toy examples for CIFAR-10 and MNIST. Training can be done by running the following:

./train.sh <config_file> -o <override_parameters>

-o overrides parameters in the <config_file>.

The following parameters are crucial:

Examples

  • This is how to train a 4-bit full qnn on CIFAR-10:

    ./train.sh config_CIFAR-10 -o lr=0.01 wbits=4 abits=4 network_type='full-qnn'

  • This is how to train a qnn with 4-bit weights and floating point activations on CIFAR-10:

    ./train.sh config_CIFAR-10 -o lr=0.01 wbits=4 network_type='qnn'

  • This is how to train a BinaryNet on CIFAR-10:

    ./train.sh config_CIFAR-10 -o lr=0.01 network_type='full-bnn'


The included networks have parametrized sizes and are split into three blocks (A-B-C), each with a number of layers (nl) and a number of filters per layer (nf).

  • This is how to train a small 2-bit network on MNIST:

    ./train.sh config_MNIST -o nla=1 nfa=64 nlb=1 nfb=64 nlc=1 nfc=64 wbits=2 abits=2 network_type='full-qnn'

  • This is how to train a large 8-bit network on CIFAR-10:

    ./train.sh config_CIFAR-10 -o nla=3 nfa=256 nlb=3 nfb=256 nlc=3 nfc=256 wbits=8 abits=8 network_type='full-qnn'

About

Quantized Neural Networks - networks trained for inference at arbitrary low precision.

Resources

License

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published

Languages

  • Python 99.1%
  • Shell 0.9%