README.md

Paper

This repository provides code for:

Interspecies Knowledge Transfer for Facial Keypoint Detection. Maheen Rashid, Xiuye Gu, Yong Jae Lee. CVPR 2017.

If you find this repo useful please cite our work:

@inproceedings{rashid2017interspecies,
  title={Interspecies Knowledge Transfer for Facial Keypoint Detection},
  author={Rashid, Maheen and Gu, Xiuye and Lee, Yong Jae},
  booktitle={Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition},
  year={2017}
}

For questions contact Maheen Rashid (mhnrashid at ucdavis dot edu)

Getting Started

Download the code from GitHub:

git clone https://github.com/menoRashid/animal_human_kp
cd animal_human_kp

Install Torch. Instructions are here

Install Torch requirements:

• torchx

luarocks install torchx

• npy4th (You may need to checkout commit from 5-10-16)

git clone https://github.com/htwaijry/npy4th.git
cd npy4th
luarocks make

Install Python requirements if needed:

• numpy
• scipy
• matplotlib
• PIL

Install the Spatial Tranformer module provided:

cd stnbhwd-master
luarocks make

It is a modification of the code from Spatial Transformer Network (Jaderberg et al.) and includes a Thin Plate Spline grid generator layer.

Dataset

Download the Horse Dataset (580 MB)

Run the following commands

cd data
unzip <path to data zip file>

Models

To download all the pretrained and untrained models go here (145 MB)

Run the following commands

cd models
unzip <path to models zip file>

Otherwise add the individual models to models/

• Full model for horses with tps warping(36 MB)
• Full model for horses with affine warping(63 MB)
• TPS Warping model for horses(34 MB)
• Affine Warping model for horses(61 MB)
• Keypoint network trained on human faces(2.3 MB)
• Untrained TPS Warping model(23 MB)
• Untrained Affine Warping model(29 MB)

Testing

To test pretrained model run the following commands

cd torch
th test.th -out_dir_images <path to results directory>

after replacing with the path to the folder where you would like the output images to be saved.

A webpage with the results, a text file with the accuracy numbers, and a bar graph, would be in the results directory.

<path to results directory>/results.html
<path to results directory>/stats.txt
<path to results directory>/bar.pdf

Training

The file for training the full model is

torch/train_full_model.th

For details on training run

cd torch
th train_full_model.th -help

To train the model with affine warping uncomment lines 377-378. Currently, all parameters are the parameters used in the paper.

The file for training the warping network is

torch/train_warping_net.th

For details on training run

cd torch
th train_warping_net.th -help

To train the model with affine warping uncomment lines 326-327.