Viewpoints And Keypoints

Installation

This code has been tested with Python 3.5, and is not intended to be used with Python 2.7. The required Python packages can be installed by command

pip install -r lib/requirements.txt

Slightly modified version of Caffe is found in the external/caffe directory, and the compilation of it is rather straightforward:

cd external/caffe
mkdir build
cd build
cmake ..
make all

Preprocessing

During the preprocessing phase, the dependencies are resolved by means of Python module Luigi. When the Luigi daemon is started by command

luigid

its web interface is available on http://localhost:8082.

For the vehicle-keypoints task, images with their annotations should be put into the data/veh_keypoints directory. Then, training and validation image sets are created by

python lib/veh_preprocess.py TrainValImageSets

Input Caffe data for coarse-to-fine training is generated by

python lib/veh_preprocess.py CreateVehKeypoints --phase train --heatmap-dims (6,6)
python lib/veh_preprocess.py CreateVehKeypoints --phase val --heatmap-dims (6,6)
python lib/veh_preprocess.py CreateVehKeypoints --phase train --heatmap-dims (12,12)
python lib/veh_preprocess.py CreateVehKeypoints --phase val --heatmap-dims (12,12)

Network training

• Download a pre-trained VGG 16-layer model from http://www.robots.ox.ac.uk/~vgg/software/very_deep/caffe/VGG_ILSVRC_16_layers.caffemodel and put it into the pretrainedModels directory.
• Update the solver files in prototxts/vgg_veh_conv6/solver.prototxt and prototxts/vgg_veh_conv12/solver.prototxt to refer to the locations of the net configuration file as well as update the directory for saving snapshots.
• Update the window file paths in the data layers of prototxts/vgg_veh_conv6/trainTest.prototxt and prototxts/vgg_veh_conv12/trainTest.prototxt to refer to the training and validation files created during the preprocessing phase.
• Train coarse and fine networks. Run the commands below:

./external/caffe/build/tools/caffe train -gpu 0 -solver prototxts/vgg_veh_conv6/solver.prototxt -weights pretrainedModels/VGG_ILSVRC_16_layers.caffemodel
./external/caffe/build/tools/caffe train -gpu 0 -solver prototxts/vgg_veh_conv12/solver.prototxt -weights cachedir/snapshots/vgg_veh_conv6_iter_70000.caffemodel

Here, vgg_veh_conv6_iter_70000.caffemodel is the final model of coarse network, from which the training of fine network starts. The GPU device ID is set through the -gpu option.

Prediction

Scenario lib/veh_predict.py demonstrates how to localize the vehicle-keypoints positions with our model. It outputs the percentage of correctly predicted keypoints on the validation image set. (A keypoint is predicted correctly if its predicted and ground-truth positions fall into the same square among the total of 12x12 squares splitting the bounding box of the vehicle.)