This is a tensorflow implementation of Deformable Convolutional Network in Faster R-CNN fashion. This project is largely built on TFFRCNN, the original implementation in mxnet and many other upstream projects. This repository is only in test phase right now, any contributions helping with bugs and compatibility issues are welcomed.
- TF_Deformable_Net
- TODO
- Requirements: software
- Requirements: Hardware
- Installation (sufficient for the demo)
- Demo
- Training
- Testing
- FAQ
-
Faster R-CNN
-
Trained on ResNet-50
-
More Networks
-
Potential bugs
-
R-FCN
Python 3 (Insufficient compatibility guaranteed for Python 2, though TFFRCNN is built on python 2, I use ilovin's refactored version as base and add some __future__
imports, so any report on compatibility issues welcomed)
Tensorflow==1.4 (Build from source recommended, or else you might need to check ./lib/cuda_config.h
to fill in some options.)
matplotlib
python-opencv
easydict
scikit-image
cython
g++ 4.9(For gcc5 users, you should check make.sh and modify it as told. But there are still chances you might failed with some undefined symbol
message, I haven't figured it out totally. Therefore g++ 4.9 should be the safest.)
Cuda 8.0
Cudnn 6.0
For cuda and cudnn of a different version, please modify ./lib/cuda_config.h
accordingly, but I don't give the guarantee it will work.
Any NVIDIA GPUs with at least 4GB memory should be OK.(Only single gpu mode supported, if you encounter any memory issue on a multi-gpu machine, try export $CUDA_VISIBLE_DEVICE=$(the gpu id you want to use)
).
-
Clone this repository
git clone https://github.com/Zardinality/TF_Deformable_Net.git cd TF_Deformable_Net
-
Before setup all the Cython module and GPU kernels, you might want to set some parameter specific to your GPU model by hand. For example, if you install Tensorflow in binary way, the CUDA_CAPABILITIES of your GPU model should be filled in in the following file:
./lib/cuda_config.h
./lib/make.sh
Cuda and Cudnn version, plus the path to Cuda toolkit should also be filled in
./lib/cuda_config.h
.Note that the build of
roi_pooling
andpsroi_pooling
inmake.sh
are not necessarily needed, becausedeformable_psroi_pooling
can work as them in many ways, also, tensorflow already hastf.image.crop_and_resize
as a faster version forroi_pooling
). -
setup all the Cython module and GPU kernels.
cd ./lib make
After successfully completing basic installation, you'll be ready to run the demo.
To run the demo
cd $TF_Deformable_Net
python ./faster_rcnn/demo.py --model model_path
# e.g. python faster_rcnn/demo.py --model ~/tf_deformable_net/restore_output/Resnet50_iter_145000.ckpt
Also, for many people work on a remote machine, an ipython notebook version demo and test are provided in case visually debug is needed. Except for that all arguments are made in an easydict object, the ipython notebook version demo and test are the same as the script version ones.
The demo performs detection using a ResNet50 network trained for detection on PASCAL VOC 2007. , where the model can be download below. Note that since TF 0.12 the checkpoint must contain more than one file, so you need to unzip the downloaded model to a folder whose path is model_path. Also, when you restore, be sure to set the checkpoint name to be the name you save them(no matter what suffix the checkpoint files now have).
-
Download the training, validation, test data and VOCdevkit
wget http://host.robots.ox.ac.uk/pascal/VOC/voc2007/VOCtrainval_06-Nov-2007.tar wget http://host.robots.ox.ac.uk/pascal/VOC/voc2007/VOCtest_06-Nov-2007.tar wget http://host.robots.ox.ac.uk/pascal/VOC/voc2007/VOCdevkit_08-Jun-2007.tar
-
Extract all of these tars into one directory named
VOCdevkit
tar xvf VOCtrainval_06-Nov-2007.tar tar xvf VOCtest_06-Nov-2007.tar tar xvf VOCdevkit_08-Jun-2007.tar
-
It should have this basic structure
$VOCdevkit/ # development kit $VOCdevkit/VOCcode/ # VOC utility code $VOCdevkit/VOC2007 # image sets, annotations, etc. # ... and several other directories ...
-
Create symlinks for the PASCAL VOC dataset
cd $TF_Deformable_Net/data ln -s $VOCdevkit VOCdevkit2007
-
Download pre-trained model VGG16 or Resnet50 and put it in the path
./data/pretrain_model
-
Run training scripts
cd $TF_Deformable_Net # for resnet-50 python ./faster_rcnn/train_net.py --gpu 0 --weights ./data/pretrain_model/Resnet50.npy --imdb voc_2007_trainval --iters 70000 --cfg ./experiments/cfgs/faster_rcnn_end2end_resnet.yml --network Resnet50_train --set EXP_DIR exp_dir # for vggnet python ./faster_rcnn/train_net.py --gpu 0 --weights ./data/pretrain_model/VGG_imagenet.npy --imdb voc_2007_trainval --iters 70000 --cfg ./experiments/cfgs/faster_rcnn_end2end.yml --network VGGnet_train --set EXP_DIR exp_dir
Or equivalently, edit scripts in
./experiments/scripts
, and start training by running shell scripts. For example:# for resnet-50 ./experiments/scripts/faster_rcnn_voc_resnet.sh # for vggnet ./experiments/scripts/faster_rcnn_vggnet.sh
-
Run a profiling
cd $TF_Deformable_Net # install a visualization tool sudo apt-get install graphviz ./experiments/profiling/run_profiling.sh # generate an image ./experiments/profiling/profile.png
After training, you could run scripts in ./experiments/eval
to evaluate on VOC2007. Or by running ./faster_rcnn/test_net.py
directly.
# for resnet-50
./experiments/eval/voc2007_test_res.sh
# for vggnet
./experiments/scripts/voc2007_test_vgg.sh
-
cudaCheckError() failed : invalid device function.
Check ./lib/make.sh and change the -arch flag accordingly. (Credit to here)
-
undefined symbol: _ZN10tensorflow8internal21CheckOpMessageBuilder9NewStringB5cxx11Ev
If you use gcc5 to build, modify
make.sh
to gcc5 version(simply adding a-D_GLIBCXX_USE_CXX11_ABI=0
flag as pointed out in this issue). If it doesn't work, try reinstall gcc4.9 as following:-
installing gcc-4.9 and g++-4.9
-
changing all nvcc related command in
make.sh
to this form:nvcc -std=c++11 -ccbin=/usr/bin/g++-4.9 -c -o deform_conv.cu.o deform_conv.cu.cc -I $TF_INC -D GOOGLE_CUDA=1 -x cu -Xcompiler -fPIC -L /usr/local/cuda-8.0/lib64/ --expt-relaxed-constexpr
-
and changing all g++ related command in
make.sh
to this form:g++-4.9 -std=c++11 -shared -o deform_conv.so deform_conv.cc deform_conv.cu.o -I TF_INC -fPIC -lcudart -L CUDA_HOME/lib64 -D GOOGLE_CUDA=1 -Wfatal-errors -I $CUDA_HOME/include -D_GLIBCXX_USE_CXX11_ABI=0
-
credit to @cotrane in this issue