CCNN: Constrained Convolutional Neural Networks for Weakly Supervised Segmentation

Deepak Pathak, Philipp Krähenbühl, Trevor Darrell

CCNN is a framework for optimizing convolutional neural networks with linear constraints.

• It has been shown to achieve state-of-the-art results on the task of weakly-supervised semantic segmentation.
• It is written in Python and C++, and based on Caffe.
• It has been published at ICCV 2015. It was initially described in the arXiv report.

If you find CCNN useful in your research, please cite:

@inproceedings{pathakICCV15ccnn,
    Author = {Pathak, Deepak and Kr\"ahenb\"uhl, Philipp and Darrell, Trevor},
    Title = {Constrained Convolutional Neural Networks for Weakly Supervised Segmentation},
    Booktitle = {International Conference on Computer Vision ({ICCV})},
    Year = {2015}
}

License

CCNN is released under academic, non-commercial UC Berkeley license (see LICENSE file for details).

Contents

1. Requirements
2. Installation
3. Usage
4. Scripts Information
5. Extra Downloads

1) Requirements

1. Requirements for Caffe and pycaffe (see: Caffe installation instructions)
2. GCC version more than 4.7
3. Boost version more than 1.53 (recommended). If system dependencies give issues, install anaconda dependencies:

$ conda install boost
$ conda install protobuf

4. A good GPU (e.g., Titan, K20, K40, ...) with at least 3G of memory is sufficient.

2) Installation

1. Clone the CCNN repository

# Make sure to clone with --recursive
git clone --recursive https://github.com/pathak22/ccnn.git

2. Build Caffe and pycaffe

• Now follow the Caffe installation instructions here
• Caffe must be built with support for Python layers!
• In your Makefile.config, make sure to have this line uncommented WITH_PYTHON_LAYER := 1
• You can download my Makefile.config for reference.

cd ccnn/caffe-ccnn
# If you have all caffe requirements installed
# and your Makefile.config in place, then simply do:
make -j8 && make pycaffe

3. Now build CCNN

   cd ccnn
   mkdir build
   cd build
   cmake ..
   make -j8

• Note: If anaconda is installed, then python paths may have been messed b/w anaconda and system python.
• I usually run this command :

cmake .. -DBOOST_ROOT=/home/pathak/anaconda -DPYTHON_LIBRARY=/home/pathak/anaconda/lib/libpython2.7.so -DPYTHON_INCLUDE_DIR=/home/pathak/anaconda/include/python2.7/ -DCMAKE_C_COMPILER=gcc-4.8 -DCMAKE_CXX_COMPILER=g++-4.8

• To verify this do : ccmake ./ inside the build folder and manually check the following things : MAKE_CXX_COMPILER, CMAKE_C_COMPILER , PYTHON_EXECUTABLE , PYTHON_INCLUDE_DIR , PYTHON_LIBRARY
• Make sure that cmake doesn't mess the anaconda boost to system boost.

4. Configure path (if needed) in src/user_config.py.

5. (Optional -- I don't do it) If everything runs fine, set CMAKE_BUILD_TYPE using ccmake . to Release. This prevents eigen from checking all assertions etc. and works faster.

3) Usage

Demo CCNN.

cd ccnn
bash ./models/scripts/download_ccnn_models.sh
# This will populate the `ccnn/models/` folder with trained models.
python ./src/demo.py

Train CCNN.

cd ccnn
bash ./models/scripts/download_pretrained_models.sh
# This will populate the `ccnn/models/` folder with imagenet pre-trained models.
python ./src/train.py 2> log.txt

Test CCNN.

cd ccnn
python ./src/test.py  # To test IOU with CRF post-processing
python ./src/test_argmax.py  # To test IOU without CRF

4) Scripts Information

Model Prototxts:

• models/fcn_8s/ : Atrous algorithm based 8-strided VGG, described here.
• models/fcn_32s/ : 32-strided VGG

Configure:

• src/config.py : Set glog-minlevel accordingly to get desired caffe output to terminal

Helper Scripts:

• src/extras/ : These scripts are not needed to run the code. They are simple helper scripts to create data, to prepare pascal test server file, to add pascal cmap to segmentation outputs etc.

5) Extra Downloads

• Pascal VOC Image List: train, val, trainval, test
• Training image-level label indicator files
• Pascal VOC 2012 validation result images
• Pascal VOC 2012 test result images