Instance Shadow Detection (CVPR’ 20)

Tianyu Wang*, Xiaowei Hu*, Qiong Wang, Pheng-Ann Heng, and Chi-Wing Fu (* Joint first authors.)

[openaccess][arXiv] [BibTeX]

News: Our new work on instance shadow detection was accepted in CVPR 2021 as Oral presentation, check here!

Instance shadow detection aims to find shadow instances paired with object instances. We present a dataset, a deep framework, and an evaluation metric to approach this new task. This repo is implemented on Detectron2.

Dependences

• python>=3.6
• torch (tested on 1.3.0+cu100 and 1.12.0+cu113)
• torchvision (tested on 0.4.1+cu100 and 0.13.0+cu113)
• tensorboard
• cython
• jupyter
• scikit-image
• numpy
• opencv-python
• pycocotools

Installation

Install LISA and pysobatools

$ cd InstanceShadowDetection
$ python setup.py install
$ cd PythonAPI
$ python setup.py install

Docker

$ cd InstanceShadowDetection/docker

$ docker build --network=host --tag="instanceshadow" -f ./Dockerfile .

$ docker run --gpus all -it --ipc=host --name=instanceshadow --network=host -v /YOURPATH:/data instanceshadow:latest

(Nvidia-docker)[https://github.com/NVIDIA/nvidia-docker] is needed.

Model, dataset and our results

Please download from Google Drive. Put the model and dataset follow the directory layout below.

.
├── ...
├── dataset
│   ├── SOBA                # put dataset here
├── InstanceShadowDetection # this repo
│   ├── projects
│   │   ├── LISA
│   │   │   ├── output_light
│   │   │   │   ├── last_checkpoint.pth
│   │   │   │   └── ...
│   │   │   └── ...
│   └── ...
└── ...

Demo

$ cd projects/LISA/
$ python demo.py --input ./demo/web-shadow0573.jpg --output ./ --config ./config/LISA_101_FPN_3x_demo.yaml

Train

$ python train_net.py --num-gpus 2 --config-file ./config/LISA_101_FPN_3x.yaml

Evaluation

$ python train_net.py --num-gpus 2 --config-file ./config/LISA_101_FPN_3x.yaml --eval-only --resume
$ python SOAP.py

Visualize

python visualize_json_results.py --ins_input ./output_light/inference/soba_instances_results.json --ass_input ./output_light/inference/soba_association_results.json --output ./output_light/results --dataset soba_cast_shadow_val_full

Citation

If you use LISA, SISS, SOBA, or SOAP, please use the following BibTeX entry.

@InProceedings{Wang_2020_CVPR,
author = {Wang, Tianyu and Hu, Xiaowei and Wang, Qiong and Heng, Pheng-Ann and Fu, Chi-Wing},
title = {Instance Shadow Detection},
booktitle = {IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)},
month = {June},
year = {2020}
}

@InProceedings{Wang_2021_CVPR,
author    = {Wang, Tianyu and Hu, Xiaowei and Fu, Chi-Wing and Heng, Pheng-Ann},
title     = {Single-Stage Instance Shadow Detection With Bidirectional Relation Learning},
booktitle = {Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)},
month     = {June},
Year      = {2021},
pages     = {1-11}
}