Deep Anomaly Discovery from Unlabeled Videos via Normality Advantage and Self-Paced Refinement

Official implementation of "Deep Anomaly Discovery from Unlabeled Videos via Normality Advantage and Self-Paced Refinement" accepted by CVPR 2022.

1. Dependencies

ubuntu 16.04
cuda 10.1
cudnn 7.6.4
python 3.7
pytorch 1.7.0
torchvision 0.8.0
numpy 1.19.2
opencv-contrib-python 4.1.1.26
mmdetection 2.11.0
mmcv-full 1.3.1

2. Preparing Data

(1) Download and organize VAD datasets: Download UCSDped1/ped2 from official source and complete pixel-wise ground truth of UCSDped1 from the website; Avenue and ShanghaiTech from OneDrive or BaiduNetdisk (code: i9b3, provided by StevenLiuWen) , and ground truth of avenue from official source. Then create a folder named raw_datasets in root directory to place the downloaded datasets. The directory structure should be organized according to tree.txt.

(2) Calculate optical flow: Follow the instructions to compile FlowNet2. Then download and move the pretrained FlowNet 2.0 (FlowNet2_checkpoint.pth.tar) to ./FlowNet2_src/pretrained. Finally run (in PyTorch 0.3.0): python calc_optical_flow.py --dataset_name {UCSDped1, UCSDped2, avenue or ShanghaiTech} --mode {train or test}. As an alternative, you can follow this repository to extract optical flow to avoid the issues related to PyTorch being too old.

(3) Localize foreground objects: Follow the instructions to install mmdetection (v2.11.0). Then download the pretrained object detectors YOLOv3 in this page (Scale 320 and 608), and move them to fore_det/obj_det_checkpoints . Finally run: python extract_bbox.py --dataset_name {UCSDped1, UCSDped2, avenue or ShanghaiTech} --mode {test or merge}.

(4) Extract STCs for training and testing: python extract_stc.py --{extract_training_stc or extract_testing_stc} --dataset_name {UCSDped1, UCSDped2, avenue or ShanghaiTech} --mode {test or merge} .

3. Training

python train.py --dataset_name {UCSDped1, UCSDped2, avenue or ShanghaiTech} --mode {test or merge}

mode: The testing set or the merged set of training and testing set of a VAD dataset. Here the training set that only contains normal videos should not be used alone due to the unsupervised setting (i.e. UVAD).

4. Testing

python test.py --dataset_name {UCSDped1, UCSDped2, avenue or ShanghaiTech} --mode {test or merge}

5. Citation

@InProceedings{Yu_2022_CVPR,
    author    = {Yu, Guang and Wang, Siqi and Cai, Zhiping and Liu, Xinwang and Xu, Chuanfu and Wu, Chengkun},
    title     = {Deep Anomaly Discovery From Unlabeled Videos via Normality Advantage and Self-Paced Refinement},
    booktitle = {Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)},
    month     = {June},
    year      = {2022},
    pages     = {13987-13998}
}