[CodeCamp2023-503] Add the DDQ algorithm to mmdetection

[jiongjiongli]

Task: CodeCamp2023-503

Motivation

Refer to the open source code DDQ and add support for training this model in mmdetection.

Modification

1. Added DDQ DETR config files:
   • configs/ddq_detr/ddq-detr-4scale_r50_8xb2-12e_coco.py
   • configs/ddq_detr/ddq-detr-5scale_r50_8xb2-12e_coco.py
   • configs/ddq_detr/ddq-detr-4scale_swinl_8xb2-30e_coco.py
2. Added classes:
   • DDQDETR
   • DDQDETRHead
   • DDQTransformerDecoder
   • DDQAuxLoss
   • TopkHungarianAssigner

Use cases

Installation

module load python/3.8.12-gcc-4.8.5-jbm
module load cuda/11.1.0-gcc-4.8.5-67q
module load gcc/9.3.0-gcc-4.8.5-bxl

mkdir ~/ddq/new
cd ~/ddq/new/

git clone -b jiongjiongli/ddq_detr https://github.com/jiongjiongli/mmdetection.git mmdetection

cd ~/ddq/new/mmdetection/
python -m venv  ~/ddq/new/detr_py38
source ~/ddq/new/detr_py38/bin/activate

pip install --upgrade pip
pip install cython numpy \
matplotlib pycocotools scipy shapely six terminaltables \
cityscapesscripts imagecorruptions scikit-learn \
pytest-runner \
ninja psutil \
addict numpy packaging Pillow pyyaml yapf coverage lmdb onnx onnxoptimizer onnxruntime pytest PyTurboJPEG scipy tifffile \
termcolor rich \
dadaptation lion-pytorch parameterized \
ipython

pip install torch==1.9.0+cu111 torchvision==0.10.0+cu111 -f https://download.pytorch.org/whl/torch_stable.html
pip install mmengine==0.7.1
pip install mmcv==2.0.0rc4 -f https://download.openmmlab.com/mmcv/dist/cu111/torch1.9/index.html

cd ~/ddq/new/mmdetection/
pip install -v -e .

Train

module load python/3.8.12-gcc-4.8.5-jbm
module load cuda/11.1.0-gcc-4.8.5-67q
module load gcc/9.3.0-gcc-4.8.5-bxl

source ~/ddq/new/detr_py38/bin/activate

cd ~/ddq/new/mmdetection/

PORT=50132 sh tools/train.sh configs/ddq_detr/ddq-detr-4scale_r50_8xb2-12e_coco.py 8 ./exp/ddq_detr

Test

module load python/3.8.12-gcc-4.8.5-jbm
module load cuda/11.1.0-gcc-4.8.5-67q
module load gcc/9.3.0-gcc-4.8.5-bxl

source ~/ddq/new/detr_py38/bin/activate

cd ~/ddq/new/mmdetection/

PORT=50136 sh tools/dist_test.sh  configs/ddq_detr/ddq-detr-4scale_r50_8xb2-12e_coco.py $pretrain_model_path 8

Checklist

1. Pre-commit or other linting tools are used to fix the potential lint issues.
2. The modification is covered by complete unit tests. If not, please add more unit test to ensure the correctness.
3. If the modification has potential influence on downstream projects, this PR should be tested with downstream projects, like MMDet or MMPreTrain.
4. The documentation has been modified accordingly, like docstring or example tutorials.

[CLAassistant]

All committers have signed the CLA.

[PhoenixZ810]

in ~ddq/new/mmdetection/mmdet/models/detectors/ddq_detr.py, Row158:
_, keep_idxs = batched_nms(single_proposals, single_scores, torch.ones(len(single_scores)), self.cache_dict['dqs_cfg'])
there will be error:
RuntimeError:indices should be either on cpu or on the same device as the indexed tensor (cpu)
should change to：
_, keep_idxs = batched_nms(single_proposals, single_scores, torch.ones(len(single_scores).device=single_scores.get_device()), self.cache_dict['dqs_cfg'])

[jiongjiongli]

in ~ddq/new/mmdetection/mmdet/models/detectors/ddq_detr.py, Row158: _, keep_idxs = batched_nms(single_proposals, single_scores, torch.ones(len(single_scores)), self.cache_dict['dqs_cfg']) there will be error: RuntimeError:indices should be either on cpu or on the same device as the indexed tensor (cpu) should change to： _, keep_idxs = batched_nms(single_proposals, single_scores, torch.ones(len(single_scores).device=single_scores.get_device()), self.cache_dict['dqs_cfg'])

Fixed this issue, thanks!

[OpenMMLab-Assistant-004]

Hi @jiongjiongli,

We'd like to express our appreciation for your valuable contributions to the mmdetection. Your efforts have significantly aided in enhancing the project's quality.
It is our pleasure to invite you to join our community thorugh Discord_Special Interest Group (SIG) channel. This is a great place to share your experiences, discuss ideas, and connect with other like-minded people. To become a part of the SIG channel, send a message to the moderator, OpenMMLab, briefly introduce yourself and mention your open-source contributions in the #introductions channel. Our team will gladly facilitate your entry. We eagerly await your presence. Please follow this link to join us: ​https://discord.gg/UjgXkPWNqA.

If you're on WeChat, we'd also love for you to join our community there. Just add our assistant using the WeChat ID: openmmlabwx. When sending the friend request, remember to include the remark "mmsig + Github ID".

Thanks again for your awesome contribution, and we're excited to have you as part of our community!