Skip to content

Latest commit

 

History

History
136 lines (92 loc) · 4.9 KB

README.md

File metadata and controls

136 lines (92 loc) · 4.9 KB
Raw

Track 5 - Robust Multi-Modal BEV Detection

Preparation

We implemented BEVFusion as the baseline model for Track 5. The baseline model was trained on the official train split of the nuScenes dataset and evaluated on our robustness probing sets under sensor failure scenarios.

This codebase provides basic instructions for the reproduction of the baseline model in the RoboDrive Challenge.

⚙️ Installation

Kindly refer to GET_STARTED.md to set up environments and download necessary checkpoints.

♨️ Datasets

We use data under the nuScenes train split as the training set and the RoboDrive robustness probing data as the evaluation sets. For training data preparation, kindly refer to NUSCENES_DET.md.

For evaluation data preparation, kindly download the dataset from the following resources:

Type Phase 1 Phase 2
Google Drive link1 or link2 link1 or link2

Uncompress the downloaded dataset and organize the folder structure as follows:

.
├── data
│   ├── nuscenes
│   └── robodrive-sensor
├── configs
├── mmdet3d
└── tools

Next, run the following command to generate the .pkl file for the evaluation sets:

bash tools/create_data.sh

🚙 Hint: You can download our generated .pkl file from this Google Drive link.

The nuscenes folder should end up looking like this:nes folder should be like this:

.
├── basemap
├── can_bus
├── can_bus.zip
├── expansion
├── lidarseg
├── maps
├── nuscenes_infos_train.pkl
├── nuscenes_infos_val.pkl
├── nuScenes-panoptic-v1.0-all
├── prediction
├── robodrive_infos_test.pkl
├── robodrive-v1.0-test
├── samples
├── sweeps
├── v1.0-mini
├── v1.0-test
└── v1.0-trainval

Getting Started

The training and evaluation instructions are summarized as follows.

🚀 Training

Kindly refer to GET_STARTED.md for the details regarding model training.

📊 Evaluation

Simply run the following command to evaluate the trained baseline model on the RoboDrive robustness probing sets:

cd BEVFusion
bash tools/test_corruption.sh

Please rename the generated results_nusc.json file to pred.json and compress it into a .zip file.

Finally, upload the compressed file to Track 5's evaluation server for model evaluation.

🚙 Hint: We provided the baseline submission file at this Google Drive link. Feel free to download and check it for reference and learn how to correctly submit the prediction files to the server.

Customized Dataset

To customize your own dataset, simply build your dataset based on RoboDriveDataset from this line. We simply modified the data path to load the image and LiDAR data.

Baseline Results

Phase 1

Model NDS mAP mATE mASE mAOE mAVE mAAE
BEVFusion 0.4285 0.2448 0.4012 0.2910 0.4928 0.5289 0.2251

Phase 2

Model NDS mAP mATE mASE mAOE mAVE mAAE
BEVFusion 0.3913 0.2159 0.4402 0.2996 0.5297 0.6462 0.2506

References

Kindly cite the corresponding paper(s) once you use the baseline model in this track.

@inproceedings{liu2023bevfusion,
    title = {BEVFusion: Multi-Task Multi-Sensor Fusion with Unified Bird's Eye View Representation},
    author = {Liu, Zhijian and Tang, Haotian and Amini, Alexander and Yang, Xinyu and Mao, Huizi and Rus, Daniela L and Han, Song},
    booktitle = {IEEE International Conference on Robotics and Automation (ICRA)},
    pages = {2774-2781},
    year = {2023}
}