Clustering based Point Cloud Representation Learning for 3D Analysis

Clustering based Point Cloud Representation Learning for 3D Analysis
Tuo Feng, Wenguan Wang, Xiaohan Wang, Yi Yang, Qinghua Zheng

This repository is the official implementation of "Clustering based Point Cloud Representation Learning for 3D Analysis".

Requirements

The implementation has been based on SPVNAS, and the installation also follows SPVNAS. The details are as follows:

Recommended Installation

For easy installation, use conda:

conda create -n torch python=3.7
conda activate torch
conda install pytorch torchvision torchaudio cudatoolkit=10.2 -c pytorch
conda install numba opencv
pip install torchpack
pip install --upgrade git+https://github.com/mit-han-lab/torchsparse.git

SemanticKITTI

Please follow the instructions from semantic-kitti to download the SemanticKITTI dataset (both KITTI Odometry dataset and SemanticKITTI labels) and extract all the files in the sequences folder to /dataset/semantic-kitti. You shall see 22 folders 00, 01, …, 21; each with subfolders named velodyne and labels.

Please replace the dataset root in spvnas/configs/semantic_kitti/default.yaml with the actual directory path of your dataset.

Control file and pretrained models

The control file and pretrained models for SPVNAS_10.8M were shared by SPVNAS. And the difference between SPVNAS_12.5M and SPVNAS_10.8M is that SPVNAS_10.8M is trained except sequence 08. As for our implementation, SPVNAS_10.8M and SPVNAS_10.8M + Ours are trained on sequences 00-10 and evaluated on sequence 11-21.

We placed the Control file and SPVNAS_10.8M pretrained model in spvnas/configs/semantic_kitti/spvnas/net.config and SemanticKITTI_val_SPVNAS @65GMACs/init, respectively.

Training

To train the models in the paper, run these commands:

CUDA_VISIBLE_DEVICES=0 python spvnas/train_spvnas108m.py

and

CUDA_VISIBLE_DEVICES=0 python spvnas/train_spvnas108m_cluster.py

Evaluate the pretrained model

To evaluate the pretrained models, run:

CUDA_VISIBLE_DEVICES=0 python spvnas/test_spvnas108m.py

and

CUDA_VISIBLE_DEVICES=0 python spvnas/test_spvnas108m_cluster.py

Results will be stored in ./out folder. Test performance can be evaluated by uploading label results onto the SemanticKITTI competition website here.

Remember to shift label number back to the original dataset format before submitting! Instruction can be found in semantic-kitti-api repo.

python remap_semantic_labels.py -p </your result path> -s test --inverse

Pre-trained Models

Our pretrained models are provided here:

• SPVNAS_10.8M is trained on sequence 00-10.
• SPVNAS_10.8M + Ours is trained on sequence 00-10 with our cluster-analysis based contrastive learning algorithm.

Results

The model achieves the following performance on SemanticKITTI Single-Scan test Challenge:

Model name	mIoU
SPVNAS10.8M	62.3
SPVNAS10.8M + Ours	64.3

• spvnas_10.8M.txt and spvnas_10.8M_cluster.txt are the detailed test results for SPVNAS_10.8M and SPVNAS_10.8M + Ours. They are downloaded from the online server.

Citation

If you find the code useful in your research, please consider citing our paper:

@inproceedings{feng2023clustering,
  title = {Clustering based Point Cloud Representation Learning for 3D Analysis},
  author = {Feng, Tuo and Wang, Wenguan and Wang, Xiaohan and Yang, Yi and Zheng, Qinghua},
  booktitle = {The IEEE International Conference on Computer Vision (ICCV)},  
  year = {2023}
}

Any comments, please email: [email protected].

Acknowledgments

We thank for the opensource codebases: ProtoSeg, self-label, and SPVNAS.