Regional-to-Local Point-Voxel Transformer (RegionPVT)

This is the official PyTorch implementation of the paper Regional-to-Local Point-Voxel Transformer for Large-scale Indoor 3D Point Cloud Semantic Segmentation

Get Started

Environment

1. Install dependencies

pip install -r requirements.txt

2. Compile pointops2

Make sure you have installed gcc and cuda, and nvcc can work (Note that if you install cuda by conda, it won't provide nvcc and you should install cuda manually.). Then, compile and install pointops2 as follows. (We have tested on gcc==9.4.0, pytorch==1.12.1 and cuda==11.3).

# pytorch > 1.12.1 is also OK
cd lib/pointops2
python3 setup.py install
cd ../..

3. Compile MinkowskiEngine

sudo apt-get install libopenblas-dev
cd thirdparty
git clone https://github.com/NVIDIA/MinkowskiEngine.git
cd MinkowskiEngine-master
python3 setup.py install --blas_include_dirs=${CONDA_PREFIX}/include --blas=openblas
cd ../..

4. Compile cuda_ops for LightWeightSelfAttention

cd libs/cuda_ops
pip3 install .
cd ../..

Datasets Preparation

S3DIS

Please refer to https://github.com/yanx27/Pointnet_Pointnet2_pytorch for S3DIS preprocessing. Then modify the data_root entry in the .yaml configuration file.

ScanNetv2

Please refer to https://github.com/dvlab-research/PointGroup for the ScanNetv2 preprocessing. Then change the data_root entry in the .yaml configuration file accordingly.

See data_utils for more details.

Training

S3DIS

python3 train_regionpvt.py --config config/s3dis/s3dis_regionpvt.yaml

ScanNetv2

python3 train_regionpvt.py --config config/scannetv2/scannetv2_regionpvt.yaml

Note: It is normal to see the the results on S3DIS fluctuate between -0.5% and +0.5% mIoU maybe because the size of S3DIS is relatively small, while the results on ScanNetv2 are relatively stable.

Testing

For testing, first change the model_path, save_folder and data_root_val (if applicable) accordingly. Then, run the following command.

python3 test_regionpvt.py --config [YOUR_CONFIG_PATH]

Acknowledgements

Our code is based on the Stratified-Transformer and FastPointTransformer. If you use our model, please consider citing them as well.