Let It Flow: Simultaneous Optimization of 3D Flow and Object Clustering

Patrik Vacek¹    David Hurych²    Karel Zimmermann¹    Tomas Svoboda<sup>1

1</sup>CTU Deparment of Cybernetics    ²Valeo AI   

🎥 1-Minute Summary
>>>>>>> Click Here <<<<<<<

✨ Performance

• 🚀 Scalability: Let It Flow does not require training or supervision as an optimization-based method and works on out-of-distribution data.
• 🌈 Real Scene Flow: Let It Flow performs consistently accross real world benchmark datasets and not only on StereoKITTI. Improvements are observable even on new dynamic/object class error metrics.
• ⚖️ Generalizability: Let It Flow does not have to adjust the parameter config for accurate results on the tested datasets.

Mechanism

Installation

Tested with Python==3.10, PyTorch3d, PyTorch Scatter See install.sh for installation of libraries or run it directly:

conda create -n let-it-flow python=3.10
conda activate let-it-flow

conda install pytorch=1.13.0 torchvision pytorch-cuda=11.6 -c pytorch -c nvidia
conda install -c fvcore -c iopath -c conda-forge fvcore iopath

git clone https://github.com/facebookresearch/pytorch3d.git
cd pytorch3d && python3 -m pip install -e .
cd ..

conda install pytorch-scatter -c pyg

Dataset

Download and unpack the Argoverse2 preprocessed sequence samples into the dataset/argoverse2/ path.

tar -xvf argoverse2.tgz datasets/argoverse2/

Set up the config.yaml to include path to the argoverse2 folder and path to store the results.

Inference

To infer the model on Argoverse2 dataset sequences, please run the following:

python run_optimization.py

The script will store the outputs (and inputs with Ground truth) into the store_path for metric calculation.

Generation of EPE metrics

After the model inference (takes about 2 hours on GPU), generate metrics of Let It Flow (lif) by running:

python compute_metrics.py lif

Result will print the End-Point-Error per class annotated in Argoverse2 and also overall ThreewayEPE.

Citation

If you find our work helpful, please consider citing our paper:

@ARTICLE{Vacek-IEEE-TIV2024,
  author={Vacek, Patrik and Hurych, David and Zimmermann, Karel and Svoboda, Tomáš},
  journal={IEEE Transactions on Intelligent Vehicles}, 
  title={Let-It-Flow: Simultaneous Optimization of 3D Flow and Object Clustering}, 
  year={2024},
  volume={},
  number={},
  pages={1-10},
  keywords={Springs;Rigidity;Point cloud compression;Estimation;Three-dimensional displays;Intelligent vehicles;Europe;Autonomous Driving;LiDAR;3D Scene Flow;Object Clustering},
  doi={10.1109/TIV.2024.3443316}}

License

This work is under the Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License.

Acknowledgement

This work is developed based on the open-source repositories and baselines: KISS-ICP, NSFP, MBNSFP, Chodosh

❤️ We thank to the contributors from the above open-source repositories.