We address the task of language-driven 6-DoF grasp detection in cluttered point clouds. We introduce a novel diffusion model incorporating the new concept of negative prompt guidance learning. Our proposed negative prompt guidance assists in tackling the fine-grained challenge of the language-driven grasp detection task, directing the detection process toward the desired object by steering away from undesired ones.
Create new CONDA environment and install necessary packages
conda create -n l6gd python=3.9
conda activate l6gd
conda install pip
pip install -r requirements.txt
You can request for our HuggingFace dataset at our project page.
To start training the model, run
python3 train.py --config <path to the config file>
Config files are stored in ./config. Remember to change dataset_path in the config files after downloading the dataset. After training, log files and model weights will be saved to ./log.
To detect grasp for test data, run
python3 generate.py --config <path to the config file> --checkpoint <path to the trained model> --data_path <path to the downloaded dataset> --n_sample 64
The detected grasp poses will be saved to an all_data.pkl file in the corresponding log directory.
For evaluation, excecute
python eval.py --data <path to all_data.pkl>
where <path to all_data.pkl> is the path to the file all_data.pkl generated after the step of detecting grasps.
If you find our work interesting or helpful for your research, please consider citing our paper as
@inproceedings{nguyen2024language,
title={Language-driven 6-dof grasp detection using negative prompt guidance},
author={Nguyen, Toan and Vu, Minh Nhat and Huang, Baoru and Vuong, An and Vuong, Quan and Le, Ngan and Vo, Thieu and Nguyen, Anh},
booktitle={ECCV},
year={2024}
}