Skip to content

ContactGen: Generative Contact Modeling for Grasp Generation (ICCV 2023)

Notifications You must be signed in to change notification settings

stevenlsw/contactgen

Repository files navigation


ContactGen: Generative Contact Modeling
for Grasp Generation

Shaowei Liu · Yang Zhou · Jimei Yang · Saurabh Gupta* · Shenlong Wang* ·

Logo

Paper PDF Project Page Youtube Video


This repository contains the pytorch implementation for the paper ContactGen: Generative Contact Modeling for Grasp Generation, ICCV 2023. In this paper, we present a novel object-centric contact representation for high-fidelity and diverse human grasp synthesis of 3D objects.

Installation

  • Clone this repository:
    git clone https://github.com/stevenlsw/contactgen.git
    cd contactgen
  • Install requirements by the following commands:
    conda create -n contactgen python=3.9
    conda activate contactgen
    pip3 install torch # install compatible version
    pip install "git+https://github.com/facebookresearch/pytorch3d.git"
    pip install -r requirements.txt
    cd pointnet_lib && python setup.py install

Demo

  • Generate grasp for toothpaste from sampled ContactGen. results are stored in save_root.

    python demo.py --obj_path assets/toothpaste.ply --n_samples=10 --save_root exp/demo_results
  • Below shows some generated samples for toothpaste:

    1 2 3 4
    Sample-1 Sample-2 Sample-3 Sample-4
  • Visualize the generated grasps in meshlab or by the following command using open3d.

    python vis_grasp.py --hand_path exp/demo_results/grasp_0.obj --obj_path assets/toothpaste.ply

Training & Inference

  • Download the processed GRAB dataset from here and unzip to current directory.

  • Train the model by the following command, experiment logs are stored in work_dir.

    python train.py --work_dir exp
  • Inference using the following command, generated samples are stored in save_root.

    python eval.py --save_root exp/results --checkpoint exp/checkpoint.pt
  • Pretrained models can be found at checkpoint/checkpoint.pt

Citation

If you find our work useful in your research, please cite:

@inproceedings{liu2023contactgen,
  title={ContactGen: Generative Contact Modeling for Grasp Generation},
  author={Liu, Shaowei and Zhou, Yang and Yang, Jimei and Gupta, Saurabh and Wang, Shenlong},
  booktitle={Proceedings of the IEEE/CVF International Conference on Computer Vision},
  year={2023}
}

Acknowledges

We thank:

  • Manopth for ManoLayer implementation
  • GrabNet for training and testing on GRAB dataset
  • ContactOpt for contact map computation
  • HALO for grasp evaluation setup
  • LatentHuman for SDF model implementation