Recent Progress in 3D Hand Tasks

A complete paper list for hand pose can be found here. This repo focuses on some subject areas, ideas, and works. Although some papers invloved do not study towards 3D hand scope, they are conceived presenting relavent and inspiring ideas. If there is any mistake or carelessness, please feel free to open an issue or send me email.

• Recent Progress in 3D Hand Tasks
  • Overview
  • Datasets
  • Papers
    • Hand Modeling
      • Forward Kinematics
      • Inverse Kinematics
      • Graph
      • Transformer
      • 2.5D
      • Texture
      • Implicit Shape
      • Neural Rendering on Generation
      • Neural Rendering on Avatar
      • Dynamics
      • Weak Supervision
      • Temporal
    • Hand-Object Interaction
      • Kinematics
      • Graph
      • Implicit Shape
      • Neural Rendering
      • Dynamics
      • Contact
    • Hand-Hand Interaction
      • Kinematics
      • 2.5D
      • Graph
      • Transformer
      • Implicit Shape
      • Neural Rendering
      • Gesture Prediction
      • Manipulation
    • Full Body Reconstruction
      • Forward Kinematics
      • Inverse Kinematics
      • Implicit Shape
      • Neural Rendering
      • Grasp
    • Multi-Modal Motion
      • In Sence
      • Text to Motion
    • Dexterous Robot Hand
      • Dynamics
      • Imitation
      • VLA
  • Tools
    • Model
    • MoCap

Overview

Datasets

This list presents recent datasets with properties of data size, shape annotation, hand-hand interaction (Hand int.), hand-object interaction (Obj. int.), motion sequence, synthetic, and physical attributes (Phy. att.).

Dataset	Link	Size	Shape	Hand int.	Obj. int.	Motion	Synthetic	Phy. att.	Affordacne
GigaHands	link	183M	✅	✅	✅	✅			✅
HOGraspNet	link	1.5M	✅		✅	✅		✅	✅
HInt	link	40.4K		✅	✅
HANDS23	link	257K		✅	✅				✅
HIMO	link	4.1M	✅	✅	✅	✅			✅
Assembly101	link	111M		✅	✅	✅
HMDO	link	21K	✅	✅	✅	✅
HOI4D	link	2.4M	✅		✅	✅			✅
TACO	link	5.2M	✅	✅	✅	✅			✅
KIT	link	4M		✅	✅	✅			✅
OakInk2	link	4M	✅	✅	✅	✅			✅
HandDiffuse12.5M	link	12.5M	✅	✅		✅
Two-hand 500K	link	500K	✅	✅			✅
AffordPose	link	26K	✅		✅		✅		✅
RenderIH	link	1.0M	✅	✅			✅
AssemblyHands	link	3.0M			✅	✅
ARCTIC	link	2.1M	✅	✅	✅	✅		✅
BlurHand	link	155K	✅	✅		✅
DART	link	800K	✅		✅		✅
CBF	link	25K	✅		✅	✅		✅
OakInk	link	50K	✅		✅	✅
Sign-Language	link	22801
MuViHand	link	402K				✅	✅
H2O	link	571K	✅	✅	✅	✅
HanCO	link	860K	✅			✅
H2O	link	5M	✅	✅	✅	✅
H2O-3D	link	62K	✅	✅	✅	✅
GRAB	link	1.6M	✅	✅	✅	✅
DexYCB	link	528K	✅		✅	✅
YoutubeHand	link	47K	✅
YCB-Affordance	link	134K	✅		✅	✅
HO3D	link	77K	✅		✅	✅
ContactPose	link	2.9M	✅	✅	✅	✅
FreiHAND	link	134K	✅
ObMan	link	154K	✅		✅		✅
Interhand2.6M	link	2.6M	✅	✅		✅
RHD	link	44K					✅
H3D	link	22K	✅
MHP	link	80K
MVHM	link	320K	✅				✅
Dexter+Object	link	3K			✅	✅
EgoDexter	link	3K				✅
STB	link	36K	✅			✅
FPHA	link	105K				✅
Tzionas et al.	link	36K		✅		✅
GANerated Hands	link	331K			✅	✅
SynthHands	link	220K			✅		✅

Papers

The below indices are references to the above overview.

Hand Modeling

Forward Kinematics

• [H.FK.15] SiMA-Hand: Boosting 3D Hand-Mesh Reconstruction by Single-to-Multi-View Adaptation. arXiv24. [PDF]
  Yinqiao Wang, Hao Xu, Pheng-Ann Heng, Chi-Wing Fu

• [H.FK.14] HMP: Hand Motion Priors for Pose and Shape Estimation from Video. WACV24. [PDF] [Project] [Code]
  Enes Duran, Muhammed Kocabas, Vasileios Choutas, Zicong Fan, Michael J. Black

• [H.FK.14] Overcoming the Trade-off Between Accuracy and Plausibility in 3D Hand Shape Reconstruction. CVPR23. [PDF]
  Ziwei Yu, Chen Li, Linlin Yang, Xiaoxu Zheng, Michael Bi Mi, Gim Hee Lee, Angela Yao

• [H.FK.13] Identity-Aware Hand Mesh Estimation and Personalization from RGB Images. ECCV22. [Code]
  Deying Kong, Linguang Zhang, Liangjian Chen, Haoyu Ma, Xiangyi Yan, Shanlin Sun, Xingwei Liu, Kun Han, Xiaohui Xie

• [H.FK.12] HandOccNet: Occlusion-Robust 3D Hand Mesh Estimation Network. CVPR22. [PDF] [Code]
  JoonKyu Park, Yeonguk Oh, Gyeongsik Moon, Hongsuk Choi, Kyoung Mu Lee

• [H.FK.11] Locally Aware Piecewise Transformation Fields for 3D Human Mesh Registration. CVPR21. [PDF] [Code]
  Shaofei Wang, Andreas Geiger, Siyu Tang

• [H.FK.10] RealisticHands: A Hybrid Model for 3D Hand Reconstruction. arXiv21. [PDF]
  Michael Seeber, Martin R. Oswald, Roi Poranne

• [H.FK.9] Contrastive Representation Learning for Hand Shape Estimation. arXiv21. [PDF] [Project] [Code]
  Christian Zimmermann, Max Argus, Thomas Brox

• [H.FK.8] Hand Image Understanding via Deep Multi-Task Learning. ICCV21. [PDF]
  Xiong Zhang, Hongsheng Huang, Jianchao Tan, Hongmin Xu, Cheng Yang, Guozhu Peng, Lei Wang, Ji Liu

• [H.FK.7] DeepHandMesh: Weakly-supervised Deep Encoder-Decoder Framework for High-fidelity Hand Mesh Modeling from a Single RGB Image. ECCV20. [PDF] [Project]
  Gyeongsik Moon, Takaaki Shiratori, Kyoung Mu Lee

• [H.FK.6] Knowledge as Priors: Cross-Modal Knowledge Generalization for Datasets without Superior Knowledge. CVPR20. [PDF]
  Long Zhao, Xi Peng, Yuxiao Chen, Mubbasir Kapadia, Dimitris N. Metaxas

• [H.FK.5] FreiHAND: A Dataset for Markerless Capture of Hand Pose and Shape from Single RGB Images. ICCV19. [PDF] [Project]Code]
  Christian Zimmermann, Duygu Ceylan, Jimei Yang, Bryan Russell, Max Argus, Thomas Brox

• [H.FK.4] End-to-end Hand Mesh Recovery from a Monocular RGB Image. ICCV19. [PDF] [Code]
  Xiong Zhang, Qiang Li, Wenbo Zhang, Wen Zheng

• [H.FK.3] 3D Hand Shape and Pose from Images in the Wild. CVPR19. [PDF] [code]
  Adnane Boukhayma, Rodrigo de Bem, Philip H.S. Torr

• [H.FK.2] Pushing the Envelope for RGB-based Dense 3D Hand Pose Estimation via Neural Rendering. CVPR19. [PDF]
  Seungryul Baek, Kwang In Kim, Tae-Kyun Kim

• [H.FK.1] GANerated Hands for Real-Time 3D Hand Tracking from Monocular RGB. CVPR18. [PDF] [Supp] [Project]
  Franziska Mueller, Florian Bernard, Oleksandr Sotnychenko, Dushyant Mehta, Srinath Sridhar, Dan Casas, Christian Theobalt

Inverse Kinematics

• [H.IK.4] HandTailor: Towards High-Precision Monocular 3D Hand Recovery. arXiv21. [PDF] [Code]
  Jun Lv, Wenqiang Xu, Lixin Yang, Sucheng Qian, Chongzhao Mao, Cewu Lu

• [H.IK.3] HybrIK: A Hybrid Analytical-Neural Inverse Kinematics Solution for 3D Human Pose and Shape Estimation. CVPR21. [PDF] [Code]
  Jiefeng Li, Chao Xu, Zhicun Chen, Siyuan Bian, Lixin Yang, Cewu Lu

• [H.IK.2] BiHand: Recovering Hand Mesh with Multi-stage Bisected Hourglass Networks. BMVC20. [PDF] [Code]
  Lixin Yang, Jiasen Li, Wenqiang Xu, Yiqun Diao, Cewu Lu

• [H.IK.1] Monocular Real-time Hand Shape and Motion Capture using Multi-modal Data. CVPR20. [PDF] [Project] [Code]
  Yuxiao Zhou, Marc Habermann, Weipeng Xu, Ikhsanul Habibie, Christian Theobalt, Feng Xu

Graph

• [H.GH.14] HandDGP: Camera-Space Hand Mesh Prediction with Differentiable Global Positioning. ECCV24. [Code] [[Project][https://nianticlabs.github.io/handdgp/]]
  Eugene Valassakis, Guillermo Garcia-Hernando

• [H.GH.13] HandR2N2: Iterative 3D Hand Pose Estimation Using a Residual Recurrent Neural Network. ICCV23. [PDF] [Code]
  Wencan Cheng, Jong Hwan Ko

• [H.GH.12] HaMuCo: Hand Pose Estimation via Multiview Collaborative Self-Supervised Learning. ICCV23. [PDF] [Project]
  [Code]
  Xiaozheng Zheng, Chao Wen, Zhou Xue, Pengfei Ren, Jingyu Wang

• [H.GH.11] High Fidelity 3D Hand Shape Reconstruction via Scalable Graph Frequency Decomposition. CVPR23. [PDF]
  Tianyu Luan, Yuanhao Zhai, Jingjing Meng, Zhong Li, Zhang Chen, Yi Xu, and Junsong Yuan

• [H.GH.10] MobRecon: Mobile-Friendly Hand Mesh Reconstruction from Monocular Image. CVPR22. [PDF] [Code]
  Xingyu Chen, Yufeng Liu, Yajiao Dong, Xiong Zhang, Chongyang Ma, Yanmin Xiong, Yuan Zhang, Xiaoyan Guo

• [H.GH.9] JGR-P2O: Joint Graph Reasoning based Pixel-to-Offset Prediction Network for 3D Hand Pose Estimation from a Single Depth Image. ECCV20 [PDF] [Code]
  Linpu Fang, Xingyan Liu, Li Liu, Hang Xu, Wenxiong Kang

• [H.GH.8] Point2Skeleton: Learning Skeletal Representations from Point Clouds. CVPR21. [PDF] [Code]
  Cheng Lin, Changjian Li, Yuan Liu, Nenglun Chen, Yi-King Choi, Wenping Wang

• [H.GH.7] DC-GNet: Deep Mesh Relation Capturing Graph Convolution Network for 3D Human Shape Reconstruction, MM21. [PDF]
  Shihao Zhou, Mengxi Jiang, Shanshan Cai, Yunqi Lei

• [H.GH.6] MVHM: A Large-Scale Multi-View Hand Mesh Benchmark for Accurate 3D Hand Pose Estimation. WACV21. [PDF]
  Liangjian Chen, Shih-Yao Lin, Yusheng Xie, Yen-Yu Lin, Xiaohui Xie

• [H.GH.5] Camera-Space Hand Mesh Recovery via Semantic Aggregation and Adaptive 2D-1D Registration. CVPR21. [PDF] [Code]
  Xingyu Chen, Yufeng Liu, Chongyang Ma, Jianlong Chang, Huayan Wang, Tian Chen, Xiaoyan Guo, Pengfei Wan, Wen Zheng

• [H.GH.4] Pose2Mesh: Graph Convolutional Network for 3D Human Pose and Mesh Recovery from a 2D Human Pose. ECCV20. [PDF] [Code]
  Hongsuk Choi, Gyeongsik Moon, Kyoung Mu Lee

• [H.GH.3] Weakly-Supervised Mesh-Convolutional Hand Reconstruction in the Wild. CVPR20. [PDF] [Project]
  Dominik Kulon, Riza Alp Güler, Iasonas Kokkinos, Michael Bronstein, Stefanos Zafeiriou

• [H.GH.2] Exploiting Spatial-temporal Relationships for 3D Pose Estimation via Graph Convolutional Networks. ICCV19. [PDF]
  Yujun Cai, Liuhao Ge, Jun Liu, Jianfei Cai, Tat-Jen Cham, Junsong Yuan, and Nadia Magnenat Thalmann

• [H.GH.1] 3D Hand Shape and Pose Estimation from a Single RGB Image. CVPR19. [PDF] [Project] [Code]
  Liuhao Ge, Zhou Ren, Yuncheng Li, Zehao Xue, Yingying Wang, Jianfei Cai, Junsong Yuan

Transformer

• [H.TR.15] HandOS: 3D Hand Reconstruction in One Stage. arXiv24. [PDF] [Project]
  Xingyu Chen, Zhuheng Song, Xiaoke Jiang, Yaoqing Hu, Junzhi Yu, Lei Zhang

• [H.TR.14] Hamba: Single-view 3D Hand Reconstruction with Graph-guided Bi-Scanning Mamba. NuerIPS24. [PDF] [Project] [Code]
  Haoye Dong, Aviral Chharia, Wenbo Gou, Francisco Vicente Carrasco, Fernando De la Torre

• [H.TR.13] SMPLer: Taming Transformers for Monocular 3D Human Shape and Pose Estimation. TPAMI24. [PDF] [Code]
  Xiangyu Xu, Lijuan Liu, Shuicheng Yan

• [H.TR.12] HandDAGT: A Denoising Adaptive Graph Transformer for 3D Hand Pose Estimation. ECCV24. [PDF] Wencan Cheng, Eunji Kim, Jong Hwan Ko

• [H.TR.11] TokenHMR: Advancing Human Mesh Recovery with a Tokenized Pose Representation. CVPR24. [PDF] [Project] [Code]
  Sai Kumar Dwivedi, Yu Sun, Priyanka Patel, Yao Feng, Michael J. Black

• [H.TR.10] Reconstructing Hands in 3D with Transformers. CVPR24. [PDF] [Project] [Code]
  Georgios Pavlakos, Dandan Shan, Ilija Radosavovic, Angjoo Kanazawa, David Fouhey, Jitendra Malik

• [H.TR.9] POEM: Reconstructing Hand in a Point Embedded Multi-view Stereo. CVPR23. [PDF] [Code]
  Lixin Yang, Jian Xu, Licheng Zhong, Xinyu Zhan, Zhicheng Wang, Kejian Wu, Cewu Lu

• [H.TR.8] Recovering 3D Hand Mesh Sequence from a Single Blurry Image: A New Dataset and Temporal Unfolding. CVPR23. [PDF] [Code]
  Yeonguk Oh, JoonKyu Park, Jaeha Kim, Gyeongsik Moon, Kyoung Mu Lee

• [H.TR.7] Deformer: Dynamic Fusion Transformer for Robust Hand Pose Estimation. ICCV23 [PDF]
  Qichen Fu,Xingyu Liu,Ran Xu,Juan Carlos Niebles,Kris M. Kitani

• [H.TR.6] Spatial-Temporal Parallel Transformer for Arm-Hand Dynamic Estimation. CVPR22. [PDF]
  Shuying Liu, Wenbin Wu, Jiaxian Wu, Yue Lin

• [H.TR.5] GraFormer: Graph Convolution Transformer for 3D Pose Estimation. arXiv21. [PDF] [Code]
  Weixi Zhao, Yunjie Tian, Qixiang Ye, Jianbin Jiao and Weiqiang Wang

• [H.TR.4] Mesh Graphormer. ICCV21. [PDF] [Code]
  Kevin Lin, Lijuan Wang, Zicheng Liu

• [H.TR.3] End-to-End Human Pose and Mesh Reconstruction with Transformers. CVPR21. [PDF] [Code]
  Kevin Lin, Lijuan Wang, Zicheng Liu

• [H.TR.2] Hand-Transformer: Non-Autoregressive Structured Modeling for 3D Hand Pose Estimation. ECCV20. [PDF]
  Lin Huang, Jianchao Tan, Ji Liu, and Junsong Yuan

• [H.TR.1] Epipolar Transformers. CVPR20. [PDF] [Code]
  Yihui He, Rui Yan, Shoou-I Yu, Katerina Fragkiadaki

2.5D

• [H.VX.7] Towards Accurate Alignment in Real-time 3D Hand-Mesh Reconstruction. ICCV21. [PDF] [Code]
  Xiao Tang, Tianyu Wang, Chi-Wing Fu

• [H.VX.6] Weakly-Supervised 3D Human Pose Learning via Multi-view Images in the Wild. CVPR20. [PDF]
  Umar Iqbal, Pavlo Molchanov, Jan Kautz

• [H.VX.5] HandVoxNet++: 3D Hand Shape and Pose Estimation using Voxel-Based Neural Networks. arXiv21. [PDF]
  Jameel Malik, Soshi Shimada, Ahmed Elhayek, Sk Aziz Ali, Christian Theobalt, Vladislav Golyanik, Didier Stricker

• [H.VX.4] Exploiting Learnable Joint Groups for Hand Pose Estimation. AAAI21. [PDF] [Code] Moran Li, Yuan Gao, Nong Sang

• [H.VX.3] I2L-MeshNet: Image-to-Lixel Prediction Network for Accurate 3D Human Pose and Mesh Estimation from a Single RGB Image. ECCV20. [PDF] [Code]
  Gyeongsik Moon, Kyoung Mu Lee

• [H.VX.2] HandVoxNet: Deep Voxel-Based Network for 3D Hand Shape and Pose Estimation from a Single Depth Map. CVPR20. [PDF]
  Jameel Malik, Ibrahim Abdelaziz, Ahmed Elhayek, Soshi Shimada, Sk Aziz Ali, Vladislav Golyanik, Christian Theobalt, Didier Stricker

• [H.VX.1] Hand Pose Estimation via Latent 2.5D Heatmap Regression. ECCV18. [PDF]
  Umar Iqbal, Pavlo Molchanov, Thomas Breuel, Juergen Gall, Jan Kautz

Texture

• [H.UV.6] A Probabilistic Attention Model with Occlusion-aware Texture Regression for 3D Hand Reconstruction from a Single RGB Image. CVPR23. [PDF] [Code]
  Zheheng Jiang, Hossein Rahmani, Sue Black, Bryan M. Williams

• [H.UV.5] Handy: Towards a high fidelity 3D hand shape and appearance model. CVPR23. [PDF] [Code]
  Rolandos Potamias Potamias, Stylianos Ploumpis, Stylianos Moschoglou, Vasileios Triantafyllou, Stefanos Zafeiriou

• [H.UV.4] Semi-supervised Hand Appearance Recovery via Structure Disentanglement and Dual Adversarial Discrimination. CVPR23. [PDF]
  Zimeng Zhao, Binghui Zuo, Zhiyu Long, Yangang Wang

• [H.UV.3] DART: Articulated Hand Model with Diverse Accessories and Rich Textures. NeurIPS22. [PDF] [Project]
  Daiheng Gao, Yuliang Xiu, Kailin Li, Lixin Yang, Feng Wang, Peng Zhang, Bang Zhang, Cewu Lu, Ping Tan

• [H.UV.2] I2UV-HandNet: Image-to-UV Prediction Network for Accurate and High-fidelity 3D Hand Mesh Modeling. ICCV21. [PDF]
  Ping Chen, Yujin Chen, Dong Yang, Fangyin Wu, Qin Li, Qingpei Xia, Yong Tan

• [H.UV.1] HTML: A Parametric Hand Texture Model for 3D Hand Reconstruction and Personalization. ECCV20. [PDF] [Project]
  Neng Qian, Jiayi Wang, Franziska Mueller, Florian Bernard, Vladislav Golyanik, Christian Theobalt

Implicit Shape

• [H.SP.7] PHRIT: Parametric Hand Representation with Implicit Template. ICCV23. [PDF]
  Zhisheng Huang, Yujin Chen, Di Kang, Jinlu Zhang, Zhigang Tu

• [H.SP.6] Neural Voting Field for Camera-Space 3D Hand Pose Estimation. CVPR23. [PDF] [Project]
  Lin Huang, Chung-Ching Lin, Kevin Lin, Lin Liang, Lijuan Wang, Junsong Yuan, Zicheng Liu

• [H.SP.5] COAP: Compositional Articulated Occupancy of People. CVPR22. [PDF] [Project] [Code]
  Marko Mihajlovic, Shunsuke Saito, Aayush Bansal, Michael Zollhoefer, Siyu Tang

• [H.SP.4] A Skeleton-Driven Neural Occupancy Representation for Articulated Hands. 3DV21. [PDF]
  Korrawe Karunratanakul, Adrian Spurr, Zicong Fan, Otmar Hilliges, Siyu Tang

• [H.SP.3] SNARF: Differentiable Forward Skinning for Animating Non-Rigid Neural Implicit Shapes. ICCV21. [PDF] [Code] [Project]
  Xu Chen, Yufeng Zheng, Michael J. Black, Otmar Hilliges, Andreas Geiger

• [H.SP.2] LEAP: Learning Articulated Occupancy of People. CVPR21 [PDF] [Code]
  Marko Mihajlovic, Yan Zhang, Michael J. Black, Siyu Tang

• [H.SP.1] NASA: Neural Articulated Shape Approximation. ECCV20. [PDF] [Project]
  Boyang Deng, JP Lewis, Timothy Jeruzalski, Gerard Pons-Moll, Geoffrey Hinton, Mohammad Norouzi, Andrea Tagliasacchi

Neural Rendering on Generation

• [H.GEN.10] FoundHand: Large-Scale Domain-Specific Learning for Controllable Hand Image Generation. [PDF] [Project]
  Kefan Chen, Chaerin Min, Linguang Zhang, Shreyas Hampali, Cem Keskin, Srinath Sridhar

• [H.GEN.9] Human 3Diffusion: Realistic Avatar Creation via Explicit 3D Consistent Diffusion Models. arXiv24. [PDF] [Project] [Code]
  Yuxuan Xue, Xianghui Xie, Riccardo Marin, Gerard Pons-Moll

• [H.GEN.8] AG3D: Learning to Generate 3D Avatars from 2D Image Collections. arXiv23. [PDF] [Project] [Code]
  Zijian Dong, Xu Chen, Jinlong Yang3, Michael J. Black3, Otmar Hilliges1, Andreas Geiger

• [H.GEN.7] Generative Neural Articulated Radiance Fields. NeurIPS22. [PDF] [Project]
  Alexander W. Bergman, Petr Kellnhofer, Wang Yifan, Eric R. Chan, David B. Lindell, Gordon Wetzstein

• [H.GEN.6] 3DHumanGAN: Towards Photo-Realistic 3D-Aware Human Image Generation. arXiv22. [PDF] [Project] [Code]
  Zhuoqian Yang, Shikai Li, Wayne Wu, Bo Dai

• [H.GEN.5] HumanGen: Generating Human Radiance Fields with Explicit Priors. arXiv22. [PDF]
  Suyi Jiang, Haoran Jiang, Ziyu Wang, Haimin Luo, Wenzheng Chen, Lan Xu

• [H.GEN.4] AvatarGen: a 3D Generative Model for Animatable Human Avatars. arXiv22. [PDF] [Code]
  Jianfeng Zhang, Zihang Jiang, Dingdong Yang, Hongyi Xu, Yichun Shi, Guoxian Song, Zhongcong Xu, Xinchao Wang, Jiashi Feng

• [H.GEN.3] Next3D: Generative Neural Texture Rasterization for 3D-Aware Head Avatars. arXiv22. [PDF] [Project] [Code]
  Jingxiang Sun, Xuan Wang, Lizhen Wang, Xiaoyu Li, Yong Zhang, Hongwen Zhang, Yebin Liu

• [H.GEN.2] EVA3D: Compositional 3D Human Generation from 2D Image Collections. ICLR23. [PDF] [Project] [Code]
  Fangzhou Hong, Zhaoxi Chen, Yushi Lan, Liang Pan, Ziwei Liu

• [H.GEN.1] Efficient Geometry-aware 3D Generative Adversarial Networks. CVPR22. [PDF] [Project] [Code]
  Eric R. Chan, Connor Z. Lin, Matthew A. Chan, Koki Nagano, Boxiao Pan, Shalini De Mello, Orazio Gallo, Leonidas Guibas, Jonathan Tremblay, Sameh Khamis, Tero Karras, Gordon Wetzstein

Neural Rendering on Avatar

• [H.AVA.30] Volume Rendering of Human Hand Anatomy. arXiv24. [PDF] Jingtao Huang, Bohan Wang, Zhiyuan Gao, Mianlun Zheng, George Matcuk, Jernej Barbic

• [H.AVA.29] XHand: Real-time Expressive Hand Avatar. arXiv24. [PDF] [Project] [Code]
  Qijun Gan, Zijie Zhou, Jianke Zhu

• [H.AVA.28] PhysAvatar: Learning the Physics of Dressed 3D Avatars from Visual Observations. arXiv24. [PDF] [Project]
  Yang Zheng, Qingqing Zhao, Guandao Yang, Wang Yifan, Donglai Xiang, Florian Dubost, Dmitry Lagun, Thabo Beeler, Federico Tombari, Leonidas Guibas, Gordon Wetzstein

• [H.AVA.27] InstructHumans: Editing Animated 3D Human Textures with Instructions. arXiv24. [PDF] [Project] [Code]
  Jiayin Zhu, Linlin Yang, Angela Yao

• [H.AVA.26] URHand: Universal Relightable Hands. arXiv24. [PDF] [Project]
  Zhaoxi Chen, Gyeongsik Moon, Kaiwen Guo, etal.

• [H.AVA.25] LiveHand: Real-time and Photorealistic Neural Hand Rendering. ICCV23. [PDF] [Project] [Code]
  Akshay Mundra, Mallikarjun B R, Jiayi Wang, Marc Habermann, Christian Theobalt, Mohamed Elgharib

• [H.AVA.24] HumanRF: High-Fidelity Neural Radiance Fields for Humans in Motion. arXiv23. [PDF] Mustafa Işık, Martin Rünz, Markos Georgopoulos, Taras Khakhulin, Jonathan Starck, Lourdes Agapito, Matthias Nießner

• [H.AVA.23] Structured 3D Features for Reconstructing Relightable and Animatable Avatars. CVPR23. [PDF] [Project]
  Enric Corona, Mihai Zanfir, Thiemo Alldieck, Eduard Gabriel Bazavan, Andrei Zanfir, Cristian Sminchisescu

• [H.AVA.22] SHERF: Generalizable Human NeRF from a Single Image. arXiv23. [PDF] [Project]
  Shoukang Hu, Fangzhou Hong, Liang Pan, Haiyi Mei, Lei Yang, Ziwei Liu

• [H.AVA.21] RelightableHands: Efficient Neural Relighting of Articulated Hand Models. CVPR23. [PDF] [Project]
  Shun Iwase, Shunsuke Saito, Tomas Simon, Stephen Lombardi, Timur Bagautdinov, Rohan Joshi, Fabian Prada, Takaaki Shiratori, Yaser Sheikh, Jason Saragih

• [H.AVA.20] HARP: Personalized Hand Reconstruction from a Monocular RGB Video. CVPR23. [PDF]
  Korrawe Karunratanakul, Sergey Prokudin, Otmar Hilliges, Siyu Tang

• [H.AVA.19] One-shot Implicit Animatable Avatars with Model-based Priors. arXiv22. [PDF]
  Yangyi Huang, Hongwei Yi, Weiyang Liu, Haofan Wang, Boxi Wu, Wenxiao Wang, Binbin Lin, Debing Zhang, Deng Cai

• [H.AVA.18] NDF: Neural Deformable Fields for Dynamic Human Modelling. ECCV22. [PDF]
  Ruiqi Zhang, Jie Chen

• [H.AVA.17] ARAH: Animatable Volume Rendering of Articulated Human SDFs. ECCV22. [PDF] [Project] [Code]
  Shaofei Wang, Katja Schwarz, Andreas Geiger, Siyu Tang

• [H.AVA.16] Hand Avatar: Free-Pose Hand Animation and Rendering from Monocular Video. CVPR23. [PDF] [Project]
  Xingyu Chen, Baoyuan Wang Heung-Yeung, Shum

• [H.AVA.15] Authentic Volumetric Avatars From a Phone Scan. SIGGRAPH22. [PDF]
  Chen Cao, Tomas Simon, Jin Kyu Kim, Gabe Schwartz, Michael Zollhoefer, Shunsuke Saito, Stephen Lombardi, Shih-En Wei, Danielle Belko, Shoou-I Yu, Yaser Sheikh, Jason Saragih

• [H.AVA.14] Mixture of volumetric primitives for efficient neural rendering. SIGGRAPH21. [PDF] [Code]
  Stephen Lombardi, Tomas Simon, Gabriel Schwartz, Michael Zollhoefer, Yaser Sheikh, Jason Saragih

• [H.AVA.13] I M Avatar: Implicit Morphable Head Avatars from Videos. CVPR22. [PDF] [Project] [Code]
  Yufeng Zheng, Victoria Fernández Abrevaya, Marcel C. Bühler, Xu Chen, Michael J. Black, Otmar Hilliges

• [H.AVA.12] Neural Head Avatars From Monocular RGB Videos. CVPR22. [PDF] [Project] [Code]
  Philip-William Grassal, Malte Prinzler, Titus Leistner, Carsten Rother, Matthias Nießner, Justus Thies

• [H.AVA.11] LISA: Learning Implicit Shape and Appearance of Hands. CVPR22. [PDF]
  Enric Corona, Tomas Hodan, Minh Vo, Francesc Moreno-Noguer, Chris Sweeney, Richard Newcombe, Lingni Ma

• [H.AVA.10] Surface-Aligned Neural Radiance Fields for Controllable 3D Human Synthesis. CVPR22. [PDF] [Project] [Code]
  Tianhan Xu, Yasuhiro Fujita, Eiichi Matsumoto

• [H.AVA.9] Structured Local Radiance Fields for Human Avatar Modeling. CVPR22. [PDF] [Project]
  Zerong Zheng, Han Huang, Tao Yu, Hongwen Zhang, Yandong Guo, Yebin Liu

• [H.AVA.8] Neural Actor: Neural Free-view Synthesis of Human Actors with Pose Control. SIGGRAPHAsia21. [PDF] [Project]
  Lingjie Liu, Marc Habermann, Viktor Rudnev, Kripasindhu Sarkar, Jiatao Gu, Christian Theobalt

• [H.AVA.7] Editable Free-viewpoint Video Using a Layered Neural Representation. SIGGRAPH21. [PDF] [Project] [Code]
  Jiakai Zhang, Xinhang Liu, Xinyi Ye, Fuqiang Zhao, Yanshun Zhang, Minye Wu, Yingliang Zhang, Lan Xu, Jingyi Yu

• [H.AVA.6] NeuMan: Neural Human Radiance Field from a Single Video. arXiv22. [PDF]
  Wei Jiang, Kwang Moo Yi, Golnoosh Samei, Oncel Tuzel, Anurag Ranjan

• [H.AVA.5] HumanNeRF: Generalizable Neural Human Radiance Field from Sparse Inputs. CVPR22. [PDF] [Project]
  Fuqiang Zhao, Wei Yang, Jiakai Zhang, Pei Lin, Yingliang Zhang, Jingyi Yu, Lan Xu

• [H.AVA.4] HumanNeRF: Free-viewpoint Rendering of Moving People from Monocular Video. CVPR22. [PDF] [Project]
  Chung-Yi Weng, Brian Curless, Pratul P. Srinivasan, Jonathan T. Barron, Ira Kemelmacher-Shlizerman

• [H.AVA.3] Vid2Actor: Free-viewpoint Animatable Person Synthesis from Video in the Wild. arXiv20. [PDF] [Project]
  Chung-Yi Weng, Brian Curless, Ira Kemelmacher-Shlizerman

• [H.AVA.2] Animatable Neural Radiance Fields for Modeling Dynamic Human Bodies. ICCV21 [PDF] [Code] [Project]
  Sida Peng, Junting Dong, Qianqian Wang, Shangzhan Zhang, Qing Shuai, Xiaowei Zhou, Hujun Bao

• [H.AVA.1] Neural Body: Implicit Neural Representations with Structured Latent Codes for Novel View Synthesis of Dynamic Humans. CVPR21. [PDF] [Code] [Project]
  Sida Peng, Yuanqing Zhang, Yinghao Xu, Qianqian Wang, Qing Shuai, Hujun Bao, Xiaowei Zhou

Dynamics

• [H.DN.5] PhysDiff: Physics-Guided Human Motion Diffusion Model. arXiv22. [PDF] [Project]
  Ye Yuan, Jiaming Song, Umar Iqbal, Arash Vahdat, Jan Kautz

• [H.DN.4] Neural MoCon: Neural Motion Control for Physically Plausible Human Motion Capture. CVPR22. [PDF] [Project]
  Buzhen Huang, Liang Pan, Yuan Yang, Jingyi Ju, Yangang Wang

• [H.DN.3] PoseTriplet: Co-evolving 3D Human Pose Estimation, Imitation, and Hallucination under Self-supervision. CVPR22. [PDF] [Code]
  Kehong Gong, Bingbing Li, Jianfeng Zhang, Tao Wang, Jing Huang, Michael Bi Mi, Jiashi Feng, Xinchao Wang

• [H.DN.2] SimPoE: Simulated Character Control for 3D Human Pose Estimation. ICCV21. [PDF] [Project]
  Ye Yuan, Shih-En Wei, Tomas Simon, Kris Kitani, Jason Saragih

• [H.DN.1] Residual Force Control for Agile Human Behavior Imitation and Extended Motion Synthesis. NeurIPS20. [PDF] [Project] [Code]
  Ye Yuan, Kris Kitani

Weak Supervision

• [H.WS.17] CycleHand: Increasing 3D Pose Estimation Ability on In-the-wild Monocular Image through Cyclic Flow. MM22. [PDF]
  Daiheng Gao, Xindi Zhang, Xingyu Chen, Andong Tan, Bang Zhang, Pan Pan, Ping Tan

• [H.WS.16] Image-free Domain Generalization via CLIP for 3D Hand Pose Estimation. arXiv22. [PDF]
  Seongyeong Lee, Hansoo Park, Dong Uk Kim, Jihyeon Kim, Muhammadjon Boboev, Seungryul Baek

• [H.WS.15] Domain Adaptive Hand Keypoint and Pixel Localization in the Wild. ECCV22. [PDF] [Project]
  Takehiko Ohkawa, Yu-Jhe Li, Qichen Fu, Ryosuke Furuta, Kris M. Kitani, and Yoichi Sato

• [H.WS.14] End-to-end Weakly-supervised Multiple 3D Hand Mesh Reconstruction from Single Image. arXiv22. [PDF]
  Jinwei Ren, Jianke Zhu, Jialiang Zhang

• [H.WS.13] Consistent 3D Hand Reconstruction in Video via Self-Supervised Learning. arXiv22. [PDF]
  Zhigang Tu, Zhisheng Huang, Yujin Chen, Di Kang, Linchao Bao, Bisheng Yang, Junsong Yuan

• [H.WS.12] Weakly-supervised 3D Hand Pose Estimation from Monocular RGB Images. ECCV18. [PDF]
  Yujun Cai, Liuhao Ge, Jianfei Cai, and Junsong Yuan

• [H.WS.11] Aligning Latent Spaces for 3D Hand Pose Estimation. ICCV19. [PDF]
  Linlin Yang, Shile Li, Dongheui Lee, Angela Yao

• [H.WS.10] Semi-Supervised 3D Hand Shape and Pose Estimation with Label Propagation. arXiv21. [PDF]
  Samira Kaviani, Amir Rahimi, Richard Hartley

• [H.WS.9] Self-supervised Transfer Learning for Hand Mesh Recovery from Binocular Images. ICCV21. [PDF]
  Zheng Chen, Sihan Wang, Yi Sun, Xiaohong Ma

• [H.WS.8] SO-HandNet: Self-Organizing Network for 3D Hand Pose Estimation with Semi-supervised Learning. ICCV19. [PDF]
  Yujin Chen, Zhigang Tu, Liuhao Ge, Dejun Zhang, Ruizhi Chen, Junsong Yuan

• [H.WS.7] PeCLR: Self-Supervised 3D Hand Pose Estimation from Monocular RGB via Contrastive Learning. ICCV21. [PDF] [Project] [Code]
  Adrian Spurr, Aneesh Dahiya, Xucong Zhang, Xi Wang, Otmar Hilliges

• [H.WS.6] Adversarial Motion Modelling helps Semi-supervised Hand Pose Estimation. arXiv21. [PDF]
  Adrian Spurr, Pavlo Molchanov, Umar Iqbal, Jan Kautz, Otmar Hilliges

• [H.WS.5] SemiHand: Semi-supervised Hand Pose Estimation with Consistency. ICCV21. [PDF]
  Linlin Yang, Shicheng Chen, Angela Yao

• [H.WS.4] S2HAND: Model-based 3D Hand Reconstruction via Self-Supervised Learning. CVPR21. [PDF] [Project] [Code]
  Yujin Chen, Zhigang Tu, Di Kang, Linchao Bao, Ying Zhang, Xuefei Zhe, Ruizhi Chen, Junsong Yuan

• [H.WS.3] Weakly-Supervised 3D Hand Pose Estimation via Biomechanical Constraints. ECCV20. [PDF]
  Adrian Spurr, Umar Iqbal, Pavlo Molchanov, Otmar Hilliges, Jan Kautz

• [H.WS.2] Dual Grid Net: hand mesh vertex regression from single depth maps. ECCV20. [PDF]
  Chengde Wan, Thomas Probst, Luc Van Gool, Angela Yao

• [H.WS.1] Adaptive Wasserstein Hourglass for Weakly Supervised Hand Pose Estimation from Monocular RGB. MM20. [PDF]
  Yumeng Zhang, Li Chen, Yufeng Liu, Junhai Yong, Wen Zheng

Temporal

• [H.TP.10] DGTR: Dual-Branch Graph Transformer Network for 3D Human Mesh Reconstruction from Video. IROS24. [PDF] [Code]
  Tao Tang, Hong Liu, Yingxuan You, Ti Wang, Wenhao Li

• [H.TP.9] Hierarchical Temporal Transformer for 3D Hand Pose Estimation and Action Recognition from Egocentric RGB Videos. CVPR23. [PDF] [Project] [Code]
  Yilin Wen, Hao Pan, Lei Yang, Jia Pan, Taku Komura, Wenping Wang

• [H.TP.8] Capturing Humans in Motion: Temporal-Attentive 3D Human Pose and Shape Estimation from Monocular Video. CVPR22. [PDF] [Project]
  Wen-Li Wei, Jen-Chun Lin, Tyng-Luh Liu, Hong-Yuan Mark Liao

• [H.TP.7] SmoothNet: A Plug-and-Play Network for Refining Human Poses in Videos. arXiv21. [PDF]
  Ailing Zeng, Lei Yang, Xuan Ju, Jiefeng Li, Jianyi Wang, Qiang Xu

• [H.TP.6] Multi-View Video-Based 3D Hand Pose Estimation. arXiv21. [PDF] [Code]
  Leyla Khaleghi, Alireza Sepas Moghaddam, Joshua Marshall, Ali Etemad

• [H.TP.5] TravelNet: Self-supervised Physically Plausible Hand Motion Learning from Monocular Color Images. ICCV21. [PDF]
  Zimeng Zhao, Xi Zhao and Yangang Wang

• [H.TP.4] Beyond Static Features for Temporally Consistent 3D Human Pose and Shape from a Video. CVPR21. [PDF] [Code]
  Hongsuk Choi, Gyeongsik Moon, Ju Yong Chang, Kyoung Mu Lee

• [H.TP.3] Temporal-Aware Self-Supervised Learning for 3D Hand Pose and Mesh Estimation in Videos. WACV21. [PDF]
  Liangjian Chen, Shih-Yao Lin, Yusheng Xie, Yen-Yu Lin, Xiaohui Xie

• [H.TP.2] Adaptive Computationally Efficient Network for Monocular 3D Hand Pose Estimation. ECCV20. [PDF]
  Zhipeng Fan, Jun Liu, Yao Wang

• [H.TP.1] SeqHAND: RGB-Sequence-Based 3D Hand Pose and Shape Estimation. ECCV20. [PDF]
  John Yang, Hyung Jin Chang, Seungeui Lee, Nojun Kwak

[back to top]

Hand-Object Interaction

Kinematics

• [HO.FK.18] FastGrasp: Efficient Grasp Synthesis with Diffusion. 3DV25. [PDF] [Code]
  Xiaofei Wu, Tao Liu, Caoji Li, Yuexin Ma, Yujiao Shi, Xuming He

• [HO.FK.17] HOGraspNet with Full Grasping Taxonomy and Dynamics. ECCV24. [PDF] [Project] [Code]

• [HO.FK.16] Diffusion-Guided Reconstruction of Everyday Hand-Object Interaction Clips. ICCV23. [PDF] [Project] [Code]
  Yufei Ye, Poorvi Hebbar, Abhinav Gupta, Shubham Tulsiani

• [HO.FK.15] Chord: Category-level Hand-held Object Reconstruction via Shape Deformation. ICCV23. [PDF] [Project]
  Kailin Li, Lixin Yang, Haoyu Zhen, Zenan Lin, Xinyu Zhan, Licheng Zhong, Jian Xu, Kejian Wu, Cewu Lu

• [HO.FK.14] Harmonious Feature Learning for Interactive Hand-Object Pose Estimation. CVPR23. [PDF] [Code]
  Zhifeng Lin, Changxing Ding, Huan Yao, Zengsheng Kuang, Shaoli Huang

• [HO.FK.13] H2O: A Benchmark for Visual Human-human Object Handover Analysis. ICCV21. [PDF]
  Ruolin Ye, Wenqiang Xu, Zhendong Xue, Tutian Tang, Yanfeng Wang, Cewu Lu

• [HO.FK.12] ArtiBoost: Boosting Articulated 3D Hand-Object Pose Estimation via Online Exploration and Synthesis. CVPR22. [PDF] [Code]
  Kailin Li, Lixin Yang, Xinyu Zhan, Jun Lv, Wenqiang Xu, Jiefeng Li, Cewu Lu

• [HO.FK.11] OakInk: A Large-scale Knowledge Repository for Understanding Hand-Object Interaction. CVPR22. [PDF] [Code]
  Lixin Yang, Kailin Li Xinyu Zhan, Fei Wu, Anran Xu, Liu Liu, Cewu Lu

• [HO.FK.10] CPF: Learning a Contact Potential Field to Model the Hand-object Interaction. ICCV21. [PDF] [Code]
  Lixin Yang, Xinyu Zhan, Kailin Li, Wenqiang Xu, Jiefeng Li, Cewu Lu

• [HO.FK.9] HO-3D_v3: Improving the Accuracy of Hand-Object Annotations of the HO-3D Dataset. arXiv21. [PDF]
  Shreyas Hampali, Sayan Deb Sarkar, Vincent Lepetit

• [HO.FK.8] Unsupervised Domain Adaptation with Temporal-Consistent Self-Training for 3D Hand-Object Joint Reconstruction. arXiv21. [PDF]
  Mengshi Qi, Edoardo Remelli, Mathieu Salzmann, Pascal Fua

• [HO.FK.7] DexYCB: A Benchmark for Capturing Hand Grasping of Objects. CVPR21. [PDF] [Project] [Code]
  Yu-Wei Chao, Wei Yang, Yu Xiang, Pavlo Molchanov, Ankur Handa, Jonathan Tremblay, Yashraj S. Narang, Karl Van Wyk, Umar Iqbal, Stan Birchfield, Jan Kautz, Dieter Fox

• [HO.TP.6] Towards unconstrained joint hand-object reconstruction from RGB videos. arXiv21. [PDF] [Project] [Code]
  Yana Hasson, Gül Varol, Ivan Laptev, Cordelia Schmid

• [HO.TP.5] Leveraging Photometric Consistency over Time for Sparsely Supervised Hand-Object Reconstruction. CVPR20. [PDF] [Project] [Code]
  Yana Hasson, Bugra Tekin, Federica Bogo, Ivan Laptev, Marc Pollefeys, Cordelia Schmid

• [HO.FK.4] Semi-Supervised 3D Hand-Object Poses Estimation with Interactions in Time. CVPR21. [PDF] [Project] [Code]
  Shaowei Liu, Hanwen Jiang*, Jiarui Xu, Sifei Liu, Xiaolong Wang*

• [HO.FK.3] Weakly-supervised Domain Adaptation via GAN and Mesh Model for Estimating 3D Hand Poses Interacting Objects. CVPR20. [PDF] [Code]
  Seungryul Baek, Kwang In Kim, Tae-Kyun Kim

• [HO.FK.2] HOnnotate: A method for 3D Annotation of Hand and Objects Poses. CVPR20. [PDF] [Project] [Code]
  Shreyas Hampali, Mahdi Rad, Markus Oberweger, Vincent Lepetit

• [HO.FK.1] Learning joint reconstruction of hands and manipulated objects. CVPR19. [PDF] [Project] [Code]
  Yana Hasson, Gül Varol, Dimitris Tzionas, Igor Kalevatykh, Michael J. Black, Ivan Laptev, and Cordelia Schmid

Graph

• [HO.GH.7] Dynamic Hyperbolic Attention Network for Fine Hand-object Reconstruction. ICCV23. [PDF]
  Zhiying Leng, Shun-Cheng Wu, Mahdi Saleh, Antonio Montanaro, Hao Yu, Yin Wang, Nassir Navab, Xiaohui Liang, Federico Tombari

• [HO.GH.7] THOR-Net: End-to-end Graformer-based Realistic Two Hands and Object Reconstruction with Self-supervision. WACV23. [PDF] [Code]
  Ahmed Tawfik Aboukhadra, Jameel Malik, Ahmed Elhayek, Nadia Robertini, Didier Stricker

• [HO.GH.6] Interacting Hand-Object Pose Estimation via Dense Mutual Attention. WACV23. [PDF] [Code]
  Rong Wang Wei Mao Hongdong Li

• [HO.GH.5] Scene Synthesis from Human Motion. SIGGRAPH Asia22. [PDF]
  Sifan Ye, Yixing Wang, Jiaman Li, Dennis Park, C. Karen Liu, Huazhe Xu, Jiajun Wu

• [HO.GH.4] S2Contact: Graph-based Network for 3D Hand-Object Contact Estimation with Semi-Supervised Learning. ECCV22. [PDF] [Project] [Code]
  Tze Ho Elden Tse, Zhongqun Zhang, Kwang In Kim, Ales Leonardis, Feng Zheng, Hyung Jin Chang

• [HO.GH.3] Collaborative Learning for Hand and Object Reconstruction with Attention-guided Graph Convolution. CVPR22. [PDF]
  Tze Ho Elden Tse, Kwang In Kim, Ales Leonardis, Hyung Jin Chang

• [HO.GH.2] H2O: Two Hands Manipulating Objects for First Person Interaction Recognition. ICCV21. [PDF] [Project] [Code]
  Taein Kwon, Bugra Tekin, Jan Stuhmer, Federica Bogo, Marc Pollefeys

• [HO.GH.1] HOPE-Net: A Graph-based Model for Hand-Object Pose Estimation. CVPR20. [PDF] [Code]
  Bardia Doosti, Shujon Naha, Majid Mirbagheri, David Crandall

Implicit Shape

• [HO.SP.4] gSDF: Geometry-Driven Signed Distance Functions for 3D Hand-Object Reconstruction. CVPR23. [PDF] [Project]
  Zerui Chen, Shizhe Chen, Cordelia Schmid, Ivan Laptev

• [HO.SP.3] AlignSDF: Pose-Aligned Signed Distance Fields for Hand-Object Reconstruction. ECCV22. [PDF] [Project] [Code]
  Zerui Chen, Yana Hasson, Cordelia Schmid, Ivan Laptev

• [HO.SP.2] What’s in your hands? 3D Reconstruction of Generic Objects in Hands. CVPR22. [PDF] [Project] [Code]
  Yufei Ye, Abhinav Gupta, Shubham Tulsiani

• [HO.SP.1] Grasping Field: Learning Implicit Representations for Human Grasps. 3DV20. [PDF] [Code]
  Korrawe Karunratanakul, Jinlong Yang, Yan Zhang, Michael Black, Krikamol Muandet, Siyu Tang

Neural Rendering

• [HO.NR.8] DreamHOI: Subject-Driven Generation of 3D Human-Object Interactions with Diffusion Priors. arXiv24. [PDF] [Project] [Code] Thomas Zhu, Ruining Li, Tomas Jakab

• [HO.NR.7] Color-NeuS: Reconstructing Neural Implicit Surfaces with Color. 3DV24. [PDF] [Code] [Project]
  Licheng Zhong, Lixin Yang, Kailin Li, Haoyu Zhen, Mei Han, Cewu Lu

• [HO.NR.6] InterFusion: Text-Driven Generation of 3D Human-Object Interaction. arXiv24. [PDF]
  Sisi Dai, Wenhao Li, Haowen Sun, Haibin Huang, Chongyang Ma, Hui Huang, Kai Xu, Ruizhen Hu

• [HO.NR.5] Affordance Diffusion: Synthesizing Hand-Object Interactions. CVPR23. [PDF] [Project]
  Yufei Ye, Xueting Li, Abhinav Gupta, Shalini De Mello, Stan Birchfield, Jiaming Song, Shubham Tulsiani, Sifei Liu

• [HO.NR.4] Hand-Object Interaction Image Generation. NeurIPS22. [PDF] [Project]
  Hezhen Hu, Weilun Wang, Wengang Zhou, Houqiang Li

• [HO.NR.3] In-Hand 3D Object Scanning from an RGB Sequence. arXiv22. [PDF]
  Shreyas Hampali, Tomas Hodan, Luan Tran, Lingni Ma, Cem Keskin, Vincent Lepetit

• [HO.NR.2] Reconstructing Hand-Held Objects from Monocular Video. SIGGRAPH Asia22. [PDF] [Project]
  Di Huang,Xiaopeng Ji,Xingyi He,Jiaming Sun,Tong He,Qing Shuai,Wanli Ouyang,Xiaowei Zhou

• [HO.NR.1] NeuralHOFusion: Neural Volumetric Rendering Under Human-Object Interactions. CVPR22. [PDF]
  Yuheng Jiang, Suyi Jiang, Guoxing Sun, Zhuo Su, Kaiwen Guo, Minye Wu, Jingyi Yu, Lan Xu

Dynamics

• [HO.DN.7] ArtiGrasp: Physically Plausible Synthesis of Bi-Manual Dexterous Grasping and Articulation. 3DV24. [PDF] [Project] [Code]
  Hui Zhang, Sammy Christen, Zicong Fan, Luocheng Zheng, Jemin Hwangbo, Jie Song, Otmar Hilliges

• [HO.DN.6] InterDiff: Generating 3D Human-Object Interactions with Physics-Informed Diffusion. ICCV23. [PDF] [Project] [Code]
  Sirui Xu, Zhengyuan Li, Yu-Xiong Wang, Liang-Yan Gui

• [HO.DN.5] MACS: Mass Conditioned 3D Hand and Object Motion Synthesis. 3DV24. [PDF] [Project]
  Soshi Shimada, Franziska Mueller, Jan Bednarik, Bardia Doosti, Bernd Bickel, Danhang Tang, Vladislav Golyanik, Jonathan Taylor, Christian Theobalt, Thabo Beeler

• [HO.DN.4] DeepSimHO: Stable Pose Estimation for Hand-Object Interaction via Physics Simulation. NeurIPS23. [PDF] [Code]
  Rong Wang, Wei Mao, Hongdong Li

• [HO.DN.3] D-Grasp: Physically Plausible Dynamic Grasp Synthesis for Hand-Object Interactions. CVPR22. [PDF] [Project]
  Sammy Christen, Muhammed Kocabas, Emre Aksan, Jemin Hwangbo, Jie Song, Otmar Hilliges

• [HO.DN.2] DexMV: Imitation Learning for Dexterous Manipulation from Human Videos. ICCVW21. [PDF] [Project]
  Yuzhe Qin, Yueh-Hua Wu, Shaowei Liu, Hanwen Jiang, Ruihan Yang, Yang Fu, Xiaolong Wang

• [HO.DN.1] Stability-driven Contact Reconstruction From Monocular Color Images. CVPR22. [PDF]
  Zimeng Zhao, Binghui Zuo, Wei Xie, Yangang Wang

Contact

• [HO.Con.19] TOCH: Spatio-Temporal Object-to-Hand Correspondence for Motion Refinement. ECCV22. [PDF] [Project] [Code]
  Keyang Zhou, Bharat Lal Bhatnagar, Jan Eric Lenssen, Gerard Pons-Moll

• [HO.Con.18] CAMS: CAnonicalized Manipulation Spaces for Category-Level Functional Hand-Object Manipulation Synthesis. CVPR23. [PDF] [Project] [Code]
  Juntian Zheng, Lixing Fang, Qingyuan Zheng, Yun Liu, Li Yi

• [HO.Con.17] ContactArt: Learning 3D Interaction Priors for Category-level Articulated Object and Hand Poses Estimation. 3DV24. [PDF] [Project]
  Zehao Zhu, Jiashun Wang, Yuzhe Qin, Deqing Sun, Varun Jampani, Xiaolong Wang

• [HO.Con.16] HMDO: Markerless multi-view hand manipulation capture with deformable objects. Graphical Models23. [PDF] [Code]
  Wei Xie, Zhipeng Yu, Zimeng Zhao, Binghui Zuo, Yangang Wang

• [HO.Con.15] GenHeld: Generating and Editing Handheld Objects. arXiv24. [PDF] [Code]
  Chaerin Min, Srinath Sridhar

• [HO.Con.14] Joint Reconstruction of 3D Human and Object via Contact-Based Refinement Transformer. CVPR24. [PDF] [Code]
  Hyeongjin Nam, Daniel Sungho Jung, Gyeongsik Moon, Kyoung Mu Lee

• [HO.Con.13] HOI-Diff: Text-Driven Synthesis of 3D Human-Object Interactions using Diffusion Models. arXiv23. [PDF] [Project] [Code]
  Xiaogang Peng, Yiming Xie, Zizhao Wu, Varun Jampani, Deqing Sun, Huaizu Jiang

• [HO.Con.12] In-Hand 3D Object Reconstruction from a Monocular RGB Video. AAAI24. [PDF] [Project]
  Shijian Jiang, Qi Ye, Rengan Xie, Yuchi Huo, Xiang Li, Yang Zhou, Jiming Chen

• [HO.Con.11] ContactGen: Generative Contact Modeling for Grasp Generation. ICCV23. [PDF] [Project] [Code]
  Shaowei Liu, Yang Zhou, Jimei Yang, Saurabh Gupta, Shenlong Wang

• [HO.Con.10] Transformer-based Unified Recognition of Two Hands Manipulating Objects. CVPR23. [PDF] Hoseong Cho, Chanwoo Kim, Jihyeon Kim, Seongyeong Lee, Elkhan Ismayilzada, Seungryul Baek

• [HO.Con.9] Visual-Tactile Sensing for In-Hand Object Reconstruction. CVPR23. [PDF] [Project]
  Wenqiang Xu, Zhenjun Yu, Han Xue, Ruolin Ye, Siqiong Yao, Cewu Lu

• [HO.Con.8] UV-Based 3D Hand-Object Reconstruction with Grasp Optimization. BMVC22. [PDF]
  Ziwei Yu,Linlin Yang,You Xie,Ping Chen,Angela Yao

• [HO.Con.7] ContactOpt: Optimizing Contact to Improve Grasps. CVPR21. [PDF] [Code]
  Patrick Grady, Chengcheng Tang, Christopher D. Twigg, Minh Vo, Samarth Brahmbhatt, Charles C. Kemp

• [HO.Con.6] CHORE: Contact, Human and Object REconstruction from a single RGB image. arXiv22. [PDF]
  Xianghui Xie, Bharat Lal Bhatnagar, Gerard Pons-Moll

• [HO.Con.5] Understanding Egocentric Hand-Object Interactions from Hand Pose Estimation. arXiv21. [PDF]
  Yao Lu, Walterio W. Mayol-Cuevas

• [HO.Con.4] Reconstructing Hand-Object Interactions in the Wild. ICCV21. [PDF] [Code]
  Zhe Cao, Ilija Radosavovic*, Angjoo Kanazawa, Jitendra Malik*

• [HO.Con.3] Perceiving 3D Human-Object Spatial Arrangements from a Single Image in the Wild. ECCV20. [PDF] [Code]
  Jason Y. Zhang, Sam Pepose, Hanbyul Joo, Deva Ramanan, Jitendra Malik, Angjoo Kanazawa

• [HO.Con.2] GanHand: Predicting Human Grasp Affordances in Multi-Object Scenes. CVPR20. [PDF] [Code]
  Enric Corona, Albert Pumarola, Guillem Alenya, Francesc Moreno-Noguer, Gregory Rogez

• [HO.Con.1] Hand-Object Contact Consistency Reasoning for Human Grasps Generation. ICCV21. [PDF] [Project]
  Hanwen Jiang, Shaowei Liu, Jiashun Wang, Xiaolong Wang

[back to top]

Hand-Hand Interaction

Kinematics

• [HH.FK.7] Monocular 3D Hand Mesh Recovery via Dual Noise Estimation. AAAI24. [PDF] [Code]
  Hanhui Li, Xiaojian Lin, Xuan Huang, Zejun Yang, Zhisheng Wang, Xiaodan Liang

• [HH.FK.6] 3D Pose Estimation of Two Interacting Hands from a Monocular Event Camera. 3DV24. [PDF] [Project]
  Christen Millerdurai, Diogo Luvizon, Viktor Rudnev, André Jonas3, Jiayi Wang, Christian Theobalt, Vladislav Golyanik

• [HH.FK.6] ACR: Attention Collaboration-based Regressor for Arbitrary Two-Hand Reconstruction. CVPR23. [PDF] [Code]
  Zhengdi Yu,Shaoli Huang,Chen Fang,Toby P. Breckon,Jue Wang

• [HH.FK.5] UmeTrack: Unified multi-view end-to-end hand tracking for VR. SIGGRAPH Asia 2022. [PDF]
  Shangchen Han, Po-chen Wu, Yubo Zhang, Beibei Liu, Linguang Zhang, Zheng Wang, Weiguang Si, Peizhao Zhang, Yujun Cai, Tomas Hodan, Randi Cabezas, Luan Tran, Muzaffer Akbay, Tsz-Ho Yu, Cem Keskin, Robert Wang

• [HH.FK.4] Constraining Dense Hand Surface Tracking with Elasticity. TOG20. [PDF]
  Breannan Smith, Chenglei Wu, He Wen, Patrick Peluse, Yaser Sheikh, Jessica Hodgins, Takaaki Shiratori

• [HH.FK.3] Coherent Reconstruction of Multiple Humans from a Single Image. CVPR20 [PDF] [Project] [Code]
  Wen Jiang, Nikos Kolotouros, Georgios Pavlakos, Xiaowei Zhou, Kostas Daniilidis

• [HH.FK.2] Monocular 3D Reconstruction of Interacting Hands via Collision-Aware Factorized Refinements. 3DV21. [PDF] [Project] [Code]
  Yu Rong, Jingbo Wang, Ziwei Liu, Chen Change Loy

• [HH.FK.1] RGB2Hands: Real-Time Tracking of 3D Hand Interactions from Monocular RGB Video. SIGGRAPHAsia20 [PDF] [Project]
  Jiayi Wang, Franziska Mueller, Florian Bernard, Suzanne Sorli, Oleksandr Sotnychenko, Neng Qian, Miguel A. Otaduy, Dan Casas, and Christian Theobalt

2.5D

• [HH.VX.5] Bringing Inputs to Shared Domains for 3D Interacting Hands Recovery in the Wild. CVPR23. [PDF] [Code]
  Gyeongsik Moon

• [HH.VX.4] End-toend detection and pose estimation of two interacting hands. ICCV21. [PDF]
  Dong Uk Kim, Kwang In Kim, Seungryul Baek

• [HH.VX.3] Learning to Disambiguate Strongly Interacting Hands via Probabilistic Per-pixel Part Segmentation. 3DV21. [PDF]
  Zicong Fan, Adrian Spurr, Muhammed Kocabas, Siyu Tang, Michael J. Black, Otmar Hilliges

• [HH.VX.2] Interacting Two-Hand 3D Pose and Shape Reconstruction from Single Color Image. ICCV21. [PDF] [Project] [Code]
  Baowen Zhang, Yangang Wang, Xiaoming Deng, Yinda Zhang, Ping Tan, Cuixia Ma and Hongan Wang

• [HH.VX.1] InterHand2.6M: A New Large-scale Dataset and Baseline for 3D Single and Interacting Hand Pose Estimation from a Single RGB Image. ECCV20. [PDF] [Project] [Code]
  Gyeongsik Moon, Shoou-i Yu, He Wen, Takaaki Shiratori, Kyoung Mu Lee

Graph

• [HH.GH.2] Decoupled Iterative Refinement Framework for Interacting Hands Reconstruction from a Single RGB Image. ICCV23. [PDF] [Project] [Code]
  Pengfei Ren, Chao Wen, Xiaozheng Zheng, Zhou Xue, Haifeng Sun, Qi Qi, Jingyu Wang, Jianxin Liao

• [HH.GH.1] Interacting Attention Graph for Single Image Two-Hand Reconstruction. CVPR22. [PDF] [Code]
  Mengcheng Li, Liang An, Hongwen Zhang, Lianpeng Wu, Feng Chen, Tao Yu, Yebin Liu

Transformer

• [HH.TR.12] 4DHands: Reconstructing Interactive Hands in 4D with Transformers. arXiv24. [PDF] [Proejct]
  Dixuan Lin, Yuxiang Zhang, Mengcheng Li, Yebin Liu, Wei Jing, Qi Yan, Qianying Wang, Hongwen Zhang

• [HH.TR.11] Extract-and-Adaptation Network for 3D Interacting Hand Mesh Recovery. ICCVW23. [PDF] [Code]
  JoonKyu Park, Daniel Sungho Jung, Gyeongsik Moon, Kyoung Mu Lee

• [HH.TR.10] Spectral Graphormer: Spectral Graph-based Transformer for Egocentric Two-Hand Reconstruction using Multi-View Color Images. ICCV23. [PDF] [Project] [Code]
  Tze Ho Elden Tse, Franziska Mueller, Zhengyang Shen, Danhang Tang, Thabo Beeler, Mingsong Dou, Yinda Zhang, Sasa Petrovic, Hyung Jin Chang, Jonathan Taylor, Bardia Doosti

• [HH.TR.9] Reconstructing Interacting Hands with Interaction Prior from Monocular Images. ICCV23. [PDF] [Project] [Code] Binghui Zuo, Zimeng Zhao, Wenqian Sun, Wei Xie, Zhou Xue, Yangang Wang

• [HH.TR.8] MeMaHand: Exploiting Mesh-Mano Interaction for Single Image Two-Hand Reconstruction. CVPR23. [PDF]
  Congyi Wang, Feida Zhu, Shilei Wen

• [HH.TR.3] A2J-Transformer: Anchor-to-Joint Transformer Network for 3D Interacting Hand Pose Estimation from a Single RGB Image. CVPR23. [PDF] [Code]
  Changlong Jiang, Yang Xiao, Cunlin Wu, Mingyang Zhang, Jinghong Zheng, Zhiguo Cao, Joey Tianyi Zhou

• [HH.TR.2] 3D Interacting Hand Pose Estimation by Hand De-occlusion and Removal. ECCV22. [PDF] [Project]
  Hao Meng, Sheng Jin, Wentao Liu, Chen Qian, Mengxiang Lin, Wanli Ouyang, Ping Luo

• [HH.TR.1] Keypoint Transformer: Solving Joint Identification in Challenging Hands and Object Interactions for Accurate 3D Pose Estimation. CVPR22. [PDF] [Project]
  Shreyas Hampali, Sayan Deb Sarkar, Mahdi Rad, Vincent Lepetit

Implicit Shape

• [HH.SP.2] MultiPly: Reconstruction of Multiple People from Monocular Video in the Wild. CVPR24. [PDF] [Project]
  Zeren Jiang, Chen Guo, Manuel Kaufmann, Tianjian Jiang, Julien Valentin, Otmar Hilliges, Jie Song

• [HH.SP.1] Im2Hands: Learning Attentive Implicit Representation of Interacting Two-Hand Shapes. CVPR23. [PDF] [Project] [Code]
  Jihyun Lee, Minhyuk Sung, Honggyu Choi, Tae-Kyun Kim

Neural Rendering

• [HH.NR.4] Learning Interaction-aware 3D Gaussian Splatting for One-shot Hand Avatars. NeurIPS24. [PDF] [Code]
  Xuan Huang, Hanhui Li, Wanquan Liu, Xiaodan Liang, Yiqiang Yan, Yuhao Cheng, Chengqiang Gao

• [HH.NR.3] 3D Visibility-aware Generalizable Neural Radiance Fields for Interacting Hands. AAAI24 [PDF] [Code]
  Xuan Huang, Hanhui Li, Zejun Yang, Zhisheng Wang, Xiaodan Liang

• [HH.NR.2] HandRefiner: Refining Malformed Hands in Generated Images by Diffusion-based Conditional Inpainting. arXiv23. [PDF] [Code]
  Wenquan Lu, Yufei Xu, Jing Zhang, Chaoyue Wang, Dacheng Tao

• [HH.NR.1] HandNeRF: Neural Radiance Fields for Animatable Interacting Hands. CVPR23. [PDF]
  Zhiyang Guo, Wengang Zhou, Min Wang, Li Li, Houqiang Li

Gesture Prediction

• [HH.GP.13] EMAGE: Towards Unified Holistic Co-Speech Gesture Generation via Expressive Masked Audio Gesture Modeling. CVPR24. [PDF] [Project] [Code]
  Haiyang Liu, Zihao Zhu, Giorgio Becherini, Yichen Peng, Mingyang Su, You Zhou, Xuefei Zhe, Naoya Iwamoto, Bo Zheng, Michael J. Black

• [HH.GP.12] BEAT: A Large-Scale Semantic and Emotional Multi-Modal Dataset for Conversational Gestures Synthesis. ECCV22. [PDF] [Project] [Code]
  Haiyang Liu, Zihao Zhu, Naoya Iwamoto, Yichen Peng, Zhengqing Li, You Zhou, Elif Bozkurt, Bo Zheng

• [HH.GP.11] Generating Holistic 3D Human Motion from Speech. CVPR23. [PDF] [Project] [Code]
  Hongwei Yi, Hualin Liang, Yifei Liu, Qiong Cao, Yandong Wen, Timo Bolkart, Dacheng Tao, Michael J. Black

• [HH.GP.10] LivelySpeaker: Towards Semantic-Aware Co-Speech Gesture Generation. ICCV23. [PDF] [Code]
  Yihao Zhi, Xiaodong Cun, Xuelin Chen, Xi Shen, Wen Guo, Shaoli Huang, Shenghua Gao

• [HH.GP.9] From Audio to Photoreal Embodiment: Synthesizing Humans in Conversations. arXiv23. [PDF] [Project] [Code]
  Evonne Ng, Javier Romero, Timur Bagautdinov, Shaojie Bai, Trevor Darrell, Angjoo Kanazawa, Alexander Richard

• [HH.GP.8] Taming Diffusion Models for Audio-Driven Co-Speech Gesture Generation. CVPR23. [PDF] [Code]
  Lingting Zhu, Xian Liu, Xuanyu Liu, Rui Qian, Ziwei Liu, Lequan Yu

• [HH.GP.7] Learning Hierarchical Cross-Modal Association for Co-Speech Gesture Generation. CVPR22. [PDF] [Project] [Code]
  Xian Liu, Qianyi Wu, Hang Zhou, Yinghao Xu, Rui Qian, Xinyi Lin, Xiaowei Zhou, Wayne Wu, Bo Dai, Bolei Zhou

• [HH.GP.6] GestureDiffuCLIP: Gesture Diffusion Model with CLIP Latents. SIGGRAPH23. [PDF] [Project]
  Tenglong Ao, Zeyi Zhang, Libin Liu

• [HH.GP.5] Rhythmic Gesticulator: Rhythm-Aware Co-Speech Gesture Synthesis with Hierarchical Neural Embeddings. SIGGRAPH22. [PDF] [Project] [Code]
  Tenglong Ao, Qingzhe Gao, Yuke Lou, Baoquan Chen, Libin Liu

• [HH.GP.4] HandDiffuse: Generative Controllers for Two-Hand Interactions via Diffusion Models. arXiv23. [PDF] [Project]
  Pei Lin, Sihang Xu, Hongdi Yang, Yiran Liu, Xin Chen, Jingya Wang, Jingyi Yu, Lan Xu

• [HH.GP.3] BodyFormer: Semantics-guided 3D Body Gesture Synthesis with Transformer. arXiv23. [PDF]
  Kunkun Pang, Dafei Qin, Yingruo Fan, Julian Habekost, Takaaki Shiratori, Junichi Yamagishi, Taku Komura

• [HH.GP.2] Diverse 3D Hand Gesture Prediction from Body Dynamics by Bilateral Hand Disentanglement. CVPR23. [PDF] [Code]
  Xingqun Qi, Chen Liu, Muyi Sun, Lincheng Li, Changjie Fan, Xin Yu

• [HH.GP.1] Body2Hands: Learning to Infer 3D Hands from Conversational Gesture Body Dynamics. CVPR21. [PDF] [Project] [Code]
  Evonne Ng, Hanbyul Joo, Shiry Ginosar, Trevor Darrell

Manipulation

• [HH.Mani.2] GigaHands: A Massive Annotated Dataset of Bimanual Hand Activities. arXiv24. [PDF] [Project]
  Rao Fu, Dingxi Zhang, Alex Jiang, Wanjia Fu, Austin Funk, Daniel Ritchie, Srinath Sridhar

• [HH.Mani.1] OAKINK2: A Dataset of Bimanual Hands-Object Manipulation in Complex Task Completion. CVPR24. [PDF] [Project] [Code]
  Xinyu Zhan, Lixin Yang, Yifei Zhao, Kangrui Mao, Hanlin Xu, Zenan Lin, Kailin Li, Cewu Lu

[back to top]

Full Body Reconstruction

Forward Kinematics

• [FBR.FK.8] AiOS: All-in-One-Stage Expressive Human Pose and Shape Estimation. CVPR24. [PDF] [Project] [Code]
  Qingping Sun, Yanjun Wang, Ailing Zeng, Wanqi Yin, Chen Wei, Wenjia Wang, Haiyi Mei, Chi Sing Leung, Ziwei Liu, Lei Yang, Zhongang Cai

• [FBR.FK.8] One-Stage 3D Whole-Body Mesh Recovery with Component Aware Transformer. CVPR23. [PDF] [Project] [Code]
  Jing Lin, Ailing Zeng, Haoqian Wang, Lei Zhang, Yu Li

• [FBR.FK.7] PyMAF-X: Towards Well-aligned Full-body Model Regression from Monocular Images. arXiv22. [PDF] [Project]
  Hongwen Zhang, Yating Tian, Yuxiang Zhang, Mengcheng Li, Liang An, Zhenan Sun, Yebin Liu

• [FBR.FK.6] Lightweight Multi-person Total Motion Capture Using Sparse Multi-view Cameras. ICCV21. [PDF]
  Yuxiang Zhang, Zhe Li, Liang An, Mengcheng Li, Tao Yu, Yebin Liu

• [FBR.FK.5] Monocular Total Capture: Posing Face, Body, and Hands in the Wild. CVPR19. [PDF] [Project] [Code]
  Donglai Xiang, Hanbyul Joo, Yaser Sheikh

• [FBR.FK.4] Total Capture: A 3D Deformation Model for Tracking Faces, Hands, and Bodies [PDF]
  Hanbyul Joo, Tomas Simon, Yaser Sheikh

• [FBR.FK.3] Expressive Body Capture: 3D Hands, Face, and Body from a Single Image. CVPR19. [PDF] [Project] [Code]
  Georgios Pavlakos, Vasileios Choutas*, Nima Ghorbani, Timo Bolkart, Ahmed A. A. Osman, Dimitrios Tzionas, Michael J. Black*

• [FBR.FK.2] FrankMocap: A Monocular 3D Whole-Body Pose Estimation System via Regression and Integration. arXiv21. [PDF] [Code]
  Yu Rong, Takaaki Shiratori, Hanbyul Joo

• [FBR.FK.1] Monocular Expressive Body Regression through Body-Driven Attention. ECCV20. [PDF] [Project] [Code]
  Vasileios Choutas, Georgios Pavlakos, Timo Bolkart, Dimitrios Tzionas , Michael J. Black

Inverse Kinematics

• [FBR.IK.2] Pose2Pose: 3D Positional Pose-Guided 3D Rotational Pose Prediction for Expressive 3D Human Pose and Mesh Estimation. arXiv20. [PDF]
  Gyeongsik Moon, Kyoung Mu Lee

• [FBR.IK.1] Monocular Real-time Full Body Capture with Inter-part Correlations. CVPR21. [PDF]
  Yuxiao Zhou, Marc Habermann, Ikhsanul Habibie, Ayush Tewari, Christian Theobalt, Feng Xu

Implicit Shape

• [FBR.SP.7] ECON: Explicit Clothed humans Obtained from Normals. arXiv22. [PDF] [Project] [Code]
  Yuliang Xiu, Jinlong Yang, Xu Cao, Dimitrios Tzionas, Michael J. Black

• [FBR.SP.6] SPAMs: Structured Implicit Parametric Models. CVPR22. [PDF] [Project]
  Pablo Palafox, Nikolaos Sarafianos, Tony Tung, Angela Dai

• [FBR.SP.5] Monocular Real-Time Volumetric Performance Capture. ECCV20. [PDF] [Code]
  Ruilong Li, Yuliang Xiu, Shunsuke Saito, Zeng Huang, Kyle Olszewski, Hao Li

• [FBR.SP.4] ARCH: Animatable Reconstruction of Clothed Humans. CVPR20. [PDF]
  Zeng Huang, Yuanlu Xu, Christoph Lassner, Hao Li, Tony Tung

• [FBR.SP.3] Geo-PIFu: Geometry and Pixel Aligned Implicit Functions for Single-view Human Reconstruction. NeurIPS20. [PDF] [Code]
  Tong He, John Collomosse, Hailin Jin, Stefano Soatto

• [FBR.SP.2] PIFuHD: Multi-Level Pixel-Aligned Implicit Function for High-Resolution 3D Human Digitization. CVPR20. [PDF] [Code]
  Shunsuke Saito, Tomas Simon, Jason Saragih, Hanbyul Joo

• [FBR.SP.1] PIFu: Pixel-Aligned Implicit Function for High-Resolution Clothed Human Digitization. CVPR19 [PDF] [Code]
  Shunsuke Saito, Zeng Huang, Ryota Natsume, Shigeo Morishima, Angjoo Kanazawa, Hao Li

Neural Rendering

• [FBR.NR.7] One Shot, One Talk: Whole-body Talking Avatar from a Single Image. arXiv24. [PDF] [Project] Jun Xiang, Yudong Guo, Leipeng Hu, Boyang Guo, Yancheng Yuan, Juyong Zhang

• [FBR.NR.6] Expressive Whole-Body 3D Gaussian Avatar. ECCV24. [PDF] [Project] Gyeongsik Moon, Takaaki Shiratori, Shunsuke Saito

• [FBR.NR.5] XAGen: 3D Expressive Human Avatars Generation. NeurIPS23. [PDF] [Project] [Code]
  Zhongcong Xu, Jianfeng Zhang, Jun Hao Liew, Jiashi Feng, Mike Zheng Shou

• [FBR.NR.4] UnitedHuman: Harnessing Multi-Source Data for High-Resolution Human Generation. ICCV23. [Project]
  Jianglin Fu, Shikai Li, Yuming Jiang, Kwan-Yee Lin, Ziwei Liu, and Wayne Wu

• [FBR,NR,4] Expressive Gaussian Human Avatars from Monocular RGB Video. arXiv24. [PDF] [Project] [Code]
  Hezhen Hu, Zhiwen Fan, Tianhao Wu, Yihan Xi, Seoyoung Lee, Georgios Pavlakos, Zhangyang Wang

• [FBR.NR.3] AvatarReX: Real-time Expressive Full-body Avatars. SIGGRAPH23. [PDF] [Project]
  Zerong Zheng, Xiaochen Zhao, Hongwen Zhang, Boning Liu, Yebin Liu

• [FBR.NR.2] X-Avatar: Expressive Human Avatars. CVPR23. [PDF] [Code] [Project]
  Kaiyue Shen, Chen Guo, Manuel Kaufmann, Juan Jose Zarate, Julien Valentin, Jie Song, Otmar Hilliges

• [FBR.NR.1] TotalSelfScan: Learning Full-body Avatars from Self-Portrait Videos of Faces, Hands, and Bodies. NeurIPS22. [PDF] [Project]
  Junting Dong, Qi Fang, Yudong Guo, Sida Peng, Qing Shuai, Xiaowei Zhou, Hujun Bao

Grasp

• [FBR.GR.4] HIMO: A New Benchmark for Full-Body Human Interacting with Multiple Objects. ECCV24. [PDF] [Project] [Code]
  Xintao Lv, Liang Xu, Yichao Yan, Xin Jin, Congsheng Xu, Shuwen Wu, Yifan Liu, Lincheng Li, Mengxiao Bi, Wenjun Zeng, Xiaokang Yang

• [FBR.GR.3] Stochastic Whole-Body Grasping with Contact. arXiv21. [PDF] [Project] [Code]
  Yan Wu, Jiahao Wang, Yan Zhang, Siwei Zhang, Otmar Hilliges, Fisher Yu, Siyu Tang

• [FBR.GR.2] GOAL: Generating 4D Whole-Body Motion for Hand-Object Grasping. CVPR22. [PDF] [Project]
  Omid Taheri, Vasileios Choutas, Michael J. Black, Dimitrios Tzionas

• [FBR.GR.1] GRAB: A Dataset of Whole-Body Human Grasping of Objects. ECCV20. [PDF] [Project] [Code]
  Omid Taheri, Nima Ghorbani, Michael J. Black, Dimitrios Tzionas

Multi-Modal Motion

In Sence

• [MM.IS.10] Human-Aware 3D Scene Generation with Spatially-constrained Diffusion Models. arXiv24. [PDF] [Project]
  Xiaolin Hong, Hongwei Yi, Fazhi He, Qiong Cao

• [MM.IS.9] GenZI: Zero-Shot 3D Human-Scene Interaction Generation. arXiv23. [PDF] [Project]
  Lei Li, Angela Dai

• [MM.IS.8] Compositional Human-Scene Interaction Synthesis with Semantic Control. ECCV22. [PDF] [Project] [Code]
  Kaifeng Zhao, Shaofei Wang, Yan Zhang, Thabo Beeler, Siyu Tang

• [MM.IS.7] Controllable Human-Object Interaction Synthesis. arXiv23. [PDF]
  Jiaman Li, Alexander Clegg, Roozbeh Mottaghi, Jiajun Wu, Xavier Puig, C. Karen Liu

• [MM.IS.6] Generating Continual Human Motion in Diverse 3D Scenes. arXiv23. [PDF]
  Aymen Mir, Xavier Puig, Angjoo Kanazawa, Gerard Pons-Moll

• [MM.IS.5] Stochastic Scene-Aware Motion Prediction. ICCV21. [PDF] [Project] [Code]
  Mohamed Hassan, Duygu Ceylan, Ruben Villegas, Jun Saito, Jimei Yang, Yi Zhou, Michael Black

• [MM.IS.4] PLACE: Proximity Learning of Articulation and Contact in 3D Environments. 3DV20. [PDF] [Project] [Code]
  Siwei Zhang, Yan Zhang, Qianli Ma, Michael J. Black, Siyu Tang

• [MM.IS.3] Populating 3D Scenes by Learning Human-Scene Interaction. CVPR21. [PDF] [Project] [Code]
  Mohamed Hassan, Partha Ghosh, Joachim Tesch, Dimitrios Tzionas, Michael J. Black

• [MM.IS.2] The Wanderings of Odysseus in 3D Scenes. CVPR22. [PDF] [Project]
  Yan Zhang, Siyu Tang

• [MM.IS.1] Generating 3D People in Scenes without People. CVPR20. [PDF] [Code]
  Yan Zhang, Mohamed Hassan, Heiko Neumann, Michael J. Black, Siyu Tang

Text to Motion

• [MM.Tex.12] ChatPose: Chatting about 3D Human Pose. CVPR24. [PDF] [Project]
  Yao Feng, Jing Lin, Sai Kumar Dwivedi, Yu Sun, Priyanka Patel, Michael J. Black

• [MM.Tex.15] SemGrasp: Semantic Grasp Generation via Language Aligned Discretization. ECCV24. [PDF] [Project] Kailin Li, Jingbo Wang,Lixin Yang, Cewu Lu, Bo Dai

• [MM.Tex.14] Human Motion Instruction Tuning. arXiv24. [PDF]
  Lei Li, Sen Jia, Wang Jianhao, Zhongyu Jiang, Feng Zhou, Ju Dai, Tianfang Zhang, Wu Zongkai, Jenq-Neng Hwang

• [MM.Tex.13] MotionCLR: Motion Generation and Training-free Editing via Understanding Attention Mechanisms. arXiv24. [PDF] [Project] [Code]
  Ling-Hao Chen, Wenxun Dai, Xuan Ju, Shunlin Lu, Lei Zhang

• [MM.Tex.12] MotionLLM: Understanding Human Behaviors from Human Motions and Videos. arXiv24. [PDF] [Project] [Code]
  Ling-Hao Chen, Shunlin Lu, Ailing Zeng, Hao Zhang, Benyou Wang, Ruimao Zhang, Lei Zhang

• [MM.Tex.11] InterGen: Diffusion-based Multi-human Motion Generation under Complex Interactions. IJCV24. [PDF] [Project] [Code]
  Han Liang, Wenqian Zhang, Wenxuan Li, Jingyi Yu, Lan Xu

• [MM.Tex.10] Inter-X: Towards Versatile Human-Human Interaction Analysis. CVPR24. [PDF] [Project] [Code]
  Liang Xu, Xintao Lv, Yichao Yan, Xin Jin, Shuwen Wu, Congsheng Xu, Yifan Liu, Yizhou Zhou, Fengyun Rao, Xingdong Sheng, Yunhui Liu, Wenjun Zeng, Xiaokang Yang

• [MM.Tex.9] MoMask: Generative Masked Modeling of 3D Human Motions. [PDF] [Project] [Code]
  Chuan Guo, Yuxuan Mu, Muhammad Gohar Javed, Sen Wang, Li Cheng

• [MM.Tex.8] TMR: Text-to-Motion Retrieval Using Contrastive 3D Human Motion Synthesis. ICCV23. [PDF] [Project] [Code]
  Mathis Petrovich, Michael J. Black, Gül Varol

• [MM.Tex.7] HumanTOMATO: Text-aligned Whole-body Motion Generation. arXiv23. [PDF] [Project] [Code]
  Shunlin Lu, Ling-Hao Chen, Ailing Zeng, Jing Lin, Ruimao Zhang, Lei Zhang, Heung-Yeung Shum

• [MM.Tex.6] Motion-X: A Large-scale 3D Expressive Whole-body Human Motion Dataset. NeurIPS23. [PDF] [Project] [Code]
  Jing Lin, Ailing Zeng, Shunlin Lu, Yuanhao Cai, Ruimao Zhang, Haoqian Wang, Lei Zhang

• [MM.Tex.5] MotionDiffuse: Text-Driven Human Motion Generation with Diffusion Model. arXiv22. [PDF] [Project] [Code]
  Mingyuan Zhang, Zhongang Cai, Liang Pan, Fangzhou Hong, Xinying Guo, Lei Yang, Ziwei Liu

• [MM.Tex.4] UDE: A Unified Driving Engine for Human Motion Generation. CVPR23. [PDF] [Project] [Code]
  Zixiang Zhou, Baoyuan Wang

• [MM.Tex.3] MotionGPT: Finetuned LLMs are General-Purpose Motion Generators. arXiv23. [PDF] [Project] [Code]
  Yaqi Zhang, Di Huang, Bin Liu, Shixiang Tang, Yan Lu, Lu Chen, Lei Bai, Qi Chu, Nenghai Yu, Wanli Ouyang

• [MM.Tex.2] MotionGPT: Human Motion as a Foreign Language. NeurIPS23. [PDF] [Project] [Code]
  Biao Jiang, Xin Chen, Wen Liu, Jingyi Yu, Gang Yu, Tao Chen

• [MM.Tex.1] GUESS: GradUally Enriching SyntheSis for Text-Driven Human Motion Generation. TVCG24. [PDF] [Code]
  Xuehao Gao, Yang Yang, Zhenyu Xie, Shaoyi Du, Zhongqian Sun, Yang Wu

Dexterous Robot Hand

Dynamics

• [RO.DN.6] DexDiffuser: Generating Dexterous Grasps with Diffusion Models. arXiv24. [PDF] [Project]
  Zehang Weng, Haofei Lu, Danica Kragic, Jens Lundell

• [RO.DN.5] Grasping Diverse Objects with Simulated Humanoids. arXiv24. [PDF] [Project]
  Zhengyi Luo, Jinkun Cao, Sammy Christen, Alexander Winkler, Kris Kitani, Weipeng Xu

• [RO.DN.3] UniGraspTransformer: Simplified Policy Distillation for Scalable Dexterous Robotic Grasping. arXiv24. [PDF] [Project] [Code]
  Wenbo Wang, Fangyun Wei, Lei Zhou, Xi Chen, Lin Luo, Xiaohan Yi, Yizhong Zhang, Yaobo Liang, Chang Xu, Yan Lu, Jiaolong Yang, Baining Guo

• [RO.DN.2] Bimanual Grasp Synthesis for Dexterous Robot Hands. RAL24. [PDF] [Project]
  Yanming Shao, Chenxi Xiao

• [RO.DN.1] DexGraspNet: A Large-Scale Robotic Dexterous Grasp Dataset for General Objects Based on Simulation. ICRA23. [PDF] [Project] [Code]
  Ruicheng Wang, Jialiang Zhang, Jiayi Chen, Yinzhen Xu, Puhao Li, Tengyu Liu, He Wang

Imitation

• [RO.IM.2] OKAMI: Teaching Humanoid Robots Manipulation Skills through Single Video Imitation. CoRL24. [PDF] [Project]
  Jinhan Li, Yifeng Zhu, Yuqi Xie, Zhenyu Jiang, Mingyo Seo, Georgios Pavlakos, Yuke Zhu

• [RO.IM.1] DexMimicGen: Automated Data Generation for Bimanual Dexterous Manipulation via Imitation Learning. arXiv24. [PDF] [Project]
  Zhenyu Jiang, Yuqi Xie, Kevin Lin, Zhenjia Xu, Weikang Wan, Ajay Mandlekar, Linxi Fan, Yuke Zhu

VLA

• [RO.VLA.2] Diffusion-VLA: Scaling Robot Foundation Models via Unified Diffusion and Autoregression. arXiv24. [PDF] [Project]
  Junjie Wen, Minjie Zhu, Yichen Zhu, Zhibin Tang, Jinming Li, Zhongyi Zhou, Chengmeng Li, Xiaoyu Liu, Yaxin Peng, Chaomin Shen, Feifei Feng

• [RO.VLA.1] CogACT: A Foundational Vision-Language-Action Model for Synergizing Cognition and Action in Robotic Manipulation. arXiv24. [PDF] [Project] [Code]
  Qixiu Li, Yaobo Liang, Zeyu Wang, Lin Luo, Xi Chen, Mozheng Liao, Fangyun Wei, Yu Deng, Sicheng Xu, Yizhong Zhang, Xiaofan Wang, Bei Liu, Jianlong Fu, Jianmin Bao, Dong Chen, Yuanchun Shi, Jiaolong Yang, Baining Guo

[back to top]

Tools

Model

• [T.MD.6] NIMBLE
• [T.MD.5] HTML
• [T.MD.4] mano
• [T.MD.3] manotorch
• [T.MD.2] manopth
• [T.MD.1] spheremesh

MoCap

• [T.MC.5] AlphaPose
• [T.MC.4] EasyMocap
• [T.MC.3] MediaPipe
• [T.MC.2] frankmocap
• [T.MC.1] openpose

[back to top]