COAP: Compositional Articulated Occupancy of People

Paper | Video | Project Page

This is the official implementation of the CVPR 2022 paper COAP: Learning Compositional Occupancy of People.

Description

This repository provides the official implementation of an implicit human body model (COAP) which implements efficient loss terms for resolving self-intersection and collisions with 3D geometries.

Installation

The necessary requirements are specified in the requrements.txt file. To install COAP, execute:

pip install git+https://github.com/markomih/COAP.git

Note that Pytorch3D may require manuall installation (see instructions here). Alternatively, we provide a conda environment file to install the dependences:

conda env create -f environment.yml
conda activate coap
pip install git+https://github.com/markomih/COAP.git

Optional Dependencies

Install the pyrender package to use the visualization/tutorial scripts and follow the additional instructions specified here if you wish to retrain COAP.

Tutorials

COAP extends the interface of the SMPL-X package (follow its instructions for the usage) via two volumetric loss terms: 1) a loss for resolving self-intersections and 2) a loss for resolving collisions with 3D geometries flexibly represented as point clouds. In the following, we provide a minimal interface to access the COAP's functionalities:

import smplx
from coap import attach_coap

# create a SMPL body and extend the SMPL body via COAP (we support: smpl, smplh, and smplx model types)
model = smplx.create(**smpl_parameters)
attach_coap(model)

smpl_output = model(**smpl_data)  # smpl forward pass
# NOTE: make sure that smpl_output contains the valid SMPL variables (pose parameters, joints, and vertices). 
assert model.joint_mapper is None, 'COAP requires valid SMPL joints as input'

# access two loss functions
model.coap.selfpen_loss(smpl_output)  # self-intersections
model.coap.collision_loss(smpl_output, scan_point_cloud)  # collisions with other geometris

Additionally, we provide two tutorials on how to use these terms to resolve self-intersections and collisions with the environment.

Pretrained Models

A respective pretrained model will be automatically fetched and loaded. All the pretrained models are available on the dev branch inside the ./models directory.

Citation

@inproceedings{Mihajlovic:CVPR:2022,
   title = {{COAP}: Compositional Articulated Occupancy of People},
   author = {Mihajlovic, Marko and Saito, Shunsuke and Bansal, Aayush and Zollhoefer, Michael and Tang, Siyu},
   booktitle = {Proceedings IEEE Conf. on Computer Vision and Pattern Recognition (CVPR)},
   month = jun,
   year = {2022}
}

Contact

For questions, please contact Marko Mihajlovic ([email protected]) or raise an issue on GitHub.