Face Reconstruction

Our face reconstruction C++ pipeline aims to obtain parameters for a morphable face model that matches a given RGB/RGB-D image. It utilizes a PCA-based Basel Face Model encompassing shape, expression, albedo, illumination, and pose parameters.

The reconstruction process follows an analysis-by-synthesis approach, updating parameters to minimize an overall energy function using Ceres. The energy function includes dense and sparse terms, incorporating geometry and color comparisons between the rendered face and the actual image.

Main Dependencies

• OpenCV
• Eigen3
• Glog
• GLFW
• PCL
• Boost
• Ceres

Expression Transfer

Transfer expression example. We find parameters independently using source (top left) and target (bottom left) actors. Then, we transfer expression coefficients from the source into the target. Finally, we project the source image into the mesh and visualize the mask above the source image.

Results

Results of the optimization pipeline using RGB only. In the center column, we apply a BFM model texture. The last column obtains texture from the projection mesh into the image.

Building

mkdir build && cd build
cmake ..
make

Method

You can read the final report where explained our method here.

Team Members

• Gökçe Şengün
• Dmitrii Pozdev
• Biray Sütçüoğlu
• David Gichev