This repository provides supporting code for my Masters thesis: On Stylistic Differences Between 3D Meshes
This fork extends the original meshCNN implementation by Rana Hanocka and Amir Hertz for their SIGGRAPH 2019 paper
MeshCNN is a general-purpose deep neural network for 3D triangular meshes, which can be used for tasks such as 3D shape classification or segmentation. This framework includes convolution, pooling and unpooling layers which are applied directly on the mesh edges.
Extensions are as follows:
- optimised pooling operation for linear space complexity instead of quadratic, greatly increasing the sizes of meshes able to be processed
- support for meshes composed of many smaller components, where pooling of small components to single faces does not break the model
- implemented edge colouring for obj file exports for viewing in external programs such as MeshLab
- changed much of the boilerplate code for model training to pytorch-lightning
- examples of back-propagated gradients to the original mesh to visualise edges/regions which contributed to classification
- gram-based style loss for identification of stylistic differences between meshes, visualising regions of dissimilarity
- example pre-processing scripts to attempt to deal with low-quality meshes using ManifoldPlus
- made the code-base slightly easier to navigate with function description comments, simplifying pytorch objects like datasets & dataloaders, and removal of empty files
- Clone this repo:
git clone https://github.com/ranahanocka/MeshCNN.git
cd MeshCNN- Install dependencies: PyTorch, Optional : tensorboardX for training plots, pytorch-lightning for training.
For the original datasets used in the meshCNN paper, please refer to the main repo
Style-based analysis uses the Co-Locating Style-Defining Elements on 3D Shapes dataset found here.
The original implementation method stores an n x n dense matrix for saving which edge features are averaged during the pooling phase. This matrix ends up being very sparse, only storing a few values per row depending on how many pooling operations are needed. This can be replaced with a Pytorch sparce matrix (still in beta) which enabled the use of much larger meshes, experimentally over an order of magnitude larger. This removed the need for hacks like splitting the mesh into parts to be passed through the network before reassembly, as is seen in the point2mesh repo. However due to sparce matrices being in beta, efficient slicing is not implemented. A crude implementation is provided however this greatly reduces performance by 8x to 10x. The change is given in the mesh union file, and the call to meshUnion in mesh pool should be changed to MeshUnionSparse to use the sparce implementation.
Pooling also does not break when sub-components are pooled to a single face. This code is modified to skip this face, and continue to pool the rest of the mesh. If the mesh cannot be pooled the necessary amount due to many unpoolable components, the mesh is left as-is and other meshes in the batch are padded to accomodate.
The design philosophy of the main reposotory aims to keep all requirements included with the project. While this can be useful, added functionality was chosen for this fork. Pytorch-lightning code replaces the functionality of the original mesh_classifier file.
A method for saving edge colourings to .obj files is implemented. Since obj files support node colourings and not edge colourings, new faces and nodes are added along each edge with the appropriate colouring. This means they can be interpreted by mesh renderers such as MeshLab for viewing. This addition can be seen from the mesh.py file.
If the original model is all that is used then the original paper should be cited, details on the main repo
If you have questions or issues running this code, please open an issue so we can know to fix it.