Annonce

Les commentaires sont clos.

Garment MeshNet: Geometric Deep Learning on 3D Garment Mesh

18 Octobre 2024

Catégorie : Stagiaire

Geometric deep learning has emerged in the fields of computer graphics and computer vision, enabling deep learning models to operate on geometric data such as graphs, meshes, manifolds, and point clouds. Some notable models in this area include Graph Convolutional Networks (GCNs), PointNet, Geodesic Neural Networks (GNNs), and specialized architectures for 3D meshes, such as MeshNet and MeshCNN.

Motivated by these recent successes, we will explore and develop geometric deep learning models for a 3D mesh dataset. In particular, we are interested in 3D garment mesh data representing garment shapes. Our specific focus will be on the development of a geometric deep learning model capable of performing various downstream tasks, such as classification and segmentation.

 

We will proceed with the following tasks:

  1. Data generation: We will augment our current garment dataset, composed primarily of T-shirts, by adding more garment types such as pants and skirts. Blender will be used to create and simulate 3D garment models, allowing for the generation of diverse and realistic training data that can improve the performance of our segmentation models.
  2. Shape representation: The first step is to investigate how to represent various mesh data in a uniform manner, irrespective of its topological structure and geometric deformation. While the representation should be invariant to rigid transformations and vertex/triangle orders, it must be able to effectively disentangle the canonical shape characteristics of the garment from the geometric shape changes caused by the physical self-interaction and the interaction with the environment. Specialized models for 3D meshes, such as MeshNet and MeshCNN, will be adopted to build the latent representation space.
  3. Mesh classification and segmentation: Based on the above representation, we will perform several downstream tasks by developing deformation-invariant convolution and pooling methods, and by designing specialized loss functions tailored to each specific task.
  4. Experiments: The developed model will be parameter-tuned, tested and compared with the state-of-the-art models. Additionally, a number of ablation studies will be conducted to assess the impact of various components on performance.

https://mlms.icube.unistra.fr/img_auth_namespace.php/6/6e/Stage-Garment_MeshNet-2025_En.pdf