StyleMM: Stylized 3D Morphable Face Model via Text-Driven Aligned Image Translation

Seungmi Lee * ,   Kwan Yun * ,   Junyong Noh ,  

KAIST, Visual Media LAB
* Contributed equally to this work

Pacific Graphics 2025 Journal Track

Disentangled control results of StyleMM on geometry, expression, and texture

TL;DR : StyleMM is the first text driven
stylized 3D Morphable Model (Stylized 3DMM)




Definition of Stylized 3DMM

1. All generated faces share dense point-to-point correspondences, ensuring consistent mesh structure across instances.

2. Facial shape and color are disentangled and can be independently controlled.

3. Expressive stylization of both geometry and texture that extends beyond realistic face models.



Abstract

We introduce StyleMM, a novel framework that can construct a stylized 3D Morphable Model (3DMM) based on user-defined text descriptions specifying a target style. Building upon a pre-trained mesh deformation network and a texture generator for original 3DMM-based realistic human faces, our approach fine-tunes these models using stylized facial images generated via text-guided image-to-image (i2i) translation with a diffusion model, which serve as stylization targets for the rendered mesh. To prevent undesired changes in identity, facial alignment, or expressions during i2i translation, we introduce a stylization method that explicitly preserves the facial attributes of the source image. By maintaining these critical attributes during image stylization, the proposed approach ensures consistent 3D style transfer across the 3DMM parameter space through image-based training. Once trained, StyleMM enables feed-forward generation of stylized face meshes with explicit control over shape, expression, and texture parameters, producing meshes with consistent vertex connectivity and animatability. Quantitative and qualitative evaluations demonstrate that our approach outperforms state-of-the-art methods in terms of identity-level facial diversity and stylization capability.


Video retargeting

High quality textures generated using StyleMM

texture generator

StyleMM 3D Stylization

3D Stylization

StyleMM 3D stylized reconstruction can be completed in less than a second and can also perform additional style enhancements.