deepgeek/LAM_Audio2Expression
2
0
Fork 0
mirror of https://github.com/aigc3d/LAM_Audio2Expression.git synced 2026-02-04 17:39:24 +08:00
Code Issues Packages Projects Releases Wiki Activity

fix: Update README

Browse Source
This commit is contained in:
fdyuandong
2025-05-06 16:21:21 +08:00
parent 4e87aa413c
commit 71ab0b4de5
1 changed files with 7 additions and 1 deletions
Download Patch File Download Diff File Expand all files Collapse all files

8
README.md
View File

@@ -4,7 +4,13 @@
[![Apache License](https://img.shields.io/badge/📃-Apache--2.0-929292)](https://www.apache.org/licenses/LICENSE-2.0)
[![ModelScope](https://img.shields.io/badge/%20ModelScope%20-Space-blue)](https://www.modelscope.cn/studios/Damo_XR_Lab/LAM-A2E)
#### This project leverages audio input to generate ARKit blendshapes-driven facial expressions in ⚡real-time⚡, powering ultra-realistic 3D avatars generated by [LAM](https://github.com/aigc3d/LAM).
## Description
#### This project leverages audio input to generate ARKit blendshapes-driven facial expressions in ⚡real-time⚡, powering ultra-realistic 3D avatars generated by [LAM](https://github.com/aigc3d/LAM).
To enable ARKit-driven animation of the LAM model, we adapted ARKit blendshapes to align with FLAME's facial topology through manual customization. The LAM-A2E network follows an encoder-decoder architecture, as shown below. We adopt the state-of-the-art pre-trained speech model Wav2Vec for the audio encoder. The features extracted from the raw audio waveform are combined with style features and fed into the decoder, which outputs stylized blendshape coefficients.
<div align="center">
<img src="./assets/images/framework.png" alt="Architecture" width="90%" align=center/>
</div>
## Demo