Visual-Guided Prosody & Emotional TTS
A text-to-speech system that conditions prosody and emotional expression on visual cues, producing more natural, context-aware synthesized speech.
An advanced TTS system that uses visual cues to drive prosody and emotional expression in synthesized speech — the core of my expressive-speech thesis work.
Overview
This project sits at the intersection of computer vision and speech synthesis. By analyzing visual information from images or video, the system generates contextually appropriate prosody and emotional intonation, rather than treating text as a flat phoneme sequence.
Key features
- Visual cue extraction and analysis
- Prosody modeling conditioned on visual context
- Emotional expression synthesis
- Context-aware TTS generation
- Multimodal conditioning
Research focus
The central challenge is generating emotionally expressive, contextually appropriate speech by incorporating non-textual signals. The multimodal approach lets the system infer emotional context and adjust prosodic features accordingly.