Reading Emotion From a Silent Face: Gemini 3 Flash vs Qwen2.5-VL-32B

Vision-language models are good at describing what's in an image. But can they read something subtler — the expressive state of a person — from video alone, with no sound to help? I ran a small controlled benchmark to find out, pitting two strong VLMs against each other on exactly the same task: watch a muted clip of an actor and name the emotion.

The two candidates are Google's Gemini 3 Flash and Alibaba's Qwen2.5-VL-32B-Instruct. Same clips, same prompt, scored against ground truth. This post documents the results — the headline accuracy, but more interestingly, the per-class behaviour, where the two models fail in completely different ways.

Setup

The test set is RAVDESS — studio-recorded actors performing emotional speech, frontal framing, clean lighting. I used a balanced subset: 25 clips × 6 emotions = 150 clips, covering the true Ekman six (happy, sad, angry, fearful, disgust, surprise). RAVDESS's neutral and calm categories were dropped so every label is a real Ekman emotion.

The critical constraint: the audio is stripped before the clip ever reaches the model. RAVDESS is a speech dataset, and the easiest way to "cheat" emotion recognition is to listen to the voice. Audio was removed with ffmpeg -an into a parallel silent directory, and clip-level assertions verified no audio stream survived. Both models see exactly the same muted video — this is purely a test of reading the face.

A few choices worth stating, because they shape how the numbers should be read:

• Forced enum output. Emotion is a constrained schema field, so neither model can emit an off-vocabulary label. Parsing is deterministic, not regex-guesswork.
• One call per clip. No self-consistency voting, no ensembling — just the model's single-shot answer, measured as you'd actually use it.
• The prompt is locked. It is not tuned per model. Any difference in performance is attributable to the model, not the instructions.
• Quarantine, never guess. Outputs that failed schema validation after three retries were set aside rather than counted as wrong — we don't know what the model would have said. Gemini produced 150 valid outputs; Qwen quarantined 15, leaving 135 clips common to both models. All numbers below are computed on that common set, so the comparison is strictly like-for-like.

Chance baseline: 16.7%.

Headline numbers

Both models are far above chance — roughly 3.7× and 3.1× respectively — so the answer to the basic question is yes: a VLM does extract a stable expressive signal from a silent face. Gemini comes out ahead, but the confidence intervals overlap, so the headline accuracy gap alone is not a knockout. The interesting differences are per-class.

Per-class recall

Where the two models actually diverge is in which emotions they get right.

Two things jump out.

Qwen cannot see disgust. Zero out of 25. Not "struggles with" — it never once predicted disgust on the entire common set. Whatever facial configuration Qwen associates with the label, acted disgust in RAVDESS does not trigger it; those clips get routed elsewhere. Gemini, by contrast, catches disgust 80% of the time. This single class accounts for a large share of the overall accuracy gap.

Fear is hard for both. Neither model clears 35% recall on fearful clips. Without the vocal fear signature — the audio that's been stripped out — the visual signature of fear (raised inner brows, widened eyes, an open mouth) overlaps heavily with surprise and with sad, and both models scatter their fearful predictions into those neighbours. This is the clearest evidence that some emotions in this dataset are carried more by voice than by face.

The confusion matrices make the divergence concrete — counts are row-wise (true emotion → predicted), so the diagonal is correct and everything off it is an error:

Read the columns and the failure modes are obvious. Qwen's disgust column is empty — its disgust clips all collapse into sad (10) and angry (15). Gemini's sad column is the brightest band in either plot: it absorbs 5 angry, 13 fearful, and a handful of others on top of its true sad clips. And the two models disagree on where fear goes — Gemini sends fearful almost entirely to sad (13), while Qwen splits it between sad (5) and surprise (10).

Prediction distribution

Recall tells you what each model catches; the prediction distribution tells you what it over-reaches for. On a balanced 135-clip set, a perfectly unbiased model would predict each emotion ~22–23 times.

Gemini predicts sad 46 times against 19 true sad clips — it more than doubles the true rate. Its sad recall is a perfect 100%, but that's because it sweeps up roughly two dozen ambiguous clips along the way. This is a recognisable pattern: when the model is visually uncertain, it collapses toward a high-prior negative-affect class rather than committing to the harder call. Qwen shows a milder version of the same lean toward sad, plus its own over-prediction of surprise (40 vs 21 true) — the flip side of having no disgust bucket to route ambiguous negative clips into.

This bias matters as much as the headline accuracy. A model that funnels many different inputs into one over-predicted label is brittle in a way a single accuracy number hides — two models can land within a few points of each other and still behave nothing alike, as Gemini and Qwen do here.

What this does and doesn't tell us

It tells us: silent-video emotion reading is real and well above chance; Gemini is the stronger classifier here, driven largely by disgust; fear is genuinely hard without audio; and both models lean on sad/surprise as uncertainty sinks — but in different proportions.

It does not tell us: how either model performs on in-the-wild footage. RAVDESS is clean, frontal, and theatrically acted — a floor, not a transfer guarantee. These numbers are a sanity check on a single 150-clip subset (per-class precision shifts a few points on a single clip flip), and the model used was a preview build of Gemini 3 Flash.

The bigger caveat is the task framing itself: forcing a model to pick one of six boxes is a lossy way to ask "what's going on in this face." The accuracy ceiling here is as much about the rigidity of categorical emotion labels as it is about the models — which is a thread worth pulling on another time.

Want to discuss this? DM me on LinkedIn or drop me an email.

Numbers are computed from a single benchmark run on the 135-clip common set (seed 42).