WoodstitchHuman evals for voice AI

Research

Blinded listening studies on speech AI, run with paid human raters. Each report states its sample size, its confidence intervals and what it does not show. We publish results that go against us.

Research report 11 November 2026 · Hawkes T

Which ElevenLabs voice is best for audiobooks?

32 Voice Library voices, chosen to span gender, age, accent, recording tier and popularity, read the same 30-second nonfiction passage. 125 listeners judge them blind, in pairs. Listen to the voices yourself, then see where each finished.

66%Marie, first
95% CI 55–76
46%professional voices vs instant clones
studio tier no better
24%Alder, last
95% CI 16–35

125 listeners · 32 voices · 1 passage · 1,122 judgments · popularity predicts nothing · louder clip wins 54%Read the report →

Research report 6 November 2026 · Hawkes T

Which TTS model is best for audiobooks?

ElevenLabs v4, Gemini 3.8 Flash TTS, Fish Audio Drama 3, Cartesia Sonic 3 and Qwen 3.1 TTS Flash read the same sixteen passages from classic novels. 78 listeners judge them blind, in pairs. Listen to the readings yourself, then see who read them.

56%ElevenLabs v4 win rate
95% CI 49–63
54% · 53%Gemini, Fish: tied with it
order not established
41%Qwen 3.1 Flash, last
95% CI 34–48

78 listeners · 5 systems · 16 passages · 481 judgments, preregistered sample reachedRead the report →

Interim research report 5 November 2026 · Hawkes T

Which AI model writes the best piano music?

Eight large language models compose short piano loops for the same twenty prompts, all played on one piano. 62 listeners judge them blind. Listen to the loops yourself, then see who wrote them.

66%Gemini 4 Argon win rate
95% CI 57–75
61%Claude Opus 5.5
95% CI 51–70
27%Llama 4 Maverick, last
95% CI 19–35

62 listeners · 8 models · 20 prompts · 514 of 1,638 judgmentsRead the report →

Interim research report 3 November 2026, updated 6 November · Hawkes T

Can listeners distinguish AI narration from human narration?

A blinded listening study of ElevenLabs v4 and human-read audiobook excerpts of 15, 30 and 60 seconds.

48.8%accuracy, 496 judgments
95% CI 44.0–53.8
0.0 pp60 s vs 15 s clips
no length effect detected
48.7%accuracy when “very confident”
confidence did not track accuracy

75 listeners · 6 AI voices · 10 narrators · 210 clips analysedRead the report →