Can listeners distinguish AI narration from human narration?

A blinded listening study of ElevenLabs v4 and human-read audiobook excerpts. Interim report, updated 6 November 2026: 496 of 1,200 planned judgments.

1What we tested

Audiobook producers, platforms and listeners increasingly ask whether synthetic narration is detectable by ear. Most public claims are anecdotal, or rest on older systems. We asked a narrower, answerable question: when an ordinary listener hears a single excerpt from a current commercial system, blind and without a reference, can they say whether it was read by a person — and does hearing more of it help?

We tested one system, ElevenLabs v4,¹ against human narrators from the Hi-Fi TTS corpus,² on ten passages of public-domain nonfiction prose. Each recording was cut at 15, 30 and 60 seconds so that clip length could be examined within the same material. Listeners also rated their confidence, which lets us ask whether people know when they are guessing. Finally, we measured each clip acoustically to ask two separate questions: what objectively distinguishes the two sets, and what listeners were responding to when they said “human”.

2Results

2.1 Overall accuracy was near chance

Listeners classified 242 of 496 judgments correctly (48.8%). Resampling listeners, the 95% interval is 44.0–53.8%; resampling listeners and speakers independently, it is 38.3–59.6% (§4.4). Both include 50%. Accuracy was 46.7% on AI clips and 51.7% on human clips (Table 1). The gap is a lean toward one answer, not discrimination: listeners answered “human” on 52.6% of judgments against a true human share of 41.7%, which raises the score on human clips and lowers it on AI clips by about the same amount. Sensitivity measured as d′³ was −0.04 (listener-resampled CI −0.28 to 0.20; listener-and-speaker −0.61 to 0.54), where zero denotes no discrimination. Of the 41 listeners who completed six or more trials, three scored 8 of 10 and none higher; with 41 listeners answering at chance, about two scores of 8 or more in 10 are expected, and 14 of the 41 scored above half against about 16 expected. We did not establish above-chance classification on these stimuli.

Table 1. Classification accuracy by ground truth. Intervals are 95% bootstrap intervals resampling listeners (5,000 replicates), conditional on this stimulus set; Wilson intervals on the raw counts are within 1 pp.

2.2 We did not detect an effect of clip length

Accuracy was 49.1% at 15 seconds, 48.1% at 30 seconds and 49.1% at 60 seconds (Table 2; Figure 1). The 60-second condition gave listeners four times the audio of the 15-second condition; the estimated difference is 0.0 percentage points with a listener-resampled 95% interval of −8.9 to +8.9 (listener-and-speaker −15.4 to +18.2). The data do not detect a clip-length effect and cannot rule out a moderate one in either direction. The point estimate was −0.2 at v1.0, +5.3 at v1.1 and 0.0 here; that movement is itself a measure of how unsettled it still is. These intervals are conditional on the selected speakers and passages.

Table 2. Accuracy by clip length. Each listener heard a mix of lengths; no listener heard the same passage or the same speaker twice.

2.3 Stated confidence was not a guide to accuracy

After each verdict, listeners chose one of four confidence labels. Accuracy was somewhat lower at the two upper levels than at the two lower ones (Table 3): 54.7% when “just guessing”, 52.5% “slightly confident”, 43.8% “confident” and 48.7% “very confident”. Pooling the two lower and two upper levels gives 53.2% versus 45.7%, a difference of 7.5 points whose interval (−1.2 to +16.1) narrowly includes zero. Judgments rated “very confident” were correct 55 of 113 times (48.7%; listener-resampled CI 37.8–59.8%; listener-and-speaker 29.3–69.2%); both intervals include 50%. At the v1.0 snapshot this category stood at 16 of 48 (33.3%) with a listener-resampled interval that excluded 50%, and at v1.1 at 34 of 73 (46.6%). The 42 very-confident judgments added since v1.1 were correct 22 times. We read the v1.0 figure as the kind of extreme that a small interim subgroup produces and that more data pulls back toward the mean, and we make only the weaker claim: confidence did not track accuracy, and if anything the more confident answers were slightly less often right. Of the 58 very-confident errors, 36 were an AI clip called human.

Table 3. Accuracy by stated confidence. The four labels are those shown to listeners.

2.4 Some voices were classified as human more often than others

Table 4 lists all sixteen speakers by the share of trials on which listeners answered “human”, regardless of the truth. One AI voice (Eric) was judged human on 37 of 46 trials (80%), level with the two most human-judged narrators (6671 at 81% and 11614 at 78%), and a second (Chris) on 62%; narrator 12787 was judged AI on 82% of trials and narrator 11697 on 77%. At v1.1 Eric stood above every narrator (29 of 33, 88%); with more trials that lead has gone. Trials per speaker range from 16 to 53 and most intervals overlap several others; the table describes the sample and should not be read as a ranking of voices.

Table 4. Share of trials on which each speaker was judged human. Narrator IDs are Hi-Fi TTS speaker IDs. Wilson 95% intervals.

2.5 Exploratory acoustic comparison of the stimulus sets

We extracted eleven acoustic measures from every clip as served (§4.5) and asked two descriptive questions: on what dimensions do the AI clips differ from the human clips, and which measures are associated with a listener answering “human”? Both analyses were specified after the behavioural results were known and are exploratory. They describe these stimulus sets, not an intrinsic property of AI versus human speech: each narrator read only one passage, so speaker and passage are confounded on the human side, and the pause detector is energy-based rather than linguistic. The spectral adjustment applied to the human recordings (§4.2) does not account for the contrasts: recomputing every measure on the unprocessed source recordings gives the same values to within 0.01 (human pause s.d. 0.045 s before and after; AI 0.142 s; pitch range 10.46 versus 10.45 semitones).

What differs. The two sets differ most on pause structure (Table 5). The AI clips pause for longer (mean 0.47 s versus 0.32 s), their longest pauses are longer (0.70 s versus 0.40 s), and their pause lengths vary far more (standard deviation 0.14 s versus 0.04 s; standardised difference d = 3.85 on speaker means). The AI clips also use a wider pitch range (15.4 versus 10.5 semitones). Median pitch, loudness and dynamic range do not differ. On these measures the AI output is more varied and more deliberate than these narrators, not flatter. A different human baseline — professional narrators, or the same readers in different material — could narrow or reverse several rows.

Table 5. Acoustic measures on the clips as served, speaker-level means (6 AI voices, 10 narrators; 210 clips). d is Cohen’s d on the 16 speaker means; rows in bold have Welch t > 2.1. Descriptive only.

Association with listeners’ answers. Table 6 relates the same measures to the listener’s answer across the 496 judgments. All associations are small (|r| ≤ 0.16). Listeners answered “human” somewhat more often for lower-pitched clips (r = −0.16; 58% below the median-pitch median versus 46% above), quieter clips (r = −0.13; 57% versus 49%), slower clips (r = −0.13) and clips with a wider pitch range (r = +0.11; 57% above the median versus 47% below). The pause measures, which separate the two sets most strongly, again showed essentially no marginal association (r = 0.01–0.05 for mean pause, longest pause and pause variability). The pitch-range association had the same sign within the AI clips and within the human clips; the loudness and median-pitch associations were carried mainly by the AI clips. These are marginal correlations on non-independent judgments; they do not establish which cues listeners used, and we draw no causal conclusion from them.

Table 6. Association between each acoustic measure and answering “human” (496 judgments). r is the point-biserial correlation; “below / above median” is the share answered human in each half. Judgments are not independent (listeners, speakers and passages recur) and eleven measures were examined, so no significance threshold is claimed.

3Interpretation and limitations

The defensible central finding is narrow: on these stimuli, these listeners did not classify above chance, more audio did not detectably help, and stated confidence carried no useful information about accuracy. The acoustic comparison adds a descriptive observation — the sets differ most on pause rhythm, and listeners’ answers were not associated with it — but not an explanation.

One hypothesis consistent with the pattern, which this study did not test, is that listeners apply a prior about synthetic voices (flat, quick, bright) in reverse, taking range and an unhurried pace as evidence of a person; since the AI clips here have the wider pitch range, such a prior would point the wrong way, and would produce errors of the kind that predominate here (AI called human, which is also the direction of the overall response bias). Alternatives are equally open: pause variability may not be perceptible in a single short hearing; listeners may be responding to features we did not measure; or the associations may reflect particular speakers rather than any general cue. Distinguishing these would need a design that manipulates cues directly.

Specific limitations:

• Interim snapshot. The study is planned at 1,200 judgments (600 on AI clips and 600 on human clips; 3–4 ratings per AI clip and 20 per human clip). This update uses the 496 available on 6 November 2026, 41% of the plan; the headline estimate was inspected at 174 and 176 judgments on 3 November for v1.0, at 346 for v1.1 on 5 November, and once more for this update. The intervals are not adjusted for data-dependent inspection. The movement across the three snapshots — the very-confident subgroup from 33% to 47% to 49%, the clip-length difference from −0.2 to +5.3 to 0.0 points — shows how far estimates built on a few dozen judgments per cell can move. A versioned update will be issued at the planned sample size whatever it shows.
• Single system. All synthetic clips are ElevenLabs v4 with default settings. Results do not transfer to other systems or versions.
• Human baseline. The narrators are LibriVox volunteers via Hi-Fi TTS, selected for recording quality, not professional audiobook narrators. Professional delivery typically has more pausing and wider range, which would likely reduce several contrasts in Table 5 and might change detectability in either direction.
• Design imbalance. 180 AI clips versus 30 human; six AI voices versus ten narrators; 16–53 trials per speaker. Per-speaker estimates are imprecise.
• Repeated measures and generalisation. Listeners contributed up to 10 judgments each, and each speaker and passage recurs across judgments. Listener-resampled intervals are conditional on this stimulus set. Intervals that also resample speakers are reported for the main estimates and are much wider (§4.4); with 16 speakers and 10 passages, intervals that resample passages as well are close to uninformative and we do not rely on them.
• Exploratory analysis. The acoustic analyses were not prespecified and were run after the behavioural results were known. Eleven measures were examined on non-independent judgments; no significance claim is made. Features come from our own extraction pipeline (§4.5), not a validated phonetic toolkit, and the pause detector is energy-based. The human recordings were spectrally adjusted before measurement (§4.2).
• Material. One genre, ten passages. Dialogue, verse and emotionally loaded prose are where synthetic narration is most often reported to fail and were not tested.
• Engagement. There were no catch trials. Interaction signals were captured for 410 of the 496 judgments (§4.3); they are consistent with the audio having been heard, and are not extrapolated to the rest.

4Methods

4.1 Stimuli

Ten passages of public-domain nonfiction prose were selected from the Hi-Fi TTS corpus² (NVIDIA; derived from LibriVox), each read by one of ten narrators, giving ten human source recordings of about 60 seconds. The same ten passages were synthesised with six ElevenLabs v4 voices (Alice, Bella, Chris, Eric, George, Matilda) using the vendor’s default voice settings, giving 60 AI source recordings. Every source recording was cut at 15, 30 and 60 seconds from its start, for 210 clips (180 AI, 30 human). Passages were chosen to be calm expository prose without humour, grief or technical vocabulary.

4.2 Levelling and spectral adjustment

All clips were rendered as 44.1 kHz mono 16-bit PCM. A single gain adjustment of +0.34 dB was applied to every clip; this sets the level of the set as a whole and leaves between-clip level differences unchanged (the AI and human clips differ by 0.4 dB on average, Table 5). On the earlier clip set, a classifier using only long-term spectral tilt separated human from AI recordings at AUC 0.815 — a microphone and recording-chain difference, not a property of the voices. To remove it, a straight-line tilt equalisation in log-frequency (−0.11 dB per decade, capped at ±4 dB) was fitted on the 60-second clips and applied to the human recordings only; the AI clips were not altered. After adjustment the same tilt classifier, evaluated on the same clips (not held out), scored AUC 0.51. Listeners therefore heard spectrally adjusted human recordings against unprocessed vendor output. Other spectral or temporal recording-chain differences were not tested for and may remain. Source recordings were screened for background music and for spoken LibriVox preambles.

4.3 Listeners and task

Recruitment, payment, blinding and assignment follow the Woodstitch platform methodology; only what is specific to this study is given here. 75 raters on the Woodstitch platform contributed to this analysis between 30 October and 6 November 2026 (data snapshot 6 November 2026, 23:05 UTC), a median of 7 judgments each (range 1–10). Raters are paid per second of audio heard; payment does not depend on the answer given, so there is no incentive to answer in any particular way. Each trial presented one clip with no reference and a forced choice, Human reading or AI-generated, followed by a mandatory confidence rating with the labels Just guessing, Slightly confident, Confident, Very confident. The player permits one playback per clip, with pause and resume but no restart; the instruction shown was “Play the clip once in full to answer · you can pause, but not replay”. Listeners were not told the proportion of AI clips. The assignment rules prevented any listener from hearing the same passage or the same speaker twice. Clip order within a sitting was randomised. No feedback on correctness was given. Of 511 trials submitted, none was skipped and all received a verdict with a confidence rating. Five were submitted from the author’s own rater account during an interface test on 3 November (five verdicts within three seconds, with randomised confidence); they were present in the v1.0 dataset and have been excluded since v1.1. Ten more came from two accounts that the platform’s location checks have since blocked: one signed in only from data-centre (VPN) addresses with a browser time zone of Africa/Nairobi (8 judgments), the other from a Nigerian mobile network (2 judgments). The panel is restricted to US residents, so these are excluded from v1.2 onward; both sets were inside the v1.1 dataset. That leaves 496. Browser interaction signals were captured for 410 of the 496 judgments: in 408 of those the recorded listening time reached at least 90% of the clip, none reported an audio error, 42 showed the tab losing focus at some point, and mean time with the tab hidden was 3.3 s. These signals are not available for the other 86 judgments and are not extrapolated to them.

4.4 Statistical analysis

Accuracy is the proportion of judgments on which the verdict matched the ground truth. Primary 95% intervals are percentile intervals from a non-parametric bootstrap that resamples listeners with replacement (5,000 replicates),⁴ because judgments from one listener are not independent; these intervals are conditional on the selected speakers and passages. To indicate uncertainty from the choice of speakers, a second bootstrap resamples listeners and speakers independently (4,000 replicates), weighting each judgment by the product of its listener’s and speaker’s resampled multiplicities; replicates in which a subgroup is empty are dropped, and d′ rates are clipped to [0.001, 0.999] before the normal inverse. Independent resampling of crossed clusters is not guaranteed to be calibrated, especially with sparse cells and with speaker confounded with passage on the human side, so these wider intervals are reported as indicative rather than exact. Wilson score intervals⁵ on raw counts are reported for per-speaker rates. Sensitivity d′ was computed as z(hit rate) − z(false-alarm rate), treating “AI” as the signal (hit rate 46.7%, false-alarm rate 48.3%).³ The five judgments from the author’s test account and the ten from two accounts blocked as outside the US were excluded (§4.3); no other judgments were excluded and no other listener was removed. No adjustment is made for this being an interim snapshot (§3).

4.5 Acoustic measures

Clips were downsampled to 16 kHz and analysed with librosa 0.10.⁶ Pitch (F0) was estimated with pYIN⁷ over 60–400 Hz in 10 ms frames; pitch variability and range are the standard deviation and the 5th–95th percentile span of voiced-frame F0 in semitones. Speech frames were those within 35 dB of the clip’s peak short-time RMS; pauses are runs of non-speech frames of at least 250 ms between the first and last speech frame. Speaking rate is the number of onset-strength peaks within speech frames per second of speech, a syllable-nucleus proxy. Loudness is overall RMS in dBFS; dynamic range is the 95th minus 10th percentile of frame level during speech. Group differences in Table 5 use speaker-level means to avoid treating the three cuts of one recording as independent. Associations in Table 6 are point-biserial correlations at the judgment level and are reported without significance tests. Measures were taken on the clips as served, i.e. after the adjustments in §4.2, and repeated on the unprocessed source recordings as a check; the two agree to within 0.01 on every measure.

5Data and reproducibility

The per-judgment dataset (trial, clip identifier, ground truth, clip length, verdict, confidence; listener identifiers replaced by anonymous indices), the per-clip acoustic feature table and the analysis scripts are available on request from support@woodstitch.com. The audio clips remain in use in a running study and will be released when it closes. The task interface and assignment rules are documented in the Woodstitch API reference.

6Funding and competing interests

This study was funded by SquireWorks Audiobooks, which produces audiobooks using ElevenLabs voices among other systems, and was conducted on Woodstitch, a human-evaluation platform operated by the author. The author therefore has a commercial interest in synthetic narration being acceptable to listeners. The finding reported here is favourable to that interest. ElevenLabs had no involvement in the design, execution, analysis or reporting of the study and was not informed of it. The author declares no other competing interests.

7References

1. ElevenLabs. Eleven v4 text-to-speech model. https://elevenlabs.io/ (accessed October 2026).
2. Bakhturina E, Lavrukhin V, Ginsburg B, Zhang Y. Hi-Fi Multi-Speaker English TTS Dataset. Proc. Interspeech 2021. arXiv:2104.01497.
3. Macmillan NA, Creelman CD. Detection Theory: A User’s Guide, 2nd ed. Lawrence Erlbaum, 2005.
4. Efron B, Tibshirani RJ. An Introduction to the Bootstrap. Chapman & Hall, 1993.
5. Wilson EB. Probable inference, the law of succession, and statistical inference. J. Am. Stat. Assoc. 1927;22(158):209–212.
6. McFee B, Raffel C, Liang D, et al. librosa: Audio and music signal analysis in Python. Proc. SciPy 2015.
7. Mauch M, Dixon S. pYIN: A fundamental frequency estimator using probabilistic threshold distributions. Proc. ICASSP 2014.

8Version history

• 1.2 (interim) — 6 November 2026: updated to 496 judgments from 75 listeners (41% of the planned 1,200). Headline accuracy 48.8% (v1.1: 48.0%). Ten judgments from two accounts since blocked by the platform’s location checks (data-centre VPN with an Africa/Nairobi time zone; a Nigerian mobile network) were inside the v1.1 dataset and are excluded from this version onward, because the panel is restricted to US residents; without them v1.1 would have read 159 of 336 (47.3%). The clip-length estimate moved from +5.3 to 0.0 points; the very-confident subgroup from 46.6% to 48.7%. In v1.1 the AI voice Eric was judged human more often than any narrator (88%); at 80% over 46 trials it is now level with the two most human-judged narrators, and §2.4 and the key findings were reworded accordingly; only one narrator, not two, is now judged AI on 80% or more of trials. A short listening quiz with six of the study’s clips was added at the top of the page. Table 5 is unchanged; Tables 1–4 and 6 and Figure 1 are recomputed.
• 1.1 (interim) — 5 November 2026: updated to 346 judgments from 63 listeners (29% of the planned 1,200); v1.0 covered 176 from 35. Headline accuracy 48.0% (v1.0: 48.9%). The v1.0 very-confident subgroup result (33.3%, interval excluding 50%) did not persist (46.6%, interval including 50%) and §2.3 has been rewritten around the weaker claim that confidence did not track accuracy; the clip-length estimate moved from −0.2 to +5.3 points, interval still spanning zero. Five judgments submitted from the author’s own rater account during an interface test on 3 November were found in the v1.0 dataset and are excluded from this version onward (without them v1.0 would have read 84 of 171, 49.1%). The planned sample was misstated in v1.0 as 1,050 (210 × 5); the study’s allocation is 1,200 judgments, 600 per ground-truth class. Table 5 (stimulus acoustics) is unchanged; Tables 1–4 and 6 and Figure 1 are recomputed.
• 1.0 (interim) — 3 November 2026; revised 4 November 2026 after external methodological review: added listener-and-speaker intervals, restated the length and confidence findings as estimates with intervals, described the human-side spectral adjustment explicitly, reframed the acoustic analysis as descriptive and the listener-cue account as an untested hypothesis, added engagement signals and the interim-snapshot status. Numerical results unchanged. A versioned update will be issued at the planned 1,050 judgments.