Woodstitch API — human ratings for TTS

Paid human listeners judge your audio and give you a number back. Use it when automated metrics (WER, speaker-sim, predicted MOS) can't answer the question.

Two templates, chosen with "template" on create:

`template`	A listener hears	and answers	you get
`tts_preference.v1` (default)	two clips	which one they prefer (forced choice, no tie)	preference for side B, 0.5 = no preference
`tts_detect.v1`	one clip	human or AI, plus confidence 1–4	share of clips taken for human, vs a real-human baseline

Use preference to pick between two systems or checkpoints. Use detection to find out whether output passes as real.

What Woodstitch does not do: MOS, CMOS, MUSHRA or any 1–5 rating scale. These two tasks are the whole product. If you need a mean opinion score, this is not the right tool; if you need to know which of two outputs people prefer, or whether output passes as human, it is. API studies are forced-choice; the ties fields in results are kept for schema stability and are 0 on API studies.

GET /v1/templates returns the same thing as JSON, no key needed.

Base URL: https://api.woodstitch.com/v1 Auth: Authorization: Bearer ws_live_... Machine-readable spec: /openapi.json (OpenAPI 3.1). This page as markdown: /docs.md. MCP server for Claude Code, Cursor and other agents: claude mcp add woodstitch -e WOODSTITCH_API_KEY=ws_live_... -- uvx woodstitch-mcp (woodstitch-mcp on PyPI).

Read this first (especially if you are an AI agent)

Always show the quote to the user before calling /start. Step 1 is free and tells you the exact cost. Step 2 spends real money and cannot be undone once raters begin answering.

Pass max_total_usd on every start. It is a hard ceiling; the call fails rather than overspending.

Send Idempotency-Key on every POST. If a call times out, retry with the same key — you get the original response back instead of a second charge.

Poll as often as you like. GET /v1/evals/{id} is cheap and returns live numbers. The first judgements usually land within minutes of starting, and the study completes when the last one does, which depends on its size and how many listeners are online. poll_after_s is a suggestion, not a limit.

Duplicate protection is automatic. Job identity is a hash of the audio content plus the question and rating count. Re-running the same script with different local paths returns the existing evaluation ("duplicate": true) rather than buying it twice.

Partial scores are published live. GET /v1/evals/{id} returns the current win rate, CI, and counts from whatever has been rated so far, flagged "provisional": true with the n it came from. Poll it as often as you like.

Know what you are looking at: deciding to stop because the number crossed a threshold you were watching is optional stopping, and it inflates false positives badly — a test built for p<0.05 behaves more like p<0.2 under continuous peeking. Treat an interim figure as a preview. Whether to act on one is your call.

No key yet? Start without one

POST /v1/start (no auth) creates an account in the background, puts the first-study credit on it, and returns an api_key to use on every later call. Send source (your git remote) so the account has a name; email is optional. Nothing is asked of the person. One account per call, rate-limited per address.

curl -sX POST https://api.woodstitch.com/v1/start -H "Content-Type: application/json" \
  -d '{"source": "github.com/your-org/your-repo"}'
# -> {"api_key":"ws_live_…","credit_usd":25,"next":"POST /v1/files, then POST /v1/evals …"}

To open the account in a browser later, attach an email with POST /v1/account/email {"email": "…"} and sign in at https://woodstitch.com/app (a sign-in link is emailed). The step-by-step recipe an agent should follow is llms.txt.

The whole flow

# 0. Upload your audio. Skip this if it already sits at public URLs. Any number of files per call.
curl -sX POST https://api.woodstitch.com/v1/files \
  -H "Authorization: Bearer $WOODSTITCH_KEY" \
  -F "file[]=@base/001.wav" -F "file[]=@cand/001.wav" -F "file[]=@base/002.wav" -F "file[]=@cand/002.wav"
# -> {"files":[{"id":"file_7c1e…","name":"001.wav","duration_s":12.4,"format":"wav"}, ...]}

# 1. Create — measures every clip, returns a price. CHARGES NOTHING.
curl -sX POST https://api.woodstitch.com/v1/evals \
  -H "Authorization: Bearer $WOODSTITCH_KEY" \
  -H "Content-Type: application/json" \
  -H "Idempotency-Key: ckpt-42-create" \
  -d '{
    "name": "ckpt-42 vs baseline",
    "question": "Which audio sample do you prefer if you were listening to an audiobook?",
    "ratings_per_pair": 5,
    "pairs": [
      {"id":"utt-001","a":"file_7c1e…","b":"file_09aa…"},
      {"id":"utt-002","a":"file_b3d0…","b":"file_55f2…"}
    ]
  }'
# -> {"eval_id":7,"quote":{"id":3,"total_usd":5.80},"nothing_charged_yet":true,
#     "dashboard_url":"https://woodstitch.com/app/evaluations/7",...}

# 2. Start — THIS SPENDS MONEY. Confirm the amount with a human first.
curl -sX POST https://api.woodstitch.com/v1/evals/7/start \
  -H "Authorization: Bearer $WOODSTITCH_KEY" \
  -H "Content-Type: application/json" \
  -H "Idempotency-Key: ckpt-42-start" \
  -d '{"quote_id":3,"max_total_usd":10.00}'

# 3. Poll as often as you like: live numbers, first judgements usually within minutes.
curl -s https://api.woodstitch.com/v1/evals/7 -H "Authorization: Bearer $WOODSTITCH_KEY"

# 4. Results
curl -s https://api.woodstitch.com/v1/evals/7/results -H "Authorization: Bearer $WOODSTITCH_KEY"

Pricing

Free to check, no key needed:

curl -s https://api.woodstitch.com/v1/pricing

Priced per judgement, by how many seconds of audio the rater hears — measured server-side from your audio, never from a number you declare, and never rounded up to a band.

usd_per_judgement = audio_seconds / 3600 × 18.00      (rounded up to the cent)

Template	`audio_seconds` is	15 s clips	30 s clips	60 s clips
`tts_preference.v1`	`dur_a + dur_b` — the rater hears both clips	$0.15	$0.30	$0.60
`tts_detect.v1`	the clip's own duration — one clip	$0.08	$0.15	$0.30

A 10 s + 50 s pair costs exactly what a 30 s + 30 s pair costs. Detection costs half of preference at the same clip length because the rater hears one clip instead of two.

Every pair or clip is priced independently. One long outlier does not reprice the rest, and dry_run returns each unit's measured listening time and its price so you can see exactly what you are buying before paying.

preference:  total = sum over pairs of (usd_per_judgement × ratings_per_pair)
detection:   total = sum over clips of (usd_per_judgement × ratings_per_clip)

200 pairs × 30 s clips × 5 ratings = 1,000 judgements = $300.00 600 clips × 30 s × 8 ratings = 4,800 judgements = $720.00

Clips must be 0.5–120 s. Up to 5,000 pairs or clips per evaluation.

Billing is metered

You hold a balance and it drains as judgements arrive — there is no up-front reservation of the whole study.

Money is committed when a task is handed to a rater, not when the answer comes back, so a rater mid-task is always paid even if you cancel.
/start needs a positive balance, not the whole quote. Run out of funds and the eval pauses (pause_reason: insufficient_funds) rather than failing. Top up and it resumes by itself.
POST /v1/evals/{id}/cancel stops it. You are charged only for work already handed out; max_additional_charge_if_canceled_now_usd on the progress response tells you that figure before you decide.
Abstentions are not charged.

How listeners are recruited, paid and screened is in the platform methodology. Rate quality is the product.

How many ratings do I need?

This is the question most people get wrong. Preference gaps are small, and required sample size explodes as the gap narrows:

True gap (Elo)	Expected preference	Judgements for 80% power
50	57%	~400
30	54%	~1,100
20	53%	~2,400
10	51.4%	~9,500
2	50.3%	~237,000

Those are idealized. Ties and repeated raters push the real number higher. If you are trying to resolve a sub-10-Elo difference, this is the wrong tool — no human panel will settle it affordably.

A good default for "is my new checkpoint better than baseline?" is 200 pairs × 5 ratings, which reliably detects moderate-to-large differences.

Endpoints

`POST /v1/files` — upload audio

Multipart. One file as -F file=@x.wav, or several as repeated file[] parts (up to 100 files or 1 GB per call, 50 MB per file; a plain repeated file= keeps only the last one, and the API tells you so). Each file is decoded, measured and stored; you get back an id to use anywhere create takes a URL.

curl -sX POST https://api.woodstitch.com/v1/files \
  -H "Authorization: Bearer $WOODSTITCH_KEY" \
  -F "file[]=@base/001.wav" -F "file[]=@cand/001.wav"

{"object":"list","count":2,"files":[
  {"id":"file_7c1e4f…","object":"file","name":"001.wav","duration_s":12.4,"format":"wav","bytes":1190456,"created_at":"2026-11-05T19:02:11Z"},
  {"id":"file_09aa31…","object":"file","name":"001.wav","duration_s":12.1,"format":"wav","bytes":1162004,"created_at":"2026-11-05T19:02:11Z"}
]}

A single-file call returns that file object directly (with the same files list attached). Files are returned in the order you sent them and carry their original name, so pairing two folders by filename is a one-liner. Ids are private to your account. If one file in a call is bad, the call fails and nothing from it is recorded; fix the file and resend the call. Re-sending the same bytes is harmless: storage is content-addressed, so nothing is duplicated.

Also: GET /v1/files (your uploads, newest first), GET /v1/files/{id}, and DELETE /v1/files/{id} to remove an upload from your account. Studies that already used a file keep their copy.

`POST /v1/evals` — create and price

Free. Resolves every clip (a file id or a URL it downloads), validates it decodes, measures duration, stores an immutable copy, and returns a quote.

Field	Required	Default	Notes
`template`	no	`tts_preference.v1`	Or `tts_detect.v1`.
`pairs`	preference	—	Array of `{id, a, b}`. Max 5000. `a`/`b` are file ids from `POST /v1/files` or public http(s) URLs.
`clips`	detection	—	Array of `{id, audio, truth, source, speaker, passage, duration_bucket}`. Max 5000. `audio` is a file id or a URL.
`ratings_per_pair`	no	5	1–50. Independent judgements per pair. (`judgments_per_item` is accepted as an alias.)
`ratings_per_clip`	no	5	Detection's name for the same thing.
`pairs[].system_a`, `pairs[].system_b`	no	—	Labels for the two sides (model ids). Never shown to raters. Needed for `per_participant` rules on `system`.
`pairs[].passage`, `pairs[].speaker`	no	—	Text id and voice id, for no-repeat rules and your own analysis.
`per_participant`	no	—	No-repeat rules for one listener; see Controlling who hears what. Works on both templates.
`allocation`	detection only	—	Study-wide balancing. Refused on preference studies, where every pair gets `ratings_per_pair`.
`question`	no	per template	Max 200 chars. Shown verbatim to raters.
`name`	no	"eval"	Your label.
`dry_run`	no	false	Price it and throw it away. Nothing is stored, no `eval_id`.

See the detection template for what each clip field means and why speaker and passage are required.

Pricing without committing: send "dry_run": true to get the quote and pair_seconds — every pair's measured listening time and its price per judgement — with nothing persisted. Use it to spot an over-long pair before you commit.

The create response also carries a warnings array: it flags an underpowered design, and flags any pair whose two sides are in different audio formats (a WAV-vs-MP3 pair risks measuring your encoder instead of your model).

Audio: wav, mp3, flac, ogg, m4a. Each clip 0.5–120s, max 50MB. Give each clip as a file id from POST /v1/files (uploaded from disk) or as a public http(s) URL. A URL must stay reachable for the duration of this call only — we keep our own copy. Redirects are not followed, so use direct or presigned links.

Which side is which: a and b are preserved exactly as you send them. Raters see them in randomized order and are never told what either one is.

Use the same format and bitrate on both sides. If a is 48kHz WAV and b is 128kbps MP3, you are partly measuring your encoder, not your model. Export both sides identically.

pairs[].id must be unique within the evaluation, max 64 characters. It is echoed back in per_pair so you can join results to your own data.

Comparing several systems at once. Send one pair per matchup, label the sides with system_a / system_b, give each text a passage id, and add a per_participant block so no listener hears one system often enough to recognise it:

"pairs": [
  {"id":"p01-x-y","a":"https://.../x_p01.wav","b":"https://.../y_p01.wav",
   "system_a":"x","system_b":"y","passage":"p01"}
],
"per_participant": {"max_judgements": 16, "unique_by": ["passage"], "max_per": {"system": 7}}

The result is still per pair; the export carries the labels so you can tally wins per system yourself.

`POST /v1/evals/{id}/start` — begin (spends money)

Field	Required	Notes
`quote_id`	no	If given, must match the current quote, else `quote_mismatch`.
`max_total_usd`	no	Hard ceiling. Fails with `over_max` rather than overspending.

Needs a positive balance and a key whose spend cap can cover the quote. Nothing is debited here; billing is metered from the first judgement.

`GET /v1/evals/{id}` — progress and live score

Returns the submitted question, pairs, ratings_per_pair, cost_usd and started_at, plus judgements.done / .target / .percent, pairs_with_data, and the same metrics / counts shape as /results — computed from whatever has been rated so far.

"provisional": true until the evaluation completes. Status values: ready, running, completed, canceled, failed.

Everything an evaluation needs to be resumed is in this response, so an agent that restarts only needs the eval_id.

Every response that names an evaluation also carries dashboard_url, the page where a person can watch the same study in the browser (https://woodstitch.com/app/evaluations/{id}, sign-in required). Hand it to the user when you report the eval_id.

`GET /v1/evals/{id}/results` — final numbers

409 not_ready while the evaluation is still running. Available once it is completed, or canceled — a canceled study returns whatever was judged before the stop, with "incomplete": true and judgements_target so you can see how far it got.

{
  "metrics": {
    "human_pref/b_win_rate": 0.7222,
    "human_pref/b_win_rate_ci95_low": 0.6667,
    "human_pref/b_win_rate_ci95_high": 0.8333,
    "human_pref/tie_rate": 0.1111,
    "human_pref/pairs": 3,
    "human_pref/judgements": 9
  },
  "counts": {"a_wins": 2, "b_wins": 6, "ties": 1},
  "per_pair": [{"id":"utt-001","a_wins":1,"b_wins":2,"ties":0}]
}

metrics is a flat numeric dict, so it drops straight into a tracker:

wandb.log(result["metrics"])

Reading the score. b_win_rate is (b_wins + 0.5*ties) / total — the share of preference for your b side. 0.5 means indistinguishable. Ties count half to each side, because a pair nobody can tell apart is evidence of parity, not evidence for either one.

The confidence interval is bootstrapped by pair, not by vote. Multiple judgements on the same pair are not independent, so a naive binomial interval would be far too narrow. If the interval spans 0.5, you have not shown a difference.

`GET /v1/evals/{id}/export` — per-judgement rows

One row per judgement, with every clustering dimension: rater (pseudonymous, stable within the eval), source, speaker, passage, duration bucket, answer, confidence, time spent. Add ?format=csv for CSV.

This is what you fit your own mixed-effects model on.

`POST /v1/evals/{id}/cancel` — stop early

Charges you only for work already handed to raters. Anyone mid-task finishes and is paid. Check max_additional_charge_if_canceled_now_usd on the progress response first if you want to know that number before deciding.

`POST /v1/start` — account + key + first-study credit, no auth

See Start without one. Returns api_key, credit_usd, account.

`POST /v1/account/email` — attach an email to the account

{"email": "…"}. Needed once before the dashboard sign-in link can find an account created by /start without an email.

`POST /v1/account/topup` — add funds

A paused evaluation resumes automatically when the money lands.

`GET /v1/account` — balance and spend

`GET /v1/evals` — your last 50 evaluations

`GET /v1/files` — your uploads · `GET /v1/files/{id}` · `DELETE /v1/files/{id}`

`GET /v1/templates` — both templates, their inputs and metrics (no auth)

`GET /v1/pricing` — rate card for both templates (no auth)

The detection template — `tts_detect.v1`

A listener hears one clip and answers "was this made by a human or AI?", then rates their confidence 1–4. Unlike preference, this template has a right answer, so you send ground truth with every clip.

What you must send per clip

Field	Required	Notes
`id`	yes	Unique within the eval. Echoed back in results.
`audio`	yes	Public http(s) URL. Same formats and limits as preference.
`truth`	yes	`"human"` or `"ai"`. Never sent to a listener.
`source`	yes*	The model id (`"eleven_v4"`). Real narration uses the reserved value `"human"` — and may omit the field.
`speaker`	yes	Voice id for AI, narrator id for human.
`passage`	yes	The text id. Use the same id for every rendering of the same text, across every source.
`duration_bucket`	no	Your design condition, e.g. `"15s"`, `"30s"`, `"60s"`. Free text, max 16 chars.

speaker and passage are required because the result resamples over them to work out its uncertainty. Without them there is no honest interval to report.

curl -sX POST https://api.woodstitch.com/v1/evals \
  -H "Authorization: Bearer $WOODSTITCH_KEY" \
  -H "Content-Type: application/json" \
  -H "Idempotency-Key: realism-001" \
  -d '{
    "template": "tts_detect.v1",
    "name": "realism vs human narration",
    "ratings_per_clip": 8,
    "clips": [
      {"id":"h-p01","audio":"https://.../human_p01.wav",
       "truth":"human","source":"human","speaker":"narrator_03","passage":"p01","duration_bucket":"30s"},
      {"id":"e-p01","audio":"https://.../eleven_p01.wav",
       "truth":"ai","source":"eleven_v4","speaker":"rachel","passage":"p01","duration_bucket":"30s"}
    ]
  }'

You need real human recordings in the same study

A fooling rate on its own means nothing: if 40% of your clips are called human, that could be a realistic model or an inattentive panel, and you cannot tell which. The real-human baseline is what separates the two, so include genuine narration of the same passages and let the API compute the gap.

Create returns a warning if you leave it out, and no baseline_deficit.

The result

{
  "metric": "human_label_rate",
  "results": [
    {"source":"eleven_v4","duration":"30s","human_label_rate":0.41,"n":480,
     "ci95":{"tested_stimuli":{"low":0.37,"high":0.46},
             "model":        {"low":0.28,"high":0.55}},
     "baseline_deficit":0.37,
     "baseline_deficit_ci95":{"tested_stimuli":{"low":0.31,"high":0.43},
                              "model":        {"low":0.19,"high":0.54}}}
  ]
}

human_label_rate is the share of judgements that called the recording human. For an AI source that is the fooling rate; for source: "human" it is the recognition rate. baseline_deficit is the human rate minus this source's rate — 0 means indistinguishable from real narration on these stimuli.

Two intervals, and why you must not mix them up

Every number comes with two confidence intervals:

`scope`	Answers	Resamples
`tested_stimuli`	"What if a different panel heard these exact clips?"	raters
`model`	"What if I re-ran this with different voices and passages from the same model?"	raters, voices and passages together

The model interval is typically 3–4× wider. That is not pessimism, it is the cost of a claim about a model rather than about a set of files.

This was measured, not assumed. Simulated against a known ground truth on a 4-source / 5-voice / 30-passage / 400-rater design, at nominal 95%:

interval	covers its own estimand	covers the model-level truth
`tested_stimuli`	96.7%	46.7%
`model`	—	93.8–96.0%

Quoting the narrow interval as a statement about a model is wrong more than half the time. If you are going to publish a sentence of the form "model X fools listeners Y% of the time", the model interval is the one that belongs next to it. Use tested_stimuli when the claim really is about the specific files you tested — a release gate on this build, for instance.

The response also carries a generalisation block stating in machine-readable form what the data does and does not support, and coverage with the counts behind it.

More voices, not more ratings

If a source has few voices, buying more judgements will not fix the model interval — between-voice variance is what makes it wide, and only more voices reduce it. With fewer than 8 voices the response says "to_the_model": "not_established" and tells you so in warnings.

The honest way to report a 5-voice study is "these five voices", not "this model".

Controlling who hears what

By default every clip gets ratings_per_clip judgements and any rater may be served any clip. Two optional blocks change that. per_participant works on both templates (for preference, the system field reads the pair's system_a / system_b labels); allocation is detection only. They are named for their scope, which is the thing to get right: a rule either applies across the whole study or within one participant.

{
  "template": "tts_detect.v1",
  "clips": [ ... ],

  "allocation": {
    "total_judgements": 2400,
    "balance_by": ["truth", "duration_bucket"]
  },

  "per_participant": {
    "max_judgements": 12,
    "counts": { "truth": { "human": 6, "ai": 6 } },
    "unique_by": ["speaker", "passage"]
  }
}

allocation — study-wide targets.

Field	Means
`total_judgements`	How many judgements to buy in total.
`balance_by`	Target equal completed judgements in each combination of these fields.

balance_by is what lets a small group of clips carry as much weight as a large one. With 36 human clips and 216 AI clips, balance_by: ["truth"] gives each human clip ~33 judgements and each AI clip ~6 — so the human baseline is estimated as precisely as the models, which a single uniform count cannot do.

Balance on every field whose comparison you care about. A field you leave out will drift, because the remainder from integer division lands wherever it happens to land. If the study compares durations, name duration_bucket.

per_participant — one person's experience.

Field	Means
`max_judgements`	Most clips this rater will ever hear in this study. Not per sitting — per person.
`counts`	Required mix within that, e.g. 6 human and 6 AI. Must sum to `max_judgements`.
`unique_by`	Fields that may not repeat for a rater.

unique_by is the one that protects a repeated-measures design. If a rater hears voice V at 60 seconds and again at 15 seconds, they recognise the voice and carry over their earlier answer — and because longer clips are more recognisable, that contamination flows from the long arm into the short one and flattens the very effect being measured. unique_by: ["speaker", "passage"] prevents it. Valid fields: speaker, passage, source, truth, duration_bucket, system.

max_per caps how often a rater may meet the same value of a field, where unique_by is the special case of a cap of 1. It exists because an A/B field like system is too scarce to be unique — five systems make only two disjoint pairs — so "max_per": {"system": 7} lets a rater work through the study while stopping anyone hearing one system often enough to start recognising it.

These two are independent: no repeated speaker and no repeated passage, not merely no repeated speaker-passage pair.

Per-participant balance is not inferred from global balance. They are different designs. If each rater must hear 6 human and 6 AI, say so in counts; balance_by alone only balances the totals.

What create tells you

The response carries effective_design — the compiled plan, so you can check what you are buying before paying:

"effective_design": {
  "judgements_total": 2400,
  "ratings_per_clip": { "min": 6, "max": 34 },
  "judgements_by": {
    "truth": { "human": 1200, "ai": 1200 },
    "duration_bucket": { "15s": 800, "30s": 800, "60s": 800 }
  },
  "per_participant": { "max_judgements": 12, ... }
}

An impossible design is refused at create time rather than stalling halfway through a paid study. If unique_by includes passage and a rater needs 12 clips but you supplied 8 passages, that is infeasible_session with the numbers in the message.

Expect a shortfall

These constraints mean some raters cannot be given a valid session and are simply turned away — a safe outcome, because the alternative is serving a clip that violates the design.

Simulated on a 12-passage, 6-voice, 3-duration study: 91% of target at 200 raters, 96% at 250, and flat after that. More raters past that point mostly get declined. Plan for a shortfall and read the completed counts in the results rather than assuming every target is met.

Confidence

Raters pick 1 (guessing) to 4 (very confident). It is exploratory only: it never affects rater pay, never filters a judgement, never gates quality — and raters are told that, which is the only reason the numbers mean anything. Results include an accuracy-by-confidence breakdown as a calibration check.

A listener who cannot judge (audio failed) records an abstention, not confidence 1. Abstentions are reported separately, the rater is still paid, and you are not charged — the clip goes back out to someone else.

Fit your own model

GET /v1/evals/{id}/export returns one row per judgement with every clustering dimension (rater, speaker, passage, duration, answer, confidence, timing). Add ?format=csv for CSV. Rater ids are pseudonyms, stable inside the eval.

If you want the hierarchical logistic model this API does not fit, that export is what you fit it on.

Design checks before you pay

Create runs these and returns anything it finds in warnings:

no human baseline, or no AI clips at all
lopsided stimulus counts (a baseline resting on far fewer recordings)
a source with only one voice — that result describes the voice, not the model
fewer than 8 voices per source, or fewer than 10 passages
no passage shared by every source — sources then get compared on different text, and an easy-text model simply looks more human
mixed audio formats, which in a detection study is the most dangerous confound there is: listeners learn to hear the codec, not the voice
too few judgements to separate anything

Run with "dry_run": true first to see all of it for free.

Errors

Always:

{"error": {"code": "insufficient_funds", "message": "Balance is $0.00. Add funds to begin.", "hint": "Top up at https://woodstitch.com/app"}}

Code	HTTP	Meaning
`no_key` / `bad_key`	401	Missing or invalid `Authorization: Bearer`.
`insufficient_funds`	402	Balance is empty. Billing is metered, so `/start` needs a positive balance, not the whole quote; a running study pauses instead if it runs dry.
`spend_cap`	402	This key's cap would be exceeded.
`over_max`	400	Quote exceeds your `max_total_usd`.
`idempotency_conflict`	409	Key reused with a different body.
`not_ready`	409	Results requested while still running.
`bad_allocation`	400	`allocation` sent on a preference study, or malformed.
`bad_audio`	400	Clip won't decode, or is outside 0.5–120s.
`fetch_failed`	400	We couldn't download the URL.
`bad_file`	400	A file id that is not on this account, or whose audio is gone.
`no_files` / `upload_failed` / `too_many_files`	400	Nothing usable in a `POST /v1/files` call; the message names the file.
`upload_too_large`	413	A call over 1 GB, or a file over 50 MB. Send fewer files per call.
`quote_expired`	400	Quotes last 24h. Create again.

What raters actually do

Sign in, hear clip A and clip B in full (the choice is locked until both finish playing), and pick A, B, or "sounds the same." They're paid per answer. They are never told what the systems are, and side order is randomized per rater, so position bias can't favour either side.

Woodstitch API — human ratings for TTS

Read this first (especially if you are an AI agent)

No key yet? Start without one

The whole flow

Pricing

Billing is metered

How many ratings do I need?

Endpoints

POST /v1/files — upload audio

POST /v1/evals — create and price

POST /v1/evals/{id}/start — begin (spends money)

GET /v1/evals/{id} — progress and live score

GET /v1/evals/{id}/results — final numbers

GET /v1/evals/{id}/export — per-judgement rows

POST /v1/evals/{id}/cancel — stop early

POST /v1/start — account + key + first-study credit, no auth

POST /v1/account/email — attach an email to the account

POST /v1/account/topup — add funds

GET /v1/account — balance and spend

GET /v1/evals — your last 50 evaluations

GET /v1/files — your uploads · GET /v1/files/{id} · DELETE /v1/files/{id}

GET /v1/templates — both templates, their inputs and metrics (no auth)

GET /v1/pricing — rate card for both templates (no auth)

The detection template — tts_detect.v1