Building a 240-million-sample voice-acting corpus

The plan, the mechanics, and what it costs — measured on this hardware, against the actual compute budgets.

What this is, in plain language

A text-to-speech model can be asked to say a sentence angrily, quietly angry, very fast, with a lot of vocal fry. This project builds the training data for that, systematically: for each of 6,000 distinct voices it renders every cell of a 832-condition grid, in English and German, and stores every sample with full measurements attached.

The end goal is a speech model that can be steered — and, further out, a speech-to-speech translation model that keeps the speaker and the emotional level across languages. That second goal is why every text exists as an EN/DE pair of the same sentence.

samples

240 M

6,000 voices × 832 conditions

audio

523k h

at 7.86 s mean take length

compute

15.5 M

core-hours, speech generation

of budget

15 %

of both budgets combined

Hear what a "condition group" actually is

Everything below is counted in "condition groups" and "samples", which means nothing until you have heard one. Three complete voice profiles are published — the same reference voice rendered through all 832 conditions, with the winning take of each group playable and its scores shown:

Profile A — translation-corpus sentences: the original text source. Every emotion at four intensity/containment settings, every VoiceNet dimension at four levels from "extremely low" to "very high" so you can hear the extremes back to back, plus edge cases and style LoRAs, in English and German.
Profile B — the same, with 25 % Mediathek LoRA: identical sentences and conditions, one adapter merged in. Kept so the null result is audible rather than merely asserted.
Profile RW — sentences rewritten to carry the emotion: the arm this whole plan is built around. Same voice, same conditions, different words.

Compare A against RW on the same emotion — Bitterness or Contempt are the clearest — and the argument for building the corpus this way stops being a table of numbers. In A the speaker is reciting a neutral sentence about a gym membership in a bitter tone of voice. In RW the sentence gives them something to actually be bitter about.

What comes out at the end, concretely

For an outsider: the deliverable is not a model. It is a dataset — roughly a quarter of a billion short speech clips, each one labelled with what it was asked to be and measured for what it actually turned out to be. Somebody else then trains on it.

Each row of that dataset is: a voice (one of 6,000 reference speakers), a sentence (English or German, paired with its counterpart in the other language), a condition (one of 832 — "intense, freely expressed anger", "very high vocal fry", "a fearful gasp mid-sentence"), the audio, and about a hundred numbers describing the result. Nothing is thrown away: the takes that scored badly are stored too, because a later change of mind about what "good" means should never require regenerating anything.

Two subsets, built for two different jobs

The corpus is deliberately split. The two halves teach opposite things, and a model needs both.

Subset A — 32 samples per condition, every one a different sentence

For each of the 832 conditions, 32 samples are generated from 32 different texts, each written so its content justifies that exact condition. This is the breadth half: it stops the model from associating "anger" with one particular sentence, and gives a pre-training corpus real lexical variety.

Why this matters more than it sounds. A controlled experiment on this pipeline showed that a dozen emotions produced almost no measurable emotion at all — Bitterness sat at −0.005 — not because the conditioning was broken but because the sentences, drawn from a translation corpus, gave the speaker nothing to feel. Rewriting the text so its content carries the emotion raised emotion strength +22 %, genuineness +27 % and vocal-burst blend +39 %, with audio quality flat and word error rate down 12 %. Text is the single most effective lever measured on this system, so the corpus is built around it.

Subset B — 16 samples per condition, all sharing one text

For each of the 125 parent groups (40 emotions, 57 VoiceNet dimensions, edge cases, character clusters, sports, explicitness), 16 texts are each rendered through all four sub-conditions. For Anger that is intense/free-flowing, intense/contained, moderate/free-flowing and moderate/contained — 16 × 4 = 64 samples per language, the same words every time.

This is the controllability half. Holding the sentence fixed and varying only the intensity gives a model minimal pairs: the only thing that differs between two samples is the dimension you want to control. That is exactly what a later fine-tune needs in order to learn "say this same line, but angrier" — and it is impossible to learn from subset A, where text and condition vary together.

Where the texts come from

Every text is written by a Gemma model before any speech is generated, and every one is seeded rather than invented from nothing — a model asked to write freely collapses each emotion onto its most convenient topic (every bitterness line about a betrayal), destroying the domain balance the corpus is supposed to have.

90 % — a different FineTranslations sentence per sample, paraphrased so its content supports the target condition. Sampling is balanced across the 14 inferred domains.
10 % — ten domains where TTS systems characteristically break, at 1 % each. At this scale 1 % is 1.60 M samples per domain, which is a substantial corpus in its own right. Within a domain, variety comes from rotating seed keywords, so the model is not handed the same twelve equations a million times.

hard domain (1 % each)	what breaks	seed keywords
Mathematics & formulae	equations, Greek letters, operators, fractions, exponents, sub/superscripts	α, ∑, x², 3/4, ∫, ≠, ±, matrix, derivative, lemma
Numbers, units & measures	large numbers, decimals, ordinals, ranges, currencies, units	1,250,000, 3.14, 21st, 1990-1995, €4.99, 37 °C, 5 km/h
Dates, times & durations	formats that expand differently depending on locale and context	3/4/25, 1900-1945, 14:05, Q3, 2h30, fortnight
Abbreviations & acronyms	letter-by-letter vs word-like, and the ones read as their expansion	NASA, e.g., i.e., Dr., GmbH, z.B., ISO 9001, PhD
Homographs & heteronyms	same spelling, different pronunciation, disambiguated only by sense	read/read, lead/lead, live/live, bow, tear, German 'ubersetzen', 'Modern'
Proper nouns & loanwords	personal, place and brand names, including foreign-origin spellings	Xiaomi, Worcestershire, Nguyen, Reykjavik, Bourgogne, Oaxaca
Code-switching EN↔DE	a mid-sentence language switch -- directly load-bearing for this project's goal	'Ich habe das feature schon deployed', 'the Umlaut is tricky'
URLs, emails, paths & code	strings that are not words and must not be read as words	https://a.io/x?q=1, snake_case, C:\\Users, <div>, npm i -g
Slang, profanity & net register	registers a filtered web corpus under-represents, including censored forms	swear words, f***, lol, innit, Digga, emoji-adjacent text
Technical, medical & legal jargon	long low-frequency terms with irregular stress	acetylsalicylic acid, §823 BGB, pneumonoultramicroscopic, myocardial infarction

Caveat on that list, stated rather than hidden. It is assembled from the standard non-standard-word taxonomy used in text-normalisation work and from known failure modes — not from a fresh literature review, because the web search tool was unavailable when this page was written. The categories are uncontroversial, but the specific choice of ten and their relative weighting deserve a review before committing compute.

Texts are written to a target length, on purpose

This is not cosmetic — it is worth real money, for a reason specific to how batched generation works. See the batching section.

Why batching cares about sentence length

Speech is generated autoregressively in batches of 64. A batch runs until every sequence in it has finished, so one long take holds up the other 63. Today all 64 samples in a batch are the same sentence, so they finish together. Give each sample a different text and the batch waits for the longest one.

Measured on 25,850 real takes (mean 7.86 s against a 32 s generation cap, so takes genuinely do stop early — if they all ran to the cap this would cost nothing):

batch composition	expected longest take	relative cost
64 × the same text (today)	12.46 s	1.00×
64 × different texts, unsorted	23.20 s	1.86×
64 × different texts, sorted by word count	16.77 s	1.35×
+ texts written to a target length (estimate)	—	~1.15×

Different token counts per sample cost almost nothing on their own — that only affects prefill, one forward pass, against ~96 sequential decode steps. The entire penalty is the spread in durations. Which is why the text generator is asked for a target length: it turns a 1.86× tax into something near 1.15×.

And per useful sample, diverse texts are far cheaper, not dearer. Today a group spends 32 generations to yield one selected take. Under this plan those same 32 generations yield 32 distinct usable samples. That is roughly 17× less compute per usable pre-training sample, at a ~35 % batching premium. The trade is that each sample is a single attempt rather than the best of 32, so per-clip quality sits at the distribution's mean instead of its maximum — the right trade for pre-training breadth, the wrong one for a showcase.

Vocal bursts, placed deliberately

A gasp, a sigh, a shaky breath, a laugh. There are 64 trained burst adapters, and the corpus should contain them inside sentences rather than as isolated sound effects.

The measured recipe is specific, and it constrains the design:

burst adapter dose	burst occurs	rest of the line survives	blend	WER
none (inline tag in the script only)	21.9 %	1.00	6.63	0.05
λ = 0.50 ← the operating point	50.3 %	0.90	4.93	0.15
λ = 1.00	72.1 %	0.45	4.07	0.57

At full merge more than half the words after the burst are never spoken. λ = 0.5 is the measured optimum for "burst and sentence and blend".

Design rules that follow:

Every second sentence carries 1–2 bursts, not every sentence. A corpus where every angry line contains a growl teaches the model that anger implies growling.
All 64 classes must end up roughly equally represented — a hard constraint, not a preference. See the section below.
The burst is written inline in the script as well as merged. The tag alone already fires 21.9 % of the time; the adapter multiplies that, it does not replace it.
No emotion adapter stacked on top when the burst is the point — measured, the emotion adapter suppresses bursts, 23.6 % → 16.7 %, below even the no-adapter baseline.

A consequence that has to be accepted up front. With one take per text and no best-of-N selection, a tagged burst fires about half the time. So of the 50 % of sentences tagged, roughly 25 % of subset A — about 40 M samples — will actually contain a burst, and the rest will be ordinary speech whose script mentioned one. That is still an enormous burst corpus, but the metadata must record requested and detected separately, or every downstream user will mistake the tag for a label. The manual's 4–8-candidate selection recipe reaches ~97 %, at 4–8× the cost; it is the right choice for a curated subset, not for all 160 M.

Every burst class, equally often

The obvious way to assign bursts — let each emotion draw the classes that fit it — produces a wildly skewed corpus. Sighs and gasps fit almost every emotion and would appear constantly; wolf_whistle, hiccup or gurgling fit almost none and would nearly vanish. A class with a few thousand examples is not learnable next to one with several million.

So the allocation is inverted: the target count per class is fixed first, and emotion-fit only decides where each class's quota gets spent.

Fix a global budget of burst-carrying samples and divide it equally: 64 classes → 0.63 M samples each.
For each class, take the set of emotions it is plausible in, and spread that class's quota across them — rather than letting each emotion pick freely.
Where a class fits only one or two emotions, it still gets its full quota; it simply concentrates there. Coverage of the class is what matters, not its spread across emotions.

All 64 shipped classes qualify. In the manual's evaluation every one of the 64 scored 2 on the blind Gemini judge — the same score as the real recorded audio control, against 0.667 for the base model with no adapter. There is no subset of "classes that work"; they all cleared the bar, so all 64 are in the balanced rotation.

Two caveats carried over from that evaluation: training-set sizes are very uneven (75 items for swallows against 1,396 for the largest, median 178), and 19 of the 64 were scored without a blind metric. Equal representation in the corpus is therefore not a claim of equal quality — per-class burst-detection rates should be measured on the output and published alongside, so a downstream user can weight them.

Order the work queue by adapter, not by condition. Each sample needs an emotion adapter and possibly a burst adapter. With 40 emotions × 64 burst classes, a naive ordering thrashes the adapter cache — and adapter eviction has already cost this project four failed runs. Sorting the queue by (emotion adapter, burst adapter) loads each combination once.

What it costs

1. Writing the texts

model	subset A texts	subset B pool	GPU-h	core-h	share of total job
Gemma 4 E4B	79.9 M pairs	200k pairs	3,767	271,222	1.7 %
Gemma 12B bf16	79.9 M pairs	200k pairs	11,301	813,665	5.0 %

Take the 12B. Going from E4B to 12B costs an extra 542,443 core-h — 3.3 % of the whole job. Text generation is a rounding error against speech generation either way, so there is no reason to economise on the component that determines what every sample actually says. Use E4B only if a side-by-side shows the two are genuinely indistinguishable on this task.

The E4B figure (1,299 tok/s at batch 128) is measured on one GH200. The 12B figure (433 tok/s) is extrapolated by FLOP ratio and is the weakest number on this page — but since the whole line item is ~4 % of the job, even a 2× error changes nothing.

2. Generating the speech

batching strategy	subset A GPU-h	subset B GPU-h	total GPU-h	core-h
naive (no bucketing)	198,083	99,041	297,124	21,392,916
word-count bucketing	143,770	71,885	215,654	15,527,117
length-controlled + bucketing	122,470	61,235	183,706	13,226,803

3. GPU-hours by condition block

Speech generation only, all 6,000 voices, both subsets. Cost is proportional to sample count, so it follows the block sizes directly — the VoiceNet block is the expensive one simply because it is the biggest.

block	what it is	groups	samples (6,000 voices)	GPU-h	share
Emotions	40 emotions x {intense, moderate} x {free, contained} x {EN, DE}	320	92.2 M	82,944	38.5 %
VoiceNet dimensions	57 dimensions x 4 levels x {EN, DE}	456	131.3 M	118,195	54.8 %
Edge cases	screams, groans, shivers, laughter, whimpering, crying	28	8.1 M	7,258	3.4 %
Character clusters	12 character LoRAs x {EN, DE}	24	6.9 M	6,221	2.9 %
Sports commentator	x {EN, DE}	2	0.6 M	518	0.2 %
Explicitness	x {EN, DE}, behind an age gate	2	0.6 M	518	0.2 %
total			240 M	215,654	100 %

4. GPU-hours by scale

The same thing per number of reference voices. These are GPU-hours; JUPITER bills whole nodes, so multiply by 72 for core-hours (288 core-h per node-hour, 4 GPUs per node).

voices	samples	audio	text GPU-h	speech GPU-h	total GPU-h	core-h	wall-clock @64 nodes
100	4.0 M	9k h	188	3,594	3,783	272,346	0.6 days
500	20.0 M	44k h	942	17,971	18,913	1,361,732	2.9 days
1,000	39.9 M	87k h	1,883	35,942	37,826	2,723,464	5.9 days
6,000	239.6 M	523k h	11,301	215,654	226,955	16,340,781	35.1 days

5. Against the budgets

scenario (speech + 12B text)	core-h	% of REFORMO left	% of LAIONIZE left	% of both
naive (no bucketing)	22,206,581	32.3 %	59.1 %	20.9 %
word-count bucketing	16,340,781	23.8 %	43.5 %	15.4 %
length-controlled + bucketing	14,040,468	20.4 %	37.4 %	13.2 %

Planning on the middle row — word-count bucketing, which is measured rather than estimated — the whole corpus costs 16,340,781 core-h, about 15 % of the two budgets combined. REFORMO expires 2026-12-31 and LAIONIZE 2027-04-30, so the split between them is a scheduling question, not a feasibility one.

nodes dedicated	wall-clock
32 nodes (×4 GPUs)	70 days
64 nodes (×4 GPUs)	35 days
128 nodes (×4 GPUs)	18 days
256 nodes (×4 GPUs)	9 days

6. Starting smaller — what a partial build costs

Nothing forces all 6,000 voices at once. The matrix is per-voice, so the corpus can be built in voice batches and stopped at any point with a complete, usable result. Every column below is the full 832-condition matrix and both subsets, priced with word-count bucketing and the 12B text model.

reference voices	samples	audio	text GPU-h	speech GPU-h	total core-h	% of both budgets	storage	wall-clock @64 nodes
100	4.0 M	9k h	188	3,594	272,346	0.26 %	0.6 TB	0.6 days
500	20.0 M	44k h	942	17,971	1,361,732	1.28 %	3.1 TB	2.9 days
1,000	39.9 M	87k h	1,883	35,942	2,723,464	2.56 %	6.3 TB	5.9 days
6,000	239.6 M	523k h	11,301	215,654	16,340,781	15.38 %	37.7 TB	35.1 days

The sensible first step is 100 voices. It costs 272,346 core-h — about 0.26 % of the budget, roughly 0.6 days on 64 nodes — and produces 4.0 M samples, already a substantial corpus. It also surfaces the things that only appear at scale: adapter-cache thrashing, length-bucket drift, and whether the hard-domain texts actually come out usable. Scaling from there is linear and carries no new risk.

7. Storage

At 160 kbps and 48 kHz — the quality the corpus is now written at — 240 M samples of 7.86 s come to about 38 TB of audio, plus metadata. Dropping to 128 kbps saves roughly a fifth. It is stored as WebDataset tar shards plus parquet, never as loose files: at this scale loose files would mean a quarter of a billion objects on a shared filesystem.

Why 48 kHz matters here. Until this week the pipeline resampled every take to 16 kHz before storing it, band-limiting the whole corpus to 8 kHz, and the demo pages then re-encoded that to 40 kbps. The model generates at 48 kHz; the corpus now keeps it. Committing 16,340,781 core-h to a corpus and then throwing away two thirds of its bandwidth on the way to disk would have been the single most expensive mistake available.

What you have at the end

240 million samples, ~523k hours of conditioned speech across 6,000 voices, English and German, every text existing as an EN/DE pair.
Subset A (160 M) — broad, lexically diverse pre-training data where the words themselves carry the condition.
Subset B (80 M) — minimal pairs: identical text, one dimension varied. The data a controllable model needs and cannot get from subset A.
~40 M samples with a real, labelled vocal burst inside the sentence, across 64 classes, with requested and detected recorded separately.
~1.6 M samples in each of ten hard-for-TTS domains — mathematics, code-switching, slang, URLs, jargon — the material a web-scraped corpus lacks.
Full per-sample metadata: 57 VoiceNet scores, 40 EmoNet scores, genuineness, vocal-burst blend, a 768-d VoiceCLAP embedding, a 192-d speaker embedding with its cosine to the reference, word-level ASR timestamps, a procedural caption, and every score component behind its rank. Re-ranking never requires regenerating anything.

Decisions already settled by measurement

No Mediathek LoRA anywhere — λ = 0. A full dose sweep over 13,760 generations found doses of 0.25/0.50/0.75 statistically indistinguishable from zero, and λ = 1.00 producing runaway takes 69 % of the time. Its apparent +0.815 genuineness gain was the scorer rewarding broken audio.
Batch 64. Measured 3.92 / 5.69 / 7.69 / 8.88× realtime at batch 8/16/32/64.
Burst adapters at λ = 0.5, no emotion adapter stacked when the burst is the point.
Store at 48 kHz / 160 kbps.

Open risks

The 1.15× length-controlled batching figure is an estimate, not a measurement. The 1.35× bucketed figure is measured and is what the budget above uses.
The 12B throughput number is extrapolated by FLOP ratio.
The hard-domain list was not verified against current literature (search unavailable).
German conditioning is weaker than English — mean speaker similarity 0.376 vs 0.578 over 323,200 candidates. Since the whole point is an EN/DE paired corpus, this is the central technical obstacle and it is not solved by any amount of text generation.
No human listening study. Every quality number here comes from automatic scorers whose agreement with a human listener on spoken material is about ρ ≈ +0.21. The Mediathek sweep is a worked example of a scorer confidently rewarding audio that is plainly broken.

Throughput measured on GH200: 1,500 generations/GPU-h at batch 64 (6,016 candidates in 4.01 GPU-h); node billing 72 core-h per GPU-hour. Budgets as of 2026-08-10.