NSynth's 305,000 notes came from sample libraries, not songs

The system

Modeling raw audio waveforms directly is hard because a single second of sound contains far more information than a comparable slice of text or image data, and in April 2017 Google's Magenta team addressed part of that problem by building a dataset scoped to a single, controllable unit: one note. The NSynth paper, submitted 5 April 2017, introduces a WaveNet-style autoencoder that conditions an autoregressive decoder on temporal codes learned directly from raw audio, alongside the dataset built to train and evaluate it.

What the documents establish

Magenta's NSynth dataset page states the set contains 305,979 musical notes generated from 1,006 instruments drawn from commercial sample libraries. Each instrument was sampled across every pitch of a standard 88-key MIDI range and five velocities, producing four-second monophonic clips at 16kHz, held for three seconds and left to decay for the last one. Since not every instrument covers the full range, the average is 65.4 pitches per instrument. Each note carries three annotations combining human evaluation with heuristics: a source label (acoustic, electronic or synthetic), one instrument family, and zero or more quality tags. The page reports a 289,205-example training split, 12,678 for validation and 4,096 for test, built so no instrument spans more than one split, released by Google Inc. under CC BY 4.0. The paper's abstract reports the autoencoder outperforms a tuned spectral-autoencoder baseline and that its embedding space supports morphing between timbres.

Craft and rights

NSynth documents single notes, not phrases or songs — nothing in either source describes generating full musical passages, so citing NSynth as evidence of a system's song-generation capability would be citing the wrong dataset. On rights, the dataset page states plainly that the notes were rendered from commercial sample libraries, but it does not name which libraries or describe how their original licences permitted this redistribution; it only states Google's own CC BY 4.0 terms for the resulting rendered-note dataset. That gap is worth flagging editorially: a CC licence on the derived dataset tells a reader what Google permits downstream, but it does not, on its own, document the chain of permission back to whoever originally sold or licensed the sampled instrument sounds.

Outcomes and open questions

Neither document covers NSynth Super, the physical hardware interface Magenta released later using ideas from this research — a separate product this entry does not describe. The timbre-morphing result is reported here as a property of the trained embedding space evaluated in the paper, not as a claim that the same model generates complete compositions.

• Does a later system citing "NSynth" mean this dataset, the WaveNet-style autoencoder, or the separate NSynth Super hardware?
• What were the original commercial sample libraries, and what did their licences permit before Google's own CC BY release of the derived note set?
• Does the reported timbre-interpolation result hold up when instruments outside the original 1,006 are introduced?

NSynth is precise about what it measures — single notes, annotated three ways — and precise readers should keep its scope that narrow too.