MuseNet named its training data, unlike many later models

The system

MuseNet is a model OpenAI published on 25 April 2019 that generates multi-instrument musical passages across styles, from a symbolic representation rather than raw audio. OpenAI's own research post describes it plainly: "We've created MuseNet, a deep neural network that can generate 4-minute musical compositions with 10 different instruments, and can combine styles from country to Mozart to the Beatles." The post states MuseNet "was not explicitly programmed with our understanding of music," but instead learned patterns of harmony and rhythm by predicting the next token in hundreds of thousands of MIDI files, using "the same general-purpose unsupervised technology as GPT-2," a large-scale transformer trained to predict the next item in a sequence.

What the documents establish

OpenAI's post specifies the architecture as a 72-layer network with 24 attention heads and full attention over a 4,096-token context, built using the Sparse Transformer, whose separate paper, posted to arXiv on 23 April 2019, introduces "sparse factorizations of the attention matrix" that cut the computational cost of long-range attention and reports the same architecture applied across images, audio and text. OpenAI names its training sources directly: donated MIDI collections from ClassicalArchives and BitMidi, additional jazz, pop, African, Indian and Arabic-style collections found online, and the MAESTRO piano-performance dataset. The post also documents specific limits: instrument requests are "strong suggestions, not requirements," and MuseNet "has a more difficult time with odd pairings of styles and instruments," such as Chopin scored for bass and drums.

Craft and rights

Because MuseNet generates symbolic note sequences rather than finished audio, and because OpenAI names donated and collected MIDI archives rather than commercial recordings as training material, its immediate rights profile differs from later raw-audio systems trained on recorded music; the open questions shift to whether any individual MIDI transcription in those collections carried its own restrictions. MuseNet should not be conflated with OpenAI's later Jukebox system: MuseNet's composer and instrumentation tokens steer a symbolic score, while Jukebox, a separate later project, generates raw audio directly, a distinction the sources maintain but that popular summaries sometimes blur.

Outcomes and open questions

OpenAI's own account frames MuseNet as an experiment for musicians and non-musicians to explore stylistic blending, streamed live in an April 2019 concert where, the post states, no one involved had heard the pieces beforehand, rather than as a finished composition tool with guaranteed stylistic fidelity.

• Did the MIDI collections OpenAI used carry any transcription-specific usage restrictions?
• Is a given "MuseNet" claim describing symbolic generation or being confused with Jukebox's raw audio?
• How does the Sparse Transformer's cross-domain design change what counts as music-specific evaluation?

MuseNet's documented architecture and named training sources make it one of the more transparent entries in this history, precisely because OpenAI published the token encoding and data sources rather than only the output samples.