Dance Diffusion opened its code before its training data

The system

Dance Diffusion is an open-source diffusion model for generating short audio clips, released by Harmonai, a group describing itself as a "Stability AI Lab releasing open-source generative audio tools." The code repository, created on 25 May 2022, is titled simply "tools to train a generative model on arbitrary audio samples" and ships as Colab notebooks: one for running inference with a pretrained Dance Diffusion checkpoint, and a separate one for fine-tuning a checkpoint on new audio. Unlike a symbolic model that outputs notes, Dance Diffusion generates audio waveforms directly through an iterative denoising process, the same general family of technique used in image diffusion models applied instead to sound.

What the documents establish

The repository is released under the MIT license, one of the more permissive open-source terms available, allowing reuse, modification and commercial redistribution with only an attribution requirement. Harmonai's Hugging Face organization page lists six published pretrained checkpoints, including glitch-440k, maestro-150k and a larger jmann-large-580k model documented at roughly 0.2 billion parameters, each hosted for direct use through Hugging Face's diffusion tooling rather than requiring users to train their own model from scratch. Harmonai's page describes its interests as "Diffusion Models, Music, Art," and frames the group's purpose around making generative audio tools "accessible and fun for everyone," a mission statement rather than a technical specification.

Craft and rights

An MIT-licensed, openly published model with downloadable weights is a materially different proposition for a producer than a closed commercial API: anyone can inspect, fine-tune or redistribute Dance Diffusion's checkpoints, which shifts the rights question away from usage terms and toward the training audio behind each specific checkpoint, since neither document reviewed here specifies what recordings trained glitch-440k, maestro-150k or the other released weights. That gap matters editorially: an open license governs what a user may do with the model, not what the model was permitted to learn from, and those are separate questions a producer should not conflate when evaluating a checkpoint for commercial use.

Outcomes and open questions

Dance Diffusion predates Stability AI's later, separately branded Stable Audio product line, and the two should not be assumed to share a license or training data simply because both originated inside the same company's audio research efforts.

• What training audio underlies a specific published Dance Diffusion checkpoint?
• Does an MIT license on model weights say anything about the data used to train them?
• How do Dance Diffusion's open terms compare with the terms of Stability's later commercial audio products?

Dance Diffusion's release pattern, open code, open weights, an unspecified training corpus, is a template worth watching as later, better-funded diffusion audio models decide how much of that openness to keep.