StyleTTS 2

StyleTTS 2 approaches text-to-speech with two ideas that pushed it to human-level quality in evaluations: style diffusion and adversarial training with large speech language models. Rather than needing reference audio at synthesis time, it models speaking style as a latent variable sampled through diffusion, so each utterance can receive natural, varied delivery. During training, a pretrained speech language model such as WavLM serves as a discriminator, paired with differentiable duration modeling for end-to-end learning. The Columbia University research project reports speech that surpasses human recordings on the single-speaker LJSpeech benchmark and matches them on the multi-speaker VCTK set, and it supports zero-shot speaker adaptation when trained on LibriTTS. The code is MIT licensed, with pretrained models carrying a condition that synthesized speech be disclosed as such unless the speaker gave permission. Researchers and developers use it for single and multi-speaker synthesis and style transfer from reference audio.