ESPnet

ESPnet is an end-to-end speech processing toolkit built on PyTorch that covers speech recognition, text-to-speech, speech translation, speech enhancement and separation, speaker diarization, spoken language understanding, voice conversion, and singing voice synthesis. Developed by an academic community including Johns Hopkins and Carnegie Mellon, it follows Kaldi-style data conventions and ships reproducible recipes across many corpora and languages. Architectures include Conformer, Branchformer, Transformer, and transducer models for ASR with streaming support, plus Tacotron 2, FastSpeech, and VITS for TTS. Training scales across nodes via Slurm or MPI, features are extracted on the fly, and experiment tracking integrates with Weights and Biases; pretrained models from FairSeq and S3PRL can be plugged in as front ends. Released under the Apache 2.0 license, ESPnet is among the most widely used research toolkits in the speech community and also serves engineers building production systems.