Whisper

Few open source releases have reshaped a field the way Whisper did for speech recognition. OpenAI's model is a Transformer sequence-to-sequence network trained on 680,000 hours of multilingual, multitask audio, which lets a single checkpoint handle transcription, speech translation into English, language identification, and voice activity detection across some 99 languages. Audio is processed as log-Mel spectrograms in sliding 30 second windows, with every task expressed as a token sequence so no separate pipeline stages are needed. Checkpoints range from tiny at 39 million parameters, needing about 1 GB of VRAM, up to large at 1.55 billion needing around 10 GB, plus English-only variants and a faster turbo model that trades away translation. Code and weights are MIT licensed and run locally on CPU or GPU through a simple Python API or CLI. Its accuracy and permissive license spawned an ecosystem of ports and optimizations, and it remains the default starting point for transcription features in both applications and research.