IBM Introduces Granite Speech 5.0 with Enhanced Transcription and Translation Capabilities

IBM has unveiled three updates to its Granite Speech collection, including a new model, granite-speech-5.0-2b, along with revised versions of two existing models, granite-speech-5.0-470m-turboctc and granite-speech-4.1-2b-plus. These models aim to enhance enterprise speech tasks, including transcription and translation, with improved accuracy and speed.

The new granite-speech-5.0-2b model offers advanced features such as noise- and reverb-robust automatic speech recognition (ASR), language-ID prompting, and keyword biasing for up to 2,000 terms. It supports transcription and translation in multiple languages, including English, French, German, Spanish, Portuguese, and Japanese, and translates between these languages and English.

Granite Speech 5.0 2B Enhancements

Compared to its predecessor, granite-speech-4.1-2b, the new model boasts:

• Improved ASR Accuracy: Enhanced performance on English benchmarks, with better handling of background noise and distant speech.
• Language ID Prompting: Users can specify the input language to guide multilingual transcription.
• Long-form Translation: Improved translation quality for longer and more natural speech.
• Keyword Lists: Support for biasing transcription towards specific terms, enhancing accuracy for product names and acronyms.
• Punctuation and Capitalization: Users can request formatted output directly in the prompt.

The model is available under the Apache 2.0 license and is compatible with various frameworks, including Transformers and vLLM.

Granite Speech 5.0 Turbo CTC Updates

The granite-speech-5.0-470m-turboctc model, initially launched as an English-only model, has been updated to include multilingual support and improved features:

• Multilingual Transcription: Now supports multiple languages, including Japanese, Spanish, German, French, Italian, and Portuguese.
• Native Punctuation and Capitalization: Transcripts are generated with proper punctuation and casing.
• Enhanced Robustness: Improved performance in challenging acoustic environments.

This model has also seen a significant speed increase, achieving 19,870 RTFx on the Open ASR Leaderboard, making it approximately 1.6 times faster than its initial launch.

Granite Speech 4.1 2B Plus Improvements

The updated granite-speech-4.1-2b-plus model focuses on improving the accuracy of speaker attribution and long-form speech performance. Key enhancements include:

• Speaker Attribution: More precise identification of speakers, with a 7% reduction in word diarization error rate.
• Timestamps: Improved timing accuracy for captions and reviews, with a significant reduction in average timestamp shift.
• Long-Speech Performance: Reduced word error rate on long-form tests by 24%.
• Keyword Biasing: Improved accuracy for biased keywords, with a 45% reduction in errors.

These models are designed to meet the needs of enterprises handling meetings, support calls, and multi-speaker recordings.

For users unsure which model to choose, IBM recommends:

• For high transcription accuracy and translation, use granite-speech-5.0-2b.
• For speed or on-device transcription, opt for granite-speech-5.0-470m-turboctc.
• For multi-speaker recordings, select granite-speech-4.1-2b-plus.

All models are released under an Apache 2.0 license, with full evaluation results and usage examples available on their respective model cards.

Source: Hugging Face Blog