Hugging Face's logo

ibm-granite
/
granite-speech-4.1-2b

Automatic Speech Recognition
Transformers
Safetensors
granite_speech
Eval Results
Model card Files
xet
 Community
5
gsaon commited on
Commit
15571a0
·
verified ·
1 Parent(s): dcaa3be

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +15 -15
README.md CHANGED
@@ -32,13 +32,13 @@ Two additional model variants explore different capabilities and inference optim
32
 
33
  We evaluated granite-speech-4.1-2b alongside other speech-language models in the less than 8b parameter range as well as dedicated ASR and AST systems on standard benchmarks. The evaluation spanned multiple public benchmarks, with particular emphasis on English ASR tasks while also including multilingual ASR and AST for X-En and En-X translations.
34
  <br>
35
- ![granite-speech-4.1-2b-wer1-crop](https://cdn-uploads.huggingface.co/production/uploads/666ec38102791b3b49f453e8/SW9c265Dw_iL0_HFhgGGD.png)
36
  <br>
37
- ![granite-speech-4.1-2b-wer2-crop](https://cdn-uploads.huggingface.co/production/uploads/666ec38102791b3b49f453e8/bkUqbRGywAoWco_nBCv9D.png)
38
  <br>
39
- ![granite-speech-4.1-2b-bleu1-crop](https://cdn-uploads.huggingface.co/production/uploads/666ec38102791b3b49f453e8/J4yjMTOqLmMI2r1-x1axx.png)
40
  <br>
41
- ![granite-speech-4.1-2b-bleu2-crop](https://cdn-uploads.huggingface.co/production/uploads/666ec38102791b3b49f453e8/FgKjUq43cID7PSmH1g3ly.png)
42
  <br>
43
 
44
  We evaluated the model’s keyword list biasing (KWB) capability by comparing performance with and without KWB applied at inference time.
@@ -49,17 +49,17 @@ We also evaluated our model on a variety of corpora to assess its punctuation an
49
 
50
  | Test Set | PER (&darr;) | Cap-F1 (&uarr;) |
51
  |:---------|:----:|:------:|
52
- | LScln | 25.95 | 89.46 |
53
- | LSoth | 22.45 | 91.32 |
54
- | VoxPopuli | 25.40 | 95.15 |
55
- | Earnings-22 | 22.69 | 94.87 |
56
- | CV-EN | 9.25 | 96.70 |
57
- | CV-DE | 3.71 | 99.45&dagger; |
58
- | CV-ES | 11.69 | 95.61 |
59
- | CV-FR | 11.12 | 97.17 |
60
- | CV-PT | 8.03 | 98.29 |
61
-
62
- &dagger; *We report a Cap-F1 of 99.45 on German, where noun capitalization is required.*
63
 
64
  <br>
65
 
 
32
 
33
  We evaluated granite-speech-4.1-2b alongside other speech-language models in the less than 8b parameter range as well as dedicated ASR and AST systems on standard benchmarks. The evaluation spanned multiple public benchmarks, with particular emphasis on English ASR tasks while also including multilingual ASR and AST for X-En and En-X translations.
34
  <br>
35
+ ![granite-speech-4.1-2b-wer1-crop](https://cdn-uploads.huggingface.co/production/uploads/666ec38102791b3b49f453e8/wY8LOsUVOcb0k204YpdL7.png)
36
  <br>
37
+ ![granite-speech-4.1-2b-wer2-crop](https://cdn-uploads.huggingface.co/production/uploads/666ec38102791b3b49f453e8/5nnbHPrni3qCNjzkHfod0.png)
38
  <br>
39
+ ![granite-speech-4.1-2b-bleu1-crop](https://cdn-uploads.huggingface.co/production/uploads/666ec38102791b3b49f453e8/vSmku2veF23Sn0JWplD3j.png)
40
  <br>
41
+ ![granite-speech-4.1-2b-bleu2-crop](https://cdn-uploads.huggingface.co/production/uploads/666ec38102791b3b49f453e8/AEasS_ygOZpqsDoQVL9BC.png)
42
  <br>
43
 
44
  We evaluated the model’s keyword list biasing (KWB) capability by comparing performance with and without KWB applied at inference time.
 
49
 
50
  | Test Set | PER (&darr;) | Cap-F1 (&uarr;) |
51
  |:---------|:----:|:------:|
52
+ | LScln | 25.70 | 89.71 |
53
+ | LSoth | 22.27 | 91.26 |
54
+ | VoxPopuli | 24.86 | 95.35 |
55
+ | Earnings-22 | 22.87 | 95.19 |
56
+ | CV-EN | 9.13 | 96.75 |
57
+ | CV-DE | 3.66 | 99.50&dagger; |
58
+ | CV-ES | 11.61 | 95.68 |
59
+ | CV-FR | 11.00 | 97.25 |
60
+ | CV-PT | 7.86 | 98.51 |
61
+
62
+ &dagger; *We report a Cap-F1 of 99.5 on German, where noun capitalization is required.*
63
 
64
  <br>
65