---
language: ca
datasets:
- projecte-aina/3catparla_asr
- projecte-aina/parlament_parla_v3
- projecte-aina/corts_valencianes_asr_a
- projecte-aina/commonvoice_benchmark_catalan_accents
tags:
- audio
- automatic-speech-recognition
- whisper-large-v3
- barcelona-supercomputing-center
license: apache-2.0
model-index:
- name: Whisper_bsc_large_v3_cat
  results:
  - task:
      name: Automatic Speech Recognition
      type: automatic-speech-recognition
    dataset:
      name: 3CatParla (Test)
      type: projecte-aina/3catparla_asr
      split: test
      args:
        language: ca
    metrics:
    - name: WER
      type: wer
      value: 4.801
  - task:
      name: Automatic Speech Recognition
      type: automatic-speech-recognition
    dataset:
      name: CV Benchmark Catalan Accents (Balearic fem)
      type: projecte-aina/commonvoice_benchmark_catalan_accents
      split: Balearic female
      args:
        language: ca
    metrics:
    - name: WER
      type: wer
      value: 5.314
  - task:
      name: Automatic Speech Recognition
      type: automatic-speech-recognition
    dataset:
      name: CV Benchmark Catalan Accents (Balearic male)
      type: projecte-aina/commonvoice_benchmark_catalan_accents
      split: Balearic male
      args:
        language: ca
    metrics:
    - name: WER
      type: wer
      value: 4.31
  - task:
      name: Automatic Speech Recognition
      type: automatic-speech-recognition
    dataset:
      name: CV Benchmark Catalan Accents (Central fem)
      type: projecte-aina/commonvoice_benchmark_catalan_accents
      split: Central female
      args:
        language: ca
    metrics:
    - name: WER
      type: wer
      value: 3.294
  - task:
      name: Automatic Speech Recognition
      type: automatic-speech-recognition
    dataset:
      name: CV Benchmark Catalan Accents (Central male)
      type: projecte-aina/commonvoice_benchmark_catalan_accents
      split: Central male
      args:
        language: ca
    metrics:
    - name: WER
      type: wer
      value: 3.602
  - task:
      name: Automatic Speech Recognition
      type: automatic-speech-recognition
    dataset:
      name: CV Benchmark Catalan Accents (Northern fem)
      type: projecte-aina/commonvoice_benchmark_catalan_accents
      split: Northern female
      args:
        language: ca
    metrics:
    - name: WER
      type: wer
      value: 3.189
  - task:
      name: Automatic Speech Recognition
      type: automatic-speech-recognition
    dataset:
      name: CV Benchmark Catalan Accents (Northern male)
      type: projecte-aina/commonvoice_benchmark_catalan_accents
      split: Northern male
      args:
        language: ca
    metrics:
    - name: WER
      type: wer
      value: 3.378
  - task:
      name: Automatic Speech Recognition
      type: automatic-speech-recognition
    dataset:
      name: CV Benchmark Catalan Accents (Northwestern fem)
      type: projecte-aina/commonvoice_benchmark_catalan_accents
      split: Northwestern female
      args:
        language: ca
    metrics:
    - name: WER
      type: wer
      value: 3.217
  - task:
      name: Automatic Speech Recognition
      type: automatic-speech-recognition
    dataset:
      name: CV Benchmark Catalan Accents (Northwestern male)
      type: projecte-aina/commonvoice_benchmark_catalan_accents
      split: Northwestern male
      args:
        language: ca
    metrics:
    - name: WER
      type: wer
      value: 3.949
  - task:
      name: Automatic Speech Recognition
      type: automatic-speech-recognition
    dataset:
      name: CV Benchmark Catalan Accents (Valencian fem)
      type: projecte-aina/commonvoice_benchmark_catalan_accents
      split: Valencian female
      args:
        language: ca
    metrics:
    - name: WER
      type: wer
      value: 3.581
  - task:
      name: Automatic Speech Recognition
      type: automatic-speech-recognition
    dataset:
      name: CV Benchmark Catalan Accents (Valencian male)
      type: projecte-aina/commonvoice_benchmark_catalan_accents
      split: Valencian male
      args:
        language: ca
    metrics:
    - name: WER
      type: wer
      value: 3.552
library_name: transformers
base_model:
- openai/whisper-large-v3
metrics:
- wer
---
# Whisper_bsc_large_v3_cat

## Table of Contents
<details>
<summary>Click to expand</summary>

- [Model Description](#model-description)
- [Intended Uses and Limitations](#intended-uses-and-limitations)
- [How to Get Started with the Model](#how-to-get-started-with-the-model)
- [Training Details](#training-details)
- [Citation](#citation)
- [Additional Information](#additional-information)

</details>


## Model Description

The "whisper-bsc-large-v3-cat" is an acoustic model suitable for Automatic Speech Recognition in Catalan. It is the result of finetuning the model ["openai/whisper-large-v3"](https://huggingface.co/openai/whisper-large-v3) with 4700 hours of Catalan data released by the [Projecte AINA](https://projecteaina.cat/) from Barcelona, Spain.

## Intended Uses and Limitations

This model can be used for Automatic Speech Recognition (ASR) in Catalan. The model intends to transcribe Catalan audio files to plain text without punctuation.

<!--
## How to Get Started with the Model

To see an updated and functional version of this code, please visit our [Notebook](https://colab.research.google.com/drive/1MHiPrffNTwiyWeUyMQvSdSbfkef_8aJC?usp=sharing)
-->
### Installation

To use this model, you may install [datasets](https://huggingface.co/docs/datasets/installation) and [transformers](https://huggingface.co/docs/transformers/installation):

Create a virtual environment:
```bash
python -m venv /path/to/venv
```
Activate the environment:
```bash
source /path/to/venv/bin/activate
```
Install the modules:
```bash
pip install datasets transformers 
```

### For Inference
To transcribe audio in Catalan using this model, you can follow this example:

```bash
#Install Prerequisites
pip install torch
pip install datasets
pip install 'transformers[torch]'
pip install evaluate
pip install jiwer
```

```python
#This code works with GPU

#Notice that: load_metric is no longer part of datasets.
#You have to remove it and use evaluate's load instead.
#(Note from November 2024)

import torch
from transformers import WhisperForConditionalGeneration, WhisperProcessor

#Load the processor and model.
MODEL_NAME="BSC-LT/whisper-bsc-large-v3-cat"
processor = WhisperProcessor.from_pretrained(MODEL_NAME)
model = WhisperForConditionalGeneration.from_pretrained(MODEL_NAME).to("cuda")

#Load the dataset
from datasets import load_dataset, load_metric, Audio
ds=load_dataset("projecte-aina/parlament_parla",split='test')

#Downsample to 16 kHz
ds = ds.cast_column("audio", Audio(sampling_rate=16_000))

#Process the dataset
def map_to_pred(batch):
	audio = batch["audio"]
	input_features = processor(audio["array"], sampling_rate=audio["sampling_rate"], return_tensors="pt").input_features
	batch["reference"] = processor.tokenizer._normalize(batch['normalized_text'])

	with torch.no_grad():
		predicted_ids = model.generate(input_features.to("cuda"))[0]
	
	transcription = processor.decode(predicted_ids)
	batch["prediction"] = processor.tokenizer._normalize(transcription)
	
	return batch
	
#Do the evaluation
result = ds.map(map_to_pred)

#Compute the overall WER now.
from evaluate import load

wer = load("wer")
WER=100 * wer.compute(references=result["reference"], predictions=result["prediction"])
print(WER)
```


## Training Details

### Training data

The specific datasets used to create the model are:
- [3CatParla](https://huggingface.co/datasets/projecte-aina/3catparla_asr). (Soon to be published)
- [commonvoice_benchmark_catalan_accents](https://huggingface.co/datasets/projecte-aina/commonvoice_benchmark_catalan_accents)
- [corts_valencianes](https://huggingface.co/datasets/projecte-aina/corts_valencianes_asr_a) (Only the anonymized version of the dataset is public. We trained the model with the non-anonymized version.)
- [parlament_parla_v3](https://huggingface.co/datasets/projecte-aina/parlament_parla_v3) (Only the anonymized version of the dataset is public. We trained the model with the non-anonymized version.)
- [IB3](https://huggingface.co/datasets/projecte-aina/ib3_ca_asr) (Soon to be published)

### Training procedure

This model is the result of fine-tuning the model ["openai/whisper-large-v3"](https://huggingface.co/openai/whisper-large-v3) by following this [tutorial](https://github.com/langtech-bsc/whisper_ft_pipeline) provided by [Language Technologies Laboratory](https://huggingface.co/BSC-LT). (Soon to be published)

### Training Hyperparameters

* language: Catalan
* hours of training audio: 4700
* learning rate: 1e-04
* sample rate: 16000
* train batch size: 16 (x4 GPUs)
* eval batch size: 16
* num_train_epochs: 10
* weight_decay: 1e-4
  
## Citation

If this model contributes to your research, please cite the work:
<!--
```bibtex
@inproceedings{hernandez20243catparla,
  title={3CatParla: A New Open-Source Corpus of Broadcast TV in Catalan for Automatic Speech Recognition},
  author={Hern{\'a}ndez Mena, Carlos Daniel and Armentano Oller, Carme and Solito, Sarah and K{\"u}lebi, Baybars},
  booktitle={Proc. IberSPEECH 2024},
  pages={176--180},
  year={2024}
}
```
-->
```bibtext
@misc{takanori2025whisperbsclarge3cat,
      title={Acoustic Model in Catalan: Whisper_bsc_large_v3_cat.}, 
      author={Sanchez Shiromizu, Lucas Takanori; Hernandez Mena, Carlos Daniel; Messaoudi, Abir; España i Bonet, Cristina; Cortada Garcia, Marti},
      organization={Barcelona Supercomputing Center},
      url={https://huggingface.co/langtech-veu/whisper-bsc-large-v3-cat},
      year={2025}
}
```

## Additional Information

### Author

The fine-tuning process was performed during Spring (2025) in the [Language Technologies Laboratory](https://huggingface.co/BSC-LT) of the [Barcelona Supercomputing Center](https://www.bsc.es/) by [Lucas Takanori Sanchez Shiromizu](https://huggingface.co/LucasTakanori).

### Contact
For further information, please send an email to <langtech@bsc.es>.

### Copyright
Copyright(c) 2025 by Language Technologies Laboratory, Barcelona Supercomputing Center.

### License

[Apache-2.0](https://www.apache.org/licenses/LICENSE-2.0)

### Funding
This work is funded by the Ministerio para la Transformación Digital y de la Función Pública - Funded by EU – NextGenerationEU within the framework of the project ILENIA with reference 2022/TL22/00215337.

The training of the model was possible thanks to the computing time provided by [Barcelona Supercomputing Center](https://www.bsc.es/) through MareNostrum 5.