Hubert-kakeiken-W-not_reverbed

This model is a fine-tuned version of rinna/japanese-hubert-base on the ORIGINAL_KAKEIKEN_W_NOT_REVERBED - JA dataset. It achieves the following results on the evaluation set:

• Loss: 0.0054
• Wer: 0.9988
• Cer: 1.0134

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 3e-05
• train_batch_size: 32
• eval_batch_size: 8
• seed: 42
• gradient_accumulation_steps: 2
• total_train_batch_size: 64
• optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: cosine
• lr_scheduler_warmup_steps: 12500
• num_epochs: 40.0
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	Wer	Cer
21.6118	1.0	820	8.9509	1.0	1.1284
7.4878	2.0	1640	6.3856	1.0	1.1284
5.9004	3.0	2460	3.9287	1.0	1.1284
3.4882	4.0	3280	2.9090	1.0	1.1284
2.6365	5.0	4100	2.3862	1.0	1.1284
2.2815	6.0	4920	1.4563	1.0	1.2581
1.0892	7.0	5740	0.4288	0.9999	1.0564
0.4741	8.0	6560	0.1896	0.9994	1.0188
0.3822	9.0	7380	0.1739	0.9990	1.0212
0.3101	10.0	8200	0.1713	0.9988	1.0288
0.2644	11.0	9020	0.1189	0.9988	1.0218
0.2476	12.0	9840	0.0540	0.9988	1.0172
0.2302	13.0	10660	0.0297	0.9988	1.0152
0.2182	14.0	11480	0.0431	0.9988	1.0188
0.2154	15.0	12300	0.0174	0.9988	1.0158
0.2072	16.0	13120	0.0157	0.9991	1.0154
0.1986	17.0	13940	0.0273	0.9990	1.0149
0.1919	18.0	14760	0.0110	0.9988	1.0145
0.1763	19.0	15580	0.0145	0.9988	1.0144
0.1759	20.0	16400	0.0701	0.9988	1.0167
0.1673	21.0	17220	0.0128	0.9988	1.0136
0.157	22.0	18040	0.0136	0.9988	1.0144
0.1642	23.0	18860	0.1375	0.9988	1.0074
0.1529	24.0	19680	0.0134	0.9988	1.0140
0.1511	25.0	20500	0.0073	0.9990	1.0138
0.1415	26.0	21320	0.0062	0.9988	1.0136
0.1338	27.0	22140	0.0063	0.9988	1.0135
0.1373	28.0	22960	0.0139	0.9988	1.0131
0.1224	29.0	23780	0.0076	0.9988	1.0135
0.1217	30.0	24600	0.0191	0.9988	1.0139
0.119	31.0	25420	0.0266	0.9988	1.0135
0.1122	32.0	26240	0.0061	0.9988	1.0136
0.1077	33.0	27060	0.0050	0.9988	1.0134
0.1058	34.0	27880	0.0068	0.9988	1.0135
0.0992	35.0	28700	0.0058	0.9988	1.0135
0.0977	36.0	29520	0.0065	0.9988	1.0135
0.093	37.0	30340	0.0058	0.9988	1.0133
0.0959	38.0	31160	0.0058	0.9988	1.0133
0.093	39.0	31980	0.0057	0.9988	1.0133
0.0951	39.9518	32760	0.0056	0.9988	1.0133

Framework versions

• Transformers 4.48.0
• Pytorch 2.5.1+cu124
• Datasets 3.1.0
• Tokenizers 0.21.0