minishlab

/

potion-multilingual-128M

like 49

Follow

Minish 169

Hey @Samoed , could you briefly clarify what these evaluation results are for? I was under the impression that MTEB eval results were now stored in https://github.com/embeddings-benchmark/results instead of individual models.

Hi! Yes, we're storing evaluation results in https://github.com/embeddings-benchmark/results. Recently HF introduced benchmarks (https://huggingface.co/changelog/dataset-leaderboards) and we've integrated a few tasks as an "HF benchmark" now: ArguAna and BRIGHT (you can see your model in tasks of this benchmark), and we will add more. Overall, these results are only for HF benchmark display and we will continue to use results from embeddings-benchmark/results as the main source of results. This PR is autogenerated from results in results repo. Results were submitted in https://github.com/embeddings-benchmark/results/pull/471.

@Samoed I see, thanks for explaining, and for adding this to the model! Will merge it in that case 👍