Papers
arxiv:2607.18432
Building a European Multilingual Evaluation Dataset: The MMLU Localisation Project within the EMT Network
Published on Jul 20
Authors:
Abstract
The collaboration localized the MMLU benchmark into 11 European languages to improve LLM evaluation inclusivity while providing professional translation training.
Generated by thinkingmachines/Inkling-Small
This paper reports on a collaboration between the Directorate-General for Translation (DGT) and the European Master's in Translation (EMT) to localise the MMLU dataset into 11 European languages. Beyond creating a more inclusive benchmark for LLM evaluation, the project offers master's students authentic, project-based professional training in translation, revision, project management, and multilingual coordination, while highlighting key methodological, administrative, and workflow challenges.
Community
Models citing this paper 0
No model linking this paper
Cite arxiv.org/abs/2607.18432 in a model README.md to link it from this page.
Datasets citing this paper 2
Spaces citing this paper 0
No Space linking this paper
Cite arxiv.org/abs/2607.18432 in a Space README.md to link it from this page.
Collections including this paper 0
No Collection including this paper
Add this paper to a collection to link it from this page.