In order to help developers globally understand large language models (LLMs) better, the company has funded and published a professional translation of the Massive Multitask Language Understanding (MMLU) benchmark in 14 languages via Hugging Face.
“Supporting those who (developers) understand the unique cultures, economies, and social dynamics of their communities will help ensure that AI applications are tailored to meet local needs,” the company explained.
The MMLU benchmark is available in several languages, including Arabic, Bengali, Chinese, French, German, Hindi, Indonesian, Italian, Japanese, Korean, Portuguese, Spanish, Swahili, and Yoruba.