Abu Dhabi Researchers Build AI Benchmark Across 13 Arabic Dialects

Researchers at Mohamed bin Zayed University of Artificial Intelligence in Abu Dhabi have developed a new benchmark to test how well AI models understand and generate everyday Arabic across 13 national dialects, rather than relying primarily on Modern Standard Arabic.

Known as ArabCulture-Dialogue, the benchmark covers culturally grounded conversations from 13 Arab countries across 12 everyday topics, including weddings, food, parenting, agriculture, arts and games.

Researchers tested models across three tasks: selecting culturally appropriate responses, translating between Modern Standard Arabic and specific dialects, and continuing conversations in a requested dialect. While stronger models performed well when identifying appropriate responses, accuracy dropped significantly when they were asked to generate dialectal Arabic themselves.

According to the researchers, North African dialects were among the most challenging, while Emirati Arabic also proved difficult. Models produced the requested country-specific dialect in only around half of cases, highlighting a gap between recognising cultural context and reproducing the language in which that context is naturally expressed.

The research also found that performance improved when prompts explicitly identified the relevant country or region, suggesting that models often contain the necessary cultural knowledge but do not reliably surface it without additional guidance.

The project was led by Assistant Professor Fajri Koto and researcher Muhammad Dehan, with 26 native Arabic speakers from 13 countries contributing authentic multi-turn dialogues. The paper, ‘Cultural Benchmarking of LLMs in Standard and Dialectal Arabic Dialogues’, was presented at ACL 2026.