1st Multilingual Model Workshop – Developing Arabic-centric Bilingual LLMs​

This talk will present an overview of our experience of training Jais and Jais-chat, a family of Arabic-centric bilingual LLMs. At 30 Billion parameters, Jais and Jais-chat are the world’s largest and best-performing Arabic-centric open LLMs.​

Neha begins by discussing the motivating factors and primary challenges of training Jais, including those of Arabic data collection and processing. Neha then dives into the supervised fine-tuning data and methodology for building Jais-chat, a bilingual Arabic-centric chat model. Neha also discusses ongoing work and preliminary results on aligning Jais-chat to human preferences. Finally, Neha concludes with the ways to access Jais, and the roadmap ahead.