How we made Cerebras-GPT with Nolan Dey and Quentin Anthony

In this episode of the Cerebras podcast we dive into the making of Cerebras-GPT. We discuss:
• The importance of open large language models
• What makes Cerebras-GPT unique among LLMs
• The tradeoffs of compute-optimal vs. inference-optimal
• The complexities of training on GPU clusters and how Cerebras simplifies it with weight-streaming
• Where the future of LLMs and AI hardware is headed

Speakers:
Nolan Dey (@DeyNolan) – Research Scientist, Cerebras
Quentin Anthony (@QuentinAnthon15) – Lead Engineer, EleutherAI
James Wang (@draecomino) – Product Marketing, Cerebras

Paper: https://arxiv.org/abs/2304.03208
Blog: https://www.cerebras.net/blog/cerebras-gpt-a-family-of-open-compute-efficient-large-language-models/
Twitter: https://twitter.com/CerebrasSystems
LinkedIn: https://www.linkedin.com/company/cerebras-systems/
Hugging Face: https://huggingface.co/cerebras