Language models are often evaluated with a metric called Perplexity. Feeling perplexed about it? Watch this video to get it all explained.
This video is part of the Hugging Face course: http://huggingface.co/course
Open in colab to run the code samples:
https://colab.research.google.com/github/huggingface/notebooks/blob/master/course/videos/perplexity.ipynb
Related videos:
– 🤗 Tasks: Causal Language Modeling: https://youtu.be/Vpjb1lu0MDk
– 🤗 Tasks: Masked Language Modeling: https://youtu.be/mqElG5QJWUg
– Data processing for Causal Language Modeling: https://youtu.be/ma1TrR7gE7I
– Data processing for Causal Language Modeling: https://youtu.be/ma1TrR7gE7I
Don’t have a Hugging Face account? Join now: http://huggingface.co/join
Have a question? Checkout the forums: https://discuss.huggingface.co/c/course/20
Subscribe to our newsletter: https://huggingface.curated.co/
Add comment