TF-IDF (term frequency, inverse document frequency) is a text representation technique in NLP that tackles the word count influence of common English words such as the, is etc (stop words) and some other generic words that are not stop words but can appear in any document. The idea is to give a high score to terms that are really relevant to a given document. In this video, I will explain TF-IDF in a very simple manner such that even a high school student can understand it easily 😊
Code: https://github.com/codebasics/nlp-tutorials/blob/main/12_tf_idf/tf_idf_tutorial_nlp_codebasics.ipynb
Exercise: https://github.com/codebasics/nlp-tutorials/blob/main/12_tf_idf/tf_idf_exercise.ipynb
Complete NLP Playlist: https://www.youtube.com/playlist?list=PLeo1K3hjS3uuvuAXhYjV2lMEShq2UYSwX
⭐️ Timestamps ⭐️
00:00 What is TF-IDF
11:32 Limitations of TF-IDF
12:17 Coding: sklearn TfidfVectorizer
21:38 Coding: Ecommerce item category classification using tf-idf
Stackoverflow question on usage of log in tf-idf formula: https://stackoverflow.com/questions/27067992/why-is-log-used-when-calculating-term-frequency-weight-and-idf-inverse-document#:~:text=Log%20is%20said%20to%20be,dampens%E2%80%9D%20the%20effect%20of%20IDF.
#tfidf #naturallanguageprocessing #textanalytics
Practical NLP Book In India: https://www.shroffpublishers.com/books/computer-science/artificial-intelligence/natural-language-processing/9789385889189/
Practical NLP Book Link For USA: https://amzn.to/3Aoeocm
🔖Hashtags🔖
#nlp #nlptutorial #nlppython #nlptfidf #whatistfidf #tfidfintuition #tfidf #tfidfnlp
Do you want to learn technology from me? Check https://codebasics.io/?utm_source=description&utm_medium=yt&utm_campaign=description&utm_id=description for my affordable video courses.
Need help building software or data analytics/AI solutions? My company https://www.atliq.com/ can help. Click on the Contact button on that website.
🎥 Codebasics Hindi channel: https://www.youtube.com/channel/UCTmFBhuhMibVoSfYom1uXEg
#️⃣ Social Media #️⃣
🔗 Discord: https://discord.gg/r42Kbuk
📸 Instagram: https://www.instagram.com/codebasicshub/
🔊 Facebook: https://www.facebook.com/codebasicshub
📱 Twitter: https://twitter.com/codebasicshub
📝 Linkedin (Personal): https://www.linkedin.com/in/dhavalsays/
📝 Linkedin (Codebasics): https://www.linkedin.com/company/codebasics/
🔗 Patreon: https://www.patreon.com/codebasics?fan_landing=true
Add comment