Speaker: Muhammad Uzair KhattakAbout Speaker: I am Muhammad Uzair, recently graduated from MBZUAI with a master’s degree in Computer Vision. During my master’s, I was fortunate to work under the supervision of Dr. Salman Khan and Dr. Fahad Khan at the IVAL lab. I am also grateful to have been supervised and mentored by Dr. Muzammal Naseer. My research focus is on adapting foundational multi-modal models for vision tasks, including image recognition, object detection, and video action recognition. The goal is to guide these foundational models for downstream tasks with limited data (few-/zero-shot) while maintaining their pre-trained generalization for novel tasks.
About the talk:ProText: Prompt Learning with Text Only Supervision, allows one to finetune CLIP by leveraging contextual knowledge derived from Large Language Models (LLMs) without relying on visual samples. ProText exhibits strong transferability towards unseen datasets and classes and effectively overcomes the transferability limitations of LLM-based Prompt Ensembling methods.Through text-only training, ProText improves over previous prompt ensembling and image-supervised methods in challenging cross-dataset transfer settings.We have open-sourced our checkpoints and source code. ProText offers a drop-in replacement for CLIP’s text transformer. We look forward to the impact of ProText on applications beyond what we tested in the manuscript.abs: https://arxiv.org/abs/2401.02418
Add comment