We love text embeddings as a critical pillar of LLM applications, but there’s so much to text embeddings beyond their value in vector search. This talk will be a grand tour through a series of experimental projects from my last two years of research for visualizing, manipulating, and interpreting embeddings. We’ll start with the basics (t-SNE, UMAP, and PCA), talk about how language models can be used to manipulate and interpret embeddings, and end by using a new tool I’ve built that lets us directly observe which features popular embedding models like to encode into their embeddings.
Recorded live in San Francisco at the AI Engineer Summit 2023. See the full schedule of talks at https://ai.engineer/summit/schedule & join us at the AI Engineer World’s Fair in 2024! Get your tickets today at https://ai.engineer/worlds-fair
About Linus Lee
Linus is a Research Engineer at Notion prototyping new software interfaces for augmenting our collaborative work and creativity with AI. He has spent the last few years experimenting with AI-augmented tools for thinking, like a canvas for exploring the latent space of neural networks and writing tools where ideas connect themselves. Before Notion, Linus spent a year as an independent researcher, during which he was Betaworks’s first Researcher in Residence.
Add comment