Today we’re joined by Kyle Roche, founder and CEO of Griptape to discuss patterns and middleware for LLM applications. We dive into the emerging patterns for developing LLM applications, such as off prompt data—which allows data retrieval without compromising the chain of thought within language models—and pipelines, which are sequential tasks that are given to LLMs that can involve different models for each task or step in the pipeline. We also explore Griptape, an open-source, Python-based middleware stack that aims to securely connect LLM applications to an organization’s internal and external data systems. We discuss the abstractions it offers, including drivers, memory management, rule sets, DAG-based workflows, and a prompt stack. Additionally, we touch on common customer concerns such as privacy, retraining, and sovereignty issues, and several use cases that leverage role-based retrieval methods to optimize human augmentation tasks.
🔔 Subscribe to our channel for more great content just like this: https://youtube.com/twimlai?sub_confirmation=1
🗣️ CONNECT WITH US!
===============================
Subscribe to the TWIML AI Podcast: https://twimlai.com/podcast/twimlai/
Join our Slack Community: https://twimlai.com/community/
Subscribe to our newsletter: https://twimlai.com/newsletter/
Want to get in touch? Send us a message: https://twimlai.com/contact/
📖 CHAPTERS
===============================
00:00 – Career background
01:40 – What is Griptape?
03:23 – Off prompt
07:34 – Customer concerns
08:51 – Real-time retrieval
12:35 – Use cases
14:57 – Abstraction: prompt stack
19:19 – Abstraction: DAG
21:06 – Integrating real-time information into LLMs
25:17 – Overlapping on the whole ecosystem
28:02 – Use cases
31:22 – Abstraction: drivers
34:13 – Difference of industrial use case vs typical RAG use case
37:08 – Thoughts on keynote
38:34 – What’s next for Griptape
🔗 LINKS & RESOURCES
===============================
Griptape.ai – https://www.griptape.ai/
AWS re:Invent 2023 – https://reinvent.awsevents.com/
For a COMPLETE LIST of links and references, head over to https://twimlai.com/go/659.
📸 Camera: https://amzn.to/3TQ3zsg
🎙️Microphone: https://amzn.to/3t5zXeV
🚦Lights: https://amzn.to/3TQlX49
🎛️ Audio Interface: https://amzn.to/3TVFAIq
🎚️ Stream Deck: https://amzn.to/3zzm7F5
Add comment