Cheating at Query Understanding with LLMs with Doug Turnbull

Traditional retrieval systems require months-long NLP projects and sophisticated teams to decompose user queries into structured properties. What if LLMs could make query understanding accessible to startups and smaller companies without massive engineering resources?

In this Lightning Lesson, Doug Turnbull (Principal Consultant) joins us to discuss query understanding with LLMs, based on lessons learned from over a decade working in search at companies like Shopify and Reddit.

We discuss:
• What query understanding is and how it compares to other retrieval systems
• Decomposing queries into properties like category, color, and brand from natural language
• Why query understanding has historically been a months-long NLP project that LLMs can now solve more efficiently
• The spectrum between highly structured e-commerce search and unstructured passage retrieval
• Content enrichment strategies to help answer the questions users actually want answered
• How reasoning agents might replace complicated search systems by working with simpler search tools
• Real-world examples from furniture search, Doug’s work improving search at scale, and backend routing challenges at Reddit

Doug shares insights from consulting with startups and smaller companies that don't have the resources to build complex retrieval systems, revealing practical approaches to query understanding that were previously only accessible to large, sophisticated teams. The discussion covers topics from controlled vocabularies to agentic search and the future of emerging approaches to LLM-powered query understanding.

About Doug: https://softwaredoug.com/

Connect with Doug:
LinkedIn: https://www.linkedin.com/in/softwaredoug
X/Twitter: https://x.com/softwaredoug

TIME STAMPS
00:00 Introduction and Doug’s Background
02:21 Query Understanding with LLMs
07:54 Challenges in Embedding Space
14:15 Multimodal Search Data
21:42 Traditional Query Understanding Methods
25:21 Using LLMs for Query Understanding
29:53 Handling Queries with No Color
30:21 Defining Task Success
33:17 Precision and Coverage in Query Classification
36:50 Business Rules in Query Classification
40:38 Managing Latency in Search Systems
45:46 Q&A Session
48:42 Building Effective Search Teams
51:23 Adapting Search for Different Data Types
54:29 The Future of Search with Agents

If you want to learn more about improving rag applications check out: https://improvingrag.com/

Stay updated:
X/Twitter: https://x.com/jxnlco
LinkedIn: https://www.linkedin.com/in/jxnlco
Site: https://jxnl.co/
Newsletter: https://subscribe.jxnl.co/