When Vectors Break Down: Graph-Based RAG for Dense Enterprise Knowledge - Sam Julien, Writer

Abstract

Enterprise knowledge bases are filled with “dense mapping,” thousands of documents where similar terms appear repeatedly, causing traditional vector retrieval to return the wrong version or irrelevant information. When our customers kept hitting this wall with their RAG systems, we knew we needed a fundamentally different approach.

In this talk, I’ll share Writer’s journey developing a graph-based RAG architecture that achieved 86.31% accuracy on the RobustQA benchmark while maintaining sub-second response times, significantly outperforming vector approaches.

I’ll survey the key techniques behind this performance leap and why graph-based approaches excel with complex enterprise information structures like product documentation, financial documents, and technical specifications that challenge traditional RAG systems. You’ll learn about using specialized LLMs to build semantic relationships, how compression techniques efficiently handle concentrated enterprise data patterns, and how infusing key data points in the memory layer of the LLM lowers hallucination.

The presentation will provide practical insights into identifying when graph-based approaches make sense for your organization’s specific data challenges, helping you make informed architectural decisions for your next enterprise RAG system.

About Sam Julien Sam Julien is the Director of Developer Relations at Writer and is passionate about helping engineers improve their effectiveness and advance their careers. He loves spending time outside with his family in the Pacific Northwest. You can find more of Sam’s work at samjulien.com.

Recorded at the AI Engineer World’s Fair in San Francisco. Stay up to date on our upcoming events and content by joining our newsletter here: https://www.ai.engineer/newsletter