To begin with, not all RAGs are of the same capacity. Content accuracy in a custom database is critical to solid output, but it's not the only variable. “It's not just content quality,” says Joel Herron, global head of AI at Thomson Reuters. “It's search quality, and retrieving the right content based on the query.” Mastering each step in the process is critical, as one mistake can completely destroy the model.
“Any attorney who has ever tried to use natural language search in a search engine will see that there are often instances where verbatim Similarity leads you to completely unrelated content.” Human-centered AI. Hu's research into AI legal tools that rely on RAG found that the output had a higher error rate than companies building the models.
Which brings us to the most thorny question of the debate: how do you define deception within the RAG implementation? Does this only happen when the chatbot produces a reference-less output and generates information? Is it also the case that the tool may ignore relevant data or misinterpret aspects of a reference?
According to Lewis, hallucinations in a RAG system boil down to whether the output matches what the model found during data retrieval. However, Stanford's research into AI tools for lawyers slightly broadens the definition of whether or not the output is grounded in the data provided and is indeed accurate—a high bar for legal professionals who Often analyzes complex cases and navigates complex hierarchies. of Nazir
While a RAG system attuned to legal issues is clearly better at answering questions on case law than OpenAI's ChatGPT or Google's Gemini, it can still overlook finer details. and may make random errors. All of the AI experts I spoke with emphasized the constant need for human interaction throughout the process to double-check references and verify the overall accuracy of results.
Law is an area where there is a lot of activity around RAG-based AI tools, but the potential for this process is not limited to a white-collar job. “Any profession or any business. You need to get answers that are anchored in real documents,” says Arredondo. “So, I think RAG is used in basically every professional application, at least near the mid-term.” Risk-averse executives seem excited about the prospect of using AI tools to better understand their proprietary data, without having to upload sensitive information to a standard, public chatbot.
However, it is important for users to understand the limitations of these tools and for companies focusing on AI to avoid over-promising the accuracy of their answers. Any AI tool user should still avoid relying entirely on the output, and should approach its answers with a healthy sense of skepticism even if the answer is improved by RAG.
“Deceptions are here to stay,” Ho says. “We don't yet have ready methods to eliminate deceptions.” Even when RAG reduces the prevalence of errors, human judgment reigns supreme. It does and it's not a lie.