How Generative AI Uses RAG and Knowledge Graphs to Tackle Hallucinations

Generative AI and Its Limitations

Generative AI depends on data to build responses to user queries. Training large language models (LLMs) uses huge volumes of data— for example, OpenAI’s GPT-3 used the CommonCrawl data set, which stood at 570 gigabytes and 400 billion tokens. But these data sets, while massive, are snapshots in time that cannot respond to queries around events happening today. AI responses can also include hallucinations, where information is provided that seems plausible but is not real. According to Vectara’s Hallucination Leaderboard, even the best performing family of LLMs (currently OpenAI’s) have hallucination rates in the range of 1.5 to 1.9 percent.

Challenges in RAG Implementation

Using LLMs on their own therefore faces two problems: the answers can be out of date, and the responses can be wrong. To overcome these potential problems, companies can use data streaming to get new information into their data set, and deploy retrieval-augmented generation (RAG) to encode business data in a way that can be used with generative AI.

RAG Explained

RAG creates a set of data that can be searched for relevant semantic matches to a user query, and those matches are then shared with the LLM for inclusion in the response. The vector data set can have new or additional data added to it over time, so relevant and timely data is available for inclusion in responses.

Conclusion: Integrating Knowledge Graphs

To use a knowledge graph with your RAG application, you can either use an existing knowledge graph with data that is tested and known to be correct in advance, or create your own. The combination of both technologies enhances the accuracy of generative AI applications in responding to user queries.