Domain-Specific Knowledge Graphs in RAG-Enhanced Healthcare LLMs

Code

• github.com/sydneyanuyah/ragcomparison
  OfficialIn paper★ 0

Abstract

Large Language Models (LLMs) generate fluent answers but can struggle with trustworthy, domain-specific reasoning. We evaluate whether domain knowledge graphs (KGs) improve Retrieval-Augmented Generation (RAG) for healthcare by constructing three PubMed-derived graphs: G_1 (T2DM), G_2 (Alzheimer's disease), and G_3 (AD+T2DM). We design two probes: Probe 1 targets merged AD T2DM knowledge, while Probe 2 targets the intersection of G_1 and G_2. Seven instruction-tuned LLMs are tested across retrieval sources No-RAG, G_1, G_2, G_1 + G_2, G_3, G_1+G_2 + G_3 and three decoding temperatures. Results show that scope alignment between probe and KG is decisive: precise, scope-matched retrieval (notably G_2) yields the most consistent gains, whereas indiscriminate graph unions often introduce distractors that reduce accuracy. Larger models frequently match or exceed KG-RAG with a No-RAG baseline on Probe 1, indicating strong parametric priors, whereas smaller/mid-sized models benefit more from well-scoped retrieval. Temperature plays a secondary role; higher values rarely help. We conclude that precision-first, scope-matched KG-RAG is preferable to breadth-first unions, and we outline practical guidelines for graph selection, model sizing, and retrieval/reranking. Code and Data available here - https://github.com/sydneyanuyah/RAGComparison

Reproductions