Better wit than wealth: Dynamic Parametric Retrieval Augmented Generation for Test-time Knowledge Enhancement

Yuqiao Tan1,2, Shizhu He†1,2, Huanxuan Liao1,2, Kang Liu1,2 Jun Zhao1,2
1The Laboratory of Cognition and Decision Intelligence for Complex Systems, Institute of Automation, Chinese Academy of Sciences, Beijing, China 2School of Artificial Intelligence, University of Chinese Academy of Sciences, Beijing, China
†Corresponding Author
ArXiv Paper Code

Abstract

Retrieval-augmented generation (RAG) enhances large language models (LLMs) by retrieving relevant documents from external sources and incorporating them into the context. While it improves reliability by providing factual texts, it significantly increases inference costs as context length grows and introduces challenging issue of RAG hallucination, primarily caused by the lack of corresponding parametric knowledge in LLMs. An efficient solution to this problem is to enhance the knowledge of LLMs at test-time. Parametric RAG (PRAG) addresses this by embedding document knowledge into LLM parameters through offline training, effectively reducing inference costs. However, its high training and storage costs, along with limited generalization ability, significant restrict it practical adoption. To address these challenges, we propose \textbf{Dynamic Parametric RAG (DyPRAG)}, a novel framework that utilizes a lightweight parameter translator model to efficiently map documents into parametric knowledge by modeling the underlying function from documents to parameters, reducing inference, training and storage costs while enhancing LLMs knowledge in a plug-and-play manner at test-time. Extensive experiments on multiple datasets demonstrate the effectiveness and generalization of DyPRAG. Notably, DyPRAG dynamically generates parameteric knowledge to resolve conflicts between contextual and parametric knowledge, offering a practical solution to mitigate RAG hallucination in real-world applications.

DyPRAG Overview

Workflow illustration of standard RAG, PRAG, and our proposed DyPRAG . DyPRAG significantly reduces costs by employing a lightweight parameter translator.

DyPRAG Method

An illustration of the DyPRAG method. In the offline phase, Stage 1 follows the same parameterization process as PRAG to collect Doc-Param pairs. In Stage 2, we train the parameter projector to learn the mapping function from documents to parameters. During the online Stage 3, the trained parameter translator dynamically generates LoRA modules, enabling LLMs to enhance their knowledge at test-time.

DyPRAG Comparison with RAG and PRAG

A five-dimensional comparison of the three methods highlights in cost and ability.

DyPRAG Evaluation

Results of our main experiment of DyPRAG.

Knowledge Internalization Comparison between DyPRAG-Combine vs standard RAG on RAGTruth benchmark judged by GPT-4o.

Case study about contextual and parametric knowledge conflict in 2WQA (Bridge sub-task).

BibTeX


@misc{tan2025betterwitwealthdynamic,
    title={Better wit than wealth: Dynamic Parametric Retrieval Augmented Generation for Test-time Knowledge Enhancement}, 
    author={Yuqiao Tan and Shizhu He and Huanxuan Liao and Jun Zhao and Kang Liu},
    year={2025},
    eprint={2503.23895},
    archivePrefix={arXiv},
    primaryClass={cs.CL},
    url={https://arxiv.org/abs/2503.23895}, 
}