PersonaRAG: Enhancing Retrieval-Augmented Generation Systems with User-Centric Agents

Source code for our paper :
PersonaRAG: Enhancing Retrieval-Augmented Generation Systems with User-Centric Agents

Overview

PersonaRAG is a novel RAG framework that incorporates user-centric agents to dynamically adapt the retrieval and generation process based on real-time user data and interaction patterns. It employs a multi-agent approach that actively engages with and adapts to user-specific needs and behaviors, enhancing the relevance and accuracy of generated content. The evaluation of PersonaRAG using ChatGPT-4 across various question answering datasets demonstrates its superiority over baseline models, with the capability of providing an answer that is tailored to the user's interaction and information need.

Quick Start

Installation

Clone the repository and install dependencies using Poetry:

git clone https://github.com/padas-lab-de/ir-rag-sigir24-persona-rag
cd ir-rag-sigir24-persona-rag
poetry install

Configuration

Set your own API keys in the environment variables:

export OPENAI_API_KEY='your-openai-api-key'
export OLLAMA_API_KEY='your-ollama-api-key' #Optional
export MIXTRAL_API_KEY='your-mixtral-api-key' #Optional

Running the Scripts

Use the main script to run different functionalities.

#To run the model
poetry run python -m scripts.main run --dataset nq --topk 5
#To build results into a CSV
poetry run python -m scripts.main build --dataset nq --topk 5
#To evaluate the model outputs
poetry run python -m scripts.main evaluate --dataset nq --topk 5

Reproduction

We provide our request logs, so the results in the paper can be quickly reproduced:

poetry run python -m logs.eval --dataset nq --topk 5

Parameters:

• dataset: dataset name (e.g. nq, webq, triviaqa).
• topk: using top-k of retrieved passages to augment (i.e. 3, 5)

Citation

If the code and the paper are useful for you, it is appreciable to cite our paper:

@article{DBLP:journals/corr/abs-2407-09394,
  author       = {Saber Zerhoudi and
                  Michael Granitzer},
  title        = {PersonaRAG: Enhancing Retrieval-Augmented Generation Systems with
                  User-Centric Agents},
  journal      = {CoRR},
  volume       = {abs/2407.09394},
  year         = {2024},
  url          = {https://doi.org/10.48550/arXiv.2407.09394},
  doi          = {10.48550/ARXIV.2407.09394},
  eprinttype    = {arXiv},
  eprint       = {2407.09394},
  timestamp    = {Thu, 15 Aug 2024 11:20:56 +0200},
  biburl       = {https://dblp.org/rec/journals/corr/abs-2407-09394.bib},
  bibsource    = {dblp computer science bibliography, https://dblp.org}
}