Using Giskard for scanning existing Q&A chat with LLM

[OrenRachmil]

Hi,

I have a .csv file with two columns: "Questions" and "Answers." This dataset was generated by collecting multiple prompts and answers from chats with my LLM (LLAMA 3).

Is there a way to use this existing dataset to run the Giskard scanners without generating new prompts?

[alexcombessie]

Hey @OrenRachmil,

The goal of the scan is precisely to generate new prompts in an adversarial fashion. If you already have a knowledge base with questions & answers, perhaps this methods (RAG evaluation toolkit) would be more fit for purpose: https://docs.giskard.ai/en/stable/open_source/testset_generation/rag_evaluation/index.html

Hope this helps,

Alex