Implement conversational memory

[tubamos]

Description

Integrate chat history awareness into the system to enable a more coherent and context-aware conversational experience. The retrieval process manages and utilizes chat history to provide answers. This includes updating the promps to support historical messages by reformoulating questions based on past interactions.

An overview of a history-aware retriever:

Links from the official documentation:

How to add chat history to a question-answering chain
https://js.langchain.com/v0.2/docs/how_to/qa_chat_history_how_to/

How to add message history
https://python.langchain.com/v0.2/docs/how_to/message_history/

How to add memory to chatbots
https://python.langchain.com/v0.2/docs/how_to/chatbots_memory/

Other links

https://www.youtube.com/watch?v=X05uK0TZozM

https://www.pinecone.io/learn/series/langchain/langchain-conversational-memory/

https://k33g.hashnode.dev/make-a-genai-conversational-chatbot-with-memory

User Story

• As a common-user,
• I want the system to remember my previous questions and the agent's responses within a session
• so that I can have a coherent and context-aware conversation without having to repeat information.

Acceptance Criteria

• (probabably refundant?) A memory store has been set to allow persistent chat history across multiple interactions / sessions.
• The prompt templates have been modified to support and include historical messages.
• The Q&A retrieval chain uses both the current query and historical context.
• The system has been tested to ensure follow-up questions are accurately answered using the chat history.
• A way to exclude memory from selected interactions has been created
• A way to reset memory has been created

Definition of Done

• The feature has been fully implemented.
• The feature has been manually tested and works as expected without critical bugs.
• The feature code is documented with clear explanations of its functionality and usage.
• The feature code has been reviewed and approved by at least one team member.
• The feature branches have been merged into the main branch and closed.
• The feature utility, function and usage have been documented in the respective project wiki on github.