Customize SearchRequest sent to VectorStore in QuestionAnswerAdvisor

[alexcheng1982]

In the current implementation of QuestionAnswerAdvisor, the user text is always used as the query sent to VectorStore's similaritySearch method. This may not be the case for some advanced RAG scenarios, where the original user query may be expanded, transformed, or rewritten to get the actual query sent to the vector store.

The new customizeSearchRequest method provides an extension point to customize the SearchRequest used by VectorStore's similaritySearch method. This allows subclasses of QuestionAnswerAdvisor to apply custom logic to update the SearchRequest.

[youngmoneee]

Why is it necessary to inherit and reimplement the implementation to return a SearchRequest method, even though QuestionAndAnswerAdvisor is already receiving a SearchRequest object through the constructor?

[markpollack]

@alexcheng1982 This is indeed a very important use case to address. Previous iterations of what became the Advisor approach better accommodated this, it in fact kept a history of the changes to the user message. Instead of a subclass approach, what if we make use of the context in the advisor and have a well known key which is 'transformedUserTextor some name like that. This way there can be a chain of advisors such that before theQuestionAnswerAdvisoradvisor executes, aQueryRewriteAdvisorexecutes and puts the new updated query under the keytransformedUserTextin the context. Then theQuestionAnswerAdvisor` would look in that area of the context first and use the text there if present and if not, fall back to the current impl.

Thoughts?

[youngmoneee]

I’m implementing a proxy object to intercept requests from the QuestionAndAnswerAdvisor to customize a Similarity Search.

.prompt()
.advisors(spec -> spec
    .param(CHAT_MEMORY_CONVERSATION_ID_KEY, chatId)
    .param(SEARCH_REQUEST, searchRequest)
    .param(QUERY, queryString)
    // pass query or SearchRequest
)
.user(userMessage)

Wouldn't it be more natural to pass the SearchRequest object through AdvisorSpec rather than using inheritance?

request.advisorParams().getOrDefault(
        SEARCH_REQUEST,
        this.searchRequest
    )