core[minor]: Enhance cache flexibility in BaseChatModel #17386
Conversation
|
The latest updates on your projects. Learn more about Vercel for Git ↗︎ 1 Ignored Deployment
|
dosubot
bot
added
size:M
|
@eyurtsev Could you please confirm whether you intend to include caching functionality in the |
|
We should iterate in smalls steps and match existing global cache functionality. If we want to extend the caching for streaming, we can do so, but let's do it as a separate task if it's not covered by existing global cache. |
dosubot
bot
added
size:L
Hi @eyurtsev I added the inline docstring and added test. Could not work on external documentation. Could anybody help me here @hasansustcse13 @Lord-Haji |
| InMemoryCache, | ||
| SQLAlchemyCache, | ||
| ) | ||
| from langchain.globals import get_llm_cache, set_llm_cache | ||
|
|
||
|
|
||
| class MockCache(BaseCache): |
There was a problem hiding this comment.
@alhridoy This is not the correct BaseCache interface.
Do you want to try and fix up the code? You can use InMemoryCache implementation to test things out.
At the moment, the included unit tests have an number of errors in them, so testing code does not work
| @@ -15,12 +16,27 @@ | |||
| from sqlalchemy.orm import Session | |||
|
|
|||
| from langchain.cache import ( | |||
| BaseCache, | |||
There was a problem hiding this comment.
BaseCache is not available from langchain.cache, instead it's in langchain_core.cache
|
I'm going to commandeer to we can land this |
eyurtsev
changed the title
Hi @eyurtsev Give me a few days. I'll try it first. Thanks for your patience and guidance. |
|
@alhridoy just saw your message -- i made the changes already. I'm going to merge for now if tests pass, feel free to review the changes. And there are a lot of contribution opportunities so we'll appreciate any help :) |
dosubot
bot
added
the
lgtm
…#17386) - **Description:** Enhanced the `BaseChatModel` to support an `Optional[Union[bool, BaseCache]]` type for the `cache` attribute, allowing for both boolean flags and custom cache implementations. Implemented logic within chat model methods to utilize the provided custom cache implementation effectively. This change aims to provide more flexibility in caching strategies for chat models. - **Issue:** Implements enhancement request langchain-ai#17242. - **Dependencies:** No additional dependencies required for this change. --------- Co-authored-by: Eugene Yurtsev <[email protected]>
…#17386) - **Description:** Enhanced the `BaseChatModel` to support an `Optional[Union[bool, BaseCache]]` type for the `cache` attribute, allowing for both boolean flags and custom cache implementations. Implemented logic within chat model methods to utilize the provided custom cache implementation effectively. This change aims to provide more flexibility in caching strategies for chat models. - **Issue:** Implements enhancement request langchain-ai#17242. - **Dependencies:** No additional dependencies required for this change. --------- Co-authored-by: Eugene Yurtsev <[email protected]>
- **Description:** Enhanced the `BaseChatModel` to support an `Optional[Union[bool, BaseCache]]` type for the `cache` attribute, allowing for both boolean flags and custom cache implementations. Implemented logic within chat model methods to utilize the provided custom cache implementation effectively. This change aims to provide more flexibility in caching strategies for chat models. - **Issue:** Implements enhancement request #17242. - **Dependencies:** No additional dependencies required for this change. --------- Co-authored-by: Eugene Yurtsev <[email protected]>
Please title your PR "core: Enhance cache flexibility in BaseChatModel".
BaseChatModelto support anOptional[Union[bool, BaseCache]]type for thecacheattribute, allowing for both boolean flags and custom cache implementations. Implemented logic within chat model methods to utilize the provided custom cache implementation effectively. This change aims to provide more flexibility in caching strategies for chat models.