Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feature: implement _acall method for GPT4All(LLM) #14495

Closed
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
42 changes: 41 additions & 1 deletion libs/community/langchain_community/llms/gpt4all.py
Original file line number Diff line number Diff line change
@@ -1,7 +1,10 @@
from functools import partial
from typing import Any, Dict, List, Mapping, Optional, Set

from langchain_core.callbacks import CallbackManagerForLLMRun
from langchain_core.callbacks import (
AsyncCallbackManagerForLLMRun,
CallbackManagerForLLMRun,
)
from langchain_core.language_models.llms import LLM
from langchain_core.pydantic_v1 import Extra, Field, root_validator

Expand Down Expand Up @@ -211,3 +214,40 @@ def _call(
if stop is not None:
text = enforce_stop_tokens(text, stop)
return text

async def _acall(
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

doesn't LLM._acall have a default implementation that does the same thing

cc @nfcampos

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Okey, that is true. It has that method implemented. I tested it also and it works the same as this one.
My PR is linked to this issue #5210, if the issue is solved already, then close both the issue and this PR.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Was the current implementation for GPT4All tested with websockets for example (to test the asynchronous behavior works as intended?) (I will double check this ASAP)

self,
prompt: str,
stop: Optional[List[str]] = None,
run_manager: Optional[AsyncCallbackManagerForLLMRun] = None,
**kwargs: Any,
) -> str:
"""Asynchronous Call out to GPT4All's generate method.

Args:
prompt: The prompt to pass into the model.
stop: A list of strings to stop generation when encountered.
run_manager: an async callback manager
Returns:
The string generated by the model.

Example:
.. code-block:: python

prompt = "Once upon a time, "
response = model(prompt, n_predict=55)
"""
text_callback = None
if run_manager:
text_callback = partial(run_manager.on_llm_new_token, verbose=self.verbose)

params = {**self._default_params(), **kwargs}

text = ""
for token in self.client.generate(prompt, **params):
if text_callback:
await text_callback(token)
text += token
if stop is not None:
text = enforce_stop_tokens(text, stop)
return text
Loading