Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

google-common: Support Gemini 2.0 Thinking model #7434

Open
1 task done
afirstenberg opened this issue Dec 27, 2024 · 0 comments
Open
1 task done

google-common: Support Gemini 2.0 Thinking model #7434

afirstenberg opened this issue Dec 27, 2024 · 0 comments
Labels
auto:improvement Medium size change to existing code to handle new use-cases

Comments

@afirstenberg
Copy link
Contributor

Privileged issue

  • I am a LangChain maintainer, or was asked directly by a LangChain maintainer to create an issue here.

Issue Content

Note this is NOT about general Gemini 2.0 compatibility. This is specifically about the "gemini-2.0-flash-thinking-exp" model (and related "reasoning" models).

There are some issues

  • The "reasoning tokens" are available in the output, but they are a different text part than the final result
    • So part[0] contains the reasoning and part[1] contains the final result
    • If using API version "v1alpha" (at least on AI Studio), there is a "thinking" attribute which is available to indicate which part is the reasoning and which part is final
    • The current logic tries to merge text parts together. This is, arguably, undesirable. At the very least - it breaks some of the tests.
    • We need to see how this works with streaming.
    • We need to discuss the best way to handle this in general - not just for Gemini.
  • Currently, this model doesn't support tools.

My thinking about handling the reasoning tokens:

  • For compatibility with other models - they should be removed from normal output.
  • They should be included in the response_metadata as a "reasoning" attribute, or something similar.
  • We should have a filter that can, optionally, put them back together.
  • Should any of this be controlled with a parameter?
  • I don't think this needs to be a new class.
@dosubot dosubot bot added the auto:improvement Medium size change to existing code to handle new use-cases label Dec 27, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
auto:improvement Medium size change to existing code to handle new use-cases
Projects
None yet
Development

No branches or pull requests

1 participant