-
Notifications
You must be signed in to change notification settings - Fork 2.3k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Gemini Vertex AI 400 error in recent update #7509
Comments
The error you're encountering is due to a change in the Gemini Vertex AI API requirements. The error message indicates that the Here's how you can update your code: llm = new ChatVertexAI({
model: model,
authOptions: {
credentials: JSON.parse(process.env.VERTEX_SERVICE_ACCOUNT_JSON!)
},
maxOutputTokens: 8192,
temperature: temperature,
responseLogprobs: true // Add this line
}); This change should align your configuration with the updated API requirements and resolve the 400 error you're experiencing. To continue talking to Dosu, mention @dosu. Help Dosu learn! Give it feedback: Great Response | Irrelevant Answer | Incorrect Sources | Too Verbose | Hallucination | Other |
Nope. |
I've faced the same issue, and it turns out that ChatVertexAI doesn't actually include the responseLogprobs parameter—only logprobs is available. I suspect that disabling logprobs will generate the same error message. |
@afirstenberg can this be caused by your changes from #7472 ? |
I am new to Langchain JS and having the same issue. Setting logprobs to true gives the following error
|
Yep. I have never touched this parameter, and the error began popping up out of nowhere. If I explicitly disable logprobs, it does nothing and still errors |
Using the ❯ AI Models integration tests / Test BaseAIService
Error: Unable to submit request because logprobs is not supported without setting response_logprobs to be true.. Learn more:
https://cloud.google.com/vertex-ai/generative-ai/docs/model-reference/gemini
⁃ Gaxios._request
node_modules/gaxios/src/gaxios.ts:169
⁃ async JWT.requestAsync
node_modules/google-auth-library/build/src/auth/oauth2client.js:382
⁃ async RetryOperation._fn
node_modules/p-retry/index.js:50
|
It sure sounds that way. I'm digging into what happened and why our tests didn't catch this. It sounds like it is trying to set
That's because Google limits the use of logprobs to one per day. I'm investigating and will get an update ASAP (@jacoblee93 - Heads up that something will be incoming imminently or that we should revert.) |
I'm not sure why the tests missed it before. |
Apologies for this! Fix is going out shortly |
Fix is live in 0.1.8! |
Checked other resources
Example Code
Error Message and Stack Trace (if applicable)
Google request failed with status code 400: [{ "error": { "code": 400, "message": "Unable to submit request because logprobs is not supported without setting response_logprobs to be true.. Learn more: https://cloud.google.com/vertex-ai/generative-ai/docs/model-reference/gemini", "status": "INVALID_ARGUMENT" } } ]
Description
I have been using the same code for months, and all has worked fine up until now. I recently updated my npm packages, and it seems like some incompatibility issue was introduced. That error message is fired any time I make a request now, and I have no way to bypass it. Setting logprobs to false does nothing
System Info
Platform:
Mac
Node version:
v21.0.0
Yarn version:
1.22.19
The text was updated successfully, but these errors were encountered: