[BUG] Token limitations don't seem to get thrown from the client library #46782
Labels
Client
This issue points to a problem in the data-plane of the library.
customer-reported
Issues that are reported by GitHub users external to the Azure organization.
needs-team-attention
Workflow: This issue needs attention from Azure service team or SDK team
OpenAI
question
The issue doesn't require a change to the product in order to be resolved. Most issues start as that
Service Attention
Workflow: This issue is responsible by Azure service team.
Library name and version
Azure.AI.OpenAI
Describe the bug
When trying to call a chat completion it seems like if there is an error nothing is ever thrown and it appears like the application just sits. I have a controller action that calls with a user request that catches a generic exception and the catch is never hit nor is anything ever returned. I was hitting the token limit (the only way i knew this was replicating the messages in the playground) and had no idea. Here is a very small snippet of the code illustrating the line causing it.
ChatClient chatClient = _azureOpenAIClient.GetChatClient("gpt-4o-mini");
chatClient.CompleteChat(conversationMessages, options); //this never returns, never throws, i have no idea if this is a slow request or it immediately crashed. conversationMessages is over the token limit.
Expected behavior
An meaningful exception should be thrown if an error occurred.
Actual behavior
Call never returns anything.
Reproduction Steps
ChatClient chatClient = _azureOpenAIClient.GetChatClient("gpt-4o-mini");
//some large request in conversation messages
chatClient.CompleteChat(conversationMessages, options); //this never returns, never throws, i have no idea if this is a slow request or it immediately crashed. conversationMessages is over the token limit.
Environment
Windows 11 .netcore 8 VS 17.11
The text was updated successfully, but these errors were encountered: