Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

StreamResponseEnumerableFromChatbotAsync wrong role? #119

Open
SemperFu opened this issue Apr 19, 2023 · 6 comments · May be fixed by #122
Open

StreamResponseEnumerableFromChatbotAsync wrong role? #119

SemperFu opened this issue Apr 19, 2023 · 6 comments · May be fixed by #122

Comments

@SemperFu
Copy link

SemperFu commented Apr 19, 2023

/// Calls the API to get a response, which is appended to the current chat's <see cref="Messages"/> as an <see cref="ChatMessageRole.Assistant"/>

_chat.AppendUserInput(prompt);
await foreach (string responseChunk in _chat.StreamResponseEnumerableFromChatbotAsync())
{
    OutputText.Text += responseChunk;
}

Trying to use the following but when looking at the messages in chat I can see that the role for the responses is marked as User instead of Assistant.

image

@Tryanks
Copy link

Tryanks commented Apr 19, 2023

I have checked the source code Conversation.cs#L193 and there seems to be no problem.

Is it possible that the OpenAI APIf return value is wrong?

@NorKaiser
Copy link

NorKaiser commented Apr 19, 2023

same problem,in openai api stream result,only the first jsonpack has the role information.

@Tryanks
Copy link

Tryanks commented Apr 19, 2023

It also looks like there are no errors in the content returned by the OpenAI API :

  "choices": [
    {
      "delta": {
        "role": "assistant"
      },
      "finish_reason": null,
      "index": 0
    }
  ]

Then I don't know what the problem is.

Maybe the behavior of Conversation.cs#L193 can be hard-coded as Assistant?

@OkGoDoIt Can you take a look at this?

sdcb added a commit to sdcb/OpenAI-API-dotnet that referenced this issue Apr 20, 2023
This is caused if upstream delta Role was not provided, then "user" will returned.
@sdcb
Copy link

sdcb commented Apr 20, 2023

I just found the causing reason, it will assign responseRole multiple times, first time it assignted to assistance, by next token response, it will reassigned to user although it's not response in API stream.
Pull request to fix: #122

Workaround before official merge:
Specify following code after every call to chat.StreamResponseEnumerableFromChatbotAsync():

chat.Messages[^1].Role = ChatMessageRole.Assistant;

example:

OpenAIAPI api = OpenAIAPI.ForAzure("*****", "*****", "******");
api.ApiVersion = "2023-03-15-preview";
Conversation chat = api.Chat.CreateConversation();

while (true)
{
	string? input = Console.ReadLine();
	if (input == "exit" || input == "q") break;


	chat.AppendUserInput(input);
	Console.WriteLine($"> {input}");

	await foreach (string res in chat.StreamResponseEnumerableFromChatbotAsync())
	{
		Console.Write(res);
	}
	Console.WriteLine();
	chat.Messages[^1].Role = ChatMessageRole.Assistant; // important
}

@Tryanks
Copy link

Tryanks commented Apr 20, 2023

by next token response, it will reassigned to user although it's not response in API stream.

Does this mean that the default value of delta.Role is not null but user?
Does it depend on the API or class ChatMessage initialization?

@sdcb
Copy link

sdcb commented Apr 20, 2023

by next token response, it will reassigned to user although it's not response in API stream.

Does this mean that the default value of delta.Role is not null but user? Does it depend on the API or class ChatMessage initialization?

Yes it seems ChatMessage's default role is user instead of null:
image

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

4 participants