Skip to content
This repository has been archived by the owner on May 24, 2024. It is now read-only.

Using this or the YARP repo for Assistants API #3

Open
juichiache opened this issue Mar 6, 2024 · 0 comments
Open

Using this or the YARP repo for Assistants API #3

juichiache opened this issue Mar 6, 2024 · 0 comments

Comments

@juichiache
Copy link

Hi! This repo and the other one you have built using YARP are so helpful, thank you!
I have been researching how to load balance between AOAI instances when using Assistants API.
So far, it seems like we can use an affinity setting to send a cookie to the client and pinging the subsequent requests to the same backend.
Do you know if we can do something like that in your APIM smart load balancer, or the YARP one?
Really appreciate your thoughts and input.

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant