[Feature Request] Add ModelManager
to schedule calls between different model backend
#1031
Open
1 of 2 tasks
Labels
call for contribution
enhancement
New feature or request
New Feature
P1
Task with middle level priority
Milestone
Required prerequisites
Motivation
This will be helpful when we are limited to the rate limit. We have use
ModelManager
inChatAgent
and have different API keys for each models. This may be also powerful if we want to do model-adaptive inference based on tasks. A pseudo code would be like:Similar ideas can be applied to other objects that requires api keys and so on. Like tools and loaders.
Solution
No response
Alternatives
No response
Additional context
No response
The text was updated successfully, but these errors were encountered: