-
Notifications
You must be signed in to change notification settings - Fork 5.4k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add Multi Modal Bedrock Integration #17451
base: main
Are you sure you want to change the base?
Conversation
I appreciate the PR @dnandha -- but actually, Im inclined to not merge this. We are in the middle of deprecating our multimodal llm classes, and building in multi modal capabilities directly into the existing LLM class For example, anthropic: Line 172 in db29c90
OpenAI: llama_index/llama-index-integrations/llms/llama-index-llms-openai/llama_index/llms/openai/utils.py Line 274 in db29c90
|
Issue we are tracking the migration with |
Hey @logan-markewich . Thanks for looking into the PR so quickly and sharing your concern. I wasn't aware about the issue you mentioned, but see your point. However, after checking your standpoint, my suggestion would still be merging the PR:
Since the work has already been done, my suggestion would be the following:
Let me know your thoughts on this. |
@dnandha the unit tests are failing, can you take a look? If these are relying on actual api calls, you might have to mock them |
Description
This PR adds support for AWS Bedrock multi-modal models in LlamaIndex. The integration allows users to interact with Bedrock's multi-modal models (currently Claude 3 family) for image and text analysis tasks.
Key features:
New Package?
Did I fill in the
tool.llamahub
section in thepyproject.toml
and provide a detailed README.md for my new integration or package?Version Bump?
Did I bump the version in the
pyproject.toml
file of the package I am updating?Type of Change
How Has This Been Tested?
Tests include:
Checklist:
make format; make lint
to appease the lint godsImplementation Details
The implementation includes:
BedrockMultiModal
class that inherits fromMultiModalLLM
Testing
The package includes unit tests that cover:
All tests are passing and the code follows LlamaIndex's coding standards.
Solves #13507.