Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Added new VertexAI Anthropic's model module #1213

Open
wants to merge 1 commit into
base: main
Choose a base branch
from

Conversation

alessiobertazzo88
Copy link

This pull request introduces a new module to the Spring AI framework, designed to integrate and expand the use of Anthropic Claude LLM models within the VertexAI platform. The integration leverages the Model Garden feature of VertexAI to provide seamless and consistent access to Claude LLMs.

@alessiobertazzo88
Copy link
Author

Hi @tzolov @markpollack just to understand if you could evaluate the merge of this PR that enables the use Anthropic's models from GCP Vertex Model Garden that has variations from both Anthropic standard endpoints but also from Vertex already implemented (Gemini) modules.

Thanks in advance!

@markpollack
Copy link
Member

Hi, sorry for letting this sit so long. This is a pretty big PR and i'm growing concerned over the number of models and variations each with different code bases. Just so verify my understanding with you, the need for this new API as compared to using the Vertex API to access the model is that the Anthropic model you want to use is different than the one exposed via Vertex API?

@alessiobertazzo88
Copy link
Author

Hi, that's correct. Initially, I tried using the Vertex API to take advantage of Google’s SDK and also benefit from the authentication component. I had also seen the possibility of changing the publisher by dynamically specifying the model in the URL. Unfortunately, it’s not possible to use custom models that the Anthropic APIs within VertexAI’s model garden require (here’s the link with the details: https://console.cloud.google.com/vertex-ai/publishers/anthropic/model-garden/claude-3-5-sonnet?_ga=2.151856633.1780909182.1726601103-828101531.1719608013). This is the reason why I created this additional version.

@ddobrin
Copy link

ddobrin commented Oct 21, 2024

Hi @alessiobertazzo88
could you please confirm the following, just to be sure that I understand this right?

You pick Sonnet 3.5 for example from the Model Garden in Vertex AI, Enable the API and would like to access it programmatically from Spring AI and a supporting SDK.

@alessiobertazzo88
Copy link
Author

alessiobertazzo88 commented Oct 22, 2024

Hi @ddobrin
Correct! The Google SDK now is only used for authentication because of lack of models for Anthropic use.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants