Support for Azure OpenAI Service Model Names in Haystack

[ulan-yisaev]

Hello Haystack maintainers and community,

I've noticed that the current code seems to be tailored for OpenAI model names that follow a specific naming convention with dots, as seen here:
https://github.com/deepset-ai/haystack/blob/c38943721fbd702367a8934cb660a72b3143eb86/haystack/utils/openai_utils.py#L89C57-L89C57

if model_tokenizer:
    # Based on OpenAI models page, 'davinci' considers having 2049 tokens,
    ## therefore, it is better to add `text-davinci` instead to the condition.
    ## Ref: https://platform.openai.com/docs/models/gpt-3-5
    ##      https://platform.openai.com/docs/models/gpt-3
    if "text-davinci" in model_name:
        max_tokens_limit = 4097
        tokenizer_name = model_tokenizer
    elif model_name.startswith("gpt-3.5-turbo-16k"):

However, I've noticed that the Azure OpenAI Service models use a slightly different naming convention without the dot, such as:

"gpt-35-turbo-16k"
"gpt-35-turbo"

Would it be possible to add support for these Azure OpenAI Service model names in Haystack? This would greatly enhance the flexibility and compatibility of the library for those of us using Azure's OpenAI Service.

Thank you for your time and consideration!

[bilgeyucel]

Hey @ulan-yisaev, there's support for Azure OpenAI Service in Haystack. See here:

haystack/haystack/utils/openai_utils.py

Line 70 in c389437

if model_name == "gpt-35-turbo":

In this docs page you can find more info about how to use Azure OpenAI Service: https://docs.haystack.deepset.ai/docs/prompt_node#using-azure-openai-service

However, I see the support is only for gpt-35-turbo, no support for gpt-35-turbo-16k just yet.
If you feel like contributing to Haystack, open a PR for the support for gpt-35-turbo-16k! Otherwise, I can open an issue for it 😊

[anakin87]

My impression is that even if we support these models,
something about tokenizer and max_tokens_limit might be missing (also gpt-35-turbo should be checked carefully).

As @bilgeyucel said, feel free to open a PR, otherwise we will try to take care of it...