The issue you're encountering is that your system is ignoring the user-specified model (claude-3-haiku) and defaults to the more expensive claude-3-opus, leading to functional and financial issues. Here's how you can go about resolving this:

1. Ensure Proper Model Selection in API Request

The most common reason this issue occurs is incorrect configuration or incorrect API request formatting. You should verify that you're correctly passing the desired model to the Anthropic API request.

Steps:

• Check the model parameter in the API request and ensure it's set to claude-3-haiku. If you're using a wrapper or a third-party library, check if it has any defaults that may override your configuration.

Example API Request:

When making an API request, the model parameter should look like this:

import requests

headers = {
    'Authorization': 'Bearer YOUR_API_KEY',
    'Content-Type': 'application/json',
}

data = {
    'model': 'claude-3-haiku',  # Ensure this is the model you want to use
    'prompt': 'Your input prompt goes here...',
    'max_tokens': 100,
}

response = requests.post('https://api.anthropic.com/v1/completions', json=data, headers=headers)

print(response.json())

• Verify the exact model name: Ensure you're passing the exact model name claude-3-haiku, as API calls can sometimes fail silently if there's a typo or misconfiguration.

2. Check for Hardcoded Defaults in Your Code

If the model is still being overridden, you may have some part of your code or the library you're using that defaults to claude-3-opus even when claude-3-haiku is specified. Look for any code in your project that might be hardcoding claude-3-opus or setting the default model incorrectly.

Steps:

• Search for any hardcoded model values in your project files, such as "claude-3-opus" or claude-3.
• Check for any fallback logic in your code that defaults to claude-3-opus if no model is explicitly passed.

Example:

# Check if the model is set correctly in all cases
model = config.get('model', 'claude-3-haiku')  # This will use 'claude-3-haiku' if no model is passed

# Ensure this is passed correctly in the API request
data = {
    'model': model,
    'prompt': 'your input prompt',
}

3. Audit Your API Usage in the Anthropic Console

Since you confirmed the issue through the Anthropic console, make sure you are correctly tracking API usage and metrics. Double-check that the model claude-3-opus isn't being used unintentionally due to some other process or fallback mechanism.

Steps:

• Review API usage logs in your Anthropic console to confirm that the system is using claude-3-opus when it shouldn't be.
• Look for patterns where the wrong model is being selected. This might provide more insights into where and why the fallback is happening.

4. Add Warnings for Expensive Model Use

To avoid unexpected costs in the future, it's a good idea to add a mechanism in your code to alert the user whenever a more expensive model (claude-3-opus) is being used. This can be done by checking the model before sending the API request and providing a warning.

Example:

if model == 'claude-3-opus':
    print("Warning: You are using the more expensive model (claude-3-opus). Consider switching to claude-3-haiku for lower costs.")

Alternatively, you could raise an alert or log the event so users are aware.

5. Implement Configuration Options for Default Model

If you want to make the default model configurable (i.e., use claude-3-haiku as the default for all users), implement a configuration system to define which model should be used by default.

Example:

# Configuration file or environment variable to set default model
DEFAULT_MODEL = 'claude-3-haiku'

def get_model_from_config():
    # Retrieve model from config or environment
    return os.getenv('MODEL', DEFAULT_MODEL)

# Make sure the correct model is being passed to the API
model = get_model_from_config()

data = {
    'model': model,
    'prompt': 'Your input prompt...',
}

response = requests.post('https://api.anthropic.com/v1/completions', json=data, headers=headers)

In this example, you can configure the default model in an environment variable or a configuration file (DEFAULT_MODEL). You can update this default value depending on your needs (e.g., to change the default from claude-3-opus to claude-3-haiku).

6. Monitor Usage for Unexpected Costs

After implementing the fix, it's important to keep monitoring the API usage to ensure that claude-3-opus is not being used unintentionally. You can set up alerts in your billing dashboard to notify you if you're exceeding a certain threshold of usage.

By following these steps, you should be able to fix the issue of incorrect model selection and prevent unexpected higher costs due to the use of the more expensive claude-3-opus model. Let me know if you need more help!

Hey @micedevai, you can see the PR here

Hi @diekotto, thanks for pointing this out!

@micedevai @diekotto fixed is merged, thanks all!