[Bug]: "Multi-Modal LLM using DashScope qwen-vl model for image reasoning" example code raised a ValidationError by pydantic

[9ovn]

Bug Description

running the Multi-Modal LLM using DashScope qwen-vl model for image reasoning expamle code got following error

ValidationError                           Traceback (most recent call last)
[<ipython-input-4-833822ee199d>](https://localhost:8080/#) in <cell line: 1>()
----> 1 complete_response = dashscope_multi_modal_llm.complete(
      2     prompt="What's in the image?",
      3     image_documents=image_documents,
      4 )
      5 print(complete_response)

5 frames
[/usr/local/lib/python3.10/dist-packages/pydantic/main.py](https://localhost:8080/#) in __init__(self, **data)
    212         # `__tracebackhide__` tells pytest and some other tools to omit this function from tracebacks
    213         __tracebackhide__ = True
--> 214         validated_self = self.__pydantic_validator__.validate_python(data, self_instance=self)
    215         if self is not validated_self:
    216             warnings.warn(

ValidationError: 2 validation errors for ChatMessage
blocks.0
  Unable to extract tag using discriminator 'block_type' [type=union_tag_not_found, input_value={'image': 'https://dashsc...ages/dog_and_girl.jpeg'}, input_type=dict]
    For further information visit https://errors.pydantic.dev/2.10/v/union_tag_not_found
blocks.1
  Unable to extract tag using discriminator 'block_type' [type=union_tag_not_found, input_value={'text': "What's in the image?"}, input_type=dict]
    For further information visit https://errors.pydantic.dev/2.10/v/union_tag_not_found

Version

latest

Steps to Reproduce

same as the example

Relevant Logs/Tracbacks

No response

[dosubot]

Hey @9ovn! I'm here to help you with bug fixes, questions, and more. Let's tackle this issue together!

I found a similar unsolved discussion regarding a Pydantic or version error in the LlamaIndex repository. You can view it here ^[1].

The error you're encountering seems to be related to the block_type discriminator in the Pydantic model. The possible values for the block_type discriminator are "text" and "image" ^[2]. It appears that the input data might not be correctly formatted to match these expected values.

To resolve this issue, ensure that the input data for the ChatMessage is structured correctly. Each block should explicitly specify the block_type as either "text" or "image". Here's a quick check you can perform:

1. Verify that each block in your input data has a block_type key with the correct value.
2. Ensure that the image_documents and prompt are correctly formatted as per the expected structure.

If the issue persists, it might be helpful to review the Pydantic model definitions to ensure compatibility with the version of Pydantic you are using.

To continue talking to Dosu, mention @dosu.

---

^{Help Dosu learn! Give it feedback: Great Response | Irrelevant Answer | Incorrect Sources | Too Verbose | Hallucination | Other}

[logan-markewich]

@9ovn can you give a specific version?

pip show llama-index-core

[9ovn]

pip show llama-index-core

0.12.5

[logan-markewich]

Hmm. I wish I could debug this but I can't figure out how to get an API key for dashcope (I'm in Canada)

[gavin7758521]

Got it, it's just that there's a problem with the source code of llama index, a bunch of bugs!