fix(Unstructured): Pin Client version before breaking changes

[lambda-science]

Related Issues

There is breaking changed in unstructured client 0.26 that are quite annoying.
See: https://docs.unstructured.io/api-reference/api-services/sdk-python#migration-guide
Mainly we get this error: ypeError: General.partition() takes 1 positional argument but 2 were given

Proposed Changes:

Solved by pinning client version to latest compatible version.

How did you test it?

In my own project

Notes for the reviewer

Don't know why test don't catch it or if it's only on my local setup with self-hosted latest version unstructured-api (0.82.0 dec 2024)

Checklist

• I have read the contributors guidelines and the code of conduct
• I have updated the related issue with new insights and changes
• I added unit tests and updated the docstrings
• I've used one of the conventional commit types for my PR title: fix:, feat:, build:, chore:, ci:, docs:, style:, refactor:, perf:, test:.

[anakin87]

Hello, @lambda-science!

Before pinning the version, I would like to reproduce the bug (so that we can also understand how to update the integration in the future)...

Could you share more details/a reproducible example?

[lambda-science]

Hi @anakin87

1. Run the unstructured API docker container on port 8002 quay.io/unstructured-io/unstructured-api:0.0.82
2. Have a python env with:

dependencies = [
    "haystack-ai==2.9.0",
    "unstructured-fileconverter-haystack>=0.4.1",
    "unstructured-client>=0.26",
]

Run this script next to sample4.docx

from haystack_integrations.components.converters.unstructured import UnstructuredFileConverter
UNSTRUCTURED_SETTINGS = {
    "skip_infer_table_types": "[]",
    "chunking_strategy": "by_title",
    "combine_under_n_chars": "1000",
    "new_after_n_chars": "1500",
    "max_characters": "2000",
    "pdf_infer_table_structure": "True",
    "languages": ["eng", "fra"],
    "strategy": "fast",
}

converter = UnstructuredFileConverter(api_url="http://localhost:8002/general/v0/general",
                                              document_creation_mode="one-doc-per-element",
                                              unstructured_kwargs=UNSTRUCTURED_SETTINGS)
documents = converter.run(paths=["sample4.docx"])
print(documents)

Results:

Converting files to Haystack Documents: 0it [00:00, ?it/s]WARNING: Unstructured could not process file sample4.docx. Error: 1 validation error for PartitionParameters
skip_infer_table_types
  Input should be a valid list [type=list_type, input_value='[]', input_type=str]
    For further information visit https://errors.pydantic.dev/2.10/v/list_type
Converting files to Haystack Documents: 1it [00:00,  1.67it/s]
{'documents': []}

If you remove the skip_infer_table_types param you get

Converting files to Haystack Documents: 0it [00:00, ?it/s]WARNING: Unstructured could not process file sample4.docx. Error: General.partition() takes 1 positional argument but 2 were given
Converting files to Haystack Documents: 1it [00:00,  1.78it/s]
{'documents': []}

instead of the list of doc if you pin <0.26

[lambda-science]

Here is the sample docx
sample4.docx

[anakin87]

Thanks for the detailed report! I'll take a look...

[anakin87]

@lambda-science
I tried both main and the latest version of the integration, with python 3.9 and 3.10.

unstructured-client==0.29.0 is automatically installed and I don't get the error (when removing skip_infer_table_types), but I get:

{'documents': [Document(id=177226da7488ee08d3eb12b7453e3b0acf010a1d2008ae0dde0a7a704d4a3a63,
content: 'Lorem ipsum dolor sit amet, consectetur adipiscing elit. Suspendisse sed ex eget diam dictum varius ...',
meta: {'file_path': 'sample4.docx', 'element_index': 0, 'filename': 'sample4.docx', 'languages': ['cat', 'ita', 'fra'], 'page_number': 1, 'orig_elements': ...', 'filetype': 'application/vnd.openxmlformats-officedocument.wordprocessingml.document', 'category': 'CompositeElement'}),
...]}

Could you please check?

[lambda-science]

I'm on 3.11 I'll check if it's 3.11 specific when I get a moment :)