[Enhancement]: Bulkinsert supports null in csv formats

[OxalisCu]

Is there an existing issue for this?

• I have searched the existing issues

What would you like to be added?

Recently, milvus has added support for the special data type of null, so it is necessary to support the parsing of null values ​​in csv format in the bulkinsert function.

csv is a text data format, and there is no specific data type corresponding to null, so the implementation idea is to use a configurable special string to represent the null value. If this special string is matched, then this field will be parsed as null.

Data types that support null values

Currently the data types that support null are

bool, int8, int16, int32, int64, float, double, varchar, string, json, array

The data types that do not support null are

binaryvector, floatvector, float16vector, bfloat16vector, sparsefloatvector

In addition, the array type supports null, but the elements in the array do not support null, as shown in the following example

// support
arr := NULL
// not support
arr := [1, 2, NULL, 4]

nullkey configuartion

In csv format, parsing of null identifiers can be configured. The field name is nullkey, which currently supports any string. If not configured, the default is nullkey = "" .

curl --request POST "http://localhost:19530/v2/vectordb/jobs/import/create" \
--header "Content-Type: application/json" \
--data-raw '{
    "files": [
        [
            "filepath"
        ]
    ],
    "collectionName": "collection_name",
    "options": {"nullkey": "NULL"}
}'

Why is this needed?

No response

Anything else?

No response

[xiaofan-luan]

/assign @smellthemoon

could you help on this?