[Bug] [seatunnel-formats] Allow the entry in the map to be null and a…

…llow the key in the entry to be null (#5277) * 🐛 "Add null check in text deserialization This commit adds a null check while splitting the kv in the TextDeserializationSchema. Not having this check could potentially lead to a crash if 'kvs' has less than two elements. Now, the function will return null in this case, significantly improving reliability." * 🐛 Refactor text deserialization, allow null value assignments Modified the TextDeserializationSchema to allow key-value pairs with null values. This adjustment helps to handle data records with missing values more accurately. The code was also reformatted for improved readability. * 🎨 Refactor Builder constructor in TextDeserializationSchema Tidied up the Builder constructor in TextDeserializationSchema by removing unnecessary empty lines. This change improves code readability and consistency with the rest of the class? * ✅ Add test case where the value is empty and the key is empty in the map --------- Co-authored-by: yangpeng <[email protected]>
apache · Sep 11, 2023 · 07105a5 · 07105a5
1 parent 04ce22a
commit 07105a5
Show file tree

Hide file tree

Showing 2 changed files with 15 additions and 4 deletions.
diff --git a/...format-text/src/main/java/org/apache/seatunnel/format/text/TextDeserializationSchema.java b/...format-text/src/main/java/org/apache/seatunnel/format/text/TextDeserializationSchema.java
@@ -187,9 +187,13 @@ private Object convert(String field, SeaTunnelDataType<?> fieldType, int level)
                 String[] kvs = field.split(separators[level + 1]);
                 for (String kv : kvs) {
                     String[] splits = kv.split(separators[level + 2]);
-                    objectMap.put(
-                            convert(splits[0], keyType, level + 1),
-                            convert(splits[1], valueType, level + 1));
+                    if (splits.length < 2) {
+                        objectMap.put(convert(splits[0], keyType, level + 1), null);
+                    } else {
+                        objectMap.put(
+                                convert(splits[0], keyType, level + 1),
+                                convert(splits[1], valueType, level + 1));
+                    }
                 }
                 return objectMap;
             case STRING:

diff --git a/...nnel-format-text/src/test/java/org/apache/seatunnel/format/text/TextFormatSchemaTest.java b/...nnel-format-text/src/test/java/org/apache/seatunnel/format/text/TextFormatSchemaTest.java
@@ -45,7 +45,14 @@ public class TextFormatSchemaTest {
                     + '\002'
                     + "Kris"
                     + '\003'
-                    + "21\001"
+                    + "21"
+                    + '\002'
+                    + "nullValueKey"
+                    + '\003'
+                    + '\002'
+                    + '\003'
+                    + "1231"
+                    + "\001"
                     + "tyrantlucifer\001"
                     + "true\001"
                     + "1\001"