Added Schema less option #25
Conversation
…ort, awsValidateCertificate optional parameters to config
|
is this project supported? im wondering what it would take to get this merged in. |
There was a problem hiding this comment.
please specify which issue this supposed to fix.
There are also changes that aren't related to schema-less data. those should be removed and if needed, they should be added in separate PR.
In general, if I understand correctly, you are attempting to resolve JSON conversion issue when schema is not defined along with the JsonNode data.
I disagree with the approach of assuming that any HashMap might be converted to a Json string.
if you would like to convert it to Json string, I'd suggest to implement new Converter, instead of using JsonConverter,
| } | ||
| if(value instanceof HashMap) { | ||
| try { | ||
| String json = new ObjectMapper().writeValueAsString(value); |
There was a problem hiding this comment.
would any HashMap object be converted to JSON string?
|
We produce JSON data to be consumed by Kafka and then send to AWS Kinesis. Our Kafka clients are running on edge computers on the shopfloor and are not easily accessible. Furthermore the producing clients are all sort of devices (pycoms,..). That is why we needed a way to send any kind of JSON data, without updating the schema, via this connector to AWS. Concerning the HashMap, of course not every HashMap is JSON serializable, but in this case the HashMap was created from JSON data by Kafka, so it would be safe to assume it is again serializable. My branch has diverged some more in the last years, since I assumed it will never be merged, and is now quite specific to our usecase. But I can clean it up and prepare a new PR if people are still interested. |
Allows to push Kafka JSON records without schema into Kinesis