You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
JSONL documents, ala http://jsonlines.org, are streams of valid JSON documents, separated by newlines ("\n").
I know it's pretty easy to parse these in Elixir itself (e.g. StringIO.open(jsonl) |> elem(1) |> IO.binstream(:line) |> Stream.map(&Jason.decode!/1)), but a convenience method would be nice.
As well, there may be an appreciable amount of redundant work done by the above approach, compared to having Jason manage document-splitting while parsing. Embedding the logic for handling top-level newline tokens in the document may be quite a lot faster. Something to benchmark!
The text was updated successfully, but these errors were encountered:
In general, I would say this is a subset of issue #25 - in here it's easy to do this manually, though, because the separator is a newline and newlines are not allowed inside JSON directly. I'll close this in favour of #25.
JSONL
documents, ala http://jsonlines.org, are streams of valid JSON documents, separated by newlines ("\n"
).I know it's pretty easy to parse these in Elixir itself (e.g.
StringIO.open(jsonl) |> elem(1) |> IO.binstream(:line) |> Stream.map(&Jason.decode!/1)
), but a convenience method would be nice.As well, there may be an appreciable amount of redundant work done by the above approach, compared to having Jason manage document-splitting while parsing. Embedding the logic for handling top-level newline tokens in the document may be quite a lot faster. Something to benchmark!
The text was updated successfully, but these errors were encountered: