Skip to content
This repository has been archived by the owner on Mar 8, 2020. It is now read-only.

Cannot handle large files #42

Open
dennwc opened this issue Mar 28, 2019 · 0 comments
Open

Cannot handle large files #42

dennwc opened this issue Mar 28, 2019 · 0 comments
Labels
bug
Milestone
v2.7.x

Comments

@dennwc
Copy link
Member

dennwc commented Mar 28, 2019

The driver currently tries to read the whole input file into memory and then writes the result into memory as well. This causes OOM kill for large files.

We should rewrite the driver to stream JSON output instead.
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Assignees
No one assigned
Labels
bug
Projects
None yet
Milestone
v2.7.x
Development

No branches or pull requests
1 participant
@dennwc