Fix some issues detected by fuzzing #24
Merged
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
I still want to do more cleanup before merging, but this has fixes for three issues detected by fuzzing. Two were places where the decoder didn't properly reject invalid bitstreams. The final issue is that the color indexing transform was implemented wrong in cases where 16 or fewer colors were in use, causing us to decode incorrectly and possibly index out of bounds. I believe that could result in valid images being decoded garbled.
With this patch applied, fuzzing no longer immediately crashes for me when running with the seed corpus from https://lcamtuf.coredump.cx/afl/demo. However, I haven't had a chance to fuzz for an extended length of time. And cargo fuzz does occasionally print messages about specific inputs taking nearly a minute to run
Edit: I'll handle the color indexing transform issue is a separate PR, hopefully with a regression test. For now, this PR just includes three smaller bitstream issues.