WEBP: Reading animated image shows what seem to be compression artifacts

[isXander]

Describe the bug
Reading animated webp frame-by-frame causes extreme image artifacts on some images.

Version information
TwelveMonkeys ImageIO 3.9.4
Java openjdk 17.0.1+12 (Temurin 64-bit)
OS Windows 11

To Reproduce
Steps to reproduce the behavior:

1. Compile the below sample code
2. Download the sample image file
3. Run the code with the sample file
4. See error

Expected behavior
A clear and concise description of what you expected to happen.

Example code

ImageReader reader = new WebPImageReaderSpi().createReaderInstance();
reader.setInput(ImageIO.createImageInputStream(is));
int frameCount = reader.getNumImages(true); 
for (int i = reader.getMinIndex(); i < frameCount - 1; i++) {
    BufferedImage bi = reader.read(i);
    ImageIO.write(bi, "png", new File("./" + i + ".png"));

Sample file(s)
Input WebP file
reach-around-placement.zip

Exported frames (yes, first two frames read as empty)

Additional context

$ webpinfo reach-around-placement.webp
File: reach-around-placement.webp
RIFF HEADER:
  File size:  14840
Chunk VP8X at offset     12, length     18
  ICCP: 0
  Alpha: 1
  EXIF: 0
  XMP: 0
  Animation: 1
  Canvas size 320 x 180
Chunk ANIM at offset     30, length     14
  Background color:(ARGB) ff ff ff ff
  Loop count      : 0
Chunk ANMF at offset     44, length   5152
  Offset_X: 0
  Offset_Y: 0
  Width: 320
  Height: 180
  Duration: 100
  Dispose: 0
  Blend: 1
Chunk VP8  at offset     68, length   5128
  Width: 320
  Height: 180
  Alpha: 0
  Animation: 0
  Format: Lossy (1)
Chunk ANMF at offset   5196, length    132
  Offset_X: 144
  Offset_Y: 60
  Width: 10
  Height: 50
  Duration: 100
  Dispose: 0
  Blend: 0
Chunk VP8  at offset   5220, length    108
  Width: 10
  Height: 50
  Alpha: 0
  Animation: 0
  Format: Lossy (1)
Chunk ANMF at offset   5328, length   1538
  Offset_X: 76
  Offset_Y: 48
  Width: 244
  Height: 132
  Duration: 100
  Dispose: 0
  Blend: 0
Chunk ALPH at offset   5352, length     72
Chunk VP8  at offset   5424, length   1442
  Width: 244
  Height: 132
  Alpha: 0
  Animation: 0
  Format: Lossy (1)
Chunk ANMF at offset   6866, length    970
  Offset_X: 76
  Offset_Y: 90
  Width: 179
  Height: 90
  Duration: 100
  Dispose: 0
  Blend: 0
Chunk ALPH at offset   6890, length     60
Chunk VP8  at offset   6950, length    886
  Width: 179
  Height: 90
  Alpha: 0
  Animation: 0
  Format: Lossy (1)
Chunk ANMF at offset   7836, length    968
  Offset_X: 48
  Offset_Y: 90
  Width: 197
  Height: 90
  Duration: 100
  Dispose: 0
  Blend: 0
Chunk ALPH at offset   7860, length     54
Chunk VP8  at offset   7914, length    890
  Width: 197
  Height: 90
  Alpha: 0
  Animation: 0
  Format: Lossy (1)
Chunk ANMF at offset   8804, length   1346
  Offset_X: 76
  Offset_Y: 48
  Width: 243
  Height: 132
  Duration: 100
  Dispose: 0
  Blend: 0
Chunk ALPH at offset   8828, length     66
Chunk VP8  at offset   8894, length   1256
  Width: 243
  Height: 132
  Alpha: 0
  Animation: 0
  Format: Lossy (1)
Chunk ANMF at offset  10150, length   1446
  Offset_X: 30
  Offset_Y: 48
  Width: 290
  Height: 132
  Duration: 100
  Dispose: 0
  Blend: 0
Chunk ALPH at offset  10174, length     78
Chunk VP8  at offset  10252, length   1344
  Width: 290
  Height: 132
  Alpha: 0
  Animation: 0
  Format: Lossy (1)
Chunk ANMF at offset  11596, length    240
  Offset_X: 30
  Offset_Y: 90
  Width: 49
  Height: 40
  Duration: 100
  Dispose: 0
  Blend: 0
Chunk ALPH at offset  11620, length     40
Chunk VP8  at offset  11660, length    176
  Width: 49
  Height: 40
  Alpha: 0
  Animation: 0
  Format: Lossy (1)
Chunk ANMF at offset  11836, length    714
  Offset_X: 30
  Offset_Y: 48
  Width: 275
  Height: 82
  Duration: 100
  Dispose: 0
  Blend: 0
Chunk ALPH at offset  11860, length     52
Chunk VP8  at offset  11912, length    638
  Width: 275
  Height: 82
  Alpha: 0
  Animation: 0
  Format: Lossy (1)
Chunk ANMF at offset  12550, length    710
  Offset_X: 30
  Offset_Y: 48
  Width: 275
  Height: 82
  Duration: 100
  Dispose: 0
  Blend: 0
Chunk ALPH at offset  12574, length     50
Chunk VP8  at offset  12624, length    636
  Width: 275
  Height: 82
  Alpha: 0
  Animation: 0
  Format: Lossy (1)
Chunk ANMF at offset  13260, length    568
  Offset_X: 48
  Offset_Y: 48
  Width: 256
  Height: 83
  Duration: 100
  Dispose: 0
  Blend: 0
Chunk ALPH at offset  13284, length     42
Chunk VP8  at offset  13326, length    502
  Width: 256
  Height: 83
  Alpha: 0
  Animation: 0
  Format: Lossy (1)
Chunk ANMF at offset  13828, length    490
  Offset_X: 48
  Offset_Y: 88
  Width: 256
  Height: 43
  Duration: 100
  Dispose: 0
  Blend: 0
Chunk ALPH at offset  13852, length     38
Chunk VP8  at offset  13890, length    428
  Width: 256
  Height: 43
  Alpha: 0
  Animation: 0
  Format: Lossy (1)
Chunk ANMF at offset  14318, length    522
  Offset_X: 48
  Offset_Y: 88
  Width: 257
  Height: 43
  Duration: 100
  Dispose: 0
  Blend: 0
Chunk ALPH at offset  14342, length     40
Chunk VP8  at offset  14382, length    458
  Width: 257
  Height: 43
  Alpha: 0
  Animation: 0
  Format: Lossy (1)
No error detected.

[haraldk]

Hi,

Thanks for reporting!

It's a bit odd that the first frames are decoded all transparent, that is most likely a bug...

However, note that WebP animation is stored delta-encoded with x/y offsets. I don't necessarily see any artifacts here. The plugin only decodes the deltas, client code needs to compute the full frame based on previous frames.

Anyway, I'll look into it.

[isXander]

Hi,

Thanks for reporting!

It's a bit odd that the first frames are decoded all transparent, that is most likely a bug...

However, note that WebP animation is stored delta-encoded with x/y offsets. I don't necessarily see any artifacts here. The plugin only decodes the deltas, client code needs to compute the full frame based on previous frames.

Anyway, I'll look into it.

So I am responsible for stitching all the delta frames together?

[haraldk]

Yes, this is similar to how the GIF plugin works for animations.

[isXander]

Yes, this is similar to how the GIF plugin works for animations.

Is there a guide for how to get the x and y offset of each frame? I don't see a clear method for it.

EDIT: appears you don't expose it outside the animation frame... reflection time!

[haraldk]

Ah, snap, yes. That is a feature we should have had, the plan is to expose it through the native metadata. Unfortunately we don't have that yet. Feel free to provide a PR. 😀

[haraldk]

I think I understand the reason the first two frames are all transparent. The WebP header in your sample file is in extended VP8X format, and specifies lossy with alpha and animation. However, the first two frames does not contain an ALPH chunk, only the VP8 chunk. This leaves the alpha uninitialized as fully transparent for these two frames.

I need to consult the specification whether this is really allowed as a special case for animation, or if this is a malformed WebP. In any case, it seems reasonable to allow this, and it's relatively easy to fix.

[haraldk]

The latest snapshot now contains the fix (the build is fixed, but I made a type in the commit message, so it doesn't show here).

Using that and some reflection magic to access the frames field, you should be able to recreate the animation.

[haraldk]

Closing this for now, the WebP animation metadata issue is kept open in #711