Problems Generating Thumbnails - ffmpeg exited with code 244

[latetedemelon]

The bug

Similar to the below issue, some of my thumbnails aren't being generated and I'm not clear as to why.

Issue: #13915
Fix: #14134
Commit: https://github.com/immich-app/immich/tree/v1.122.0

The OS that Immich Server is running on

Debian 12

Version of Immich Server

v1.222.3

Version of Immich Mobile App

v1.222.3

Platform with the issue

• Server
• Web
• Mobile

Your docker-compose.yml content

#
# WARNING: Make sure to use the docker-compose.yml of the current release:
#
# https://github.com/immich-app/immich/releases/latest/download/docker-compose.yml
#
# The compose file on main may not be compatible with the latest release.
#

name: immich

services:
  immich-server:
    container_name: immich_server
    image: ghcr.io/immich-app/immich-server:${IMMICH_VERSION:-release}
    # extends:
    #   file: hwaccel.transcoding.yml
    #   service: cpu # set to one of [nvenc, quicksync, rkmpp, vaapi, vaapi-wsl] for accelerated transcoding
    volumes:
      # Do not edit the next line. If you want to change the media storage location on your system, edit the value of UPLOAD_LOCATION in the .env file
      - ${UPLOAD_LOCATION}:/usr/src/app/upload
      - /etc/localtime:/etc/localtime:ro
    env_file:
      - .env
    ports:
      - '2283:2283'
    depends_on:
      - redis
      - database
    restart: always
    healthcheck:
      disable: false

  immich-machine-learning:
    container_name: immich_machine_learning
    # For hardware acceleration, add one of -[armnn, cuda, openvino] to the image tag.
    # Example tag: ${IMMICH_VERSION:-release}-cuda
    image: ghcr.io/immich-app/immich-machine-learning:${IMMICH_VERSION:-release}
    # extends: # uncomment this section for hardware acceleration - see https://immich.app/docs/features/ml-hardware-acceleration
    #   file: hwaccel.ml.yml
    #   service: cpu # set to one of [armnn, cuda, openvino, openvino-wsl] for accelerated inference - use the `-wsl` version for WSL2 where applicable
    volumes:
      - ./model-cache:/cache
    env_file:
      - .env
    restart: always
    healthcheck:
      disable: false

  redis:
    container_name: immich_redis
    image: docker.io/redis:6.2-alpine@sha256:eaba718fecd1196d88533de7ba49bf903ad33664a92debb24660a922ecd9cac8
    healthcheck:
      test: redis-cli ping || exit 1
    restart: always

  database:
    container_name: immich_postgres
    image: docker.io/tensorchord/pgvecto-rs:pg14-v0.2.0@sha256:90724186f0a3517cf6914295b5ab410db9ce23190a2d9d0b9dd6463e3fa298f0
    environment:
      POSTGRES_PASSWORD: ${DB_PASSWORD}
      POSTGRES_USER: ${DB_USERNAME}
      POSTGRES_DB: ${DB_DATABASE_NAME}
      POSTGRES_INITDB_ARGS: '--data-checksums'
    volumes:
      # Do not edit the next line. If you want to change the database storage location on your system, edit the value of DB_DATA_LOCATION in the .env file
      - ${DB_DATA_LOCATION}:/var/lib/postgresql/data
    ports:
      - '11223:5432'
    healthcheck:
      test: >-
        pg_isready --dbname="$${POSTGRES_DB}" --username="$${POSTGRES_USER}" || exit 1;
        Chksum="$$(psql --dbname="$${POSTGRES_DB}" --username="$${POSTGRES_USER}" --tuples-only --no-align
        --command='SELECT COALESCE(SUM(checksum_failures), 0) FROM pg_stat_database')";
        echo "checksum failure count is $$Chksum";
        [ "$$Chksum" = '0' ] || exit 1
      interval: 5m
      start_interval: 30s
      start_period: 5m
    command: >-
      postgres
      -c shared_preload_libraries=vectors.so
      -c 'search_path="$$user", public, vectors'
      -c logging_collector=on
      -c max_wal_size=2GB
      -c shared_buffers=512MB
      -c wal_compression=on
    restart: always

Your .env content

# Immich Version
# IMMICH_VERSION=release

# Paths for uploads and database storage
UPLOAD_LOCATION={redacted}/immich/media
DB_DATA_LOCATION=./postgres

# General Settings
TZ={redacted}
# IMMICH_ENV=production
# IMMICH_LOG_LEVEL=log
# IMMICH_MEDIA_LOCATION=./upload
# IMMICH_CONFIG_FILE=
# NO_COLOR=false
# CPU_CORES=
# IMMICH_API_METRICS_PORT=8081
# IMMICH_MICROSERVICES_METRICS_PORT=8082
# IMMICH_PROCESS_INVALID_IMAGES=
# IMMICH_TRUSTED_PROXIES=
# IMMICH_IGNORE_MOUNT_CHECK_ERRORS=

# Workers Configuration
# IMMICH_WORKERS_INCLUDE=
# IMMICH_WORKERS_EXCLUDE=

# Network Configuration
# IMMICH_HOST=0.0.0.0
# IMMICH_PORT=2283

# Database Configuration
# DB_URL=
DB_HOSTNAME=immich_postgres
DB_PORT=5432
DB_PORTE=11223
DB_USERNAME=postgres
DB_PASSWORD={redacted}
DB_DATABASE_NAME=immich
DB_NAME=immich
# DB_VECTOR_EXTENSION=pgvecto.rs
# DB_SKIP_MIGRATIONS=false

# Redis Configuration
# REDIS_URL=
# REDIS_SOCKET=
REDIS_HOSTNAME=immich_redis
# REDIS_PORT=6379
# REDIS_USERNAME=
# REDIS_PASSWORD=
# REDIS_DBINDEX=0

# Machine Learning Configuration
# MACHINE_LEARNING_MODEL_TTL=300
# MACHINE_LEARNING_MODEL_TTL_POLL_S=10
# MACHINE_LEARNING_CACHE_FOLDER=/cache
# MACHINE_LEARNING_REQUEST_THREADS=
# MACHINE_LEARNING_MODEL_INTER_OP_THREADS=1
# MACHINE_LEARNING_MODEL_INTRA_OP_THREADS=2
# MACHINE_LEARNING_WORKERS=1
# MACHINE_LEARNING_HTTP_KEEPALIVE_TIMEOUT_S=2
# MACHINE_LEARNING_WORKER_TIMEOUT=120
# MACHINE_LEARNING_PRELOAD__CLIP=

Reproduction steps

No special steps, just some of the thumbnails are not generated, even when triggering the process manually.

Relevant log output

immich_server            | [Nest] 7  - 12/16/2024, 1:01:10 PM   ERROR [Microservices:MediaRepository] ffmpeg version 7.0.2-Jellyfin Copyright (c) 2000-2024 the FFmpeg developers
immich_server            |   built with gcc 12 (Debian 12.2.0-14)
immich_server            |   configuration: --prefix=/usr/lib/jellyfin-ffmpeg --target-os=linux --extra-version=Jellyfin --disable-doc --disable-ffplay --disable-ptx-compression --disable-static --disable-libxcb --disable-sdl2 --disable-xlib --enable-lto=auto --enable-gpl --enable-version3 --enable-shared --enable-gmp --enable-gnutls --enable-chromaprint --enable-opencl --enable-libdrm --enable-libxml2 --enable-libass --enable-libfreetype --enable-libfribidi --enable-libfontconfig --enable-libharfbuzz --enable-libbluray --enable-libmp3lame --enable-libopus --enable-libtheora --enable-libvorbis --enable-libopenmpt --enable-libdav1d --enable-libsvtav1 --enable-libwebp --enable-libvpx --enable-libx264 --enable-libx265 --enable-libzvbi --enable-libzimg --enable-libfdk-aac --arch=amd64 --enable-libshaderc --enable-libplacebo --enable-vulkan --enable-vaapi --enable-amf --enable-libvpl --enable-ffnvcodec --enable-cuda --enable-cuda-llvm --enable-cuvid --enable-nvdec --enable-nvenc
immich_server            |   libavutil      59.  8.100 / 59.  8.100
immich_server            |   libavcodec     61.  3.100 / 61.  3.100
immich_server            |   libavformat    61.  1.100 / 61.  1.100
immich_server            |   libavdevice    61.  1.100 / 61.  1.100
immich_server            |   libavfilter    10.  1.100 / 10.  1.100
immich_server            |   libswscale      8.  1.100 /  8.  1.100
immich_server            |   libswresample   5.  1.100 /  5.  1.100
immich_server            |   libpostproc    58.  1.100 / 58.  1.100
immich_server            | [h264 @ 0x45c001f0580] Reinit context to 1088x1920, pix_fmt: yuv420p
immich_server            | Input #0, mov,mp4,m4a,3gp,3g2,mj2, from 'upload/upload/30645676-e275-438b-a633-4cdfe3dce10c/87/ae/87aef6cc-0d97-4d5c-a72a-fa52067284e8.MOV':
immich_server            |   Metadata:
immich_server            |     major_brand     : mp42
immich_server            |     minor_version   : 0
immich_server            |     compatible_brands: isommp42
immich_server            |     creation_time   : 1970-01-01T00:00:00.000000Z
immich_server            |   Duration: 00:00:06.06, start: 0.000000, bitrate: 5088 kb/s
--
immich_server            | [in#0/mov,mp4,m4a,3gp,3g2,mj2 @ 0x45c00120280] Terminating thread with return code 0 (success)
immich_server            | [vist#0:0/h264 @ 0x45c00240780] [dec:h264 @ 0x45c00240f00] Decoder thread received EOF packet
immich_server            | [vist#0:0/h264 @ 0x45c00240780] [dec:h264 @ 0x45c00240f00] Decoder returned EOF, finishing
immich_server            | [Parsed_thumbnail_1 @ 0x45c040506c0] batch size: 12 frames
immich_server            | [Parsed_scale_5 @ 0x45c040509c0] w:250 h:-2 flags:'lanczos+accurate_rnd+full_chroma_int' interl:0
immich_server            | [graph 0 input from stream 0:0 @ 0x45c04050a80] w:1080 h:1920 pixfmt:yuv420p tb:1/30000 fr:30000/1001 sar:1/1 csp:bt709 range:tv
immich_server            | [Parsed_fps_0 @ 0x45c04050540] fps=12/1
immich_server            | [Parsed_scale_5 @ 0x45c040509c0] w:1080 h:1920 fmt:yuv420p csp:bt709 range:tv sar:1/1 -> w:250 h:444 fmt:yuv420p csp:bt709 range:pc sar:999/1000 flags:0x00042200
immich_server            | [graph 0 input from stream 0:0 @ 0x45c04050a80] video frame properties congruent with link at pts_time: 5.672333
immich_server            | [Parsed_fps_0 @ 0x45c04050540] Set first pts to 68
immich_server            | [vist#0:0/h264 @ 0x45c00240780] [dec:h264 @ 0x45c00240f00] Terminating thread with return code 0 (success)
immich_server            | [vf#0:0 @ 0x45c00148880] Filtergraph returned EOF, finishing
immich_server            | [vf#0:0 @ 0x45c00148880] All consumers returned EOF
immich_server            | [vost#0:0/libwebp_anim @ 0x45c00171c80] No filtered frames for output stream, trying to initialize anyway.
immich_server            | Output #0, webp, to 'upload/thumbs/30645676-e275-438b-a633-4cdfe3dce10c/e0/08/e008affb-3cc4-4c50-9407-caf50d53ffac-thumbnail.webp':
immich_server            |   Metadata:
immich_server            |     major_brand     : mp42
immich_server            |     minor_version   : 0
immich_server            |     compatible_brands: isommp42
immich_server            |     encoder         : Lavf61.1.100
immich_server            |   Stream #0:0(und): Video: webp, 1 reference frame, yuv420p(progressive), 250x444 [SAR 999:1000 DAR 9:16], q=2-31, 200 kb/s, 12 fps, 1k tbn (default)
immich_server            |       Metadata:
immich_server            |         creation_time   : 1970-01-01T00:00:00.000000Z
immich_server            |         handler_name    : ISO Media file produced by Google Inc.
immich_server            |         vendor_id       : [0][0][0][0]
immich_server            |         encoder         : Lavc61.3.100 libwebp_anim
immich_server            | [out#0/webp @ 0x45c00162580] Starting thread...
immich_server            | [Parsed_fps_0 @ 0x45c04050540] 2 frames in, 0 frames out; 2 frames dropped, 0 frames duplicated.
immich_server            | [vost#0:0/libwebp_anim @ 0x45c00171c80] Encoder thread received EOF
immich_server            | [libwebp encoder @ 0x45c00060e00] WebPAnimEncoderAssemble() failed with error: 1
immich_server            | [vost#0:0/libwebp_anim @ 0x45c00171c80] Error submitting video frame to the encoder
immich_server            | [vost#0:0/libwebp_anim @ 0x45c00171c80] Error flushing encoder: Cannot allocate memory
immich_server            | [vost#0:0/libwebp_anim @ 0x45c00171c80] Task finished with error code: -12 (Cannot allocate memory)
immich_server            | [vost#0:0/libwebp_anim @ 0x45c00171c80] Terminating thread with return code -12 (Cannot allocate memory)
immich_server            | [out#0/webp @ 0x45c00162580] All streams finished
immich_server            | [out#0/webp @ 0x45c00162580] Terminating thread with return code 0 (success)
immich_server            | [vf#0:0 @ 0x45c00148880] Terminating thread with return code 0 (success)
immich_server            | [AVIOContext @ 0x45c00061e40] Statistics: 0 bytes written, 0 seeks, 0 writeouts
immich_server            | [out#0/webp @ 0x45c00162580] Output file #0 (upload/thumbs/30645676-e275-438b-a633-4cdfe3dce10c/e0/08/e008affb-3cc4-4c50-9407-caf50d53ffac-thumbnail.webp):
immich_server            | [out#0/webp @ 0x45c00162580]   Output stream #0:0 (video): 0 frames encoded; 0 packets muxed (0 bytes); 
immich_server            | [out#0/webp @ 0x45c00162580]   Total: 0 packets (0 bytes) muxed
immich_server            | [out#0/webp @ 0x45c00162580] video:0KiB audio:0KiB subtitle:0KiB other streams:0KiB global headers:0KiB muxing overhead: unknown
immich_server            | [out#0/webp @ 0x45c00162580] Output file is empty, nothing was encoded(check -ss / -t / -frames parameters if used)
immich_server            | frame=    0 fps=0.0 q=0.0 Lsize=       0KiB time=N/A bitrate=N/A speed=N/A    
immich_server            | [in#0/mov,mp4,m4a,3gp,3g2,mj2 @ 0x45c00120280] Input file #0 (upload/upload/30645676-e275-438b-a633-4cdfe3dce10c/87/ae/87aef6cc-0d97-4d5c-a72a-fa52067284e8.MOV):
immich_server            | [in#0/mov,mp4,m4a,3gp,3g2,mj2 @ 0x45c00120280]   Input stream #0:0 (video): 179 packets read (3753315 bytes); 2 frames decoded; 0 decode errors; 
immich_server            | [in#0/mov,mp4,m4a,3gp,3g2,mj2 @ 0x45c00120280]   Total: 179 packets (3753315 bytes) demuxed
immich_server            | [AVIOContext @ 0x45c00060680] Statistics: 3854454 bytes read, 0 seeks
immich_server            | Conversion failed!
immich_server            | 
immich_server            | [Nest] 7  - 12/16/2024, 1:01:10 PM   ERROR [Microservices:JobService] Unable to run job handler (thumbnailGeneration/generate-thumbnails): Error: ffmpeg exited with code 244: Conversion failed!
immich_server            | 
immich_server            | [Nest] 7  - 12/16/2024, 1:01:10 PM   ERROR [Microservices:JobService] Error: ffmpeg exited with code 244: Conversion failed!

Additional information

I'm running on Proxmox, for this LXC I'm using 4 GB RAM and have 96 GB total available.

[latetedemelon]

Thanks for taking a look!

[danieldietzler]

That file seems to be broken. Can you try opening it and see if it works locally?

[JuanToronDoe]

I have exactly the same issue and error logs

[mertalev]

Error flushing encoder: Cannot allocate memory

Maybe the server ran out of memory?

[latetedemelon]

Sorry, I got distracted over the holidays. I’ll add some logs but this may be related to images “created” by Google Photos. I’ll add some logs / images shortly. * Your server ran out of memory Definitely not the case, maybe some sort of internal memory configuration/ limitation?

…

________________________________ From: Mert ***@***.***> Sent: Tuesday, January 7, 2025 9:15:13 AM To: immich-app/immich ***@***.***> Cc: Rob ***@***.***>; Author ***@***.***> Subject: Re: [immich-app/immich] Problems Generating Thumbnails - ffmpeg exited with code 244 (Discussion #15133) Error flushing encoder: Cannot allocate memory Your server ran out of memory — Reply to this email directly, view it on GitHub<#15133 (comment)>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/ACDYM2IUTR6SP3PXTHCBJT32JQDSDAVCNFSM6AAAAABUYEVB4WVHI2DSMVQWIX3LMV43URDJONRXK43TNFXW4Q3PNVWWK3TUHMYTCNZWGM2TSMA>. You are receiving this because you authored the thread.Message ID: ***@***.***>

[bo0tzz]

Definitely not the case

What makes you say this? Earlier you mentioned having 4GB RAM allocated - that's pretty bare minimum for an Immich installation and it's not unlikely to run up against that limit.

[latetedemelon]

Definitely not the case

What makes you say this? Earlier you mentioned having 4GB RAM allocated - that's pretty bare minimum for an Immich installation and it's not unlikely to run up against that limit.

I'm using 4 GB, I have 96 GB available to use.

[bo0tzz]

for this LXC I'm using 4 GB RAM and have 96 GB total available.

This suggests to me that the LXC (and thus Immich) only gets to use 4GB

[latetedemelon]

No, that’s not the case. The 4GB mentioned was simply stating I was using 4GB total of the 96 available.

…

On Tue, Jan 7, 2025 at 11:18 bo0tzz ***@***.***> wrote: for this LXC I'm using 4 GB RAM and have 96 GB total available. This suggests to me that the LXC (and thus Immich) only gets to use 4GB — Reply to this email directly, view it on GitHub <#15133 (reply in thread)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ACDYM2POW64HLFQUAYSIZLD2JQSBNAVCNFSM6AAAAABUYEVB4WVHI2DSMVQWIX3LMV43URDJONRXK43TNFXW4Q3PNVWWK3TUHMYTCNZWGQ4DMMQ> . You are receiving this because you authored the thread.Message ID: ***@***.***>

[hemirunner426]

I see the same behavior. My system has 16GB of RAM. Containers are set for unlimited resource usage.
The video plays fine in Windows Media Player directly from the library.

The system is currently in the middle of processing a 70k library through the machine learning container. I certainly see no memory exhausting issues right now. 8GB are free according to the system resource monitor during this process. 3GB are in use by the machine learning container, 1GB in use on the server container.