When will fixes for ffmpeg invalid display handle (-17) be pushed to Intel drivers?

[TCROC]

Ffmpeg is still erroring with "invalid display handle(-17) when trying to do hardware encoding: intel/cartwheel-ffmpeg#233

While using intel's latest drivers from here:

https://repositories.intel.com/graphics/ubuntu jammy arc

Installation instructions:

https://dgpu-docs.intel.com/installation-guides/ubuntu/ubuntu-jammy-arc.html

I'm wondering when I can expect the patched drivers to be pushed out? The issue I linked above is closed claiming to have fixed it.

[TCROC]

These are the current drivers when I search with apt

apt search intel-media
Sorting... Done
Full Text Search... Done
intel-media-va-driver/jammy-updates 22.3.1+dfsg1-1ubuntu1 amd64
  VAAPI driver for the Intel GEN8+ Graphics family

intel-media-va-driver-non-free/jammy,now 23.1.4-601~22.04 amd64 [installed]
  VAAPI driver for the Intel GEN8+ Graphics family

[TCROC]

And these are the errors I get when running ffmpeg:

./ffmpeg -start_number 31 -framerate 30 -i imgs/frame%d.jpg -c:v h264_qsv -s 1920x1080 clip_0.mp4
ffmpeg version n6.0-17-g8f61cbf1b9-20230419 Copyright (c) 2000-2023 the FFmpeg developers
  built with gcc 12.2.0 (crosstool-NG 1.25.0.152_89671bf)
  configuration: --prefix=/ffbuild/prefix --pkg-config-flags=--static --pkg-config=pkg-config --cross-prefix=x86_64-ffbuild-linux-gnu- --arch=x86_64 --target-os=linux --enable-version3 --disable-debug --enable-iconv --enable-libxml2 --enable-zlib --enable-libfreetype --enable-libfribidi --enable-gmp --enable-lzma --enable-fontconfig --enable-libvorbis --enable-opencl --enable-libpulse --enable-libvmaf --enable-libxcb --enable-xlib --enable-amf --enable-libaom --enable-libaribb24 --disable-avisynth --enable-chromaprint --enable-libdav1d --disable-libdavs2 --disable-libfdk-aac --enable-ffnvcodec --enable-cuda-llvm --disable-frei0r --enable-libgme --enable-libkvazaar --enable-libass --enable-libbluray --enable-libjxl --enable-libmp3lame --enable-libopus --enable-mbedtls --enable-librist --enable-libssh --enable-libtheora --enable-libvpx --enable-libwebp --enable-lv2 --disable-libmfx --enable-libvpl --enable-openal --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-libopenh264 --enable-libopenjpeg --enable-libopenmpt --enable-librav1e --disable-librubberband --disable-schannel --enable-sdl2 --enable-libsoxr --enable-libsrt --enable-libsvtav1 --enable-libtwolame --enable-libuavs3d --enable-libdrm --enable-vaapi --disable-libvidstab --enable-vulkan --enable-libshaderc --disable-libplacebo --disable-libx264 --disable-libx265 --disable-libxavs2 --disable-libxvid --enable-libzimg --enable-libzvbi --extra-cflags=-DLIBTWOLAME_STATIC --extra-cxxflags= --extra-ldflags=-pthread --extra-ldexeflags=-pie --extra-libs='-ldl -lgomp' --extra-version=20230419
  libavutil      58.  2.100 / 58.  2.100
  libavcodec     60.  3.100 / 60.  3.100
  libavformat    60.  3.100 / 60.  3.100
  libavdevice    60.  1.100 / 60.  1.100
  libavfilter     9.  3.100 /  9.  3.100
  libswscale      7.  1.100 /  7.  1.100
  libswresample   4. 10.100 /  4. 10.100
Input #0, image2, from 'imgs/frame%d.jpg':
  Duration: 00:00:07.50, start: 0.000000, bitrate: N/A
  Stream #0:0: Video: mjpeg (Baseline), yuvj420p(pc, bt470bg/unknown/unknown), 1090x400 [SAR 1:1 DAR 109:40], 30 fps, 30 tbr, 30 tbn
File 'clip_0.mp4' already exists. Overwrite? [y/N] y
Stream mapping:
  Stream #0:0 -> #0:0 (mjpeg (native) -> h264 (h264_qsv))
Press [q] to stop, [?] for help
[swscaler @ 0x5562427484c0] deprecated pixel format used, make sure you did set range correctly
    Last message repeated 3 times
[h264_qsv @ 0x5562424219c0] Error during set display handle
: device failed (-17)
[vost#0:0/h264_qsv @ 0x5562424216c0] Error initializing output stream: Error while opening encoder for output stream #0:0 - maybe incorrect parameters such as bit_rate, rate, width or height
Conversion failed!

[TCROC]

Ok so I simply missed a step. I needed to install the runtime dependencies:

https://dgpu-docs.intel.com/installation-guides/ubuntu/ubuntu-jammy-arc.html#step-4-install-run-time-packages

Are these going to be merged into the kernel so it just works out of the box?

[Sherry-Lin]

Media stack depends on Kernel, so step1&2 are necessary. If you intent to run BRC encode cases, step3 is also needed.

1. step-1-add-package-repository
2. step-2-install-linux-kernel
3. step-3-install-intel-gpu-fws

After that, you could follow Install-from-Agama#install-run-time-packages to install necessary driver and runtime packages, then use vainfo to validate the environment.

[TCROC]

Media stack depends on Kernel, so step1&2 are necessary. If you intent to run BRC encode cases, step3 is also needed.

1. [step-1-add-package-repository](https://dgpu-docs.intel.com/installation-guides/ubuntu/ubuntu-jammy-arc.html#step-1-add-package-repository)

2. [step-2-install-linux-kernel](https://dgpu-docs.intel.com/installation-guides/ubuntu/ubuntu-jammy-arc.html#step-2-install-linux-kernel)

3. [step-3-install-intel-gpu-fws](https://dgpu-docs.intel.com/installation-guides/ubuntu/ubuntu-jammy-arc.html#step-3-install-intel-gpu-fws)

After that, you could follow Install-from-Agama#install-run-time-packages to install necessary driver and runtime packages, then use vainfo to validate the environment.

Step 1, 2, and 3 were already completed for us. We are running Pop OS (an Ubuntu derivative). It was step 4 that we were missing.

Was wondering if there is any plan to make it more seamless? I can run NVIDIA and AMD GPUS out of the box and they work with encoding / decoding. Is there any plan to make Intel run out of the box as well?

[TCROC]

this is the kernel it comes prepackaged with:

uname -r
6.2.6-76060206-generic

[nyanmisaka]

You will never get out of the box support on Ubuntu 22.04 LTS (jammy) since it was released before the Arc graphics.
The Ubuntu 23.04 (Linux 6.2 and updated iHD/oneVPL) released yesterday should support Arc out of the box.

[TCROC]

You will never get out of the box support on Ubuntu 22.04 LTS (jammy) since it was released before the Arc graphics. The Ubuntu 23.04 (Linux 6.2 and updated iHD/oneVPL) released yesterday should support Arc out of the box.

The Ubuntu 23.04 (Linux 6.2 and updated iHD/oneVPL) released yesterday should support Arc out of the box.

Pop OS updates the kernel. They must not have updated the iHD/oneVPL. I'll open an issue with them and link this issue. I suspect its probably on their end if 23.04 does indeed work out of the box.

Then I'll close this issue (or leave it open) once we confirm where the actual issue resides.

[Sherry-Lin]

Are you looking for pre-build packages? Probably you could try w/ the MediaStack.tar.gz from https://github.com/oneapi-src/oneVPL-intel-gpu/releases for quarterly release. Please let us know if any issues found.

[TCROC]

Are you looking for pre-build packages? Probably you could try w/ the MediaStack.tar.gz from https://github.com/oneapi-src/oneVPL-intel-gpu/releases for quarterly release. Please let us know if any issues found.

I installed prebuilt packages from here: https://dgpu-docs.intel.com/installation-guides/ubuntu/ubuntu-jammy-arc.html#step-4-install-run-time-packages

What I'm actually looking for is not just prebuilt packages, but seamless out of the box functionality. My experience with AMD and NVIDIA on Pop OS and Ubuntu has been pretty seamless.

AMD worked out of the box. I didn't need to install anything.

NVIDIA, I simply had to check the box for proprietary drivers at installation. Easy and painless.

Intel Arc required updating ppa, changing from open to closed drivers, installing a lot of runtime packages and dependencies. Definitely felt like more of a hassle compared to AMD and NVIDIA.

For context, my studio develops Blocky Ball: https://store.steampowered.com/app/1343040/Blocky_Ball/

We want to make sure we support the big 3 graphics cards on Windows and Linux. AMD and NVIDIA were easy. Intel Arc took us a few days to get up and running. We almost gave up on it to pursue other pressing development but fortunately we got it working! :)

I'm very much enjoying the arc card now that it's working! I'm currently using it as my daily driver. I think out of box functionality is needed to increase adoption.

[nyanmisaka]

So it seems intel needs a meta package like intel-media-all to install all media components by one command,
e.g. sudo apt-get install intel-media-all

Instead of installing each of them, because they scattered everywhere, which makes it not that user friendly.
Otherwise you have to wildcard package names,
e.g. sudo apt-get install libmfx* *vpl* intel-media-va-driver-non-free

[TCROC]

This would make things A LOT easier! :)

[nyanmisaka]

Btw i've just tried Ubuntu 23.04, as said before, Arc works out of the box in terms of both media(iHD/VPL) and graphics(Mesa).

But it still requires you to run sudo apt-get install libmfx* *vpl* intel-media-va-driver-non-free before using VA-API and QSV.

[TCROC]

Ok so this helps. And brings another question to mind:

What needs to be done to make sure these packages are preinstalled on this distributions? It would be nice for vaapi and qsv to also work out of the box.

Maybe I just got lucky and I installed the dependencies for AMD not realizing it, but AMD vaapi worked out of box for me.

[nyanmisaka]

Kernel 6.2+ (Ubuntu 23.04)

Mesa(OpenGL & Vulkan) drivers are preinstalled on nearly all Linux desktop distros.

AMD only requires va-driver-all, its a meta package.

The intel-media-va-driver is a part of the the va-driver-all but it lacks of some media features comparing to its non-free variant.

libmfx* *vpl* are required by the QSV so you have to install them manually.

[TCROC]

If this is the case, it sounds like an intel-media-all would be a very good solution! :)

I wonder if va-driver-all comes preinstalled on Ubuntu? It seems like you would just want all of these critical hardware dependencies preinstalled.

[Sherry-Lin]

Thank you for your suggestions. In Agama package repository, we only provide intel-media-va-driver-non-free so probably we could add it to va-driver-all considering no free-kernel media-driver provided. Let me check w/ team for next plan.

[TCROC]

Sounds great! What is the difference between intel-media-va-driver and intel-media-va-driver-non-free?

[nyanmisaka]

Refer to Full vs Free Feature Build https://github.com/intel/media-driver#components-and-features

[TCROC]

Thanks! Definitely a good idea to include the non-free variant in va-driver-all.

[Sherry-Lin]

However libmfx* vpl* are GPU Runtime libraries so they would not be included in va-driver-all or you folks are looking for one big meta package to include libmfx* vpl* media-driver libva together?

[TCROC]

or you folks are looking for one big meta package to include libmfx* vpl* media-driver libva together?

That is exactly what I'm looking for :).

[TCROC]

Basically I'm looking for one big meta package that includes everything my gpu needs to operate.

[Feierkuula]

ffmpeg invalid display handle (-17) is caused by missing MediaSDK runtimes.

You need to install these as additional too: https://github.com/Intel-Media-SDK/MediaSDK

ffmpeg with oneVPL does still depend on it.

[Feierkuula]

This error message is caused by missing MediaSDK lib/libmfx* libs and in some systems such as Ubuntu or Debian, it must be under /usr/lib instead of /opt/intel also /usr/local/lib was not possible.

FFmpeg with oneVPL does still depend on extra MediaSDK runtimes.

if both MediaSDK and oneVPL-intel-gpu libs is installed under /usr/lib , the message will be gone.

I hope it is clear now.