Access Violation on Windows Build when encoding

[EraYaN]

When building this wheel on Windows using:

vcpkg install libjxl --triplet x64-windows

pip wheel --use-feature=in-tree-build ./jxlpy

delvewheel repair -v --add-path C:\tools\vcpkg\installed\x64-windows\bin --no-dll msvcp140.dll --no-mangle-all jxlpy-0.9.1-cp39-cp39-win_amd64.whl

pip uninstall jxlpy -y
pip install wheelhouse/jxlpy-0.9.1-cp39-cp39-win_amd64.whl

python decode_simple.py

python encode_decode_pillow.py

echo %errorlevel%

This gives an access violation on calling im.save, with the call stack

Exception thrown at 0x00007FFFDF941372 (jxl.dll) in python.exe: 0xC0000005: Access violation reading location 0x0000000000003C80.
[Inline Frame] jxl.dll!hwy::N_AVX2::LoadDup128(hwy::N_AVX2::Simd<float,8>) Line 1994
	at C:\tools\vcpkg\installed\x64-windows\include\hwy\ops\x86_256-inl.h(1994)
jxl.dll!jxl::N_AVX2::Symmetric5Interior<jxl::WrapUnchanged>(const jxl::Plane<float> & in, const jxl::Rect & rect, const __int64 ix, const __int64 iy, const jxl::WeightsSymmetric5 & weights, float * row_out) Line 95
	at C:\tools\vcpkg\buildtrees\libjxl\src\v0.6.1-d6e48761e1.clean\lib\jxl\convolve.cc(95)
jxl.dll!jxl::N_AVX2::Symmetric5Row<jxl::WrapUnchanged>(const jxl::Plane<float> & in, const jxl::Rect & rect, const __int64 iy, const jxl::WeightsSymmetric5 & weights, float * row_out) Line 131
	at C:\tools\vcpkg\buildtrees\libjxl\src\v0.6.1-d6e48761e1.clean\lib\jxl\convolve.cc(131)
[Inline Frame] jxl.dll!jxl::N_AVX2::Symmetric5::__l2::<lambda_89f43947037a03667304206744c598ea>::operator()(const int) Line 960
	at C:\tools\vcpkg\buildtrees\libjxl\src\v0.6.1-d6e48761e1.clean\lib\jxl\convolve.cc(960)
jxl.dll!jxl::ThreadPool::RunCallState<jxl::Status __cdecl(unsigned __int64),<lambda_89f43947037a03667304206744c598ea>>::CallDataFunc(void * jpegxl_opaque, unsigned int value, unsigned __int64 thread_id) Line 89
	at C:\tools\vcpkg\buildtrees\libjxl\src\v0.6.1-d6e48761e1.clean\lib\jxl\base\data_parallel.h(89)
jxl_threads.dll!jpegxl::ThreadParallelRunner::Runner(void * runner_opaque, void * jpegxl_opaque, int(*)(void *, unsigned __int64) init, void(*)(void *, unsigned int, unsigned __int64) func, unsigned int start_range, unsigned int end_range) Line 72
	at C:\tools\vcpkg\buildtrees\libjxl\src\v0.6.1-d6e48761e1.clean\lib\threads\thread_parallel_runner_internal.cc(72)
[Inline Frame] jxl.dll!jxl::ThreadPool::Run(unsigned int) Line 51
	at C:\tools\vcpkg\buildtrees\libjxl\src\v0.6.1-d6e48761e1.clean\lib\jxl\base\data_parallel.h(51)
[Inline Frame] jxl.dll!jxl::ThreadPool::Run(unsigned int) Line 60
	at C:\tools\vcpkg\buildtrees\libjxl\src\v0.6.1-d6e48761e1.clean\lib\jxl\base\data_parallel.h(60)
[Inline Frame] jxl.dll!jxl::RunOnPool(jxl::ThreadPool *) Line 117
	at C:\tools\vcpkg\buildtrees\libjxl\src\v0.6.1-d6e48761e1.clean\lib\jxl\base\data_parallel.h(117)
jxl.dll!jxl::N_AVX2::Symmetric5(const jxl::Plane<float> & in, const jxl::Rect & rect, const jxl::WeightsSymmetric5 & weights, jxl::ThreadPool * pool, jxl::Plane<float> * out) Line 964
	at C:\tools\vcpkg\buildtrees\libjxl\src\v0.6.1-d6e48761e1.clean\lib\jxl\convolve.cc(964)
jxl.dll!jxl::GaborishInverse(jxl::Image3<float> * in_out, float mul, jxl::ThreadPool * pool) Line 62
	at C:\tools\vcpkg\buildtrees\libjxl\src\v0.6.1-d6e48761e1.clean\lib\jxl\gaborish.cc(62)
jxl.dll!jxl::DefaultEncoderHeuristics::LossyFrameHeuristics(jxl::PassesEncoderState * enc_state, jxl::ModularFrameEncoder * modular_frame_encoder, const jxl::ImageBundle * original_pixels, jxl::Image3<float> * opsin, jxl::ThreadPool * pool, jxl::AuxOut * aux_out) Line 865
	at C:\tools\vcpkg\buildtrees\libjxl\src\v0.6.1-d6e48761e1.clean\lib\jxl\enc_heuristics.cc(865)
jxl.dll!jxl::LossyFrameEncoder::ComputeEncodingData(const jxl::ImageBundle * linear, jxl::Image3<float> * opsin, jxl::ThreadPool * pool, jxl::ModularFrameEncoder * modular_frame_encoder, jxl::BitWriter * writer, jxl::FrameHeader * frame_header) Line 497
	at C:\tools\vcpkg\buildtrees\libjxl\src\v0.6.1-d6e48761e1.clean\lib\jxl\enc_frame.cc(497)
jxl.dll!jxl::EncodeFrame(const jxl::CompressParams & cparams_orig, const jxl::FrameInfo & frame_info, const jxl::CodecMetadata * metadata, const jxl::ImageBundle & ib, jxl::PassesEncoderState * passes_enc_state, jxl::ThreadPool * pool, jxl::BitWriter * writer, jxl::AuxOut * aux_out) Line 1187
	at C:\tools\vcpkg\buildtrees\libjxl\src\v0.6.1-d6e48761e1.clean\lib\jxl\enc_frame.cc(1187)
jxl.dll!JxlEncoderStruct::RefillOutputByteQueue() Line 94
	at C:\tools\vcpkg\buildtrees\libjxl\src\v0.6.1-d6e48761e1.clean\lib\jxl\encode.cc(94)
jxl.dll!JxlEncoderProcessOutput(JxlEncoderStruct * enc, unsigned char * * next_out, unsigned __int64 * avail_out) Line 458
	at C:\tools\vcpkg\buildtrees\libjxl\src\v0.6.1-d6e48761e1.clean\lib\jxl\encode.cc(458)
_jxlpy.cp39-win_amd64.pyd!__pyx_pf_6_jxlpy_12JXLPyEncoder_4get_output(__pyx_obj_6_jxlpy_JXLPyEncoder * __pyx_v_self) Line 5423
	at C:\Users\ErwindeHaan\jxlpy\jxlpy\_jxlpy\_jxl.cpp(5423)
python39.dll!method_vectorcall_NOARGS(_object * func, _object * const * args, unsigned __int64 nargsf, _object * kwnames) Line 435
	at \objects\descrobject.c(435)
[Inline Frame] python39.dll!_PyObject_VectorcallTstate(_ts *) Line 118
	at \include\cpython\abstract.h(118)
[Inline Frame] python39.dll!PyObject_Vectorcall(_object * nargsf, _object * const *) Line 127
	at \include\cpython\abstract.h(127)
python39.dll!call_function(_ts * tstate, _object * * * pp_stack, __int64 oparg, _object * kwnames) Line 5072
	at \python\ceval.c(5072)
python39.dll!_PyEval_EvalFrameDefault(_ts * tstate, _frame * f, int throwflag) Line 3505
	at \python\ceval.c(3505)
[Inline Frame] python39.dll!_PyEval_EvalFrame(_ts *) Line 40
	at \include\internal\pycore_ceval.h(40)
python39.dll!_PyEval_EvalCode(_ts * tstate, _object * _co, _object * globals, _object * locals, _object * const * args, __int64 argcount, _object * const * kwnames, _object * const * kwargs, __int64 kwcount, int kwstep, _object * const * defs, __int64 defcount, _object * kwdefs, _object * closure, _object * name, _object * qualname) Line 4327
	at \python\ceval.c(4327)
[Inline Frame] python39.dll!_PyFunction_Vectorcall(_object * stack, _object * const *) Line 396
	at \objects\call.c(396)
[Inline Frame] python39.dll!_PyObject_VectorcallTstate(_ts *) Line 118
	at \include\cpython\abstract.h(118)
[Inline Frame] python39.dll!PyObject_Vectorcall(_object * nargsf, _object * const *) Line 127
	at \include\cpython\abstract.h(127)
python39.dll!call_function(_ts * tstate, _object * * * pp_stack, __int64 oparg, _object * kwnames) Line 5072
	at \python\ceval.c(5072)
python39.dll!_PyEval_EvalFrameDefault(_ts * tstate, _frame * f, int throwflag) Line 3519
	at \python\ceval.c(3519)
[Inline Frame] python39.dll!_PyEval_EvalFrame(_ts *) Line 40
	at \include\internal\pycore_ceval.h(40)
python39.dll!_PyEval_EvalCode(_ts * tstate, _object * _co, _object * globals, _object * locals, _object * const * args, __int64 argcount, _object * const * kwnames, _object * const * kwargs, __int64 kwcount, int kwstep, _object * const * defs, __int64 defcount, _object * kwdefs, _object * closure, _object * name, _object * qualname) Line 4327
	at \python\ceval.c(4327)
[Inline Frame] python39.dll!_PyFunction_Vectorcall(_object * stack, _object * const *) Line 396
	at \objects\call.c(396)
[Inline Frame] python39.dll!_PyObject_VectorcallTstate(_ts *) Line 118
	at \include\cpython\abstract.h(118)
python39.dll!method_vectorcall(_object * method, _object * const * args, unsigned __int64 nargsf, _object * kwnames) Line 53
	at \objects\classobject.c(53)
[Inline Frame] python39.dll!_PyObject_VectorcallTstate(_ts *) Line 118
	at \include\cpython\abstract.h(118)
[Inline Frame] python39.dll!PyObject_Vectorcall(_object * nargsf, _object * const *) Line 127
	at \include\cpython\abstract.h(127)
python39.dll!call_function(_ts * tstate, _object * * * pp_stack, __int64 oparg, _object * kwnames) Line 5072
	at \python\ceval.c(5072)
python39.dll!_PyEval_EvalFrameDefault(_ts * tstate, _frame * f, int throwflag) Line 3536
	at \python\ceval.c(3536)
[Inline Frame] python39.dll!_PyEval_EvalFrame(_ts *) Line 40
	at \include\internal\pycore_ceval.h(40)
python39.dll!_PyEval_EvalCode(_ts * tstate, _object * _co, _object * globals, _object * locals, _object * const * args, __int64 argcount, _object * const * kwnames, _object * const * kwargs, __int64 kwcount, int kwstep, _object * const * defs, __int64 defcount, _object * kwdefs, _object * closure, _object * name, _object * qualname) Line 4327
	at \python\ceval.c(4327)
python39.dll!_PyEval_EvalCodeWithName(_object * _co, _object * globals, _object * locals, _object * const * args, __int64 argcount, _object * const * kwnames, _object * const * kwargs, __int64 kwcount, int kwstep, _object * const * defs, __int64 defcount, _object * kwdefs, _object * closure, _object * name, _object * qualname) Line 4366
	at \python\ceval.c(4366)
python39.dll!PyEval_EvalCodeEx(_object * _co, _object * globals, _object * locals, _object * const * args, int argcount, _object * const * kws, int kwcount, _object * const * defs, int defcount, _object * kwdefs, _object * closure) Line 4382
	at \python\ceval.c(4382)
python39.dll!PyEval_EvalCode(_object * co, _object * globals, _object * locals) Line 832
	at \python\ceval.c(832)
python39.dll!run_eval_code_obj(_ts * tstate, PyCodeObject * co, _object * globals, _object * locals) Line 1220
	at \python\pythonrun.c(1220)
python39.dll!run_mod(_mod * mod, _object * filename, _object * globals, _object * locals, PyCompilerFlags * flags, _arena * arena) Line 1241
	at \python\pythonrun.c(1241)
python39.dll!pyrun_file(_iobuf * fp, _object * filename, int start, _object * globals, _object * locals, int closeit, PyCompilerFlags * flags) Line 1138
	at \python\pythonrun.c(1138)
python39.dll!pyrun_simple_file(_iobuf * fp, _object * filename, int closeit, PyCompilerFlags * flags) Line 452
	at \python\pythonrun.c(452)
python39.dll!PyRun_SimpleFileExFlags(_iobuf * fp, const char * filename, int closeit, PyCompilerFlags * flags) Line 483
	at \python\pythonrun.c(483)
python39.dll!PyRun_AnyFileExFlags(_iobuf * fp, const char * filename, int closeit, PyCompilerFlags * flags) Line 92
	at \python\pythonrun.c(92)
python39.dll!pymain_run_file(const PyConfig * config, PyCompilerFlags * cf) Line 379
	at \modules\main.c(379)
python39.dll!pymain_run_python(int * exitcode) Line 605
	at \modules\main.c(605)
python39.dll!Py_RunMain() Line 685
	at \modules\main.c(685)
python39.dll!pymain_main(_PyArgv * args) Line 1106
	at \modules\main.c(1106)
python39.dll!Py_Main(int argc, wchar_t * * argv) Line 1118
	at \modules\main.c(1118)

So it seems to happen on the call here.

/* "_jxlpy/_jxl.pyx":804
 *         while self.status == JXL_ENC_NEED_MORE_OUTPUT:
 * 
 *             self.status = JxlEncoderProcessOutput(             # <<<<<<<<<<<<<<
 *                 self.encoder, &next_out, &avail_out
 *             )
 */

Do note to make this work you need to add the follwing to setup.py

jxlpy_ext = Extension(
    name="_jxlpy",
    sources=["_jxlpy/_jxl.pyx"],
    libraries=['jxl','jxl_threads'],
    extra_compile_args=['-Ox','-Zi'], # For debugging only, use O2 for release build
    extra_link_args=['-debug:full'], # For debugging only
    language='c++'
)
from sys import platform
if platform == "win32":
    jxlpy_ext.include_dirs=['C:\\tools\\vcpkg\\installed\\x64-windows\\include']
    jxlpy_ext.library_dirs=['C:\\tools\\vcpkg\\installed\\x64-windows\\lib']

Any ideas?

[olokelo]

Hi, thank you very much for detailed instructions. I was able to follow along even though I'm not into Windows development.
It might help creating windows prebuilt wheels of jxlpy.

As for your issue, unfortunately I wasn't able to reproduce it. For me, all examples work fine.
I really hate being but it works on my pc guy but that's what it is :/

In the meantime, I did a quick search for this issue and found out, it might be related to Cython. (source 1, 2)

I will attach my prebuilt and repaired wheel below if that helps, maybe try using it.

jxlpy-0.9.1-cp39-cp39-win_amd64.whl.zip

[EraYaN]

Right that gives the same result here, so it's probably some problem with my Cython version or environment
Would you be able to tell me how you installed cython and what version you are running? here it installed Cython-0.29.26-py2.py3-none-any.whl from pypi

EDIT; Seems like lossless mode does work, but for example quality=75, effort=6 does not. Also tried is on an other Windows 10 machine and it also does not work there, not sure what is wrong. Did you build with VS2022?

[olokelo]

My Cython version is the same as yours, installed automatically as a dependency of jxlpy.
I'm running Windows 10 build 19042 on VirtualBox. Python 3.9.10, cython 0.29.26. My CPU (amd64) supports avx2 (in your error log there was something about libhwy executing some avx2 instructions but I doubt it's relevant). I installed libjxl with VS2022 using vcpkg.

I did some encoding/decoding tests with lossless and lossy mode and everything seems fine..

[EraYaN]

Alright I'm running python 3.9.5 maybe if I upgrade that and I just tried with 3.9.10 and same problem my windows version is 21H1 (build 19043). But still the same behavior, the actual site of the access violation is within preprocessor guards for the MSVC compiler. (using some intrinsics) Can cython build with clang on windows?

I'm stumped as to why this is happening.