version 0.10.4a2 on on old macOS machines can crash Jupyter Python kernel if the first Sketch uses JAVA2D and any later Sketch uses OpenGL

[hx2A]

This issue is here to track progress on a bug found in release 0.10.4a2. This bug only happens on old macOS machines (Intel CPUs? macOS Monterey?).

When py5 is run in Jupyter or IPython, if the first Sketch uses JAVA2D, no later Sketch can use OpenGL. Using OpenGL would cause the Python kernel to crash. If the first Sketch uses OpenGL, any sequence of JAVA2D and OpenGL Sketches will work just fine. The crash only occurs if the first Sketch uses JAVA2D and a later Sketch uses OpenGL. This is clearly an obscure bug, but is a bug nevertheless.

The code change that causes this bug also fixed #5 and #564 for all macOS users. The significance and importance of this bug fix far exceeds the a bug caused by an obscure sequence of events on old macOS machines.

This problem was observed in my old MacBook Pro laptop, with an Intel i7 CPU, running macOS 12.7.6 (Monterey, which is no longer supported by Apple). I don't have this problem on my newer 2020 MacBook Pro with an M1 chip. Everything worked just fine with macOS 14.7.2 (Sonoma). I upgraded it to macOS 15.2 (Sequoia), and it works fine there too.

What is the scope of the problem? Is it the Intel CPU? Is it the old macOS version?

Release 0.10.4a1 has a safety check to prevent the sequence of events that would lead to a crash.

It would be great if macOS users in the py5 community could test this and report back. Please run the following code with py5 version 0.10.4a1 installed. Use a Jupyter notebook and execute each line of code in its own cell. Comment below to let us know if it crashed or not.

from py5 import macos_problem, test

macos_problem.disable_safety_check()

# run a Sketch with the default renderer
test.test_java2d()

# run a Sketch with an opengl renderer
# does this cause a crash???
test.test_p2d()

Once the following table is filled out, the safety check referenced by disable_safety_check() will be calibrated to just the machines that need it.

Summary of results so far:

macOS version	Intel CPU	Apple Silicon
Big Sur (11)	Crash	?
Monterey (12)	Crash	?
Ventura (13)	?	No crash
Sonoma (14)	?	No crash
Sequoia (15)	?	No crash

Thank you!

[villares]

Testing on Mac Intel Core i7 - macOS Big Sur 11.7.9 - Crash

eta@MAC315479 Downloads % ipython
Python 3.10.11 (v3.10.11:7d4cc5aa85, Apr  4 2023, 19:05:19) [Clang 13.0.0 (clang-1300.0.29.30)]
Type 'copyright', 'credits' or 'license' for more information
IPython 8.31.0 -- An enhanced Interactive Python. Type '?' for help.

In [1]: from py5 import macos_problem, test
   ...: 
Importing py5 on macOS but the necessary Jupyter macOS event loop has not been activated. I'll activate it for you, but next time, execute `%gui osx` before importing this library.

In [2]: macos_problem.disable_safety_check()

In [3]: test.test_java2d()

In [4]: test.test_p2d()

In [5]: #
# A fatal error has been detected by the Java Runtime Environment:
#
#  SIGILL (0x4) at pc=0x00007fff2037625d, pid=3755, tid=22815
#
# JRE version: OpenJDK Runtime Environment Temurin-17.0.9+9 (17.0.9+9) (build 17.0.9+9)
# Java VM: OpenJDK 64-Bit Server VM Temurin-17.0.9+9 (17.0.9+9, mixed mode, tiered, compressed oops, compressed class ptrs, g1 gc, bsd-amd64)
# Problematic frame:
# C  [libdispatch.dylib+0x525d]  _dispatch_assert_queue_fail+0x63
#
# No core dump will be written. Core dumps have been disabled. To enable core dumping, try "ulimit -c unlimited" before starting Java again
#
# An error report file with more information is saved as:
# /Users/eta/Downloads/hs_err_pid3755.log
#
# If you would like to submit a bug report, please visit:
#   https://github.com/adoptium/adoptium-support/issues
# The crash happened outside the Java Virtual Machine in native code.
# See problematic frame for where to report the bug.
#
zsh: abort      ipython
eta@MAC315479 Downloads %

Out of curiosity I ran without disabling the check 😄

eta@MAC315479 Downloads % ipython
Python 3.10.11 (v3.10.11:7d4cc5aa85, Apr  4 2023, 19:05:19) [Clang 13.0.0 (clang-1300.0.29.30)]
Type 'copyright', 'credits' or 'license' for more information
IPython 8.31.0 -- An enhanced Interactive Python. Type '?' for help.

In [1]: from py5 import macos_problem, test
Importing py5 on macOS but the necessary Jupyter macOS event loop has not been activated. I'll activate it for you, but next time, execute `%gui osx` before importing this library.

In [2]: test.test_java2d()

In [3]: test.test_p2d()

Sorry, but you can't use an OpenGL renderer in your Sketch right now. Doing so might cause Python to crash.

Here's the problem: On macOS machines with Intel CPUs and/or older macOS versions, this version of py5 seems to crash when you use an OpenGL renderer in an IPython or Jupyter session if the first Sketch run in that Python session used the default (JAVA2D) renderer. Sorry if that sounds crazy. This is an unfortunate side effect of an important code change that significantly improved py5 for all macOS users.

The root issue is somewhere in native macOS code that Processing and py5 both depend on. Hopefully in the future we will find a real fix or a better workaround.

You are seeing this message because this version of py5 has a safety feature that detects the sequence of events that might lead to this crash. However, if you'd like to disable this safety feature (and risk Python crashing), use the following code:

    from py5 import macos_problem

    macos_problem.disable_safety_check()

But before doing that, it would be great if you could do a quick test for us. Please run the following code in a new Jupyter Notebook or IPython REPL, with each line of code executed separately:

    from py5 import macos_problem, test

    macos_problem.disable_safety_check()

    # run a Sketch with the default renderer
    test.test_java2d()

    # run a Sketch with an opengl renderer
    # does this cause a crash???
    test.test_p2d()

Then report your findings to the below GitHub issue thread. Include your macOS version and CPU type. (For your convenience, this information will be displayed at the end of this message.) Your feedback will help us understand the problem better and more accurately calibrate this crash protection feature.

https://github.com/py5coding/py5generator/issues/578

If the above test code doesn't cause Python to crash on your machine, great! You can keep using that `disable_safety_check()` function so you never see this warning again. But please take the time report your findings to the GitHub issue thread. The next version of py5 will incorporate your feedback and the safety feature will be adjusted accordingly.

If the above test code does cause Python to crash on your machine, it's OK. If you really need to mix Java2D and OpenGL renderers together in one Python session, you just need to make sure that the first executed Sketch is always an OpenGL Sketch. For convenience, you can use the following code to open a quick Sketch right after importing py5. This will ensure the first Sketch is always an OpenGL Sketch, eliminating the problem (and this warning) entirely:

    import py5
    from py5 import test

    test.test_p2d()

If you'd like to read about our progress understanding this issue, please visit the above GitHub issue thread.

Sorry again for the weird limitation. We're doing our best to make py5 as stable as possible. This safety feature is here because we don't want users to become upset because Python crashed for confusing reasons. Thank you for your understanding.

macOS version: 11.7.9
macOS CPU type: i386
py5 encountered an error in your code:

RuntimeError: Halting Sketch startup to prevent Python from crashing

[hx2A]

Testing on Mac Intel Core i7 - macOS Big Sur 11.7.9 - Crash

@villares , thank you for the test. And also now I know this isn't just a weird fluke of my computer.

I'm very curious how this goes for Intel CPU laptops with more current versions of macOS.

[hx2A]

@villares also, none of this impacts Thonny users. This issue only applies for Jupyter Notebooks or IPython where someone can run more than one Sketch.

[villares]

I'm very curious how this goes for Intel CPU laptops with more current versions of macOS.

I can't update macOS on these computers, but I'll keep my eyes open and ask around!

[hx2A]

Recently I discovered I could run macOS Virtual Machines on my Mac M1 computer using UTM. I've been able to run Monterey (12) and Ventura (13) on this laptop that is currently running Sequoia (15). In both cases, the OS in the virtual machine was using the same CPU architecture as the host (arm64).

I ran the above test for both and there was no crash. So this suggests that the crash problem is limited to Intel CPU macs. However, when I import py5 in the virtual machine I get a message "WARNING: GL pipe is running in software mode". If OpenGL isn't using the GPU hardware, this might not be a valid test.

In theory UTM can do CPU emulation so I should be able to emulate an Intel CPU using my M1 computer. But I haven't gotten that to work yet, and I am not sure if it would even be a valid test anyway, because the GL pipe would still be in software mode.

If I can't find someone who can do these tests on their own hardware for us, I can also try using a cloud computing service and "rent" machines. I've also read that github actions might work. I'll keep investigating.

[mj-mahzoon]

@hx2A - Thanks for tracking issues related to the macOS. Are you also tracking crashes when using python standalone (as opposed to using Jupyter)? When using your suggested tests standalone, test.test_p2d() crashes for me. Also tried test.test_p3d() and that crashes too. I use M2 pro chip with macOS Sequoia 15.2.

Also, I only get zsh: trace trap python <filename> and don't know how to get the stacktrace or more info on the error details. Is there a way to get more on the error?

[hx2A]

Hi, @mj-mahzoon ! Thanks for stopping by. In this issue I am investigating a specific bug related to macOS and Jupyter. On macOS, Jupyter is the only way to run a JAVA2D sketch and later a P2D sketch. When you run a Sketch in the generic python interpreter, exiting the Sketch will always cause Python to also exit. That isn't a crash, that's just a side effect of how py5 works on macOS in Python.

If Python is crashing for you when run test.test_p2d() in the generic python interpreter, that sounds like a bug that I don't know about. If that is the case, can you open a new issue for that, and include whatever error message you are seeing? Thanks!

[SableRaf]

@hx2A: I tried running the crash test on macOS Ventura 13.4 (22F66) on my M1 Max MBP from 2021 and got the following error:

2025-01-15 13:21:28.307 Python[30148:435824] NewtNSView::dealloc: softLock still hold @ dealloc!

I have no experience with Jupyter notebooks so I'm not sure I did the steps right. Let me know if I missed something.

[hx2A]

Thank you, @SableRaf ! I really appreciate it.

The crash we are testing for here is a segmentation fault. The message you got there is a warning that I've traced to within native macOS code found within Processing's JOGL jars. I'm not further banging my head against the wall to eliminate that warning because I know Processing is moving away from JOGL eventually.

I have no experience with Jupyter notebooks so I'm not sure I did the steps right. Let me know if I missed something.

You did it perfectly. And you should give Jupyter a try, it is an excellent platform for combining code with documentation.