PicklingError when importing cs50 library in checks

[kzidane]

Traceback (most recent call last):
  File "/usr/local/bin/check50", line 11, in <module>
    sys.exit(main())
  File "/usr/local/lib/python3.7/dist-packages/check50/__main__.py", line 379, in main
    check_results = check_runner.run(args.target)
  File "/usr/local/lib/python3.7/dist-packages/check50/runner.py", line 201, in run
    result, state = future.result()
  File "/usr/lib/python3.7/concurrent/futures/_base.py", line 428, in result
    return self.__get_result()
  File "/usr/lib/python3.7/concurrent/futures/_base.py", line 384, in __get_result
    raise self._exception
  File "/usr/lib/python3.7/multiprocessing/queues.py", line 236, in _feed
    obj = _ForkingPickler.dumps(obj)
  File "/usr/lib/python3.7/multiprocessing/reduction.py", line 51, in dumps
    cls(buf, protocol).dump(obj)
_pickle.PicklingError: Can't pickle <function <lambda> at 0x7ff58294add0>: attribute lookup <lambda> on cs50.cs50 failed

[Jelleas]

Do you have the relevant slug by any chance?

[kzidane]

cs50/problems/2020/fall/movies

[MarijnDoeve]

Seems to be fixed in the online version of check50, with -l this error still exists.

[Jelleas]

This happens because the cs50 library sets the sys.excepthook itself unconditionally to a lambda function. That famously are not pickle-able, because where'd be the fun in that.

As it happens, check50 now treats sys.excepthook as something special, because from Python 3.8 on OS X check50 is forced to support the spawn method for multiprocessing. Long story short, this line:

check50/check50/runner.py

Line 333 in 03d5bb0

"sys.excepthook",

now causes check50 to copy w/e excepthook is set to each check process.

In part this problem stems from the awkwardness of a library changing sys.excepthook. No one expects tracebacks to change thanks to this import:

from cs50 import SQL

I think the only approach here is to accept that sys.excepthook can be set externally, either by the checks file, or a library such as python-cs50. If that happens then either under spawn pass nothing, because that imported library should be imported again and set sys.excepthook again. Or under fork don't set sys.excepthook.

Until then, either move from cs50 import SQL from the top of the test file to each test/function where it's needed. Moving that import to line 119 fixes movies for instance.

[Jelleas]

https://github.com/cs50/check50/tree/unpickleable_attributes