Running on Windows Results in error

[JohnPeng47]

C:\Users\jpeng\Documents\business\ai\marsha\marsha>python -m marsha tesseract.mrsh
Compiling functions for tesseract...
Generating Python code...
Chat query took 15sec 553.156ms, started at 44.45ms, ms/chars = 12.363398965856797
Chat query took 24sec 196.805ms, started at 26.371ms, ms/chars = 12.661855049871887
Writing generated code to temporary files...
FILENAME: tesseract_0/tesseract.py
FILENAME: tesseract_0/requirements.txt
FILENAME: tesseract_0/tesseract_test.py
Writing generated code to temporary files...
FILENAME: tesseract_1/tesseract.py
FILENAME: tesseract_1/requirements.txt
FILENAME: tesseract_1/tesseract_test.py
Writing generated code to temporary files...
FILENAME: tesseract_2/tesseract.py
FILENAME: tesseract_2/requirements.txt
FILENAME: tesseract_2/tesseract_test.py
Running tasks in parallel...
Parsing generated code...
Parsing generated code...
Verifying and correcting generated code...
Creating virtual environment...
Parsing generated code...
Verifying and correcting generated code...
Creating virtual environment...
Chat query took 10sec 55.1715ms, started at 24sec 377.311ms, ms/chars = 13.371238683132415
Installing requirements...
TESTFILE: tesseract_1\tesseract_test.py
Failed to run test suite... [WinError 2] The system cannot find the file specified
Third stage failure
argument of type 'NoneType' is not iterable
Task completed with error. Waiting for pending tasks to finish...
Running tasks in parallel...
Installing requirements...
TESTFILE: tesseract_2\tesseract_test.py
Failed to run test suite... [WinError 2] The system cannot find the file specified
Third stage failure
argument of type 'NoneType' is not iterable
Task completed with error. Waiting for pending tasks to finish...
Running tasks in parallel...
Chat query took 14sec 694.8ms, started at 34sec 434.483ms, ms/chars = 19.884708931062153
FILENAME: tesseract_0\tesseract_test.py
Chat query took 28sec 31.8413ms, started at 49sec 133.387ms, ms/chars = 38.03506279250498
FILENAME: tesseract_0\tesseract_test.py
Chat query took 40sec 667.973ms, started at 1min 17sec, ms/chars = 54.882554711922296
FILENAME: tesseract_0\tesseract_test.py
Second stage failure
('Failed to generate code', "tesseract_0\tesseract_test.py:35:5 [E] E0602 undefined name 'unittest' [pyflakes]")
All tasks failed. Raising exception...
Failed to generate working code.
('Failed to generate code', "tesseract_0\tesseract_test.py:35:5 [E] E0602 undefined name 'unittest' [pyflakes]")
Retrying...
Traceback (most recent call last):
File "C:\Program Files\WindowsApps\PythonSoftwareFoundation.Python.3.10_3.10.3056.0_x64__qbz5n2kfra8p0\lib\runpy.py", line 196, in _run_module_as_main
return _run_code(code, main_globals, None,
File "C:\Program Files\WindowsApps\PythonSoftwareFoundation.Python.3.10_3.10.3056.0_x64__qbz5n2kfra8p0\lib\runpy.py", line 86, in run_code
exec(code, run_globals)
File "C:\Users\jpeng\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.10_qbz5n2kfra8p0\LocalCache\local-packages\Python310\site-packages\marsha_main.py", line 6, in
asyncio.run(main())
File "C:\Program Files\WindowsApps\PythonSoftwareFoundation.Python.3.10_3.10.3056.0_x64__qbz5n2kfra8p0\lib\asyncio\runners.py", line 44, in run
return loop.run_until_complete(main)
File "C:\Program Files\WindowsApps\PythonSoftwareFoundation.Python.3.10_3.10.3056.0_x64__qbz5n2kfra8p0\lib\asyncio\base_events.py", line 649, in run_until_complete
return future.result()
File "C:\Users\jpeng\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.10_qbz5n2kfra8p0\LocalCache\local-packages\Python310\site-packages\marsha\base.py", line 217, in main
raise Exception(
Exception: Failed to generate working code for tesseract. Total time elapsed: 1min 59sec. Total cost: 0.02.

[JohnPeng47]

Tried to run the ocr/tesseract.mrsh file and got the error above

[dfellis]

So this is one of the more complicated examples, as you need to install and have available on your $PATH the tesseract and poppler libraries before you run that example.

The exact phrasing there is a bit misleading, now that I think about it, because in the third stage of the compiler it runs the code against the test suite it generated. So that comment is technically correct (the best kind!) but implies when you manually run the code, not when you indirectly run the code during the compile process itself.

Can you try again with those libraries installed? I don't personally know how to install them on Windows, though. Perhaps @aguillenv does, though he's asleep right now, so we'd have to wait until he wakes up if you can't figure that part out.

[aguillenv]

The poppler and tesseract packages are probably needed running the generated code not during marsha execution.

From the logs shared this seems an issue with marsha handling the file system Failed to run test suite... [WinError 2] The system cannot find the file specified.

I'll dig and see if I find what could be the issue since I'm not working on Windows.

[aguillenv]

@JohnPeng47 let us know if the issue is solved for you! If you pip-installed Marsha, make sure to run it again with the --upgrade flag to get the latest version.

[JohnPeng47]

Uhh now Im getting two errors instead of one:
1st run:

C:\Users\jpeng\Documents\business\ai\marsha\examples\general-purpose>python -m marsha fibonacci.mrsh
Compiling functions for fibonacci...
Generating Python code...
Chat query took 5sec 53.683ms, started at 35.4121ms, ms/chars = 6.454256759292778
Chat query took 5sec 396.657ms, started at 56.5364ms, ms/chars = 7.6009256739011954
Writing generated code to temporary files...
FILENAME: fibonacci_0/fibonacci.py
FILENAME: fibonacci_0/requirements.txt
FILENAME: fibonacci_0/fibonacci_test.py
Writing generated code to temporary files...
FILENAME: fibonacci_1/fibonacci.py
FILENAME: fibonacci_1/requirements.txt
FILENAME: fibonacci_1/fibonacci_test.py
Writing generated code to temporary files...
FILENAME: fibonacci_2/fibonacci.py
FILENAME: fibonacci_2/requirements.txt
FILENAME: fibonacci_2/fibonacci_test.py
Running tasks in parallel...
Parsing generated code...
Verifying and correcting generated code...
Parsing generated code...
Verifying and correcting generated code...
Creating virtual environment...
Parsing generated code...
Verifying and correcting generated code...
Creating virtual environment...
Formatting code...
Task completed successfully. Cancelling pending tasks...
Writing generated code to files...
fibonacci done! Total time elapsed: 5sec 637.325ms. Total cost: 0.0.
Exception ignored in: <function BaseSubprocessTransport.del at 0x000002B801248430>
Traceback (most recent call last):
File "C:\Program Files\WindowsApps\PythonSoftwareFoundation.Python.3.10_3.10.3056.0_x64__qbz5n2kfra8p0\lib\asyncio\base_subprocess.py", line 126, in del
self.close()
File "C:\Program Files\WindowsApps\PythonSoftwareFoundation.Python.3.10_3.10.3056.0_x64__qbz5n2kfra8p0\lib\asyncio\base_subprocess.py", line 104, in close
proto.pipe.close()
File "C:\Program Files\WindowsApps\PythonSoftwareFoundation.Python.3.10_3.10.3056.0_x64__qbz5n2kfra8p0\lib\asyncio\proactor_events.py", line 109, in close
self._loop.call_soon(self._call_connection_lost, None)
File "C:\Program Files\WindowsApps\PythonSoftwareFoundation.Python.3.10_3.10.3056.0_x64__qbz5n2kfra8p0\lib\asyncio\base_events.py", line 753, in call_soon
self._check_closed()
File "C:\Program Files\WindowsApps\PythonSoftwareFoundation.Python.3.10_3.10.3056.0_x64__qbz5n2kfra8p0\lib\asyncio\base_events.py", line 515, in _check_closed
raise RuntimeError('Event loop is closed')
RuntimeError: Event loop is closed
Exception ignored in: <function _ProactorBasePipeTransport.del at 0x000002B801249E10>
Traceback (most recent call last):
File "C:\Program Files\WindowsApps\PythonSoftwareFoundation.Python.3.10_3.10.3056.0_x64__qbz5n2kfra8p0\lib\asyncio\proactor_events.py", line 116, in del
_warn(f"unclosed transport {self!r}", ResourceWarning, source=self)
File "C:\Program Files\WindowsApps\PythonSoftwareFoundation.Python.3.10_3.10.3056.0_x64__qbz5n2kfra8p0\lib\asyncio\proactor_events.py", line 80, in repr
info.append(f'fd={self._sock.fileno()}')
File "C:\Program Files\WindowsApps\PythonSoftwareFoundation.Python.3.10_3.10.3056.0_x64__qbz5n2kfra8p0\lib\asyncio\windows_utils.py", line 102, in fileno
raise ValueError("I/O operation on closed pipe")
ValueError: I/O operation on closed pipe
Exception ignored in: <function _ProactorBasePipeTransport.del at 0x000002B801249E10>
Traceback (most recent call last):
File "C:\Program Files\WindowsApps\PythonSoftwareFoundation.Python.3.10_3.10.3056.0_x64__qbz5n2kfra8p0\lib\asyncio\proactor_events.py", line 116, in del
_warn(f"unclosed transport {self!r}", ResourceWarning, source=self)
File "C:\Program Files\WindowsApps\PythonSoftwareFoundation.Python.3.10_3.10.3056.0_x64__qbz5n2kfra8p0\lib\asyncio\proactor_events.py", line 80, in repr
info.append(f'fd={self._sock.fileno()}')
File "C:\Program Files\WindowsApps\PythonSoftwareFoundation.Python.3.10_3.10.3056.0_x64__qbz5n2kfra8p0\lib\asyncio\windows_utils.py", line 102, in fileno
raise ValueError("I/O operation on closed pipe")
ValueError: I/O operation on closed pipe
Exception ignored in: <function BaseSubprocessTransport.del at 0x000002B801248430>
Traceback (most recent call last):
File "C:\Program Files\WindowsApps\PythonSoftwareFoundation.Python.3.10_3.10.3056.0_x64__qbz5n2kfra8p0\lib\asyncio\base_subprocess.py", line 125, in del
_warn(f"unclosed transport {self!r}", ResourceWarning, source=self)
File "C:\Program Files\WindowsApps\PythonSoftwareFoundation.Python.3.10_3.10.3056.0_x64__qbz5n2kfra8p0\lib\asyncio\base_subprocess.py", line 78, in repr
info.append(f'stdout={stdout.pipe}')
File "C:\Program Files\WindowsApps\PythonSoftwareFoundation.Python.3.10_3.10.3056.0_x64__qbz5n2kfra8p0\lib\asyncio\proactor_events.py", line 80, in repr
info.append(f'fd={self._sock.fileno()}')
File "C:\Program Files\WindowsApps\PythonSoftwareFoundation.Python.3.10_3.10.3056.0_x64__qbz5n2kfra8p0\lib\asyncio\windows_utils.py", line 102, in fileno
raise ValueError("I/O operation on closed pipe")
ValueError: I/O operation on closed pipe
Exception ignored in: <function _ProactorBasePipeTransport.del at 0x000002B801249E10>
Traceback (most recent call last):
File "C:\Program Files\WindowsApps\PythonSoftwareFoundation.Python.3.10_3.10.3056.0_x64__qbz5n2kfra8p0\lib\asyncio\proactor_events.py", line 116, in del
_warn(f"unclosed transport {self!r}", ResourceWarning, source=self)
File "C:\Program Files\WindowsApps\PythonSoftwareFoundation.Python.3.10_3.10.3056.0_x64__qbz5n2kfra8p0\lib\asyncio\proactor_events.py", line 80, in repr
info.append(f'fd={self._sock.fileno()}')
File "C:\Program Files\WindowsApps\PythonSoftwareFoundation.Python.3.10_3.10.3056.0_x64__qbz5n2kfra8p0\lib\asyncio\windows_utils.py", line 102, in fileno
raise ValueError("I/O operation on closed pipe")
ValueError: I/O operation on closed pipe
Exception ignored in: <function _ProactorBasePipeTransport.del at 0x000002B801249E10>
Traceback (most recent call last):
File "C:\Program Files\WindowsApps\PythonSoftwareFoundation.Python.3.10_3.10.3056.0_x64__qbz5n2kfra8p0\lib\asyncio\proactor_events.py", line 116, in del
_warn(f"unclosed transport {self!r}", ResourceWarning, source=self)
File "C:\Program Files\WindowsApps\PythonSoftwareFoundation.Python.3.10_3.10.3056.0_x64__qbz5n2kfra8p0\lib\asyncio\proactor_events.py", line 80, in repr
info.append(f'fd={self._sock.fileno()}')
File "C:\Program Files\WindowsApps\PythonSoftwareFoundation.Python.3.10_3.10.3056.0_x64__qbz5n2kfra8p0\lib\asyncio\windows_utils.py", line 102, in fileno
raise ValueError("I/O operation on closed pipe")
ValueError: I/O operation on closed pipe

Second run:

C:\Users\jpeng\Documents\business\ai\marsha\examples\general-purpose>python -m marsha fibonacci.mrsh
Compiling functions for fibonacci...
Generating Python code...
Chat query took 4sec 677.319ms, started at 48.6438ms, ms/chars = 6.606382165251478
Chat query took 5sec 407.894ms, started at 29.7339ms, ms/chars = 6.310261812065418
Writing generated code to temporary files...
FILENAME: fibonacci_0/fibonacci.py
FILENAME: fibonacci_0/requirements.txt
FILENAME: fibonacci_0/fibonacci_test.py
Writing generated code to temporary files...
FILENAME: fibonacci_1/fibonacci.py
FILENAME: fibonacci_1/requirements.txt
FILENAME: fibonacci_1/fibonacci_test.py
Writing generated code to temporary files...
FILENAME: fibonacci_2/fibonacci.py
FILENAME: fibonacci_2/requirements.txt
FILENAME: fibonacci_2/fibonacci_test.py
Running tasks in parallel...
Parsing generated code...
Verifying and correcting generated code...
Creating virtual environment...
Parsing generated code...
Verifying and correcting generated code...
Creating virtual environment...
Parsing generated code...
Verifying and correcting generated code...
Installing requirements...
TESTFILE: fibonacci_2\fibonacci_test.py
Failed to run test suite... [WinError 2] The system cannot find the file specified
Third stage failure
argument of type 'NoneType' is not iterable
Task completed with error. Waiting for pending tasks to finish...
Running tasks in parallel...
Installing requirements...
TESTFILE: fibonacci_0\fibonacci_test.py
Failed to run test suite... [WinError 2] The system cannot find the file specified
Third stage failure
argument of type 'NoneType' is not iterable
Installing requirements...
TESTFILE: fibonacci_1\fibonacci_test.py
Failed to run test suite... [WinError 2] The system cannot find the file specified
Third stage failure
argument of type 'NoneType' is not iterable
All tasks failed. Raising exception...
Failed to generate working code.
argument of type 'NoneType' is not iterable
Retrying...
Traceback (most recent call last):
File "C:\Program Files\WindowsApps\PythonSoftwareFoundation.Python.3.10_3.10.3056.0_x64__qbz5n2kfra8p0\lib\runpy.py", line 196, in _run_module_as_main
return _run_code(code, main_globals, None,
File "C:\Program Files\WindowsApps\PythonSoftwareFoundation.Python.3.10_3.10.3056.0_x64__qbz5n2kfra8p0\lib\runpy.py", line 86, in run_code
exec(code, run_globals)
File "C:\Users\jpeng\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.10_qbz5n2kfra8p0\LocalCache\local-packages\Python310\site-packages\marsha_main.py", line 6, in
asyncio.run(main())
File "C:\Program Files\WindowsApps\PythonSoftwareFoundation.Python.3.10_3.10.3056.0_x64__qbz5n2kfra8p0\lib\asyncio\runners.py", line 44, in run
return loop.run_until_complete(main)
File "C:\Program Files\WindowsApps\PythonSoftwareFoundation.Python.3.10_3.10.3056.0_x64__qbz5n2kfra8p0\lib\asyncio\base_events.py", line 649, in run_until_complete
return future.result()
File "C:\Users\jpeng\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.10_qbz5n2kfra8p0\LocalCache\local-packages\Python310\site-packages\marsha\base.py", line 217, in main
raise Exception(
Exception: Failed to generate working code for fibonacci. Total time elapsed: 18sec 189.47ms. Total cost: 0.0.
Task exception was never retrieved
future: <Task finished name='fibonacci_0\fibonacci.py' coro=<review_and_fix() done, defined at C:\Users\jpeng\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.10_qbz5n2kfra8p0\LocalCache\local-packages\Python310\site-packages\marsha\base.py:281> exception=TypeError("argument of type 'NoneType' is not iterable")>
Traceback (most recent call last):
File "C:\Users\jpeng\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.10_qbz5n2kfra8p0\LocalCache\local-packages\Python310\site-packages\marsha\base.py", line 304, in review_and_fix
raise e
File "C:\Users\jpeng\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.10_qbz5n2kfra8p0\LocalCache\local-packages\Python310\site-packages\marsha\base.py", line 300, in review_and_fix
await test_and_fix_files(marsha_filename, functions, defined_types, void_functions, files, stats, debug=debug)
File "C:\Users\jpeng\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.10_qbz5n2kfra8p0\LocalCache\local-packages\Python310\site-packages\marsha\llm.py", line 444, in test_and_fix_files
if "FAILED" in test_results or "Traceback" in test_results:
TypeError: argument of type 'NoneType' is not iterable

[JohnPeng47]

Also.. not saying this to be an asshole, because personally I think the idea behind this project is really cool, and much needed to push the edge of LLM codegen. But your code is insanely hard to read. Would suggest doing a refactor to improve readability, modularity, all that good stuff, before pushing out any features. OSS contributions is hard, but doing it on a codebase like this makes the task Herculean

[dfellis]

ESTFILE: tesseract_1\tesseract_test.py
Failed to run test suite... [WinError 2] The system cannot find the file specified
Third stage failure
argument of type 'NoneType' is not iterable

This is a quote from your first run, same error as before. I'm spinning up a Windows VM to see if I can reproduce on my end, since the earlier change didn't fix things for you (though it did fix it for Github's Windows runner)

As for your second comment, it's pretty vague. What exactly is hard to read for you? The logic for parsing the .mrsh files is in parse.py, anything directly talking to the LLM is in llm.py, base.py has the overall compilation flow laid out, and utils.py has a few functions that are reused between multiple of the other files. It's not a big codebase, yet, and each file is on a single "topic" so I don't see how a refactor would improve anything?

[dfellis]

I cannot reproduce your error @JohnPeng47

Steps I took on a fresh Win10 VM:

1. Install all Windows updates (why this response took so long)
2. Install git for windows, using the defaults most of the time (I disabled checkout in Windows line ending format)
3. Install python for windows, using the defaults plus the following two options:
   i. Put python.exe on the PATH
   ii. Enabled support for filesystem paths >260 chars long
4. Fired up the git bash prompt (not cmd)
5. python -m venv venv
6. source venv/Scripts/activate
7. pip install git+https://github.com/alantech/marsha
8. git clone https://github.com/alantech/marsha
9. cd marsha/examples/general-purpose
10. export OPENAI_ORG=..., export OPENAI_SECRET_KEY=...
11. python -m marsha fibonacci.mrsh
12. python -m fibonacci 6 (and etc)