[Bug]: asyncio ERROR:Future exception was never retrieved

[juju4]

What happened?

I recently did a refresh of my ansible roles, among them one for lookyloo.
It was bumped to latest version (1.27.0, Ubuntu 24.04) but there is still some issue when asking for screenshot that I did not figure out for now.

$ tail -n50 /var/_lookyloo/lookyloo/logs/warning.log
[...]
2024-12-07 11:39:59,841 Pandora WARNING:Not up.
2024-12-07 11:40:09,369 asyncio ERROR:Task was destroyed but it is pending!
task: <Task pending name='Task-3' coro=<Connection.run.<locals>.init() running at /var/_lookyloo/.cache/pypoetry/virtualenvs/lookyloo-Lfrz0c9E-py3.12/lib/python3.12/site-packages/playwright/_impl/_connection.py:274> wait_for=<Future finished result=None> cb=[ProtocolCallback.__init__.<locals>.cb() at /var/_lookyloo/.cache/pypoetry/virtualenvs/lookyloo-Lfrz0c9E-py3.12/lib/python3.12/site-packages/playwright/_impl/_connection.py:195]>
2024-12-07 11:40:09,369 asyncio ERROR:Future exception was never retrieved
future: <Future finished exception=Exception('Connection closed while reading from the driver')>
Exception: Connection closed while reading from the driver
2024-12-07 11:40:29,289 asyncio ERROR:Task was destroyed but it is pending!
task: <Task pending name='Task-3' coro=<Connection.run.<locals>.init() running at /var/_lookyloo/.cache/pypoetry/virtualenvs/lookyloo-Lfrz0c9E-py3.12/lib/python3.12/site-packages/playwright/_impl/_connection.py:274> wait_for=<Future finished result=None> cb=[ProtocolCallback.__init__.<locals>.cb() at /var/_lookyloo/.cache/pypoetry/virtualenvs/lookyloo-Lfrz0c9E-py3.12/lib/python3.12/site-packages/playwright/_impl/_connection.py:195]>
2024-12-07 11:40:29,289 asyncio ERROR:Future exception was never retrieved
future: <Future finished exception=Exception('Connection closed while reading from the driver')>
Exception: Connection closed while reading from the driver
2024-12-07 11:40:49,076 asyncio ERROR:Task was destroyed but it is pending!

System is behind a proxy which is configured in lookyloo generic.json (global_proxy).
Is there any other place that needs to be configured for proxy?
Where else can I check to debug this?

Without proxy, it fails elsewhere per github action:
https://github.com/juju4/ansible-lookyloo/actions/runs/12215830703/job/34078270688#step:15:344

AttributeError: 'PlaywrightContextManager' object has no attribute '_playwright'

Possibly related to microsoft/playwright-python#2276

If the issue is related to the web interface, what browsers are you seeing the problem on?

No response

Relevant log output

No response

[Rafiot]

That's a confusing set of issues you're getting there, and I have no idea where to start tbh. All the issues seem to be within playwright, and not directly related to code in lacus/lookyloo/playwrightcapture. The only context I sometimes see asyncio exceptions like that is when the app is killed with signals instead of running stop. It for sure has nothing to do with proxies.

What do you mean by issue when asking for screenshot?

[juju4]

I mean submitting a request to a screen capture.
I will continue digging.
Thanks for the quick update

[juju4]

lookyloo-background_processing logs has more but not really easy to interpret

#
# Fatal error in , line 0
# Check failed: 12 == (*__errno_location ()).
#
#
#
#FailureMessage Object: 0x7ffdb1710b60
----- Native stack trace -----
  
 1: 0xd45071  [/var/_lookyloo/.cache/pypoetry/virtualenvs/lookyloo-Lfrz0c9E-py3.12/lib/python3.12/site-packages/playwright/driver/node]
 2: 0x218d121 V8_Fatal(char const*, ...) [/var/_lookyloo/.cache/pypoetry/virtualenvs/lookyloo-Lfrz0c9E-py3.12/lib/python3.12/site-packages/playwright/driver/node]
 3: 0x2197f47 v8::base::OS::SetPermissions(void*, unsigned long, v8::base::OS::MemoryPermission) [/var/_lookyloo/.cache/pypoetry/virtualenvs/lookyloo-Lfrz0c9E-py3.12/lib/python3.12/site-packages/playwright/driver/node] 
 4: 0x1180f9a v8::internal::MemoryAllocator::SetPermissionsOnExecutableMemoryChunk(v8::internal::VirtualMemory*, unsigned long, unsigned long, unsigned long) [/var/_lookyloo/.cache/pypoetry/virtualenvs/lookyloo-Lfrz0c9E-py3.12/lib/python3.12/site-packages/playwright/driver/node]
 5: 0x118124e v8::internal::MemoryAllocator::AllocateAlignedMemory(unsigned long, unsigned long, unsigned long, v8::internal::AllocationSpace, v8::internal::Executability, void*, v8::internal::VirtualMemory*) [/var/_lookyloo/.cache/pypoetry/virtualenvs/lookyloo-Lfrz0c9E-py3.12/lib/python3.12/site-packages/playwright/driver/node]
 6: 0x11814c4 v8::internal::MemoryAllocator::AllocateUninitializedChunkAt(v8::internal::BaseSpace*, unsigned long, v8::internal::Executability, unsigned long, v8::internal::PageSize) [/var/_lookyloo/.cache/pypoetry/virtualenvs/lookyloo-Lfrz0c9E-py3.12/lib/python3.12/site-packages/playwright/driver/node]
 7: 0x1181beb v8::internal::MemoryAllocator::AllocatePage(v8::internal::MemoryAllocator::AllocationMode, v8::internal::Space*, v8::internal::Executability) [/var/_lookyloo/.cache/pypoetry/virtualenvs/lookyloo-Lfrz0c9E-py3.12/lib/python3.12/site-packages/playwright/driver/node]
 8: 0x1196abc v8::internal::PagedSpaceBase::TryExpandImpl() [/var/_lookyloo/.cache/pypoetry/virtualenvs/lookyloo-Lfrz0c9E-py3.12/lib/python3.12/site-packages/playwright/driver/node]
 9: 0x11994e4  [/var/_lookyloo/.cache/pypoetry/virtualenvs/lookyloo-Lfrz0c9E-py3.12/lib/python3.12/site-packages/playwright/driver/node]
10: 0x1199a92 v8::internal::PagedSpaceBase::RefillLabMain(int, v8::internal::AllocationOrigin) [/var/_lookyloo/.cache/pypoetry/virtualenvs/lookyloo-Lfrz0c9E-py3.12/lib/python3.12/site-packages/playwright/driver/node]
11: 0x1105f65 v8::internal::HeapAllocator::AllocateRawWithLightRetrySlowPath(int, v8::internal::AllocationType, v8::internal::AllocationOrigin, v8::internal::AllocationAlignment) [/var/_lookyloo/.cache/pypoetry/virtualenvs/lookyloo-Lfrz0c9E-py3.12/lib/python3.12/site-packages/playwright/driver/node]
12: 0x10e2ea5 v8::internal::Factory::CodeBuilder::AllocateInstructionStream(bool) [/var/_lookyloo/.cache/pypoetry/virtualenvs/lookyloo-Lfrz0c9E-py3.12/lib/python3.12/site-packages/playwright/driver/node]
13: 0x10e327a v8::internal::Factory::CodeBuilder::BuildInternal(bool) [/var/_lookyloo/.cache/pypoetry/virtualenvs/lookyloo-Lfrz0c9E-py3.12/lib/python3.12/site-packages/playwright/driver/node]
14: 0xf5b2e7 v8::internal::baseline::BaselineCompiler::Build(v8::internal::LocalIsolate*) [/var/_lookyloo/.cache/pypoetry/virtualenvs/lookyloo-Lfrz0c9E-py3.12/lib/python3.12/site-packages/playwright/driver/node]
15: 0xf6b4a5 v8::internal::GenerateBaselineCode(v8::internal::Isolate*, v8::internal::Handle<v8::internal::SharedFunctionInfo>) [/var/_lookyloo/.cache/pypoetry/virtualenvs/lookyloo-Lfrz0c9E-py3.12/lib/python3.12/site-packages/playwright/driver/node]
16: 0xfc729b v8::internal::Compiler::CompileSharedWithBaseline(v8::internal::Isolate*, v8::internal::Handle<v8::internal::SharedFunctionInfo>, v8::internal::Compiler::ClearExceptionFlag, v8::internal::IsCompiledScope*) [/var/_lookyloo/.cache/pypoetry/virtualenvs/lookyloo-Lfrz0c9E-py3.12/lib/python3.12/site-packages/playwright/driver/node]
17: 0xfc77ed v8::internal::Compiler::CompileBaseline(v8::internal::Isolate*, v8::internal::Handle<v8::internal::JSFunction>, v8::internal::Compiler::ClearExceptionFlag, v8::internal::IsCompiledScope*) [/var/_lookyloo/.cache/pypoetry/virtualenvs/lookyloo-Lfrz0c9E-py3.12/lib/python3.12/site-packages/playwright/driver/node]
18: 0xf59a58 v8::internal::baseline::BaselineBatchCompiler::CompileBatch(v8::internal::Handle<v8::internal::JSFunction>) [/var/_lookyloo/.cache/pypoetry/virtualenvs/lookyloo-Lfrz0c9E-py3.12/lib/python3.12/site-packages/playwright/driver/node]
19: 0x109f674 v8::internal::TieringManager::OnInterruptTick(v8::internal::Handle<v8::internal::JSFunction>, v8::internal::CodeKind) [/var/_lookyloo/.cache/pypoetry/virtualenvs/lookyloo-Lfrz0c9E-py3.12/lib/python3.12/site-packages/playwright/driver/node]
20: 0x153e993  [/var/_lookyloo/.cache/pypoetry/virtualenvs/lookyloo-Lfrz0c9E-py3.12/lib/python3.12/site-packages/playwright/driver/node]
21: 0x1540184 v8::internal::Runtime_BytecodeBudgetInterruptWithStackCheck_Ignition(int, unsigned long*, v8::internal::Isolate*) [/var/_lookyloo/.cache/pypoetry/virtualenvs/lookyloo-Lfrz0c9E-py3.12/lib/python3.12/site-packages/playwright/driver/node]
22: 0x1979ef6  [/var/_lookyloo/.cache/pypoetry/virtualenvs/lookyloo-Lfrz0c9E-py3.12/lib/python3.12/site-packages/playwright/driver/node]
Traceback (most recent call last):
  File "/var/_lookyloo/lookyloo/bin/background_processing.py", line 164, in <module>
    main()
  File "/var/_lookyloo/lookyloo/bin/background_processing.py", line 159, in main
    p = Processing()
        ^^^^^^^^^^^^
  File "/var/_lookyloo/lookyloo/bin/background_processing.py", line 28, in __init__
    self.lookyloo = Lookyloo()
                    ^^^^^^^^^^
  File "/var/_lookyloo/lookyloo/lookyloo/lookyloo.py", line 84, in __init__
    self.user_agents = UserAgents()
                       ^^^^^^^^^^^^
  File "/var/_lookyloo/lookyloo/lookyloo/helpers.py", line 184, in __init__
    self.playwright_devices = get_devices()
                              ^^^^^^^^^^^^^
  File "/var/_lookyloo/.cache/pypoetry/virtualenvs/lookyloo-Lfrz0c9E-py3.12/lib/python3.12/site-packages/playwrightcapture/helpers.py", line 25, in get_devices
    playwright = sync_playwright().start()
                 ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/var/_lookyloo/.cache/pypoetry/virtualenvs/lookyloo-Lfrz0c9E-py3.12/lib/python3.12/site-packages/playwright/sync_api/_context_manager.py", line 84, in start
    return self.__enter__()
           ^^^^^^^^^^^^^^^^
  File "/var/_lookyloo/.cache/pypoetry/virtualenvs/lookyloo-Lfrz0c9E-py3.12/lib/python3.12/site-packages/playwright/sync_api/_context_manager.py", line 79, in __enter__
    playwright = self._playwright
                 ^^^^^^^^^^^^^^^^
AttributeError: 'PlaywrightContextManager' object has no attribute '_playwright'

doing the basic aka stupid extract, but it works

(lookyloo-py3.12) _lookyloo@lookyloo004:~$ cat test1.py 
#!/usr/bin/env python3

from playwrightcapture import get_devices

playwright_devices = get_devices()
(lookyloo-py3.12) _lookyloo@lookyloo004:~$ python3 test1.py ; echo $?
0

are there some global variables that could influence this function?

lookyloo-async_capture_message.log just has the asyncio error future exception with the 30s timeout.

cli playwright screenshot https://github.com /tmp/test.png is working under lookyloo user.
Is there a way to have a minimal reproduction test that can be shared with playwright team?

I also tried aside where no proxy is required and

• also got "AttributeError: 'PlaywrightContextManager' object has no attribute '_playwright'" in lookyloo-background_processing.log
• other errors

2024-12-16 01:09:47,478 AsyncCapture WARNING:Unable to setup the FOX module
2024-12-16 01:09:47,479 AsyncCapture INFO:Launching AsyncCapture
2024-12-16 01:32:50,842 AsyncCapture INFO:Max amount of captures in parallel reached (1)
2024-12-16 01:32:51,452 LacusCore ERROR:[84b5d3cd-c83f-4906-a87f-b44cdf0ca948] Something went poorly https://github.com - Capture.__aenter__: Invalid URL
Traceback (most recent call last):
  File "/var/_lookyloo/.cache/pypoetry/virtualenvs/lookyloo-Lfrz0c9E-py3.12/lib/python3.12/site-packages/lacuscore/lacuscore.py", line 485, in _capture
    async with Capture(
  File "/var/_lookyloo/.cache/pypoetry/virtualenvs/lookyloo-Lfrz0c9E-py3.12/lib/python3.12/site-packages/playwrightcapture/capture.py", line 228, in __aenter__
    self.browser = await self.playwright[self.browser_name].launch(
                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/var/_lookyloo/.cache/pypoetry/virtualenvs/lookyloo-Lfrz0c9E-py3.12/lib/python3.12/site-packages/playwright/async_api/_generated.py", line 14404, in launch
    await self._impl_obj.launch(
  File "/var/_lookyloo/.cache/pypoetry/virtualenvs/lookyloo-Lfrz0c9E-py3.12/lib/python3.12/site-packages/playwright/_impl/_browser_type.py", line 95, in launch
    Browser, from_channel(await self._channel.send("launch", params))
                          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/var/_lookyloo/.cache/pypoetry/virtualenvs/lookyloo-Lfrz0c9E-py3.12/lib/python3.12/site-packages/playwright/_impl/_connection.py", line 61, in send
    return await self._connection.wrap_api_call(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/var/_lookyloo/.cache/pypoetry/virtualenvs/lookyloo-Lfrz0c9E-py3.12/lib/python3.12/site-packages/playwright/_impl/_connection.py", line 528, in wrap_api_call
    raise rewrite_error(error, f"{parsed_st['apiName']}: {error}") from None
playwright._impl._errors.Error: Capture.__aenter__: Invalid URL
2024-12-16 01:32:51,453 LacusCore WARNING:[84b5d3cd-c83f-4906-a87f-b44cdf0ca948] Unable to capture https://github.com: Something went poorly https://github.com - Capture.__aenter__: Invalid URL
2024-12-16 01:32:51,846 AsyncCapture INFO:Got the capture for 84b5d3cd-c83f-4906-a87f-b44cdf0ca948 from Lacus
[...]
2024-12-16 01:45:39,021 Processing INFO:Initializing Processing
2024-12-16 01:45:39,418 asyncio ERROR:Task exception was never retrieved
future: <Task finished name='Task-3' coro=<Connection.run.<locals>.init() done, defined at /var/_lookyloo/.cache/pypoetry/virtualenvs/lookyloo-Lfrz0c9E-py3.12/lib/python3.12/site-packages/playwright/_impl/_connection.py:276> exception=Exception('Connection.init: Connection closed while reading from the driver')>
Traceback (most recent call last):
  File "/var/_lookyloo/.cache/pypoetry/virtualenvs/lookyloo-Lfrz0c9E-py3.12/lib/python3.12/site-packages/playwright/_impl/_connection.py", line 277, in init
    self.playwright_future.set_result(await self._root_object.initialize())
                                      ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/var/_lookyloo/.cache/pypoetry/virtualenvs/lookyloo-Lfrz0c9E-py3.12/lib/python3.12/site-packages/playwright/_impl/_connection.py", line 219, in initialize
    await self._channel.send(
  File "/var/_lookyloo/.cache/pypoetry/virtualenvs/lookyloo-Lfrz0c9E-py3.12/lib/python3.12/site-packages/playwright/_impl/_connection.py", line 61, in send
    return await self._connection.wrap_api_call(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/var/_lookyloo/.cache/pypoetry/virtualenvs/lookyloo-Lfrz0c9E-py3.12/lib/python3.12/site-packages/playwright/_impl/_connection.py", line 528, in wrap_api_call
    raise rewrite_error(error, f"{parsed_st['apiName']}: {error}") from None
Exception: Connection.init: Connection closed while reading from the driver

I also tried to disable the systemd restrictions that are included in my role, just in case, but that does not change anything.

[Rafiot]

I'm running lookyloo, on anything from Ubuntu 22.04 to 24.10 and have absolutely no issue. The one you're having is pretty fondamental and I have no idea where it can come from.

All the mentions of similar errors in the issues of playwright seem to be linked to somewhat broken virtual environments and I have no clue how ansible is doing all that.
Are you absolutely sure ansible is properly running the scripts from right python environment? And the virtual environment is not moved after it is created or anything along these lines?