Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Unable to run Promptflow in AWS LAmbda due to _multiprocessing.SemLock error #3922

Open
WillsonAtJHG opened this issue Jan 15, 2025 · 0 comments
Labels
bug Something isn't working

Comments

@WillsonAtJHG
Copy link

Is there a way we can disable multiprocessing in promptflow?
Looking at the stack trace the error happens when importing PFClient and particularly the tracing utils within it. I have tried setting PF_DISABLE_TRACING and PF_DISABLE_SERIALIZATION but they didn't seem to help. Same flow works fine in my local environment.

[ERROR] OSError: [Errno 38] Function not implemented
Traceback (most recent call last):
File "/var/lang/lib/python3.11/importlib/init.py", line 126, in import_module
return _bootstrap._gcd_import(name[level:], package, level)
File "", line 1204, in _gcd_import
File "", line 1176, in _find_and_load
File "", line 1147, in _find_and_load_unlocked
File "", line 690, in _load_unlocked
File "", line 940, in exec_module
File "", line 241, in _call_with_frames_removed
File "/var/task/lambda_function.py", line 4, in
from promptflow.client import PFClient
File "/var/lang/lib/python3.11/site-packages/promptflow/init.py", line 19, in
from promptflow._sdk._pf_client import PFClient as _PFClient
File "/var/lang/lib/python3.11/site-packages/promptflow/_sdk/_pf_client.py", line 24, in
from .operations import RunOperations
File "/var/lang/lib/python3.11/site-packages/promptflow/_sdk/operations/init.py", line 7, in
from ._flow_operations import FlowOperations
File "/var/lang/lib/python3.11/site-packages/promptflow/_sdk/operations/_flow_operations.py", line 36, in
from promptflow._sdk._orchestrator import TestSubmitter
File "/var/lang/lib/python3.11/site-packages/promptflow/_sdk/_orchestrator/init.py", line 5, in
from .test_submitter import TestSubmitter
File "/var/lang/lib/python3.11/site-packages/promptflow/_sdk/_orchestrator/test_submitter.py", line 15, in
from promptflow._internal import ConnectionManager
File "/var/lang/lib/python3.11/site-packages/promptflow/_internal/init.py", line 52, in
from promptflow._sdk._service.apis.collector import trace_collector
File "/var/lang/lib/python3.11/site-packages/promptflow/_sdk/_service/apis/collector.py", line 17, in
from promptflow._sdk._tracing import _is_azure_ext_installed, process_otlp_trace_request
File "/var/lang/lib/python3.11/site-packages/promptflow/_sdk/_tracing.py", line 61, in
from promptflow._sdk._utilities.tracing_utils import get_workspace_kind, parse_kv_from_pb_attribute, parse_protobuf_span
File "/var/lang/lib/python3.11/site-packages/promptflow/_sdk/_utilities/tracing_utils.py", line 380, in
_telemetry_helper = TraceTelemetryHelper()
File "/var/lang/lib/python3.11/site-packages/promptflow/_sdk/_utilities/tracing_utils.py", line 353, in init
self._lock = multiprocessing.Lock()
File "/var/lang/lib/python3.11/multiprocessing/context.py", line 68, in Lock
return Lock(ctx=self.get_context())
File "/var/lang/lib/python3.11/multiprocessing/synchronize.py", line 169, in init
SemLock.init(self, SEMAPHORE, 1, 1, ctx=ctx)
File "/var/lang/lib/python3.11/multiprocessing/synchronize.py", line 57, in init
sl = self._semlock = _multiprocessing.SemLock(

@WillsonAtJHG WillsonAtJHG added the bug Something isn't working label Jan 15, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

1 participant