Connection to Application Insights breaks on using PFClient.run() versus flow-as-a-function

[Flowhill]

Discussed in #3921

^{Originally posted by Flowhill January 14, 2025}
I'm having trouble when switching starting my flow using the Promptflow SDK from flow-as-a-function to PFClient.run()
I want to start using the PFClient because it allows for logging the system metrics such as the prompt_tokens and completions_tokens.

I initialize my Application insights through this code

if os.getenv("APPLICATIONINSIGHTS_CONNECTION_STRING"):
    configure_azure_monitor(
        connection_string=os.getenv("APPLICATIONINSIGHTS_CONNECTION_STRING")
    )

And in the instance of flow-as-a-function I run the flow like this

flow = 'path/to/flow/folder'
f = load_flow(flow)
utterance = request.message.content
try:
    if contains_json_html_structure(utterance):
        raise ValueError("Utterance contains json/html-like values")  # noqa: E501
    # post the request and get the result
    result = f(
        webhook_input=request.webhook_input,
        session_id=completion_input.context.message.session_id,
        role=completion_input.context.message.role,
        # Filter JSON elements out of utterance
        utterance=utterance,
    )

    result = result["output"]
    # Log results for application insights
    logger.info(json.dumps(result))
except Exception as e:
    <....>

In this case all my logs both in python code in the flow as well as after get neatly uploaded to Application Insights

If I replace the flow-as-a-function with a PFClient.run() like so:

flow = 'path/to/flow/folder'
pf = PFClient()
<... preparing data.jsonl file ...>
# Run the flow
run = pf.run(
    flow=flow,
    data="data.jsonl",
    column_mapping={
        "webhook_input": "${data.webhook_input}",
        "session_id": "${data.session_id}",
        "role": "${data.role}",
        "utterance": "${data.utterance}",
    },
)

logger.debug(pf.get_details(run))

Then the logs stop as soon as the line run = pf.run(... starts. What also changes in the output in terminal is the following additions:

• Multiple instances of trace detail URL being created as such: You can view the trace detail from the following URL: http://127.0.0.1:23336/v1.0/ui/traces/?#collection=src&uiTraceId=0xb0e9328dedd7df8f0e29fe305d371f24 with differing TraceId's
• At the beginning and end of a flow this ERROR appears: 2025-01-14 15:30:45 +0100 azure.monitor.opentelemetry.exporter.export._base:_transmit ERROR Non-retryable server side error: Operation returned an invalid status 'Bad Request'.

Package versions for promptflow and telemetry related packages is:

promptflow                               1.17.1
promptflow-azure                         1.17.1
promptflow-core                          1.17.1
promptflow-devkit                        1.17.1
promptflow-tools                         1.4.0
promptflow-tracing                       1.17.1
openai                                   1.59.7
azure-core-tracing-opentelemetry         1.0.0b11
azure-monitor-opentelemetry              1.6.4
azure-monitor-opentelemetry-exporter     1.0.0b32
opentelemetry-api                        1.29.0
opentelemetry-exporter-otlp-proto-common 1.29.0
opentelemetry-exporter-otlp-proto-http   1.29.0
opentelemetry-instrumentation            0.50b0
opentelemetry-instrumentation-asgi       0.50b0
opentelemetry-instrumentation-dbapi      0.50b0
opentelemetry-instrumentation-django     0.50b0
opentelemetry-instrumentation-fastapi    0.50b0
opentelemetry-instrumentation-flask      0.50b0
opentelemetry-instrumentation-psycopg2   0.50b0
opentelemetry-instrumentation-requests   0.50b0
opentelemetry-instrumentation-urllib     0.50b0
opentelemetry-instrumentation-urllib3    0.50b0
opentelemetry-instrumentation-wsgi       0.50b0
opentelemetry-proto                      1.29.0
opentelemetry-resource-detector-azure    0.1.5
opentelemetry-sdk                        1.29.0
opentelemetry-semantic-conventions       0.50b0
opentelemetry-util-http                  0.50b0

I have tried to configure Application Insights through the flow.dag.yaml as well by adding all combinations of

app_insights_enabled: true
environment_variables:
  APPLICATIONINSIGHTS_CONNECTION_STRING: <connection_string>

None of which seem to make a difference.

The flow and surrounding code is started through a call to a fastapi endpoint.