Skip to content

Commit

Permalink
feat(langgraph): submit spans from langgraph to APM and LLMObs (#11730)
Browse files Browse the repository at this point in the history
This PR adds a simple integration and corresponding LLMObs plugin for
the LangGraph framework.

### Enablement

For both APM and LLMObs, this integration is disabled by default, and
only enabled with `DD_TRACE_LANGGRAPH_ENABLED=true` or
`patch(langgraph=True)`, along with `_DD_TRACE_LANGGRAPH_ENABLED=true`,
due to its experimental nature.

### APM Tracing

The APM integration adds basic tracing for LangGraph `Pregel` graph
invocations, as well as the execution of each node/step in the graph
(traced as `RunnableSeq`).

### LLM Observability Plugin

The LLMObs plugin submits the traces to LLM Observability.

#### Graph Invocation/Streaming Output Tagging

Because we patch the `(a)stream` method for both streams and
`(a)invoke`, since they rely on `(a)stream` under the hood, we are bound
to the yields from the stream. The object shape and value depend on the
`stream_mode` passed in to the stream function. For `invoke`, it
defaults to `"values"`, which is perfect for our use case. However, if
one were to call `stream` directly, and either pass in nothing or
something like `stream_mode=["updates", "debug"]`, the output we would
have for the graph span would appear as:

```
{
  "type": "task_result",
  "timestamp": "2025-01-03T14:45:26.087608+00:00",
  "step": "3",
  "payload": {
    "id": "399cfe0a-aefd-f67c-6909-fcea80ef508b",
    "name": "e",
    "error": "None",
    "result": [
      "('a_list', ['e'])"
    ],
    "interrupts": []
  }
}
```

This is a known limitation for now, and a more thought-out solution for
either adding on `"values"` to `stream_mode` and omitting their yields
if it wasn't already present, or having some kind of clever joining, can
be added in a later PR or by feature request.

## Checklist
- [x] PR author has checked that all the criteria below are met
- The PR description includes an overview of the change
- The PR description articulates the motivation for the change
- The change includes tests OR the PR description describes a testing
strategy
- The PR description notes risks associated with the change, if any
- Newly-added code is easy to change
- The change follows the [library release note
guidelines](https://ddtrace.readthedocs.io/en/stable/releasenotes.html)
- The change includes or references documentation updates if necessary
- Backport labels are set (if
[applicable](https://ddtrace.readthedocs.io/en/latest/contributing.html#backporting))

## Reviewer Checklist
- [x] Reviewer has checked that all the criteria below are met 
- Title is accurate
- All changes are related to the pull request's stated goal
- Avoids breaking
[API](https://ddtrace.readthedocs.io/en/stable/versioning.html#interfaces)
changes
- Testing strategy adequately addresses listed risks
- Newly-added code is easy to change
- Release note makes sense to a user of the library
- If necessary, author has acknowledged and discussed the performance
implications of this PR as reported in the benchmarks PR comment
- Backport labels are set in a manner that is consistent with the
[release branch maintenance
policy](https://ddtrace.readthedocs.io/en/latest/contributing.html#backporting)

---------

Co-authored-by: Yun Kim <[email protected]>
Co-authored-by: Yun Kim <[email protected]>
  • Loading branch information
3 people authored Jan 17, 2025
1 parent 6141812 commit b614655
Show file tree
Hide file tree
Showing 23 changed files with 1,314 additions and 39 deletions.
3 changes: 3 additions & 0 deletions .github/CODEOWNERS
Original file line number Diff line number Diff line change
Expand Up @@ -145,6 +145,8 @@ ddtrace/contrib/internal/google_generativeai @DataDog/ml-observabilit
ddtrace/contrib/google_generativeai @DataDog/ml-observability
ddtrace/contrib/internal/vertexai @DataDog/ml-observability
ddtrace/contrib/vertexai @DataDog/ml-observability
ddtrace/contrib/langgraph @DataDog/ml-observability
ddtrace/contrib/internal/langgraph @DataDog/ml-observability
tests/llmobs @DataDog/ml-observability
tests/contrib/openai @DataDog/ml-observability
tests/contrib/langchain @DataDog/ml-observability
Expand All @@ -154,6 +156,7 @@ tests/contrib/botocore/bedrock_cassettes @DataDog/ml-observabilit
tests/contrib/anthropic @DataDog/ml-observability
tests/contrib/google_generativeai @DataDog/ml-observability
tests/contrib/vertexai @DataDog/ml-observability
tests/contrib/langgraph @DataDog/ml-observability
.gitlab/tests/llmobs.yml @DataDog/ml-observability

# Remote Config
Expand Down
42 changes: 42 additions & 0 deletions .riot/requirements/115283d.txt
Original file line number Diff line number Diff line change
@@ -0,0 +1,42 @@
#
# This file is autogenerated by pip-compile with Python 3.12
# by the following command:
#
# pip-compile --allow-unsafe --no-annotate .riot/requirements/115283d.in
#
anyio==4.7.0
attrs==24.3.0
certifi==2024.12.14
charset-normalizer==3.4.1
coverage[toml]==7.6.10
h11==0.14.0
httpcore==1.0.7
httpx==0.28.1
hypothesis==6.45.0
idna==3.10
iniconfig==2.0.0
jsonpatch==1.33
jsonpointer==3.0.0
langchain-core==0.3.28
langgraph==0.2.60
langgraph-checkpoint==2.0.9
langgraph-sdk==0.1.48
langsmith==0.2.6
mock==5.1.0
msgpack==1.1.0
opentracing==2.4.0
orjson==3.10.12
packaging==24.2
pluggy==1.5.0
pytest==8.3.4
pytest-asyncio==0.25.0
pytest-cov==6.0.0
pytest-mock==3.14.0
pyyaml==6.0.2
requests==2.32.3
requests-toolbelt==1.0.0
sniffio==1.3.1
sortedcontainers==2.4.0
tenacity==9.0.0
typing-extensions==4.12.2
urllib3==2.3.0
45 changes: 45 additions & 0 deletions .riot/requirements/18551a1.txt
Original file line number Diff line number Diff line change
@@ -0,0 +1,45 @@
#
# This file is autogenerated by pip-compile with Python 3.13
# by the following command:
#
# pip-compile --allow-unsafe --no-annotate .riot/requirements/18551a1.in
#
annotated-types==0.7.0
anyio==4.8.0
attrs==24.3.0
certifi==2024.12.14
charset-normalizer==3.4.1
coverage[toml]==7.6.10
h11==0.14.0
httpcore==1.0.7
httpx==0.28.1
hypothesis==6.45.0
idna==3.10
iniconfig==2.0.0
jsonpatch==1.33
jsonpointer==3.0.0
langchain-core==0.3.29
langgraph==0.2.61
langgraph-checkpoint==2.0.9
langgraph-sdk==0.1.48
langsmith==0.2.10
mock==5.1.0
msgpack==1.1.0
opentracing==2.4.0
orjson==3.10.14
packaging==24.2
pluggy==1.5.0
pydantic==2.10.5
pydantic-core==2.27.2
pytest==8.3.4
pytest-asyncio==0.25.2
pytest-cov==6.0.0
pytest-mock==3.14.0
pyyaml==6.0.2
requests==2.32.3
requests-toolbelt==1.0.0
sniffio==1.3.1
sortedcontainers==2.4.0
tenacity==9.0.0
typing-extensions==4.12.2
urllib3==2.3.0
45 changes: 45 additions & 0 deletions .riot/requirements/2bd0151.txt
Original file line number Diff line number Diff line change
@@ -0,0 +1,45 @@
#
# This file is autogenerated by pip-compile with Python 3.11
# by the following command:
#
# pip-compile --allow-unsafe --no-annotate --resolver=backtracking .riot/requirements/2bd0151.in
#
annotated-types==0.7.0
anyio==4.7.0
attrs==24.3.0
certifi==2024.12.14
charset-normalizer==3.4.0
coverage[toml]==7.6.9
h11==0.14.0
httpcore==1.0.7
httpx==0.28.1
hypothesis==6.45.0
idna==3.10
iniconfig==2.0.0
jsonpatch==1.33
jsonpointer==3.0.0
langchain-core==0.3.28
langgraph==0.2.60
langgraph-checkpoint==2.0.9
langgraph-sdk==0.1.48
langsmith==0.2.4
mock==5.1.0
msgpack==1.1.0
opentracing==2.4.0
orjson==3.10.12
packaging==24.2
pluggy==1.5.0
pydantic==2.10.4
pydantic-core==2.27.2
pytest==8.3.4
pytest-asyncio==0.25.0
pytest-cov==6.0.0
pytest-mock==3.14.0
pyyaml==6.0.2
requests==2.32.3
requests-toolbelt==1.0.0
sniffio==1.3.1
sortedcontainers==2.4.0
tenacity==9.0.0
typing-extensions==4.12.2
urllib3==2.3.0
47 changes: 47 additions & 0 deletions .riot/requirements/2e4fbd3.txt
Original file line number Diff line number Diff line change
@@ -0,0 +1,47 @@
#
# This file is autogenerated by pip-compile with Python 3.9
# by the following command:
#
# pip-compile --allow-unsafe --no-annotate --resolver=backtracking .riot/requirements/2e4fbd3.in
#
annotated-types==0.7.0
anyio==4.7.0
attrs==24.3.0
certifi==2024.12.14
charset-normalizer==3.4.1
coverage[toml]==7.6.10
exceptiongroup==1.2.2
h11==0.14.0
httpcore==1.0.7
httpx==0.28.1
hypothesis==6.45.0
idna==3.10
iniconfig==2.0.0
jsonpatch==1.33
jsonpointer==3.0.0
langchain-core==0.3.28
langgraph==0.2.60
langgraph-checkpoint==2.0.9
langgraph-sdk==0.1.48
langsmith==0.2.6
mock==5.1.0
msgpack==1.1.0
opentracing==2.4.0
orjson==3.10.12
packaging==24.2
pluggy==1.5.0
pydantic==2.10.4
pydantic-core==2.27.2
pytest==8.3.4
pytest-asyncio==0.25.0
pytest-cov==6.0.0
pytest-mock==3.14.0
pyyaml==6.0.2
requests==2.32.3
requests-toolbelt==1.0.0
sniffio==1.3.1
sortedcontainers==2.4.0
tenacity==9.0.0
tomli==2.2.1
typing-extensions==4.12.2
urllib3==2.3.0
47 changes: 47 additions & 0 deletions .riot/requirements/7dd4bcd.txt
Original file line number Diff line number Diff line change
@@ -0,0 +1,47 @@
#
# This file is autogenerated by pip-compile with Python 3.10
# by the following command:
#
# pip-compile --allow-unsafe --no-annotate .riot/requirements/7dd4bcd.in
#
annotated-types==0.7.0
anyio==4.7.0
attrs==24.3.0
certifi==2024.12.14
charset-normalizer==3.4.1
coverage[toml]==7.6.10
exceptiongroup==1.2.2
h11==0.14.0
httpcore==1.0.7
httpx==0.28.1
hypothesis==6.45.0
idna==3.10
iniconfig==2.0.0
jsonpatch==1.33
jsonpointer==3.0.0
langchain-core==0.3.28
langgraph==0.2.60
langgraph-checkpoint==2.0.9
langgraph-sdk==0.1.48
langsmith==0.2.6
mock==5.1.0
msgpack==1.1.0
opentracing==2.4.0
orjson==3.10.12
packaging==24.2
pluggy==1.5.0
pydantic==2.10.4
pydantic-core==2.27.2
pytest==8.3.4
pytest-asyncio==0.25.0
pytest-cov==6.0.0
pytest-mock==3.14.0
pyyaml==6.0.2
requests==2.32.3
requests-toolbelt==1.0.0
sniffio==1.3.1
sortedcontainers==2.4.0
tenacity==9.0.0
tomli==2.2.1
typing-extensions==4.12.2
urllib3==2.3.0
5 changes: 5 additions & 0 deletions ddtrace/_monkey.py
Original file line number Diff line number Diff line change
Expand Up @@ -49,6 +49,7 @@
"grpc": True,
"httpx": True,
"kafka": True,
"langgraph": False,
"mongoengine": True,
"mysql": True,
"mysqldb": True,
Expand Down Expand Up @@ -149,6 +150,10 @@
"httplib": ("http.client",),
"kafka": ("confluent_kafka",),
"google_generativeai": ("google.generativeai",),
"langgraph": (
"langgraph",
"langgraph.graph",
),
}


Expand Down
Loading

0 comments on commit b614655

Please sign in to comment.