Skip to content

Commit

Permalink
SFN Troubleshooting Updates (#25634)
Browse files Browse the repository at this point in the history
* SFN Troubleshooting Updates

- Add note to recommend updating Step Functions and Lambda Forwarder
- Add warning for log pipelines

* Update troubleshooting.md

---------

Co-authored-by: Bryce Eadie <[email protected]>
  • Loading branch information
avedmala and buraizu authored Oct 8, 2024
1 parent 81d4134 commit 431b056
Showing 1 changed file with 9 additions and 2 deletions.
11 changes: 9 additions & 2 deletions content/en/serverless/step_functions/troubleshooting.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,8 +5,7 @@ title: Troubleshooting Serverless Monitoring for AWS Step Functions
## I cannot see any traces

### Verify that your Step Function is configured to send all logs

- Ensure that the `DD_TRACE_ENABLED` environment variable is set to `true` on the Lambda function in your AWS console.
- Ensure that the `DD_TRACE_ENABLED` tag is set to `true` on the Step Function in your AWS console.
- In your AWS console, open your Step Function's logging tab. Ensure that _Log level_ is set to `ALL`, and that _Include execution data_ is selected.
- Ensure that the CloudWatch log group (also found on the logging tab) has a subscription filter to the Datadog Lambda Forwarder in the same region.

Expand All @@ -30,6 +29,14 @@ If your organization has an existing all-encompassing index with a low limit, pl

**Note**: Indexing logs is not a requirement for getting traces and may incur additional cost. If you are troubleshooting a specific issue, you may wish to temporarily send logs to an index, debug, and delete the index afterwards. See [Indexes][6] for more information.

### Verify that your Step Function is using the latest version
- AWS may release updates to the Step Function API or introduce newer versions of the Step Function definitions. Older versions may result in unexpected log formatting or behavior.
- It's also recommended that you are using the latest version of the Datadog Lambda Forwarder to avoid discrepancies in how logs are forwarded.

### Caution when using custom log pipelines
- Custom log pipelines can offer flexibility in processing logs, but altering the log format too much can lead to issues downstream, such as logs not being parsed or recognized.
- Avoid making significant changes to the Step Function log structure that change the JSON format.

## Lambda traces are not merging with Step Function traces
- Verify that you can see both Lambda traces and Step Function traces in Datadog.
- Verify that you are using Python layer v95+ or Node.js layer v112+.
Expand Down

0 comments on commit 431b056

Please sign in to comment.