Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

New Azure Functions logs format? #11729

Open
zmoog opened this issue Nov 13, 2024 · 0 comments
Open

New Azure Functions logs format? #11729

zmoog opened this issue Nov 13, 2024 · 0 comments
Assignees
Labels
Integration:azure Azure Logs Team:obs-ds-hosted-services Label for the Observability Hosted Services team [elastic/obs-ds-hosted-services]

Comments

@zmoog
Copy link
Contributor

zmoog commented Nov 13, 2024

Users reported receiving a different log format for Azure Functions.

Depending on the hosting plan, Azure Functions logs may contain invalid JSON documents like the following:

{
    "category": "FunctionAppLogs",
    "location": "useast",
    "properties": {
        'appName': 'myapp'
    }
}

However, on some deployments, it seems that Azure Functions now sends invalid JSON wrapped in a string:

{
    "category": "FunctionAppLogs",
    "location": "useast",
    "properties": "{'appName': 'myapp'}"
}

This makes the JSON document formally valid. However, the sanitizer won't change the single quote inside the JSON string.

When Azure wraps the invalid JSON in a string, the platformlogs pipeline fails with the following error:

expecting token of type [START_OBJECT] but found [VALUE_STRING] 
@zmoog zmoog added Team:obs-ds-hosted-services Label for the Observability Hosted Services team [elastic/obs-ds-hosted-services] Integration:azure Azure Logs labels Nov 13, 2024
@zmoog zmoog self-assigned this Nov 13, 2024
zmoog added a commit that referenced this issue Nov 14, 2024
… a string value (#11732)

Mitigates #11729

If the `azure.platformlogs.properties` field contains a string instead of the expected object, the pipeline renames it as `azure.platformlogs.properties.raw` field. 

This allows:

- Elasticsearch to index the log events instead of dropping them (status 400)
- Users to customize parsing of the `.raw` field in the `logs-azure.platformlogs@custom pipeline`

This avoids dropping log events while we build a complete and more robust [invalid JSON](#11729) handling solution with #11728
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Integration:azure Azure Logs Team:obs-ds-hosted-services Label for the Observability Hosted Services team [elastic/obs-ds-hosted-services]
Projects
None yet
Development

No branches or pull requests

1 participant