Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[azure] rename azure.platformlogs.properties as .raw when it contains a string value #11732

Merged

Conversation

zmoog
Copy link
Contributor

@zmoog zmoog commented Nov 13, 2024

Proposed commit message

Mitigates #11729

If the azure.platformlogs.properties field contains a string instead of the expected object, the pipeline renames it as azure.platformlogs.properties.raw field.

This allows:

  • Elasticsearch to index the log events instead of dropping them (status 400)
  • Users to customize parsing of the .raw field in the logs-azure.platformlogs@custom pipeline

This avoids dropping log events while we build a complete and more robust invalid JSON handling solution with #11728

Checklist

  • I have reviewed tips for building integrations and this pull request is aligned with them.
  • I have verified that all data streams collect metrics or logs.
  • I have added an entry to my package's changelog.yml file.
  • I have verified that Kibana version constraints are current according to guidelines.
  • I have verified that any added dashboard complies with Kibana's Dashboard good practices

Related issues

If the azure.platformlogs.properties field contains a string, the
pipeline renames it a azure.platformlogs.properties.raw, so users can
customize how to parse it in the logs-azure.platformlogs@custom
pipeline.
@zmoog zmoog changed the title [azure] rename azure.platformlogs.properties as .raw when it contains a string [azure] rename azure.platformlogs.properties as .raw when it contains a string value Nov 13, 2024
@zmoog zmoog added Integration:azure Azure Logs bugfix Pull request that fixes a bug issue Team:obs-ds-hosted-services Label for the Observability Hosted Services team [elastic/obs-ds-hosted-services] labels Nov 13, 2024
@zmoog zmoog self-assigned this Nov 13, 2024
@elastic-vault-github-plugin-prod
Copy link

elastic-vault-github-plugin-prod bot commented Nov 13, 2024

🚀 Benchmarks report

Package azure 👍(8) 💚(2) 💔(1)

Expand to view
Data stream Previous EPS New EPS Diff (%) Result
platformlogs 5649.72 4651.16 -998.56 (-17.67%) 💔

To see the full report comment with /test benchmark fullreport

@zmoog zmoog marked this pull request as ready for review November 13, 2024 22:34
@zmoog zmoog requested review from a team as code owners November 13, 2024 22:34
@elasticmachine
Copy link

💚 Build Succeeded

cc @zmoog

@zmoog zmoog merged commit baa3937 into elastic:main Nov 14, 2024
5 checks passed
@zmoog zmoog deleted the zmoog/azure-logs-platformlogs-fallback-as-raw branch November 14, 2024 17:12
@elastic-vault-github-plugin-prod

Package azure - 1.19.2 containing this change is available at https://epr.elastic.co/package/azure/1.19.2/

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bugfix Pull request that fixes a bug issue Integration:azure Azure Logs Team:obs-ds-hosted-services Label for the Observability Hosted Services team [elastic/obs-ds-hosted-services]
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants