OKHttp instrumentation metrics high cardinality #10535
Replies: 4 comments 5 replies
-
@danielbanks there are known issues with Trino JDBC, which uses okhttp, and https://github.com/open-telemetry/opentelemetry-java-contrib/tree/main/micrometer-meter-provider that could cause this, are you using either of these? |
Beta Was this translation helpful? Give feedback.
-
@laurit thanks for the quick reply :) I don't think I'm using what you're referring to. Specifically, I'm using this as part of the OKHttp Instrumentation lib. Under the hood it appears to be using this. Does this make sense? |
Beta Was this translation helpful? Give feedback.
-
OK, so thanks to your reply I found where HTTP_URL is being set and it seems it is outdated code, but looking at URL_FULL won't this also have high cardinality for urls containing GUIDs? The URL_FULL doesn't seem to be sanitised further than the authority.
|
Beta Was this translation helpful? Give feedback.
-
OK thanks for your feedback :). So it seems an update to semconv 2.0.0 will resolve my issue. Then you're suggesting that if I want to include the url in a redacted form I would need to add a view to the metric and provide my own implementation there, is that correct? |
Beta Was this translation helpful? Give feedback.
-
Hi all,
The default OKHttp instrumentation client metrics can result in high cardinality and cost. Specifically, I'm referring to the metrics labeled
http_client_duration_millisecond
.These metrics include a label of
http_url
, which has values of the exact URLs called by the client. For URLs that contain GUIDS or unique identifiers in the path, very high numbers of time series are generated, leading to high costs.Our current solution is to filter these metrics in our collector. However, I see a lot of value in them if we can tweak the implementation slightly.
Is it possible to intercept and rename these labels to replace the GUIDs with a character such as
*
? Or better yet do this by default in the lib? I think that many people will experience this issue.Thanks in advance for your reply!
Dan
Beta Was this translation helpful? Give feedback.
All reactions