-
Notifications
You must be signed in to change notification settings - Fork 5.7k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[serve] Reduce cardinality for the route
metric tag
#48290
base: master
Are you sure you want to change the base?
Conversation
Signed-off-by: Edward Oakes <[email protected]>
Signed-off-by: Edward Oakes <[email protected]>
Signed-off-by: Edward Oakes <[email protected]>
Signed-off-by: Edward Oakes <[email protected]>
Signed-off-by: Edward Oakes <[email protected]>
Signed-off-by: Edward Oakes <[email protected]>
Signed-off-by: Edward Oakes <[email protected]>
Signed-off-by: Edward Oakes <[email protected]>
Signed-off-by: Edward Oakes <[email protected]>
Signed-off-by: Edward Oakes <[email protected]>
Signed-off-by: Edward Oakes <[email protected]>
route
metric tag route
metric tag
Signed-off-by: Edward Oakes <[email protected]>
Signed-off-by: Edward Oakes <[email protected]>
Signed-off-by: Edward Oakes <[email protected]>
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Some nits, but overall LGTM!
only Python primitive types. Vanilla `pickle` is much faster than cloudpickle. | ||
""" | ||
self.asgi_scope = pickle.loads(state["pickled_asgi_scope"]) | ||
self.receive_asgi_messages = state["receive_asgi_messages"] |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Non-blocker: We can probably create a named tuple for state object to ensure those required keys exist and accessible.
# NOTE(edoakes): we use the route_prefix instead of the full HTTP path | ||
# for logs & metrics to avoid high cardinality. | ||
# See: https://github.com/ray-project/ray/issues/47999 | ||
logs_and_metrics_route = ( |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Non-blocker: We can probably refactor this into ProxyRequest
and implement them in ASGIProxyRequest
and gRPCProxyRequest
differently
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
yeah I was highly confused at the fact that our "route" was just an app name for gRPC. the "method" field also seems to be getting overwritten to something else that actually looks like a route..
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yea, I think it depends on the context. HTTP we have methods such as "GET", "POST"...etc. gRPC has concept of service methods so we used in those in metrics/ logs. Also, in gRPC we use the app name to route so that's why the equivalent route_path
was set to app names.
Signed-off-by: Edward Oakes <[email protected]>
Signed-off-by: Edward Oakes <[email protected]>
Why are these changes needed?
Modifies the
route
attribute used for logs and metrics to limit its cardinality (see linked issue).After this change, the behavior will be:
route
will always be theroute_prefix
.route
will be theroute_prefix
+ the matched route string.Take the following example:
Previously, this could have infinite cardinality in the metrics tags (a unique tag value for each user). With this change, the tag would instead be:
"/prefix/{user_id}"
.The "not found" response has also been modified to not include a
route
tag.TODO:
get_asgi_route_name
test_metrics.py
Related issue number
Closes #47999
Checks
git commit -s
) in this PR.scripts/format.sh
to lint the changes in this PR.method in Tune, I've added it in
doc/source/tune/api/
under thecorresponding
.rst
file.