Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add fine control on what consumer groups metrics are exposed to reduce scraped metrics in large clusters #108

Open
amuraru opened this issue Aug 3, 2021 · 0 comments

Comments

@amuraru
Copy link
Contributor

amuraru commented Aug 3, 2021

For large clusters with tens of thousands of consumer groups, the number of metrics exposed may be quite large.
This in turn poses extra pressure on scraping Prometheus as all these series names need to kept in RAM.

Few suggestions:

  • add flag to disable exposing metrics for inactive consumer groups (Empty or Dead)
  • List offsets for Stable consumer groups only : reduce the number of metrics for clusters where lots are consumer groups are short-lived and empty.
  • Report consumer_group_empty_members metrics for stable groups only

See #105 (comment)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant