Prometheus Elasticsearch query exporter

The exporter runs Elasticsearch queries and transforms query results to Prometheus metrics.

Queries

Queries are defined in own files similar to how alerts rules are defined in Prometheus.

They are divided by evaluation mode into two types:

• Scheduled queries are evaluated at a certain intervals in background. When Prometheus comes for metrics the last evaluated result retured.
• On-demand queries are evaluated on scrape request.

Also queries are divided by type:

• Raw - contains raw Elasticsearch request with minimal configuration.
• Default - contains only query Elasticsearch request part but provides more convenient configuration options.

Query results automatically turn into a series of metrics.

Metrics

Name	Description	Type	Labels
elastic_query_hits	Number of records satisfied by query	Gauge	query
elastic_query_duration_milliseconds	Query execution Time in milliseconds	Gauge	query
elastic_query_exceptions_total	Total number of exceptions occured on query evaluation	Counter	query
elastic_query_timeouts_total	Total number of timeouts occured on query evaluation	Counter	query
elastic_query_value_aggregation	Elasticsearch single-value aggregation result	Counter	query, aggregation

Aggregations support

Aggregations partially supported only for raw queries. Supported single-value aggregations:

1. Min
2. Max
3. Sum
4. Cardinality
5. Average
6. ValueCount
7. AverageBucket
8. Derivative
9. SumBucket
10. MovingAverage
11. CumulativeSum
12. CumulativeCardinality
13. BucketScript
14. SerialDifferencing
15. WeightedAverage
16. MaxBucket (Limited: keys are not exposed as metric part)
17. MinBucket (Limited: keys are not exposed as metric part)
18. MedianAbsoluteDeviation
19. T-Test

Quickstart

1. Create configuration file appsettings.Release.yaml and query files.
2. Run exporter as docker container

Replace:

• ${CONFIG_PATH} with path to appsettings.Release.yaml;
• ${QUERIES_PATH} with path to queries directory;
• ${HOST_PORT} with desired host port where you want to run exporter.

docker run --rm --name es-query-exporter \
   -v ${CONFIG_PATH}:/app/appsettings.Release.yaml \
   -v ${QUERIES_PATH}:/app/queries -p ${HOST_PORT}:80 \
   -e ASPNETCORE_ENVIRONMENT=Release \
   bitshift/elastic-query-exporter:latest

Note: with the presented startup format, query files will be available at app/queries dir, so in configuration file same file path should be specified.

Other launch formats and configuration files are presented in the examples. See documentation for configuration details.

Endpoints

Exporter provides several endpoints:

1. /metrics - Prometheus metrics endpoint
2. /scheduler - Scheduler dashboard.
3. /health/live - Liveness probe.
4. /health/ready - Readiness probe.