support _in operator with string and update readme

hasura · Sep 24, 2024 · c585147 · c585147
1 parent 4e02489
commit c585147
Show file tree

Hide file tree

Showing 378 changed files with 7,350 additions and 478,711 deletions.
diff --git a/README.md b/README.md
@@ -6,12 +6,105 @@ This connector is built using the [Go Data Connector SDK](https://github.com/has
 
 ## Features
 
-### Instant Metrics
+### Metrics Collection
 
-The connector can introspect and automatically transform available metrics on the Prometheus server to instant collection queries.
+#### How it works
+
+The connector can introspect and automatically transform available metrics on the Prometheus server to collection queries. Each collection has the common structure:
+
+```gql
+{
+  <label_1>
+  <label_2>
+  # ...
+  timestamp
+  value
+  labels
+  values {
+    timestamp
+    value
+  }
+}
+```
+
+> [!NOTE]
+> Labels and metrics are introspected from the Prometheus server at the current time. You need to introspect again whenever there are new labels or metrics.
+
+The configuration plugin introspects labels of each metric and defines them as collection columns that enable the ability of Hasura permissions and remote join. The connector supports basic comparison filters for labels.
+
+```gql
+{
+  process_cpu_seconds_total(
+    where: {
+      timestamp: { _gt: "2024-09-24T10:00:00Z" }
+      job: {
+        _eq: "node"
+        _neq: "prometheus"
+        _in: ["node", "prometheus"]
+        _nin: ["ndc-prometheus"]
+        _regex: "prometheus.*"
+        _nregex: "foo.*"
+      }
+    }
+    args: { step: "5m", offset: "5m", timeout: "30s" }
+  ) {
+    job
+    instance
+    timestamp
+    value
+    values {
+      timestamp
+      value
+    }
+  }
+}
+```
+
+The connector can detect if you want to request instant or range query via the `timestamp` column:
+
+- `_eq`: instant query at the exact timestamp.
+- `_gt` < `_lt`: range query.
+
+The range query mode is default If none of timestamp operator is set.
+
+#### Common arguments
+
+- `step`: the query resolution step width in duration format or float number of seconds. The step should be explicitly set for range queries. Even though the connector can estimate the approximate step width the result may be empty due to too far interval.
+- `offset`: the offset modifier allows changing the time offset for individual instant and range vectors in a query.
+- `timeout`: the evaluation timeout of the request.
+- `fn`: the array of composable PromQL functions.
+
+#### Aggregation
+
+The `fn` argument is an array of [PromQL function](https://prometheus.io/docs/prometheus/latest/querying/functions/) parameters. You can set multiple functions that can be composed into the query. For example, with this PromQL query:
+
+```
+sum by (job) (rate(process_cpu_seconds_total[1m]))
+```
+
+The equivalent GraphQL query will be:
+
+```gql
+{
+  process_cpu_seconds_total(
+    where: { timestamp: { _gt: "2024-09-24T10:00:00Z" } }
+    args: { step: "5m", fn: [{ rate: "5m" }, { sum: [job] }] }
+  ) {
+    job
+    timestamp
+    value
+    values {
+      timestamp
+      value
+    }
+  }
+}
+```
 
 ### Native Query
 
+#### How it works
+
 When simple queries don't meet your need you can define native queries in [the configuration file](./tests/configuration/configuration.yaml) with prepared variables with the `${<name>}` template.
 
 ```yaml
@@ -20,14 +113,103 @@ metadata:
     queries:
       service_up:
         query: up{job="${job}", instance="${instance}"}
-        labels: {}
+        labels:
+          instance: {}
+          job: {}
         arguments:
           instance:
             type: String
           job:
             type: String
 ```
 
+The native query is exposed as a read-only function with 2 required fields `job` and `instance`.
+
+```gql
+{
+  service_up(
+    start: "2024-09-24T00:00:00Z"
+    job: "node"
+    instance: "localhost:9090"
+  ) {
+    timestamp
+    value
+    labels
+    values {
+      value
+      timestamp
+    }
+  }
+}
+```
+
+> [!NOTE]
+> Labels aren't automatically added. You need to define them manually.
+
+#### Common arguments
+
+- `start` & `end`: time range arguments for the range query.
+- `time`: Evaluation timestamp. Use this argument if you want to run an instant query.
+- `step`: the query resolution step width in duration format or float number of seconds. The step should be explicitly set for range queries. Even though the connector can estimate the approximate step width the result may be empty due to too far interval.
+- `timeout`: the evaluation timeout of the request.
+
 ### Prometheus APIs
 
-WIP
+#### Raw PromQL query
+
+Execute a raw PromQL query directly. This API should be used by admin only. The result contains labels and values only.
+
+```gql
+{
+  promql_query(
+    query: "process_cpu_seconds_total{job=\"node\"}"
+    start: "2024-09-24T10:00:00Z"
+    step: "5m"
+  ) {
+    labels
+    values {
+      timestamp
+      value
+    }
+  }
+}
+```
+
+#### Metadata APIs
+
+| Operation Name              | REST Prometheus API                                                                                                   |
+| --------------------------- | --------------------------------------------------------------------------------------------------------------------- |
+| prometheus_alertmanagers    | [/api/v1/alertmanagers](https://prometheus.io/docs/prometheus/latest/querying/api/#alertmanagers)                     |
+| prometheus_alerts           | [/api/v1/alerts](https://prometheus.io/docs/prometheus/latest/querying/api/#alerts)                                   |
+| prometheus_label_names      | [/api/v1/labels](https://prometheus.io/docs/prometheus/latest/querying/api/#getting-label-names)                      |
+| prometheus_label_values     | [/api/v1/label/<label_name>/values](https://prometheus.io/docs/prometheus/latest/querying/api/#querying-label-values) |
+| prometheus_rules            | [/api/v1/rules](https://prometheus.io/docs/prometheus/latest/querying/api/#rules)                                     |
+| prometheus_series           | [/api/v1/series](https://prometheus.io/docs/prometheus/latest/querying/api/#finding-series-by-label-matchers)         |
+| prometheus_targets          | [/api/v1/targets](https://prometheus.io/docs/prometheus/latest/querying/api/#targets)                                 |
+| prometheus_targets_metadata | [/api/v1/targets/metadata](https://prometheus.io/docs/prometheus/latest/querying/api/#querying-target-metadata)       |
+
+## Development
+
+### Get started
+
+#### Start Docker services
+
+```sh
+docker composes up -d
+```
+
+#### Introspect the configuration file
+
+```sh
+make generate-test-config
+docker compose restart ndc-prometheus
+```
+
+#### Introspect and build DDN metadata
+
+```sh
+make build-supergraph-test
+docker compose up -d --build engine
+```
+
+Browse the engine console at http://localhost:3000.