Managed Resources: Monitoring

Opni Manager

Controllers in the Opni Manager are responsible for reconciling the following resources:

• core.opni.io/MonitoringCluster (source)

  Deployed on the upstream cluster by the metrics backend plugin.

  Responsibilities:

  • Deploy and configure Cortex in the upstream cluster. Can deploy Cortex in either standalone mode (all components run in a single pod) or HA mode (components run in their own pods and scale individually).
  • Deploy and configure a Grafana custom resource (see below), which the Grafana Operator reconciles to deploy Grafana in the upstream cluster. The Grafana instance is automatically configured to connect to Opni, install plugins, datasources, and dashboards, and configure authentication.

  The Opni Cortex configuration can be found here. For reference, see Cortex Configuration File docs. The reconciliation logic for Cortex workloads and other associated Kubernetes resources can be found in pkg/resources/monitoring/cortex.

  The MonitoringCluster controller deploys Cortex in a very similar way to the Cortex Helm chart.

• grafana.opni.io/Grafana (source)

  (CRDs renamed from integreatly.org)

  Responsibilities:

  • Deploy and configure Grafana in the upstream cluster. This is handled by the Grafana Operator, whose controllers are embedded in the Opni manager.

  Grafana Operator Docs

  The Grafana resources are reconciled using logic in pkg/resources/monitoring/grafana.go. For configuration reference, see Grafana docs.

Prometheus Operator

Controllers in the Prometheus Operator, which is deployed on downstream clusters, are responsible for reconciling the following resources:

• monitoring.coreos.com/Prometheus (source)

  Responsibilities:

  • Deploy and configure a Prometheus instance in the downstream cluster. The metrics plugin configures a Prometheus agent-mode instance, which is configured to send remote-write requests to the agent.

Deployment of managed resources

Deployment of managed monitoring resources, namely the MonitoringCluster and Prometheus resources, is handled by the metrics backend plugin and metrics capability node plugin, respectively. The plugins are responsible for deploying the resources, and the Opni manager is responsible for reconciling them.

The code responsible for interacting with the Kubernetes cluster to deploy these resources is abstracted by a Cluster Driver interface on the gateway side, and an associated Node Driver interface on the agent side. These drivers can be swapped out for different implementations depending on the runtime environment. For example, during integration tests, the test framework installs and configures drivers that spawn subprocesses for Cortex (using all-in-one mode) and Prometheus agent.