Skip to content

Latest commit

 

History

History
95 lines (73 loc) · 4.16 KB

README.md

File metadata and controls

95 lines (73 loc) · 4.16 KB
Raw

logs-to-vector

A log collection agent that ships logs from the balena engine to a downstream log aggregator. It uses Vector as an agent to collect the logs from journald, add labels to the logs, and send the logs to a log aggregator.

Important: Starting v1.0.0, logs-to-vector now uses Vector API v2 which is incompatible with Vector API v1. If you are using a Vector API v1 endpoint, please use logs-to-vector v0.2.1 instead.

The agent currently ships logs to the following log aggregators:

Usage

To use this image, create a service in your docker-compose.yml as shown below:

version: "2.1"

volumes:
  logs-to-vector: {}

logs-to-vector:
    image: bh.cr/balena/logs-to-vector-aarch64
    labels:
      io.balena.features.journal-logs: '1'
    restart: unless_stopped
    volumes:
      - logs-to-vector:/var/lib/logs-to-vector

You can also set your docker-compose.yml to build from a Dockerfile.template file. You may add your own Vector sinks that takes balena as input.

docker-compose.yml:

version: "2.1"

volumes:
  logs-to-vector: {}

services:
  logs-to-vector:
    build: ./
    labels:
      io.balena.features.journal-logs: '1'
    restart: unless_stopped
    volumes:
      - logs-to-vector:/var/lib/logs-to-vector

Dockerfile.template

FROM bh.cr/balena/logs-to-vector-aarch64

COPY sink.yaml /etc/vector

sink.yaml

sinks:
  console:
    type: console
    inputs:
      - balena

Customisation

logs-to-vector can be configured via the following variables:

Environment Variable Default Description
DISABLE false Disables the logshipper service
VECTOR_ACKNOWLEDGEMENTS_ENABLED true Sources will wait for this sink to deliver the events before acknowledging receipt
VECTOR_BUFFER_DISK_MAX_SIZE 268435488 The maximum size of the buffer on the disk
VECTOR_BUFFER_MEMORY_MAX_EVENTS 1000 The maximum number of events allowed in the buffer
VECTOR_BUFFER_TYPE memory The type of buffer to use (Options: disk, memory)
VECTOR_BUFFER_WHEN_FULL drop_newest The behavior when the buffer becomes full (Options: block, drop_newest)
VECTOR_COMPRESSION_ENABLED true Enables gRPC compression with gzip
VECTOR_ENDPOINT `` The endpoint of the vector log aggregator. The value can be a string template that will be substituted with environment variables (e.g. vector.$DOMAIN_NAME).
VECTOR_REQUEST_TIMEOUT_SECS 300 The maximum time a request can take before being aborted
VECTOR_TLS_CA_FILE `` An additional CA certificate file. The value can be a path to a file, a multi-line string, or a string encoded in base64. The value can be a string template that will be substituted with environment variables (e.g. $CERTS_DIR/ca.pem).
VECTOR_TLS_CRT_FILE `` The client certificate file. The value can be a path to a file, a multi-line string, or a string encoded in base64. The value can be a string template that will be substituted with environment variables (e.g. $CERTS_DIR/cert.pem).
VECTOR_TLS_KEY_FILE `` The client certificate key file. The value can be a path to a file, a multi-line string, or a string encoded in base64. The value can be a string template that will be substituted with environment variables (e.g. $CERTS_DIR/cert.key).
VECTOR_TLS_VERIFY_CERTIFICATE true Verifies the TLS certificate of the remote hos
VECTOR_TLS_VERIFY_HOSTNAME true Verifies the endpoint hostname against the TLS certificate

You can refer to the docs on how to set environment or service variables

Alternatively, you can set them in the docker-compose.yml or Dockerfile.template files.