Skip to content

Ops: Running Duffy

Jump to bottom
Nils Philippsen edited this page Jun 23, 2022 · 1 revision

Overview

Services

Duffy consists of a bunch of related services. Together, they manage the lifecycle of differently configured nodes, making them available for use in CI tests.

Technically, all these services can run on different hosts. Because the actual Duffy services share common configuration and are relatively lightweight it’s practical to run them on a single node, with the database optionally on an existing server.

Web API

This service offers a self-documenting CRUD/REST API and is the outward-facing element of a Duffy deployment. Users (tenants) can create sessions of nodes, specifying how many of what configuration they want to use for a limited time in their tests. The service then attempts to allocate matching nodes out of pools of alike nodes, reserves them for a tenant and installs their configured SSH key to enable access. After a tenant is done using these nodes, they retire the session and the service marks the nodes to be deprovisioned and possibly reused.

Legacy Metaclient

This optional service acts as an intermediary between legacy API clients and the current web API. It accepts requests in the legacy format, transforms them for the new API, forwards them and does the same in reverse for the responses.

Task Worker

This service performs background tasks: maintain levels of node pools by provisioning new nodes if they’re understrength, deprovision nodes after use, expire sessions past their lifetime.

Database (3rd party)

Both the web API and task workers access a relational database which stores information about sessions, nodes, tenants and how they are related. Records in this database are kept indefinitely, instead of deletion they are marked as retired.

Fpr production use, it’s recommended to use PostgreSQL as the underlying RDBMS.

Installation

Package Extras

The different use cases of Duffy (services, the client CLI app) have different sets of dependencies. The Duffy Python package defines several package extras which pull in the dependencies necessary for these use cases, you should select at least one extra, otherwise your Duffy installation won't be very useful.

The most important extras are:

  • app: Serve the web API.
  • tasks: Run the Celery backend task workers.
  • sqlite or postgresql: Additional dependencies for app and tasks, needed when using one of these RDBMS systems.
  • legacy: Run the legacy web API metaclient.
  • client: The CLI tool for users of Duffy.
  • all: All dependencies of all extras combined, useful for development and testing.

These extras are defined in pyproject.toml in the [tool.poetry.extras] section.

PyPi

Duffy is available in the Python Package Index and can be installed using pip.

For example, install the web API service and Celery task workers so they can operate with a PostgreSQL database like this:

$ pip --user install 'duffy[app,tasks,postgresql]'

Users of Duffy can install only the CLI client like this:

$ pip --user install 'duffy[client]'

Using poetry

Duffy can be installed from source (e.g. a checkout of the GIT repository) using poetry. If you want to develop Duffy, you can install Duffy and all its dependencies into a Python virtual environment in the directory venv like this:

$ source venv/bin/activate
$ poetry install -E all

Running the Legacy API Metaclient

Authentication

The Legacy API used a proprietary way to authenticate tenants: a tenant's API key is submitted as a query variable in the HTTP request, the key both identifies the tenant and acts as the secret for authentication. The current version of Duffy uses basic HTTP authentication headers, separating identity from the secret proving it. In order for the metaclient to work, operators have to map API keys of users of the legacy API to their tenant names, e.g.:

...
metaclient:
  usermap:
    "3d3b0f96-0739-455c-8752-cf5d4fbcdc3d": "tenant-name"
...

Caution: The order of using the mapping the API key to the tenant name is probably contrary to the intuitive order of these elements.

Because this contains API keys in clear text, the file containing this mapping should only be readable to the user running the legacy metaclient service.

Mapping properties to pools

The legacy API selected nodes to hand out to tenants by the specified OS version (ver), hardware architecture (arch) and flavor (which encoded the "size" of a machine in terms of storage and memory). The new API only knows pools and playbooks which provision a machine into a pool, i.e. the former must be mapped to the latter like this:

...
metaclient:
  poolmap:
  # Valid selection keys: ver, arch, flavor
  # Values can be simple strings or regexes (surrounded by "^...$")
  # and must map to a pool which can use Jinja macros.
  # First match wins.
  - arch: "^(aarch64|ppc64|ppc64le)$"
    pool: "virtual-centos{{ ver | replace('-', '') }}-{{ arch }}-{{ flavor | default('medium') }}"
  - arch: "x86_64"
    pool: "physical-centos{{ ver | replace('-', '') }}-{{ arch }}"
...
Clone this wiki locally