Skip to content

Commit

Permalink
refactor: Use "daemon" and "cli" instead of "server/client" (#220)
Browse files Browse the repository at this point in the history
Talking to the CGEM folks, I think our (my) use of the terms "server" to
refer to the long-running process that manages alpenhorn nodes, and
"client" to refer to the command-line interface to the data index are
unnecessarily confusing.

I want to move to using "daemon" and "cli" for these pieces of alpenhorn
instead.

The terms "server" and "client" imply that the two pieces of alpenhorn
are a normal server-client system, which they are not. To over-simplify
things, normally a "server" is a long-running process that produces data
for users. The users use the "client" to connect to the server and
retrieve the data.

The two parts of alpenhorn do neither of these things. The daemon is not
at all user-facing. It doesn't "serve" anything that anyone could
retrieve. It's concerned with managing files on disk (or tape).

Neither is the alpenhorn CLI a "client" in the traditional sense. It
doesn't interact with the server directly at all, and there's no API or
protocol defined for communication between the two parts. The CLI is a
tool for reading and writing the data index. By design, the two pieces
can only communicate indrectly through the database.

AFAICT, the alpenhorn daemon was originally called a "daemon" (that's
why it's `alpenhornd`. Later this migrated to "service" (the alpenhorn-1
daemon is implemented in the module `alpenhorn.service`) and then later
still to "server". A lot of the "server" nomenclature shows up during my
multithreaded rewrite of the daemon. So, using "daemon" is something of
a return to the original name.

The CLI has been referred to as both the "CLI" and the "client"
interchangably since we started. (alpenhorn-1's CLI is implemented in
the module `alpenhorn.client`). Here I want to push "CLI" over "client"
to avoid slipping back into "client-server" terms in the future.

Though, I am willing to listen to other options for the names of these
two pieces. At some point we need to write some documentation for this
system, and it would be good to have some concrete terms to work with
for that.
  • Loading branch information
ketiltrout authored Nov 13, 2024
1 parent 001249c commit cfdbeda
Show file tree
Hide file tree
Showing 74 changed files with 432 additions and 432 deletions.
4 changes: 4 additions & 0 deletions alpenhorn/cli/__init__.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
"""Alpenhorn CLI."""

from .. import __version__
from .entry import entry
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
"""Alpenhorn client interface for operations on `ArchiveAcq`s."""
"""Alpenhorn CLI for operations on `ArchiveAcq`s."""

import re
import sys
Expand Down
6 changes: 3 additions & 3 deletions alpenhorn/client/acq/files.py → alpenhorn/cli/acq/files.py
Original file line number Diff line number Diff line change
Expand Up @@ -7,13 +7,13 @@
from ...common.util import pretty_bytes
from ...db import ArchiveAcq, ArchiveFile, ArchiveFileCopy, StorageGroup, StorageNode
from ..cli import echo
from ..options import client_option, not_both
from ..options import cli_option, not_both


@click.command()
@click.argument("acq", metavar="ACQ")
@client_option("group")
@client_option("node")
@cli_option("group")
@cli_option("node")
@click.option(
"--show-removed",
is_flag=True,
Expand Down
6 changes: 3 additions & 3 deletions alpenhorn/client/acq/list.py → alpenhorn/cli/acq/list.py
Original file line number Diff line number Diff line change
Expand Up @@ -7,16 +7,16 @@

from ...db import ArchiveAcq, ArchiveFile, ArchiveFileCopy, StorageGroup, StorageNode
from ..cli import echo
from ..options import client_option, not_both
from ..options import cli_option, not_both


@click.command()
@client_option(
@cli_option(
"group",
help="Limit to acquisitions with files existing on Storage Group named "
"GROUP. In this case, only files in GROUP are counted.",
)
@client_option(
@cli_option(
"node",
help="Limit to acquisitions with files existing on Storage Node named "
"NODE. In this case, only files on NODE are counted.",
Expand Down
File renamed without changes.
2 changes: 1 addition & 1 deletion alpenhorn/client/cli.py → alpenhorn/cli/cli.py
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
"""Alpenhorn client command-line interface."""
"""Alpenhorn command-line interface."""

from __future__ import annotations

Expand Down
17 changes: 10 additions & 7 deletions alpenhorn/client/entry.py → alpenhorn/cli/entry.py
Original file line number Diff line number Diff line change
Expand Up @@ -65,16 +65,19 @@ def _verbosity_from_cli(verbose: int, debug: bool, quiet: int) -> int:
show_default=False,
default=False,
)
def cli(conf, quiet, verbose, debug):
"""Client interface for alpenhorn."""
def entry(conf, quiet, verbose, debug):
"""Alpenhorn data management system.
This is the command-line interface to the alpenhorn data index.
"""

# Initialise alpenhorn
start_alpenhorn(
conf, client=True, verbosity=_verbosity_from_cli(verbose, debug, quiet)
conf, cli=True, verbosity=_verbosity_from_cli(verbose, debug, quiet)
)


cli.add_command(acq.cli, "acq")
cli.add_command(group.cli, "group")
cli.add_command(node.cli, "node")
cli.add_command(transport.cli, "transport")
entry.add_command(acq.cli, "acq")
entry.add_command(group.cli, "group")
entry.add_command(node.cli, "node")
entry.add_command(transport.cli, "transport")
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
"""Alpenhorn client interface for operations on `StorageGroup`s."""
"""Alpenhorn CLI for operations on `StorageGroup`s."""

import click
import peewee as pw
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -5,16 +5,16 @@
import peewee as pw

from ...db import database_proxy, StorageGroup
from ..options import client_option, set_io_config
from ..options import cli_option, set_io_config
from ..cli import echo


@click.command()
@click.argument("group_name", metavar="NAME")
@client_option("io_class", default="Default", show_default=True)
@client_option("io_config")
@client_option("io_var")
@client_option("notes")
@cli_option("io_class", default="Default", show_default=True)
@cli_option("io_config")
@cli_option("io_var")
@cli_option("notes")
def create(group_name, io_class, io_config, io_var, notes):
"""Create a new storage group.
Expand Down
File renamed without changes.
Original file line number Diff line number Diff line change
Expand Up @@ -6,15 +6,15 @@

from ...db import StorageGroup, database_proxy
from ..cli import echo
from ..options import client_option, set_io_config
from ..options import cli_option, set_io_config


@click.command()
@click.argument("group_name", metavar="GROUP")
@client_option("io_class")
@client_option("io_config")
@client_option("io_var")
@client_option("notes")
@cli_option("io_class")
@cli_option("io_config")
@cli_option("io_var")
@cli_option("notes")
def modify(group_name, io_class, io_config, io_var, notes):
"""Modify storage group metadata.
Expand Down
File renamed without changes.
File renamed without changes.
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
"""Alpenhorn client interface for operations on `StorageNode`s."""
"""Alpenhorn CLI interface for operations on `StorageNode`s."""

import os
import re
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -6,15 +6,15 @@

from ...db import database_proxy, StorageGroup, StorageNode
from ..cli import echo, update_or_remove
from ..options import client_option, exactly_one, set_storage_type, set_io_config
from ..options import cli_option, exactly_one, set_storage_type, set_io_config


@click.command()
@click.argument("name", metavar="NODE")
@client_option("address", help="Set the node address to ADDR before activation.")
@client_option("host", help="Set the node host to HOST before activation.")
@client_option("root", help="Set the node root to ROOT before activation.")
@client_option("username", help="Set the node username to USER before activation.")
@cli_option("address", help="Set the node address to ADDR before activation.")
@cli_option("host", help="Set the node host to HOST before activation.")
@cli_option("root", help="Set the node root to ROOT before activation.")
@cli_option("username", help="Set the node username to USER before activation.")
def activate(name, address, host, root, username):
"""Activate a node.
Expand Down
32 changes: 16 additions & 16 deletions alpenhorn/client/node/create.py → alpenhorn/cli/node/create.py
Original file line number Diff line number Diff line change
Expand Up @@ -5,59 +5,59 @@
import peewee as pw

from ...db import database_proxy, StorageGroup, StorageNode
from ..options import client_option, exactly_one, set_storage_type, set_io_config
from ..options import cli_option, exactly_one, set_storage_type, set_io_config
from ..cli import echo


@click.command()
@click.argument("node_name", metavar="NAME")
@client_option("address")
@cli_option("address")
@click.option(
"--activate",
help="Activate the node immediately upon creation.",
is_flag=True,
)
@client_option("archive")
@cli_option("archive")
@click.option(
"--auto-import",
help="Turn on auto-import for the node. [default: off]",
is_flag=True,
default=False,
)
@client_option(
@cli_option(
"auto_verify",
help="If COUNT is non-zero, turn on auto-verify for the node node, with "
"COUNT as the maximum verified copies per iteration.",
default=0,
show_default=True,
)
@client_option("io_class")
@cli_option("io_class")
@click.option(
"--create-group",
help="Create a new Storage Group for this node. The group will have the "
"same name as the node (NAME) and use the Default group I/O class, which "
"only allows one node in it. Incompatible with --group.",
is_flag=True,
)
@client_option(
@cli_option(
"field",
help="Make node a field node (i.e. neither an archive node nor a transport "
"node). This is the default. Incompatible with --archive or --transport.",
)
@client_option(
@cli_option(
"group",
help="Add node to Storage Group GROUP, which must already exist. "
"Incompatible with --create-group",
)
@client_option("host")
@client_option("io_config")
@client_option("io_var")
@client_option("max_total")
@client_option("min_avail", default=0, show_default=True)
@client_option("notes")
@client_option("root")
@client_option("transport")
@client_option("username")
@cli_option("host")
@cli_option("io_config")
@cli_option("io_var")
@cli_option("max_total")
@cli_option("min_avail", default=0, show_default=True)
@cli_option("notes")
@cli_option("root")
@cli_option("transport")
@cli_option("username")
def create(
node_name,
address,
Expand Down
File renamed without changes.
6 changes: 3 additions & 3 deletions alpenhorn/client/node/list.py → alpenhorn/cli/node/list.py
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@
from tabulate import tabulate

from ...db import StorageGroup, StorageNode
from ..options import client_option
from ..options import cli_option
from ..cli import echo


Expand All @@ -16,8 +16,8 @@
is_flag=True,
default=None,
)
@client_option("group", help="List only nodes in Storage Group GROUP.")
@client_option("host", help="List only nodes on HOST.", metavar="HOST")
@cli_option("group", help="List only nodes in Storage Group GROUP.")
@cli_option("host", help="List only nodes on HOST.", metavar="HOST")
def list_(active, group, host):
"""List Storage Nodes."""

Expand Down
34 changes: 16 additions & 18 deletions alpenhorn/client/node/modify.py → alpenhorn/cli/node/modify.py
Original file line number Diff line number Diff line change
Expand Up @@ -5,38 +5,36 @@
import peewee as pw

from ...db import database_proxy, StorageGroup, StorageNode
from ..options import client_option, not_both, set_storage_type, set_io_config
from ..options import cli_option, not_both, set_storage_type, set_io_config
from ..cli import echo, update_or_remove


@click.command()
@click.argument("name", metavar="NAME")
@client_option("address")
@client_option("archive")
@cli_option("address")
@cli_option("archive")
@click.option(
"--auto-import/--no-auto-import",
help="Turn on/off auto-import for the node.",
is_flag=True,
default=None,
)
@client_option("auto_verify")
@client_option("io_class")
@client_option("field")
@client_option(
"group", help="Move node to Storage Group GROUP, which must already exist"
)
@client_option("host")
@client_option("io_config")
@client_option("io_var")
@client_option("max_total")
@cli_option("auto_verify")
@cli_option("io_class")
@cli_option("field")
@cli_option("group", help="Move node to Storage Group GROUP, which must already exist")
@cli_option("host")
@cli_option("io_config")
@cli_option("io_var")
@cli_option("max_total")
@click.option(
"--no-max-total", is_flag=True, help="Remove any existing max-total limit"
)
@client_option("min_avail")
@client_option("notes")
@client_option("root")
@client_option("transport")
@client_option("username")
@cli_option("min_avail")
@cli_option("notes")
@cli_option("root")
@cli_option("transport")
@cli_option("username")
def modify(
name,
address,
Expand Down
File renamed without changes.
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@

from ...db import StorageGroup, StorageNode, ArchiveFile, ArchiveFileCopy
from ...common.util import pretty_bytes
from ..options import client_option
from ..options import cli_option
from ..cli import echo


Expand All @@ -17,8 +17,8 @@
is_flag=True,
default=None,
)
@client_option("group", help="List only nodes in Storage Group GROUP.")
@client_option("host", help="List only nodes on HOST.", metavar="HOST")
@cli_option("group", help="List only nodes in Storage Group GROUP.")
@cli_option("host", help="List only nodes on HOST.", metavar="HOST")
@click.option("--extra-stats", help="Show extra stats.", is_flag=True)
def stats(active, group, host, extra_stats):
"""Show Storage Node stats.
Expand Down Expand Up @@ -100,7 +100,7 @@ def stats(active, group, host, extra_stats):
# We could make this a huge, nasty SQL query
# by employing multiple subqueries, but I think it's
# probably more readable if we do it one-by-one, even
# though that's going to be a bit more work for the client
# though that's going to be a bit more work for the CLI
# itself

# Corrupt counts
Expand Down
8 changes: 4 additions & 4 deletions alpenhorn/client/options.py → alpenhorn/cli/options.py
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
"""Common client options and option processing code."""
"""Common CLI options and option processing code."""

from __future__ import annotations

Expand All @@ -11,10 +11,10 @@
del TYPE_CHECKING


def client_option(option: str, **extra_kwargs):
"""Provide common client options.
def cli_option(option: str, **extra_kwargs):
"""Provide common CLI options.
Returns a click.option decorator for the common client option called
Returns a click.option decorator for the common CLI option called
`option`. Other keyword arguments are passed on to click.option.
"""

Expand Down
File renamed without changes.
4 changes: 0 additions & 4 deletions alpenhorn/client/__init__.py

This file was deleted.

2 changes: 1 addition & 1 deletion alpenhorn/common/__init__.py
Original file line number Diff line number Diff line change
@@ -1,3 +1,3 @@
"""Alpenhorn common modules.
Modules here are used by both client and server."""
Modules here are used by both the alpenhorn CLI and the daemon."""
Loading

0 comments on commit cfdbeda

Please sign in to comment.