This document describes the high-level architecture of gopass. If you want to get familiar with the code base you are in the right place.
On the highest level gopass manages directories (called stores or mounts)
that contain (mostly) GPG encrypted text files. gopass transparently handles
encryption and decryption when accessing these files. It applies some heuristics
to parse the file content and support certain operations on that content.
gopass is licensed under the terms of the MIT license and we require
compatible licenses from our dependencies as well (when we link against them).
For licensing reasons and security considerations we try to keep the number of external dependencies (libraries) well-arranged. Try to avoid adding new dependencies unless absolutely necessary.
This flow chart shows a high level control flow common to most operations. It leaves out a lot of details but should give a better understanding how information flows within the program and where changes might be necessary.
This section talks briefly about the various directories and some data structures.
We're trying to clearly separate between our public API and implementation
details. To that extent we're in the process of moving packages to internal/
(and sometimes back to pkg/, if necessary).
A note on semantic versioning: gopass is both an CLI and an API (Go module).
The expectations around semantic versioning and Go modules make it difficult
to express both concerns in the same versioning scheme, e.g. does a breaking
change in the API require a major version bump even if nothing about the tool
(CLI) has changed? What about the other way round? Thus we have decided to
apply semantic versioning only to the CLI tool, not the Go module. This is not
ideal and might change with sufficient active contributors.
This folder contains documentation about each of our supported backends. See
internal/backend below for more information about our backend design.
This folder contains the specification of each sub command the tool offers. We have many sub commands with sometimes dozens of flags each. In the past we did encounter some inconsistencies and decided to introduce specifications for each command. If the specification and the implementation disagree this should be reported as a bug and fixed or the specification needs to be changed (but the general assumptions should be that the specification is correct, not the code).
This directory contains an (incomplete) list of our core use cases, i.e. the
critical user journeys we aim to support. gopass can be used in various ways
and try to remain flexible and extensible, but if we encounter a conflict
between a blessed use case and a corner case we prefer the former.
This directory contains some release automation tooling that is supposed to be
invoked with go run. The changelog generator in helpers/changelog is used
by our GitHub Action based release automation and shouldn't be invoked manually.
The tooling in helpers/release will prepare a new release and helps to file a
release pull request will all the required updates in place.
gopass used to not have either of these and all our packages were rooted
directly in the repository. However we began to notice that other projects
were starting to depend directly on our internal packages and we sometimes
broke them. This put us and the other project into an unpleasant
situation so we tried to clarify the expectations by using Go's internal/
visibility rule to keep other projects from depending on our implementation
details.
Note: If we have a good reasons to use one of our internal/ packages either
copy it (our license should rarely be an issue) or nicely ask us and explain
why something should move to pkg/.
As we are in the process of formalizing a proper API surface we sometimes need
to move packages from internal/ to pkg/. The other direction might also
occur, but much less often.
This directory contains one file, and sometimes sub folders, for each command
gopass supports. These are mostly self-contained, but some (e.g. show / edit
/ find) need to depend on each other.
TODO: There is a lot to be said about this package, e.g. custom errors.
gopass is built around the concept of multiple independent password stores
that can be mounted into one namespace, much like regular file systems. Each
of these stores can have a different storage and crypto backend. We used to
have independent revision control backends as well, but since the RCS (e.g.
git) interacts so closely with the storage (you can't use regular git w/o a
filesystem-based storage) we have merged storage and RCS backends.
The backend package defines the interfaces for the backend implementation
and provides a registry that returns the concrete backend from the list of
registered ones. Registration happens through blank imports of either the
internal/backend/crypto and internal/backend/storage packages.
Each backend needs to have a loader implementation in its loader.go (please
stick to this name). We try to auto-detect the most applicable backend when
initializing the process, but some backends look alike (e.g. a fs and an
uninitialized gitfs). So the loader comes with a priority which is respected
during lookup.
The config package implements support for a simple YAML-based configuration
format for gopass. Most of the code in this package is for backwards
compatibility. Whenever we introduce or remove a config option we need to
introduce a new fallback version that is automatically attempted when loading
a config file. To resolve ambiguities when parsing different config versions
we use a "catch-all" field to catch any unused keys and check that this is
empty after parsing - otherwise we need to try a different config version.
NOTE: We did support nested configurations for sub-stores but removed this because the maintenance cost did not justify the benefits of this feature.
The name cui is an abbreviation for console-user-interface and contains
several helper functions to interact with humans over a text based interface.
Most of these ask the user to select some item from a selection or provide some input.
NOTE: We used to support rich terminal UIs with arrow navigation and such. However all existing libraries that were available without CGO were either abandoned or buggy on some platforms and we didn't have any capacity to fix them. So we had to remove support for this feature.
The queue package implements a FIFO queue that executes
in the background. This allows for certain operations, like a git push, to be
taken out of the critical path wrt. user interactions. The queue will be fully
processed before the process exits.
The root store package implements an internal password store API that (only)
supports mounting leaf stores. It will forward (almost) all operations to
its leaf stores (moves across stores being a notable exception) and do the
necessary manipulations of the affected path components (e.g. removing/adding
the mount prefix from the secret name as needed).
This package makes gopass multi-store capable.
The leaf store package implements a password store that is mostly compatible
with any other password store implementation (while aiming for interoperability,
not at 100% feature parity). The low-level operations like filesystem and / or
version control and crypto operations are passed to the configured storage
or crypto backend.
The tree package implements a simple tree structure that prints an output
similar to the output of the Unix tool tree. It does support different
gopass specific properties (like mounts or templates) not easily implemented
with other tree packages.
The updater package implements a secure and anonymous self updater.
Note: The self updater contacts GitHub. If this is a concern one should use other sources, e.g. distro packages.
It retrieves the latest stable release from GitHub, fetches its metadata and verifies the signature against the built-in release signing keyring.
It tries to avoid conflicting with any gopass binary managed by the OS
and refuse to update these.
The package pkg/ contains our public API surface, i.e. packages we want or
have to expose externally. Some packages (e.g. otp) are only exposed because
they are being used by some of our integrations. Others (e.g. pinentry or
pwgen) are designed for wider use. We are considering to split some of the
more widely used packages into their own repositories to work better with
Go module and semantic versioning expectations.
The appdir package contains a set of XDG Base Directory Specification
compatible implementations with some gopass specifics. For testing purposes
we want to honor the setting of GOPASS_HOMEDIR before everything else, so our
implementation has to take this into account before following the XDG spec.
The clipboard package is a wrapper around a clipboard package that adds
support for clearing the clipboard.
The ctxutil is the pragmatic (read: non-idiomatic) approach to pass very
specific configuration options through multiple layers of abstraction. This is
arguably not the best design, but it works well and avoids bloated interfaces.
This package contains the gopass API interface. We provide a concrete implementation that should work with any properly initialized gopass setup and a mock for tests.
This package is designed as the main entry point for any integration that wants to integrate with gopass.
gopass comes with a comprehensive set of integration tests, i.e. tests that
are executed by running a newly compiled gopass binary without access to any
kind of internal state. These tests can't be as exhaustive as the unit tests
but they exist to ensure basic functionality aren't broken by a change.