-
Notifications
You must be signed in to change notification settings - Fork 8
Contacts
This document attempts to lay out my current understanding, which is almost certainly wrong in many particulars, of the state of contacts & identities in Glow/GloUI, and where we would like to get to by the end of June 2021. Please feel free to comment or edit directly with corrections, questions, clarifications, etc.
Everything in this document should be taken as augmenting #150, which enumerates specific application & schema requirements.
A contact is a person or service that may possess one or more identities. It may have a (nick)name, email address, etc. associated with it. An identity is a particular key or address on a network that is (believed to be) associated with a particular contact. Identities may also have names associated with them, and we may want to allow compositions of such names, such as "contact @ identity".
The end-point that Faré envisions is a "Google Contacts"-like interface running in the browser. The list of contacts is the primary view, and clicking on a contact brings up its associated identities and allowed actions (e.g., send assets from/to). This is a "Subject-Verb-Object" workflow rather than the "Verb-Subject-Object" order that GloUI currently uses.
The command-line glow program currently has an independent system
for maintaining contacts & identities from that of GloUI. I think they
should be the same, primarily because a name should refer to the same
identity in the GUI as with the CLI.
The basic contacts functionality of the current CLI seems roughly
fine, but possibly confusing re: contact vs. identity (e.g., TODO at
glow/cli/identities.ss:12: "always populate contacts as well as identity
and/or check consistency between nicknames of the two"). Reworking the
code in glow/cli/identities.ss and glow/cli/contacts.ss to use an
HTTP API does not seem like it would be very difficult (famous last words).
The current version of GloUI uses a server written in Gerbil that delivers
JSON to a JavaScript front-end written with Quasar/Vue. Contacts may be
added (and edited? maybe not yet) and referred to in the front-end, and
are stored using IndexedDB storage in the browser.
The question of where to store contacts has apparently been problematic
and shifted several times. My proposal is that it live "server-side",
i.e., wherever the glow CLI lives, and that both the GUI front-end
and the CLI use JSON over HTTP (JSON-RPC?) to communicate with a simple
glow-contacts server.
The main reason for this is that we need a "server" to communicate with
the consensus. (Is this correct? Is there any reasonable way for JS in
the browser to effectively communicate with a blockchain?) Another is
that if we want to store secret keys in ssh-agent or gpg-agent, we
will need to run where it is running.
Like the location, the storage engine used for contacts has apparently
been problematic and subject to change. Right now, the CLI uses a simple
persistence layer on top of a local LevelDB for most storage, and a
seperate local secret-key-ring.json file for secrets, and the GUI uses
IndexedDB storage in the browser.
If the requirement to unify the contacts database between GUI & CLI is assumed and we move storage back to the "server", then we can use nearly any database we like.
Aside from the storage engine, we can also choose what kinds of objects or relations we would like to store.
One way to view contacts & identities is as objects linked into a DAG or tree. This is a very natural view in both object-oriented Scheme and also JavaScript. The current storage engines for both the GUI & CLI implement this abstraction more or less directly, since they are persistent object stores.
One way to unify the stores would be to adopt one or the other of the
current stores and label it canonical. The browser is probably not the
right choice of location, though, since the CLI can't reliably talk to it.
If we chose to retain the current LevelDB storage engine the CLI uses,
we could possibly layer a glow-contacts server on top of it, modifying
as needed to support the required features. (What, specifically, are the
problems with LevelDB and the current object store? Are they enumerated
anywhere?)
Another approach which Drew has advocated is switching to a server-side
Postgres instance and using its built-in class and JSON features to build
an object store. Coupled with Postgres' replication feature, this could be
turned into a distributed object store, in theory capable of syncing
between multiple glow instances.
(Question: is sync between glow instances currently a requirement?
My understanding was "no", but that it would be nice to have in the
future.)
The Postgres object store could also potentially help with the problem of logging & replaying changes to the store, though I admit I don't understand exactly how. Handling object additions would seem to be simple, and edits could be handled as "new" objects with the same ID as the old ones, but deletions and schema changes are not obvious to me. I'm sure it's all solvable, though.
Another way to implement contact storage is with a traditional relational database, i.e., SQL tables. The main disadvantage over an object store is that if you want to serialize data as objects (and we do, since JSON), they have to be "reconstituted" from the table contents. This is not difficult to do, but it does add a layer of data translation.
The advantages are that (1) relational database tables may be interactively
queried and explored, (2) features like foreign keys, unique indexes,
and CHECK conditions can be used to ensure semantic data integrity,
and (3) transactions can be used to make related sets of changes atomically.
Change logging is potentially problematic (as it is with an object store),
but I propose a simple (stupid) solution: if the only interface to
glow-contacts is JSON over HTTP requests, simply log every such request
in the database, and use very simple client code to replay such requests
as needed, starting with a previous snapshot or empty database. More
complete or complex logging strategies are obviously possible.
If we choose to use a relational store, I suggest there are two reasonable
choices: Postgres or SQLite. Both are supported out of the box by Gerbil's
dbi library, and both are fast and robust with the small number of
writers we expect (≤ 2, I think).
I think that if we require the data store to have built-in synchronization or a native class/object system, the Postgres would be a better choice than SQLite. However, I do not believe that we currently need those features, and that the lack of installation, configuration, daemon, and port/socket give SQLite the edge for our use case. I suggest that we start with SQLite, but make an effort to use only SQL and features that are trivially portable to Postgres. I don't believe this will be difficult for our application.
Do we want a database abstraction layer to allow running in either a CLI or a stand-alone GUI?
In any scenario except the one where we continue to use the current LevelDB storage, we will have the issue of whether and how to migrate current Glow contacts to the new system. My feeling is that it should be automatic, but that implementing it is a lower priority than essentially all other features.
We must also think about future migrations and schema changes. These are not difficult problems, but require thought and work usually specific to a particular engine.
Our Milestone 3 Deliverables include UIs for not just asset transfer,
but also buy_sig, rps, basket swap, and also some sort of generic
API. Milestone 4 includes a browser-based REPL of some kind. Whatever
we do for contacts, we must make sure that it will in no way hinder
achieving those goals, too.