objfs.1

'\" t
.\"     Title: objfs
.\"    Author: [see the "AUTHOR(S)" section]
.\" Generator: Asciidoctor 1.5.7
.\"      Date: 2018-05-25
.\"    Manual: \ \&
.\"    Source: \ \&
.\"  Language: English
.\"
.TH "OBJFS" "1" "2018-05-25" "\ \&" "\ \&"
.ie \n(.g .ds Aq \(aq
.el       .ds Aq '
.ss \n[.ss] 0
.nh
.ad l
.de URL
\fI\\$2\fP <\\$1>\\$3
..
.als MTO URL
.if \n[.g] \{\
.  mso www.tmac
.  am URL
.    ad l
.  .
.  am MTO
.    ad l
.  .
.  LINKSTYLE blue R < >
.\}
.SH "NAME"
objfs \- object storage file system
.SH "SYNOPSIS"
.sp
\f(CRobjfs [\-options] command args...\fP

.br
.SH "DESCRIPTION"
.sp
The objfs program implements the \(lqobject storage file system\(rq.
.sp
Objfs exposes objects from an object storage, such as a cloud drive, etc. as files in a file system that is fully integrated with the operating system. Programs that run on the operating system are able to access these files as if they are stored in a local "drive" (perhaps with some delay due to network operations).
.sp
Objfs accepts commands such as \f(CRauth\fP and \f(CRmount\fP, but also shell\-like commands, such as \f(CRls\fP, \f(CRstat\fP, etc.
.sp
Objfs configures itself on every run according to these sources of configuration in order of precedence:
.sp
.RS 4
.ie n \{\
\h'-04'\(bu\h'+03'\c
.\}
.el \{\
.  sp -1
.  IP \(bu 2.3
.\}
Command line options.
.RE
.sp
.RS 4
.ie n \{\
\h'-04'\(bu\h'+03'\c
.\}
.el \{\
.  sp -1
.  IP \(bu 2.3
.\}
A configuration file, which is found in a platform\-specific location (unless overriden by the \f(CR\-config\fP option).
.RE
.sp
.RS 4
.ie n \{\
\h'-04'\(bu\h'+03'\c
.\}
.el \{\
.  sp -1
.  IP \(bu 2.3
.\}
Sensible defaults.

.br
.RE
.SS "Default Storage"
.sp
Objfs uses defaults to simplify command line invocation. In the default build of objfs, the default storage is \f(CRonedrive\fP.
.SS "Auth"
.sp
Objfs supports multiple "auth" (authentication or authorization) mechanisms through the \f(CR\-credentials path\fP option and the \f(CRauth\fP command.
.sp
In general before an object storage service can be used it requires auth. The specific auth mechanism used depends on the service and it ranges from no auth, to username/password, to Oauth2, etc. Auth mechanisms require credentials, which can be supplied using the \f(CR\-credentials path\fP option.
.sp
In some cases the object storage service cannot readily accept the supplied credentials, they must be converted to other credentials first. As an authentication example, a particular service may require username/password credentials to be converted to some form of service\-level token before they can be used. As an authorization example Oauth2 requires application\-level credentials together with user consent to form a service\-level token that can be used to access the service.
.sp
The \f(CRauth\fP command can be used for this purpose. It takes user\-level or application\-level credentials and converts them to service\-level credentials.
.sp
Credentials can be stored in the local file system or the system keyring. The syntax \f(CR/file/path\fP is used to name credentials stored in the file system. The syntax \f(CRkeyring:service/user\fP is used to name credentials stored in the system keyring.
.SS "Example \- Oauth2 Flow"
.sp
.RS 4
.ie n \{\
\h'-04'\(bu\h'+03'\c
.\}
.el \{\
.  sp -1
.  IP \(bu 2.3
.\}
Prepare the Oauth2 \f(CRclient_secret\fP credentials in a file or the system keyring:
.sp
.if n .RS 4
.nf
client_id="XXXXXXXX"
client_secret="XXXXXXXX"
redirect_uri="http://localhost:xxxxx"
scope="files.readwrite.all offline_access"
.fi
.if n .RE
.RE
.sp
.RS 4
.ie n \{\
\h'-04'\(bu\h'+03'\c
.\}
.el \{\
.  sp -1
.  IP \(bu 2.3
.\}
Issue the command:
.sp
.if n .RS 4
.nf
$ ./objfs \-credentials=CLIENT_SECRET_PATH auth TOKEN_PATH
.fi
.if n .RE
.RE
.sp
.RS 4
.ie n \{\
\h'-04'\(bu\h'+03'\c
.\}
.el \{\
.  sp -1
.  IP \(bu 2.3
.\}
This will launch your browser and ask for authorization. If the access is authorized the Oauth2 \f(CRaccess_token\fP and \f(CRrefresh_token\fP will be stored in the specified path.
.RE
.sp
.RS 4
.ie n \{\
\h'-04'\(bu\h'+03'\c
.\}
.el \{\
.  sp -1
.  IP \(bu 2.3
.\}
The object storage can now be mounted using the command:
.sp
.if n .RS 4
.nf
$ ./objfs \-credentials=TOKEN_PATH mount MOUNTPOINT
.fi
.if n .RE
.RE
.SS "Mount"
.sp
The objfs \f(CRmount\fP command is used to mount an object storage as a file system on a mountpoint. On Windows the mount point must be a non\-existing drive or directory; it is recommended that an object storage is only mounted as a drive when the object storage is case\-sensitive. On macOS and Linux the mount point must be an existing directory.
.sp
To mount on Windows:
.sp
.if n .RS 4
.nf
> objfs \-credentials=TOKEN_PATH mount \-o uid=\-1,gid=\-1 mount X:
.fi
.if n .RE
.sp
To mount on macOS and Linux:
.sp
.if n .RS 4
.nf
$ ./objfs \-credentials=TOKEN_PATH mount MOUNTPOINT
.fi
.if n .RE
.sp
Objfs uses a local file cache to speed up file system operations. This caches files locally when they are first opened; subsequent I/O operations will be performed against the local file and are therefore fast. Modified files will be uploaded to the object storage when they are closed. File system operations such as creating and deleting files and listing directories are sent directly to the object storage and are therefore slow (although some of their results are cached).
.sp
The Objfs cache was inspired by an early version of the Andrew File System (AFS). For more information see the paper \c
.URL "http://pages.cs.wisc.edu/~remzi/OSTEP/dist\-afs.pdf" "" "."

.br
.SS "Diagnostics"
.sp
Objfs includes a tracing facility that can be used to troubleshoot problems, to gain insights into its internal workings, etc. This facility is enabled when the \f(CR\-v\fP option is used.
.sp
The environment variable \f(CRGOLIB_TRACE\fP controls which traces are enabled. This variable accepts a comma separated list of file\-style patterns containing wildcards such as \f(CR*\fP and \f(CR?\fP.
.sp
.if n .RS 4
.nf
$ export GOLIB_TRACE=pattern1,...,patternN
.fi
.if n .RE
.sp
Examples:
.sp
.if n .RS 4
.nf
$ export GOLIB_TRACE=github.com/billziss\-gh/objfs/fs.*      # file system traces
$ export GOLIB_TRACE=github.com/billziss\-gh/objfs/objio.*   # object storage traces
$ export GOLIB_TRACE=github.com/billziss\-gh/objfs/fs.*,github.com/billziss\-gh/objfs/objio.*
$ ./objfs \-v \-credentials=TOKEN_PATH mount MOUNTPOINT
.fi
.if n .RE
.sp

.br
.SH "COMMANDS"
.sp
The following commands may be used:
.sp
\f(CRversion\fP
.RS 4
get current version information
.RE
.sp
\f(CRconfig\fP
.RS 4
get or set configuration options
.RE
.sp
\f(CRkeyring\fP
.RS 4
get or set keys
.RE
.sp
\f(CRauth output\-credentials\fP
.RS 4
perform authentication/authorization
.RE
.sp
\f(CRmount [\-o option...] mountpoint\fP
.RS 4
mount file system
.RE
.sp
\f(CRstatfs\fP
.RS 4
get storage information
.RE
.sp
\f(CRls [\-l][\-n count] path...\fP
.RS 4
list files
.RE
.sp
\f(CRstat [\-l] path...\fP
.RS 4
display file information
.RE
.sp
\f(CRmkdir path...\fP
.RS 4
make directories
.RE
.sp
\f(CRrmdir path...\fP
.RS 4
remove directories
.RE
.sp
\f(CRrm path...\fP
.RS 4
remove files
.RE
.sp
\f(CRmv oldpath newpath\fP
.RS 4
move (rename) files
.RE
.sp
\f(CRget [\-r range][\-s signature] path [local\-path]\fP
.RS 4
get (download) files
.RE
.sp
\f(CRput [local\-path] path\fP
.RS 4
put (upload) files
.RE
.sp
\f(CRcache\-pending\fP
.RS 4
list pending cache files
.RE
.sp
\f(CRcache\-reset\fP
.RS 4
    reset cache (upload and evict files)

.br
.RE
.SH "GENERAL OPTIONS"
.sp
The following options apply to all commands:
.sp
\f(CR\-accept\-tls\-cert\fP
.RS 4
accept any TLS certificate presented by the server (insecure)
.RE
.sp
\f(CR\-auth name\fP
.RS 4
auth name to use
.RE
.sp
\f(CR\-config path\fP
.RS 4
path to configuration file
.RE
.sp
\f(CR\-credentials path\fP
.RS 4
auth credentials path (keyring:service/user or /file/path)
.RE
.sp
\f(CR\-datadir path\fP
.RS 4
path to supporting data and caches
.RE
.sp
\f(CR\-keyring string\fP
.RS 4
keyring type to use: system, private (default "private")
.RE
.sp
\f(CR\-storage name\fP
.RS 4
storage name to access (default "onedrive")
.RE
.sp
\f(CR\-storage\-uri uri\fP
.RS 4
storage uri to access
.RE
.sp
\f(CR\-v\fP
.RS 4
    verbose

.br
.RE
.SH "CONFIGURATION FILE"
.sp
During startup objfs consults a congifuration file from a platform\-specific location (see the \fBFILES\fP section); this location can be overriden with the \f(CR\-config\fP option.
.sp
The configuration file stores a list of properties (key/value) pairs, that may also be grouped into sections. The basic syntax of the configuration file is as follows:
.sp
.if n .RS 4
.nf
name1=value1
name2=value2
\&...
[section]
name3=value3
name4=value4
\&...
.fi
.if n .RE
.sp
The valid property names are a subset of the command\-line options: \f(CRauth\fP, \f(CRcredentials\fP, \f(CRstorage\fP, \f(CRstorage\-uri\fP. They specify the same value as the equivalent command\-line option.
.sp
The command line option or property \f(CRstorage\fP may specify the name of a storage service (e.g. \f(CRonedrive\fP), but it may also specify a section within the configuration file, which should be used to retrieve additional configuration options. For example, given the configuration file below and a command line option \f(CR\-storage=onedrive2\fP, it will instruct objfs to act on the OneDrive storage identified by the credentials \f(CRkeyring:objfs/onedrive2\fP:
.sp
.if n .RS 4
.nf
[onedrive1]
storage=onedrive
credentials=keyring:objfs/onedrive1

[onedrive2]
storage=onedrive
credentials=keyring:objfs/onedrive2
.fi
.if n .RE
.sp

.br
.SH "FILES"
.sp
Windows
.RS 4
.sp
.RS 4
.ie n \{\
\h'-04'\(bu\h'+03'\c
.\}
.el \{\
.  sp -1
.  IP \(bu 2.3
.\}
\fBconfig\fP: \f(CR%USERPROFILE%\(rsAppData\(rsRoaming\(rsobjfs.conf\fP
.RE
.sp
.RS 4
.ie n \{\
\h'-04'\(bu\h'+03'\c
.\}
.el \{\
.  sp -1
.  IP \(bu 2.3
.\}
\fBdatadir\fP: \f(CR%USERPROFILE%\(rsAppData\(rsRoaming\(rsobjfs\fP
.RE
.RE
.sp
macOS
.RS 4
.sp
.RS 4
.ie n \{\
\h'-04'\(bu\h'+03'\c
.\}
.el \{\
.  sp -1
.  IP \(bu 2.3
.\}
\fBconfig\fP: \f(CR~/Library/Preferences/objfs.conf\fP
.RE
.sp
.RS 4
.ie n \{\
\h'-04'\(bu\h'+03'\c
.\}
.el \{\
.  sp -1
.  IP \(bu 2.3
.\}
\fBdatadir\fP: \f(CR~/Library/Application Support/objfs\fP
.RE
.RE
.sp
Linux
.RS 4
.sp
.RS 4
.ie n \{\
\h'-04'\(bu\h'+03'\c
.\}
.el \{\
.  sp -1
.  IP \(bu 2.3
.\}
\fBconfig\fP: \f(CR~/.config/objfs.conf\fP
.RE
.sp
.RS 4
.ie n \{\
\h'-04'\(bu\h'+03'\c
.\}
.el \{\
.  sp -1
.  IP \(bu 2.3
.\}
\fBdatadir\fP: \f(CR~/.local/share/objfs\fP
.RE
.RE
.sp

.br
.SH "COPYRIGHT"
.sp
\(co 2018 Bill Zissimopoulos