ddr_crypt.1

.TH ddr_crypt 1 "2015-04-15" "Kurt Garloff" "En/Decryption plugin for dd_rescue"
.
.SH NAME
ddr_crypt \- Data de/encryption plugin for dd_rescue
.
.SH SYNOPSIS
.na
.nh
.B -L crypt[=option[:option[:...]]]
.br
or
.br
.B -L /path/to/libddr_crypt.so[=option[:option[:...]]]
.
.SH DESCRIPTION
.SS About
This plugin allows to en/decrypt data on the fly as it is written out
by
.B dd_rescue.
It supports a variance of AES ciphers and uses the hardware acceleration
on x86 (AESNI) and ARMv8 -- if available -- to provide good performance.
.PP
This plugin has been written for
.B dd_rescue
and uses the plugin interface from it. See the
.BR dd_rescue(1)
man page for more information on
.B dd_rescue.
.
.SH OPTIONS
Options are passed using
.B dd_rescue
option passing syntax: The name of the plugin (crypt) is
followed by an equal sign (=) and options are separated by a colon (:).
the
.B crypt
plugin also allows for some options to be abbreviated. At least encryption
or decryption and a way to determine the key (and IV) needs
to be passed.
.
.SS Help
Pass
.B help
to have ddr_crypt output a short list of options.
.
.SS Encryption or Decryption
The crypt dd_rescue plugin (subsequently referred to as just ddr_crypt which
reflects the variable parts of the file name libddr_crypt.so) need to be told
whether to encrypt or decrypt. This is done by specifying
.B enc[rypt]
or 
.B dec[rypt]
parameters on the
command line.
.
.SS De/encryption algorithm
The crypt plugin supports a number of the de/encryption variants of AES.
You can specify which one you want to use by passing 
.TP 4
.BR algo=AESxxx-yyy ,
where xxx can be 128, 192, 256, 128+, 192+, 256+, 128x2, 192x2, or 256x2,
and yyy can be ECB, CBC, or CTR. 
Pass 
.BR algo=help 
to get a list of available algorithms.
See section ALGORITHMS AND STREAM MODES for a discussion of the options. 
Note that decrypting
a file can only be successful if the exact same algorithms is chosen as was
used for encryption. There is no way to tell from an encrypted file which
algorithm has been chosen.
.br
The default (AES192-CTR) is a good choice if you can ensure that the
key/IV combination is NEVER reused.
.
.SS Crypto engine
There are several implementations of the AES algorithms that this plugin
can currently use:
.TP 4
.BR engine=aesni
This will use an own AES implementation using the AESNI instructions of
the intel x86 core CPUs since Westmere. If possible, this should be used,
as it provides both superior performance and avoids some of the cache
timing attacks that the lookup tables used in discrete implementations
may be prone to. This engine supports all algorithms.
.TP 4
.BR engine=aesarm64
This will use an own AES implementation using the AESv8 crypto extensions of
the ARMv8 (AArch64 = ARM 64bit) architecture (available on CPUs/SOCs based on 
ARM's Cortex A53,A57,A72 or Qualcomm's Kryo or Apple's A7x or newer designs). 
If possible, this should be used,
as it provides both superior performance and avoids some of the cache
timing attacks that the lookup tables used in discrete implementations
may be prone to. This engine supports all algorithms.
.TP 4
.BR engine=aes_c
This uses an AES implementation in C which is based on the rijndael
reference code. It is available on all CPUs and supports all algorithms.
Some prefetching is done on the tables to make cache timing attacks
a bit harder, but this is most likely insufficient to thwart sophisticated
attackers.
.TP 4
.BR engine=openssl
This uses the openssl library to implement the AES routines, which should
take advantage of hardware acceleration where available and have received
some hardening against attacks. Note that the openssl implementation
does not support the + variants (increased number of rounds) that the
other engines provide.
.PP
The engines are used in this order if available, which means that 
.I aesni
will be used by default on x86 if supported,
.I aesarm64
will be used on ARMv8 and
.I aes_c
by default on all other platforms.
.
.SS Padding
As AES is a block cipher (with a block size of 16 bytes), files
which are not a multiple of the block size need padding at the
end to have a full block.
.TP 4
.BR pad=zero
The last block is filled up with zeroes and then encrypted.
Note that on decryption, these zeroes can't be automatically
stripped, as the decryptor has no way to tell whether there
are true trailing zeros or whether those had been added by padding.
This option is thus not recommended for copying files of
arbitrary length.
.TP 4
.BR pad=always
This uses the PKCS7 scheme used by openssl, appending one to
sixteen bytes with the number of bytes to discard when
decrypting. This is always safe, but always makes the output file
larger than the input file. Use this when copying files. This
is the default.
.TP 4
.BR pad=asneeded
This is a hybrid between zero and always. This does PKCS7
padding when the file size does not fill a complete block,
but does no padding when it does. This mechanism has in the
worst case (no pad bytes) a chance of 1/255 to produce an
incorrect decryption result by wrong unpadding (which means
that it has an overall chance of misrepresenting the final
file size of 1/4080 for arbitrary file sizes). Don't use this
unless you can deal with such errors (or exclude them)!
.PP
Note that the CTR stream mode does NOT require padding and indeed
ddr_crypt does not apply any padding regardless of the pad option
when it is used.
.
.SS Debugging and Benchmarking
The
.B debug
parameter makes the ddr_crypt module output some debugging information.
With
.B outkeyiv
the key and IV will be output.
The
.B benchmark
parameter will result in reporting the en/decryption speed.
.
.SH Setting key and IV
There are many ways to set the key and IV (for -CBC and -CTR 
stream modes).
.SS Setting directly
They can be set directly using 
.B keyhex= 
and 
.B ivhex=
on the command line. The argument is interpreted as a hex
ASCII representation of the key/IV (without leading 0x).
This way to specify keys/IVs should only be used for
testing - the key will be visible to all local users by looking
at the command line (unless specific measures are taken to
lock down access to /proc). Not normally a good idea on a multi-user
system. ddr_crypt does overwrite the sensitive data with X to make
the attack a bit harder, but this still leaks the length and 
-- more importantly -- there is still a time window where the sensitive
data is visible.
.
.SS Reading from file of file descriptor
It's better to pass in the key and IV via a file descriptor via
.B keyfd=
and 
.B ivfd= .
If the integer is preceded by an x, the key/IV will be read as
hex string, otherwise binary data will be used. Optionally, @OFF@LEN
can be appended, designating the offset and length (in bytes) to be 
read in the file passed via the file descriptor. Note that a
file descriptor of 0 without shell redirection will result in an 
interactive prompt to the user and the answer won't be echoed to 
the screen of course.
.br
This is useful mainly when dd_rescue is called from another program.
.P
Alternatively, with
.B keyfile=
and 
.B ivfile=
a file name to be opened and read from can be specified.
The syntax does support the same optional @OFF@LEN designation,
but the key and IV will always be read in binary form.
(See below, Index files for a way to read in hex form.)
Currently (unlike with salt) there is no way to use ddr_crypt
to write out binary key and IV data with these options.
.
.SS Generating random key and IV 
The Operating System's random number generator can be used to
generate key and IV on the fly; if your system offers good random
numbers, this is the most secure way to specify and encryption
key. The options to specify are
.B keygen
and 
.B ivgen .
You need to save the key/IV somehow, otherwise you can not
decrypt again later. (The program will warn you!) Best way is to
use the next options.
.
.SS Index files
Keys and IVs can be stored as hex strings in index files;
the file format is the same as the one used in MD5SUMS:
The hex representation of the key/IV is followed by the file name.
Obviously, appropriate care needs to be taken to keep those files
confidential.
.P
If the ddr_crypt plugin gets the option 
.B keysfile
and
.B ivsfile
it will store already created keys/IVs (from the other options)
to files names KEYS.algname and IVS.algname in the MD5SUMS format.
(The files will be created or updated accordingly.)
If key/IV have not been created yet, ddr_crypt will try to retrieve
the key/IV from those files and error out upon failure.
.br
These options combine well with keygen and ivgen on encryption
(and should be used alone on decryption).
.
.SS Extended attributes
Similar to index files, keys and IVs can also be stored in and
retrieved from the encrypted file's extended attributes. This can
be achieved using the options
.B keyxattr
and
.B ivxattr .
Please review the comments in the main dd_rescue (1) man page for
general considerations about using extended attributes (xattrs).
.br
Note that storing the key in the xattr is normally
.B not
a good idea.
A user who can access the encrypted file can (locally) also read the
xattrs -- so the secrecy normally achieved by encryption is defeated 
this way. (There may be valid scenarios, though, e.g. when the file
tself is only accessible via a remote protocol that does not expose
the xattrs, such as http or DAV or NFS.)
.br
You can specify
.B kxfallb[ack]
and
.B ixfallb[ack]
in addition if you want ddr_crypt to try using xattrs and falling back
to keysfile and ivsfile in case the file system does not support the
extended attributes.
.
.SS Password based key and IV generation
Using the same key/IV for many files harms security severely (see
below in ALGORITHMS). So using a directly specified (non-generated)
key is not a good idea. However, if you prefer to have something
memorable rather than stored, you can use a password and salt
to generate many keys from one password.
.P
The key and IV are derived from an expensive to compute (and
even more expensive to revert) function of password and salt.
By default, ddr_crypt uses 17000 rounds of pbkdf2() for the
key (and a third for the IV),
although a more compute intense function (like scrypt) is planned
for the future. The expensiveness of this function is a protection
against brute forcing passwords. To use pbkdf2, you need to specify
.B pbkdf2
or
.B pbkdf2=rounds .
The latter format allows overriding the number of iterations for
key generation. (IV generation will be done with a third again.)
.P
For compatibility with openssl, key and IV can also be derived
using an openssl compatible key derivation function with
.B opbkdf .
Note that this is not recommended; only one round of md5 hashing
is used which makes brute-forcing very effective. Using this option
also has the side-effect of writing (encryption) or parsing
(decryption) an openSSL style Salted__ header. Note the openssl
version 1.1 started to default to one round of sha256 hashing
instead which can be forced on older openssl versions with
-md sha256 and overriden by specifying -md md5 on the openssl
command line. You can instruct dd_rescue to use an openssl
compatible KDF with sha256 by specifying
.B opbkdf11 .
One round of sha256 can of course still be very efficiently
brute-forced, so use high-entropy passwords if you really
need to use this.
.B nosalthdr .
Openssl starting with version 3.0 does no longer write a Salted__
header if the salt is passed explicitly. For compatibility with 3.0+,
this option allows to no longer write this header (on encryption)
nor to look for it (on decryption).
.P
The salt can be derived automatically from the name (and length)
of the encrypted file; this allows to work with just one password
to be memorized. However, be aware that file size or name changes
will result in a different salt and thus different key/IV which
render your encrypted file undecryptable. If there is a risk of this
to happen, rather memorize one salt per file (or better save key
and IV using keysfile and ivsfile options or save the salt using
saltsfile or saltxattr, see below). Remember that file names
are case sensitive (as always with Un*x). Of course the keysfile
needs to be well protected from being read by unauthorized persons.
.P
Password and salt can be specified with a string
.B pass=
and 
.B salt=
or using the 
.B passfd=
.B passfile=
and 
.B salthex=
.B saltfd=
.B saltfile=
options with the same possible parameters as above for direct specification
of key and IV. (Note that the salt is hashed, like when derived from file name
and length.) The password/passphrase is treated as a string, null-terminated
and with a trailing CRLF stripped off.
.br
The warnings about passing confidential data (here: pass, salt, salthex) on 
the command line apply -- only do it for testing or in a single-user
environment. Using passfd=0 allows to pass in a password via a pipe.
.P
If the file name based automatic salt derivation is used, the assumed file
length for salt generation can be overridden by
.B saltlen= .
.P
Alternatively, the salts can also be stored and retrieved from an MD5SUMS style
index file (like with keysfile and ivsfile) by specifying the option
.B saltsfile .
When saltsfile is used to store salts, using random salts on encryption
becomes a good idea. This can be achieved by specifying the
.B saltgen
option.
.P
Instead of a salt index file (saltsfile), the salt can also be stored in 
(and retrieved from) an extended attribute. This can be done using the
.B saltxattr[=xattr_name]
option. The attribute name is optional and defaults to user.salt.ALGNAME
(with ALGNAME replaced by the algorithm). Since ddr_crypt 1.99, the
password-based key derivation function (and the number of iterations) is
also stored and retrieved in the xattr user.pbkdf with this option.
.br
It's also possible to try using the xattr feature and fall back to using
the index file (saltsfile) if your file system does not support extended attributes.
Use the
.B sxfallback
option to tell ddr_crypt to do this. Note that the pbkdf can not be stored
(or retrieved) if the fallback actually takes place.
.br
See the main dd_rescue (1) man page for a discussion of advantages and
disadvantages of using extended attributes.
.
.SH ALGORITHMS AND STREAM MODES
The AES (Rijndael) family of algorithms is considered cryptographically
safe at the time of writing, as no practicable attacks have been published
against it. It is up to the reader to judge whether (s)he believes that
the worst criminals or intelligence agencies are significantly ahead
of common (published) knowledge. In reality, it is typically easier to
use social engineering or flaws in key handling and random number generation
to carry out attacks.
.
.SS Plus modes
Given that the best known attacks are against AES versions with a reduced 
number of rounds with only small round number reductions, it appears
that increasing the number of rounds would seem a reasonable countermeasure
against cryptographic attacks. (This has been inspired by a comment from
Bruce Schneier who the author of this document has very high respect for.)
.P
The C and AESNI implementations support AES128,192,256 modes with 2,3,4
additional rounds respectively, resulting in 12, 15, 18 rounds. These
modes are named AES128+, AES192+, and AES256+ (plus modes) respectively. 
They do offer a computationally relatively cheap
way to enhance security. The author of this document e.g. would
chose AES192+ over AES256. While the author of this document would never
judge himself as a cryptography expert strong enough to create new
algorithms or even devise significant changes to existing ones, he 
considers this variation a choice that is more secure than the original.
Please note however, that these custom algorithms result in files that
can not be decrypted using any other tools. Also, the openSSL engine does
not support the plus modes.
.
.SS Double modes
A computationally more expensive method to enhance security is doubling
the number of rounds. This is equivalent to encrypting twice (where the
second key is a simple derivation of the first).
These methods are supported by all engines and are named AES128x2, AES192x2,
and AES256x2.
.
.SS Stream modes
The AES algorithm is a block cipher -- it transforms 16 byte blocks.
The trivial application to a file of arbitrary size is to apply this
to every block in the file. This is called ECB (electronic codebook)
mode. This is very insecure ... the same input will always result
in the same output. Patterns can be easily recognized and known
plain text attacks are trivial.
.P
It's better to make the transformation dependent on the previous
content of the file or the position within it. This is what the
CBC (chained block cipher) and CTR modes do.
.P
The CBC mode has several disadvantages: It can't be parallelized
(every block depends on all previous blocks for encryption; things
are better for decryption) and random access is impossible.
.P 
The CTR mode has many desirable properties. It is basically a stream
of (reproducible) pseudo random numbers that are XORed with the input
for encryption. Decryption is just another XOR of course. It's a
one time pad -- which has been proven to be secure, if the pad is
unknown to an attacker and only used once.
.br
The latter can't be stressed enough: Don't ever use the same key/IV
combination for two files. Mathematically spoken:
c1 = r1 XOR p1 and
c2 = r2 XOR p2
(c = ciphertext, r = AES random numbers, p = plaintext).
With r == r1 == r2, it can be trivially seen that the attacker can calculate
c1 XOR c2 = r XOR p1 XOR r XOR p2 = p1 XOR p2. If the plaintext of one of
the files is partially known, so is the other.
.P
The CTR mode has more nice properties: It allows random access as the
AES random numbers (belonging to a key/IV combination) with a certain 
offset can be directly calculated and the last block does not require
padding, as partial blocks can be processed.
.P
The author of this documents prefers CTR stream mode and ensures
that keys/IVs are not reused.
.
.SH Supported dd_rescue features
With CTR mode, you can do partial writes to encrypted files and
the result will still be a consistent file (of course assuming
that the used key and IV are the same). Same with appending (-x)
or reverse direction copies.
.br
With ECB mode, this will only work, if file size and offsets
are all block (16byte) aligned. With CBC, none of this is possible.
.P
The ddr_crypt plugin has support for encoding holes;
if holes happen through read errors and skiphole is set, the
areas will be passed over and nothing gets written at the
corresponding place in the output file, leaving intact what
was there before (or zeroes if nothing was there). This only
works for CTR and ECB modes, not CBC.
.br
You can pass the option
.B skiphole
to make sure ddr_crypt leaves 512byte-sized blocks of zeros untouched.
This will reveal blocks of zeros and may thus disclose valuable
information to an attacker, so use with care. Also note that you
need to use this with en- and decryption and with the same 
alignment (mod 512) for encryption to be reversible. You have
been warned. (You don't need to be worried about misdetecting
zeros on decrypting -- the chances of non-zero plaintext resulting
in an aligned 512byte block of zeros is smaller than 2^-4096. So
this option is safe on decrypting -- if some of the ciphertext
has been overwritten with blocks of zeros, you might even prefer
to have zeros in the decrypted file rather than random gibberish.)
If you detect empty blocks (dd_rescue option -a), you may use
skiphole to make ddr_crypt skip over the holes in sparse files.
If you don't, the zeroes will be properly encrypted (since 1.99.19),
resulting in output that can be decrypted also with openssl for
example.
So, you have three options to encrypt files with holes:
.br
(A) You may use -a and skiphole (in CTR or ECB mode) for both
encryption and decryption. This results in an encrypted file
that only dd_rescue with ddr_crypt can decrypt.
.br
(B) You do not use any of these options on encryption. For
decryption, you pass -a though to detect empty blocks before
they are written to disk and produce holes rather than zeroes.
.br
(C) (New with 1.99.19:) You may pass -a on encryption without
skiphole and still create output with the zeros from the holes
properly encrypted. They will be decrypted to 0 again and
with -a will even result in holes in the decrypted file.
.P
Note that you can also compress and encode holes with ddr_lzo and then
pass to ddr_crypt to encrypt and pass through ddr_crypt to decrypt
and ddr_lzo to uncompress and extract holes again. This only works
with CTR mode. 
.P
The option
.B weakrnd
is provided for testing in environments, where strong random
numbers are not available. It will cause weaker random numbers
to be used for key generation. Don't use it if you want security.
.
.SH openssl compatibility
Files that are encrypted with openssl enc where you specify
the key (with -K) and the IV (with -iv) result in the same
output that ddr_crypt generates for -ECB and -CBC modes.
ddr_crypt uses a 64bit counter in -CTR modes.
.P
With the option
.B opbkdf
ddr_crypt also reads/writes the openSSL Salted__ header to
be compatible with openssl. This function needs more testing
and better error handling though.
.
.SH BUGS/LIMITATIONS
.SS Maturity
The plugin was introduced with dd_rescue 1.98. It has seen quite
some usage and testing since, but some care may still be warranted
relying on it as only backup for very valuable data.
.P
Due to an issue in ddr_crypt's initialization of the IV for
CTR mode in dd_rescue 1.98, the last 32bits would always be zeroed
out prior to adding the counter. This has been fixed in 1.99.
It order to be compatible with 1.98, the option
.B ctrbug198
can be specified on the command line.
.br
.
.SS Security
While care has been applied to check the result of memory allocations ...,
the code has not been audited and only limited fuzzing
has been applied to ensure it's not vulnerable to malicious data -- 
be careful when you process data from untrusted sources.
.br
Key handling is a tricky business -- the author may have screwed up
in non obvious ways, resulting in some ways to use this program to
encrypt data may not result in the level of secrecy that is desired.
.
.SS Testing
The crypt plugin has meanwhile seen reasonable test coverage, but
some more is certainly desirable.
.
.SS Plugin chain ordering
If you chain several plugins (dd_rescue supports this), please ensure
that no plugin changes the size/offsets after ddr_crypt on encryption
and none changes the size/offsets before ddr_crypt on decryption.
If you do, the calculation of the position in the file will be broken
and a wrong value will be used for the counter value in CTR mode.
This does not affect ECB, but ECB is not a good choice normally.
This limitation means that you can compress and encrypt and that you
can decrypt and decompress. This was a design choice -- as encrypted
date is incompressible, this other order also does not make any
sense.
.
.SS Future work
Except for more testing and auditing a few more features are envisioned
for this plugin:
.br
Support for other (non-AES) algorithms such as twofish (and
possibly also threefish).
.br
Stronger function to derive keys/IVs from passwords than pbkdf2.
.br
Support for other streaming modes (XTS, GCM, ...)
.
.SH EXAMPLES
.TP
.BI dd_rescue\ \-ptAL\ crypt=algo=AES256-CTR:enc:keygen:ivgen:keysfile:ivsfile\ infile\ outfile
encrypts data from
.IR infile
with AES256 in CTR mode using a generated (random) key and IV and writes the result to
.IR outfile
.
It adds a line to KEYS.AES256-CTR and to IVS.AES256-CTR where the used key
and IV are written to respectively. (Please ensure that this file is not accessible
by any unauthorized person!) Decryption can be performed by
.P
.BI dd_rescue\ \-ptAL\ crypt=algo=AES256-CTR:dec:keysfile:ivsfile\ outfile\ infile
.
.TP
.BI dd_rescue\ \-AL\ crypt=AES192+-CTR:enc:saltgen:saltxattr:sxfallback:passfd=0:pbkdf2\ infile\ outfile
will ask for a password, generate a random salt and store it in the extended attribute of
.B outfile 
(and fallback to SALTS.AES192+-CTR index file) and uses pbkdf2 function to produce a key
and IV for encrypting the data. For decrypting, just omit the
.B saltgen 
parameter.
.TP
.BI dd_rescue\ \-ptL\ lzo=compr,crypt=AES256-CTR:enc:keygen:ivgen:keysfile:ivsfile\ infile\ outfile
will compress the data (using lzo) and then encrypt. Use the reverse order and omit keygen and ivgen
to decrypt and uncompress. Compression has the nice side effect of dealing with holes, which otherwise
get compressed to non-zero values (unless you specify skiphole). Feel free to add the hash plugin
at the beginning and/or the end to produce cryptographic checksums for both the original file and the
end result.
.
.SH SEE ALSO
.BR dd_rescue (1)
.
.SH AUTHOR
Kurt Garloff <kurt@garloff.de>
.
.SH CREDITS
The x86 AESNI optimized AES implementation has been inspired by
an intel whitepaper from 2009:
https://software.intel.com/sites/default/files/article/165683/aes-wp-2012-09-22-v01.pdf
.br
The ARMv8 AES support has been inspired by studying openSSL assembly as well
as Linaro's in-kernel implementation.
. 
.SH COPYRIGHT
This plugin is under the same license as dd_rescue: The GNU General 
Public License (GPL) v2 or v3 - at your option.
.
.SH HISTORY
ddr_crypt plugin was first introduced with dd_rescue 1.98 (May 2015).
.br
Version 1.99 brought support for ARMv8 crypto acceleration and support
for openssl style key derivation and Salted__ headers. It also added storing
pbkdf related infos in xattrs and added support for storing and retrieving
keys (not recommended!) and IVs in/from xattrs. A bug with CTR initialization
was resolved (see ctrbug198 option). 1.99.19 brought clean handling of holes
also in reverse file direction (for CTR modes). reverse files could differ
from forward files before 1.99.19.
.PP
Some additional information can be found on
.br
http://garloff.de/kurt/linux/ddrescue/