-
Notifications
You must be signed in to change notification settings - Fork 3
3_1 Linux
This page describes how file handles are structured so you can understand attacks that can be performed on misconfigured servers.
A Linux file handle consists of three parts.
The first part is always 4 bytes long and contains information on how to parse the other two parts.
The second part is the file system identifier, referred to as fsid which has a variable length. It is used to identify the file system and the export containing a file.
The third part is the fileid which also has a variable length and identifies a file or directory within an export. Its structure depends on how the underlying file system identifies files.
As an example, we will look at the following file handle which has been generated by a Linux server:
meta fsid fileid
|======||================================================||==============================|
0x0100070201fd05000000000065e9e814e14742e0b4b70279933adc3778e006009b9d890402fd0500706bf7e7
It is interpreted in the following way.
| Data | Len | Meaning |
|---|---|---|
| 0x01 | 1 | Version, always 0x01 |
| 0x00 | 1 | Authentication, always 0x00 |
| 0x07 | 1 |
fsid type, specifies which type of identifier is used to identify the file system and export |
| 0x02 | 1 |
fileid type, specifies how files are identified within the export |
Since the third byte specifies that the file system id has type 7, it consists of the following fields:
| Data | Len | Meaning |
|---|---|---|
| 0x01fd050000000000 | 8 | inode of export root directory |
| 0x65e9e814e14742e0b4b70279933adc37 | 16 | uuid of the file system |
As specified in the fourth byte, the fileid has type 2 and can be broken up into the following parts.
| Data | Len | Meaning | |
|---|---|---|---|
| 0x78e00600 | 4 | inode | |
| 0x9b9d8904 | 4 | generation | |
| 0x02fd0500 | 4 | inode of parent directory | |
| 0x706bf7e7 | 4 | generation of parent directory |
This fileid structure is used to identify files and directories in exports on ext4 and xfs partitions.
In some cases, fileid type 1 which does not contain information about the parent is used to refer to directories.
The inode and generation number are stored in little endian. Since file handles are not supposed to be interpreted by clients, these fields are not converted to network byte order.
For more information on the file handle structure, see fs/nfsd/nfsfh.h and include/linux/exportfs.h.
As we could see, a file handle contains an identifier for the export and one for the file within the export.
In our previous example, the export was identified by the file system uuid and the inode of the export directory.
There is a special case when the export is not the root directory of the file system.
By default Linux does not check if the file a handle refers to is inside the exported directory.
A client could request an operation on a file handle that refers to any other file on the same file system.
The check to protect against this has to be explicitly enabled by setting the subtree_check option on the export.
This means that by default it is possible to read other files that are outside the export directory but on the same file system.
The check works by starting at the requested file and then walking up the file system tree until either the export directory or the system root is reached or there is a permission error.
If the export directory is reached, the check is successful.
The implementation and the acceptance conditions of subtree_check can be found in nfsd_acceptable in fs/nfsd/nfsfh.c
Since one file can be in multiple directories, the file handle contains the inode and generation of the parent directory in order to identify which hardlink it refers to.
A file handle should remain valid and point to the same file even after it has been moved to a different directory.
When subtree_check is enabled, this property is not fulfilled because the information about the parent directory in the file handle is not correct anymore.
This will cause operations on files that have been moved to fail.
For this reason it is recommended to always use the root directory of a file system as an export.
If you are sure that this access pattern does not occur, e.g. in a readonly export, you can use subtree_check.
A common way to log file accesses on Linux is auditd.
However, auditd cannot be used to log file accesses performed by NFS clients because the NFS server runs in the kernel and bypasses the auditing.
There are some tracepoints in the NFS request handlers which could in theory be used to monitor accesses, however there does not seem to be a tool for this.
This section shows a few things that can go wrong when configuring NFS exports on a Linux server. On Linux exports are configured in the /etc/exports file where each line has the following format:
/PATH/TO/EXPORT HOST1(OPTIONS1) HOST2(OPTIONS2) ...
Line starting with # are comments
Let's look at a configuration that is vulnerable to the attack described previously.
Consider the following configuration where /mnt/storage is the root directory of a file system.
# Give write access to the private directory to the specified subnet using Kerberos authentication with encryption
/mnt/storage/private 192.168.2.0/24(rw,subtree_check,sec=krb5p)
#Gives read access to the public directory to everyone, no_subtree_check and sec=sys by default
/mnt/storage/public *(ro)
In this case anyone can read files from the private directory because the export for public does not have the option subtree_check set.
A similar problem occurs with the following configuration. It is documented in the man page of exports.
/mnt/storage 192.168.2.0/24(rw,subtree_check)
/mnt/storage/private 192.168.2.0/24(rw,subtree_check,sec=krb5p)
While the export /mnt/storage/private can only be mounted with Kerberos enabled, it is still possible to mount /mnt/storage with Unix authentication and access files in the private directory.
It is also possible to mount /mnt/storage and replace the private directory with a symlink to any other directory on the server. When the NFS service restarts, it will follow the symlink and export the directory it points to.
The subtree_check option does not prevent these two attacks.
The problem can also occur when bind mounts are used, therefore it is important to note (despite others claiming something different on the internet) that bind mounts do not provide isolation.
Consider the following fstab.
/dev/sda2 / ext4
/dev/sdb1 /mnt/storage ext4
/mnt/storage/public /srv/nfs/public bind
The /etc/exports files is defined as follows.
/srv/nfs *(rw,fsid=root)
/srv/nfs/public *(rw)
The first export /srv/nfs is a subdirectory of the filesystem on /dev/sda2, therefore all files on the operating system partition are accessible.
The other export /srv/nfs/public is a bind mount to /mnt/storage/public which is a subdirectory of /dev/sdb1. This means that all files in /mnt/storage are accessible.
Both exports are subdirectory exports and should use the subtree_check option.
The fsid=root will cause the NFS file handles to refer to the file system using the user-defined identifier 0 instead of the volume uuid but it doesn't prevent clients from accessing files outside the exported subdirectory.
Another potential error is mentioned in the Red Had Documentation
/mnt/storage 192.168.2.123 (rw)
This configuration gives everyone write access and 192.168.2.123 read access to the export.
Because of the space between the IP address and the parentheses, this is interpreted as two different entries.
One for 192.168.2.123 with default settings because there are no parentheses and one for everyone with option rw.
After making changes to /etc/exports and restarting nfs-server.service to apply the changes, it can be helpful to run journalctl -u nfs-server.service and look at the output in order to check for errors.
In some cases the NFS server may appear to behave normally even if there are errors present in the exports file.
If the output contains the following line:
exportfs: No options for /EXPORT/PATH : suggest (sync) to avoid warning
This means that there might be an export with space between a host and the options on that export.
If the output contains the line:
Neither 'subree_check' or 'no_subtree_check' specified for export "CLIENT:/EXPORT/PATH"
This means that for some export neither subtree_check nor no_subtree_check is set and that by default no_subtree_check is set.
If the export is the root of a file system, it is safe to add the no_subtree_check option.