The DFS Protocol

What is DFS?

• Distributed FileSystem.
• Mechanism that allows clients to access files shared on different servers.
• Like symbolic links but where the target of the link can point to path on a different host.

What is DFS?

• example DFS path:
  • \\dfs-server\namespace\link
• can point to:
  • \\fileserver\share\dir

Use cases

• Unify different file server under the same namespace.
  • Users only need to remember one server.
• Recover from server failure.
  • A link can point to multiple possible targets.
• Automatically use most efficient link
  • e.g. connect to closest mirror, geographically.
• The synchronization between mirrors is handled by a different protocol. See DFS-R.

The DFS protocol

• The protocol allows multiple kinds of referrals, but we are only interested in those 2:
  • Root referrals, which identify the DFS root targets of a specific DFS namespace.
  • Link referrals, which identity the DFS link targets of a specific link in a DFS namespace.

The DFS protocol

• Described in [MS-DFSC].
• Theoretically usable with different protocols (SMB, NFS, NCP, ...)
• [MS-CIFS] and [MS-SMB2] have a section that explains how the DFS queries are encapsulated.
• [MSDFS] also gives a pretty good overview.

DFS versions

• The DFS payload has 4 versions.
• Minimal changes between versions, some flags are added but the basic layout is the same.

DFS namespaces

• 2 modes:
  • Stand-alone DFS namespace.
  • Domain-based namespace.

Stand-alone DFS namespace

• No domains/DC are required.
• The client connects directly to the server that holds the links and queries it to access resolve the DFS path to the final path to the server that holds the file.

Domain-based DFS namespace

• Adds an indirection level.
• DFS path use the domain as the path hostname and the client queries the DC to get the server that holds the links of a namespace which in turn is queried to fully resolve the path to the final server that has the file.

Resolving DFS paths in SMB1

• Negotiation: server sets CAP_DFS.
• Session setup: nothing required.
• Tree connect response: server sets "share is in DFS" flag.
• If the client makes a request with a file path DIR\FOO it should instead use \HOST\SHARE\DIR\FOO and set the SMB_FLAGS2_DFS bit in the SMB header.
• If the file is on another host, the server fails with STATUS_PATH_NOT_COVERED.
• When a client query fails with such code, the client does a DFS referral query to ask the server what the path resolves to.
• Server responds with a list of targets.
• Client connects to one of the targets.

Resolving DFS paths in SMB2+

• The DFS querying can now be made on any connection. Previously it was on the IPC share only.
• The query is now a FSCTL (previously it was a TRANS2 sub-command).
• Same payload.

DFS in the CIFS kernel module

cifs.ko

• cifs.ko is the kernel module that allows users to mount SMB shares.
• A mounted share exposes a (mostly) POSIX filesystem.
• Some features are faked and some system calls are not supported, depending on what the server supports.

Linux FS data structures

• cifs.ko uses the kernel VFS system.

• genericity is achieved through structures of function pointers for operations.

• superblocks are the metadata structures associated with mount points, they also store a pointer to the root directory entry.

• directory entries (dentry) store the hierarchy of the fs (file names, pointers to siblings and children, ...) and hold a pointer to an inode.
• inodes store file metadata (permissions, uid/gid, size, ...).
• file handles are the context associated with opened files.
• Each of these structures have associated operations that a filesystem must implement.

superblock operations

    struct super_operations {
        struct inode *(*alloc_inode)(...);
        void (*destroy_inode)(...);
        void (*dirty_inode) (...);
        int (*write_inode) (...);
        int (*drop_inode) (...);
        void (*evict_inode) (...);
        void (*put_super) (...);
        int (*show_path)(...);
        int (*show_stats)(...);
        /* ... */
    };

dentry operations

    struct dentry_operations {
        int (*d_delete)(const struct dentry *);
        int (*d_init)(...);
        void (*d_release)(...);
        void (*d_prune)(...);
        void (*d_iput)(...);
        char *(*d_dname)(...);
        struct vfsmount *(*d_automount)(...);
        int (*d_manage)(...);
        struct dentry *(*d_real)(...);
        /* ... */

    };

inode operations

    struct inode_operations {
        struct dentry * (*lookup) (...);
        const char * (*get_link) (...);
        int (*permission) (...);
        struct posix_acl * (*get_acl)(...);
        int (*readlink) (...);
        int (*create) (...);
        int (*link) (...);
        int (*unlink) (...);
        int (*symlink) (...);
        int (*mkdir) (...);
        /* ... */
    };

file operations

    struct file_operations {
        loff_t (*llseek) (...);
        ssize_t (*read) (...);
        ssize_t (*write) (...);
        uint (*poll) (...);
        long (*unlocked_ioctl) (...);
        long (*compat_ioctl) (...);
        int (*mmap) (...);
        int (*open) (...);
        int (*flush) (...);
        /* ... */
    };

CIFS virtual layer

• On top of this, cifs.ko also tries to reuse code to abstract differences between protocol version and adds another virtual layer.
  • version operations (session setup, tree connect, open, read, close, ...;).
  • version values (header sizes, capability flags, ...;).

CIFS operations

• There are a lot of operations.
• There is one for DFS referral.

    struct smb_version_operations {
        int (*sess_setup)(...);
        int (*logoff)(...);
        int (*tree_connect)(...);
        int (*tree_disconnect)(...);
        int (*get_dfs_refer)(...);
        /* ... */
    };

DFS in cifs.ko: entry points

• The DFS querying can be triggered in 2 ways:

  • At mount time, if the user gives a DFS path.
  • Or while accessing a subfolder which is actually a DFS link.

• In the later, cifs.ko marks the DFS link as a directory.

• The corresponding dentry has a special AUTOMOUNT flag and is actually "empty".

DFS in cifs.ko: AUTOMOUNT

• The automount flag is a mechanism provided by the kernel VFS API.
• Introduced specially for networked filesystems.
• Allows to lazily mount folders when they are accessed.
• The dentry is filled via a user-provided callback function.

DFS in cifs.ko

• Prior to 4.11, get_dfs_referral() CIFS operation was only implemented for SMB1.
• Only stand-alone DFS namespace are supported.
• Following commits added support for it in SMB2 and above.
• Let's go over some of the changes needed.

    284316dd42a2 CIFS: Fix sparse warnings
    9d49640a21bf CIFS: implement get_dfs_refer for SMB2+
    f0712928be1a CIFS: use DFS pathnames in SMB2+ Create requests
    b9043cc5b99e CIFS: set signing flag in SMB2+ TreeConnect if needed
    7f9f6d2ec514 CIFS: let ses->ipc_tid hold smb2 TreeIds
    51146625f34b CIFS: add use_ipc flag to SMB2_ioctl()
    268a635d414d CIFS: add build_path_from_dentry_optional_prefix()
    4ecce920e13a CIFS: move DFS response parsing out of SMB1 code

DFS in cifs.ko: parsing DFS responses

• The DFS payload is independent of the SMB version.
• First step was to make the code that parses the response generic.

DFS in cifs.ko: paths

• cifs.ko originally only supported SMB1.
• Many parts of the code still assumes it.
• e.g. build_path_from_dentry() prefixes the path with the tree name (host + share) when connected to a DFS share, as required by SMB1 on packets containing paths.
• Sometime we want the path without it.
• Sometime we want a path with it, no matter the type of share.
• We don't want to only check for the SMB1 flag.

    @@ -92,7 +103,7 @@ build_path_from_dentry(struct dentry *direntry)
            unsigned seq;

            dirsep = CIFS_DIR_SEP(cifs_sb);
    -       if (tcon->Flags & SMB_SHARE_IS_IN_DFS)
    +       if (prefix)
                    dfsplen = strnlen(tcon->treeName, MAX_TREE_SIZE + 1);
            else
                    dfsplen = 0;

DFS in cifs.ko: connection

• In SMB1, the DFS query had to be made on the IPC share.
• Not a requirement anymore (ioctl on special fid NO_FILE_ID).
• Since samba was also requiring the call to be made on IPC:
  • Added a use_ipc flag to SMB2_ioctl().
  • Try to use the IPC first, fallback on regular connection otherwise.
• ...and fix samba

DFS in cifs.ko: Tree Id storage

• In SMB1, the Tree Id is stored on 2 bytes.
• In SMB2+, it's on 4 bytes.
• Somehow this hadn't been updated yet!
• Large Tree Id where truncated when stored and thus rejected.

DFS in cifs.ko: implement get_dfs_refer

• Everything ready, we can now write the actual operation.
• Craft the request packet.
• Send it over on the connection.
• Parse the output.
• Write node list in the output.
• DFS available in SMB2!

Future work

Remaining issues: domain-based DFS

• cifs.ko only supports querying stand-alone DFS servers.
• When the DFS server is not hosted on the DC it won't work.

Remaining issues: high availability

• The DFS protocol provides high availability but...
• cifs.ko doesn't really take advantage of it.
• It connects and only keeps the first target that successfully mounts.
• In case of failure, remounting will fail.

Remaining issues: referral cache

• Making DFS queries can be slow.
• In multiuser mode, cifs.ko can make many redundant DFS queries for every single mount.
• MS-DFSC describes in detail client caching.
• How it works, when the cache should expire, etc.
• None of this is implemented.
• cifs.ko drops all results once it is done with it.