MacFUSE: New Frontiers in File Systems
Pages: 1, 2
read
Next, we'll implement the read function so that users can read from our file system.
static int
hello_read(const char *path, char *buf, size_t size, off_t offset,
struct fuse_file_info *fi)
When we get a read call, we need to do what the parameters tell us to do. The path parameter gives the name of the file, buf is a buffer to put bytes into, size gives the number of bytes to return, and offset lets the read begin at a specific byte offset within the file.
Here's the first part of the read code:
{
if (strcmp(path, file_path) != 0) {
return -ENOENT;
}
We first ensure that we're reading from our one and only file, else we return an error.
Next, we check the number of bytes to be read:
if (offset >= file_size) { /* Trying to read past the end of file. */
return 0;
}
Here we test whether we're trying to read at an offset that's greater than the number of bytes in the file. If so, there are no bytes to return, but also no error, so we return 0 to indicate that no bytes were read.
The next test makes sure we're not going to read more bytes than are available:
if (offset + size > file_size) { /* Trim the read to the file size. */
size = file_size - offset;
}
This code checks to see whether reading the given number of bytes from the desired offset will take us past the end of the file. If so, we modify the number of bytes to be read so that we read to the end of file and no further.
Now we're ready to read the file:
memcpy(buf, file_content + offset, size); /* Provide the content. */
Finally, we read the requested bytes. We use memcpy() to move bytes into the buffer from the file, starting at the given offset location in the file.
return size;
}
And we're done, returning the number of bytes read.
getattr
The last operation we need to implement is getattr, which simply returns the attributes of a file or directory:
static int
hello_getattr(const char *path, struct stat *stbuf)
We start by zeroing out the buffer we've been given that will hold the attributes structure:
memset(stbuf, 0, sizeof(struct stat));
This prevents uninitialized memory from ruining our day.
Next, we fill in the attributes structure, pretty much by brute force. There are only two possible entities we can be asked about: the root directory and the L file. We'll deal with the root directory first:
if (strcmp(path, "/") == 0) { /* The root directory of our file system. */
stbuf->st_mode = S_IFDIR | 0755;
stbuf->st_nlink = 3;
After learning that the first parameter is the root directory (it's just a slash), we set certain fields of the incoming stat structure. Specifically, we set the st_mode field to S_IFDIR | 0755, which specifies that this is a directory with its Unix permissions being 0755 (or "rwxr-xr-x"). We set the st_nlink field to 3. For a directory, the st_nlink field should be set to the total number of entries within the directory. Here, we say 3 because besides our hello.txt file, which accounts for one entry, we technically also have the "dot" and "dot-dot" entries, for a total of 3. If you think that's strange, well, that's just Unix convention.
The only other legal case is hello.txt. Let's handle that one now.
} else if (strcmp(path, file_path) == 0) { /* The only file we have. */
stbuf->st_mode = S_IFREG | 0444;
stbuf->st_nlink = 1;
stbuf->st_size = file_size;
First, we make sure the path parameter is hello.txt. If it is, we return S_IFREG | 0444 for the st_mode field (regular file, permissions being 0444 or "r--r--r--"), 1 for the st_nlink field, and our file_size constant as the size. Note that in the case of a regular file, the st_nlink field should contain the number of hard links to the file. Here, we don't even implement hard links in our file system, so we have to say 1 as the only possible sane value.
Because those are the only files we have, we need to return an error for any other filenames we might somehow be asked about:
} else { /* We reject everything else. */
return -ENOENT;
}
Otherwise, all is well, so we return 0, and we're done:
return 0;
}
And the Rest
Here's the final bit of code for our HelloWorld file system. First is a structure that points to our file system's operations (note that we only enumerate the operations we implement):
static struct fuse_operations hello_filesystem_operations = {
.getattr = hello_getattr, /* To provide size, permissions, etc. */
.open = hello_open, /* To enforce read-only access. */
.read = hello_read, /* To provide file content. */
.readdir = hello_readdir, /* To provide directory listing. */
};
We kick things off with our main function, which takes the operations structure as a parameter:
int
main(int argc, char **argv)
{
return fuse_main(argc, argv, &hello_filesystem_operations, NULL);
}
That's it. We've implemented an actual file system in very little code. Although our file system is just a demo that doesn't do anything useful, it's real and it works, and its simplicity has been known to make kernel extension programmers weep with joy.
Trying It Out
OK, enough looking at source: it's time to actually use this stuff. The first step is to get and install MacFUSE. This part is very easy: there's even a true Mac installer available on the MacFUSE project site's download section. You should download the latest version of the "MacFUSE Core Installer Package" from the aforementioned page, and run the installer to get MacFUSE up and running.
Adding our HelloWorldFS requires a little more work, because we have to build it. First, grab the hellofs.c source by going to http://code.google.com/p/macfuse/wiki/HELLOWORLDFS and copying the source, then pasting it into a text file named hellofs.c. Then, assuming you have Apple's Developer Tools (Xcode) installed (version 2.4.x), compile it like this from the Terminal command line:
$ gcc -o hellofs hellofs.c -lfuse
That's it! Now we're ready to create a volume and mount it with HelloWorldFS:
$ mkdir /tmp/hello
$ ./hellofs /tmp/hello -oping_diskarb,volname=Hello
Note that we're specifying a couple of options when we mount the volume. The ping_diskarb option causes the volume to appear in the Finder, and the volname option gives the name to use for the volume.
After executing these commands, you should see the volume Hello in the Finder. If you look at the volume, it contains one file: hello.txt. You can double-click hello.txt to open it in TextEdit and see that it contains the text "Hello World!". You can even edit the text, but if you try to save the document, TextEdit complains, because HelloWorldFS is a read-only file system. (Of course, you can save it to a regular volume.)
When you're done having fun with HelloWorldFS, you can unmount the volume in any of the usual ways from the Finder, or like this from Terminal:
$ umount /tmp/hello
MacFUSE Belongs to You
Now that MacFUSE has been released (although it's not quite final quality yet), you can start using it on your own Mac. You can begin with the MacFUSE Quicker Start Guide and the Downloads page. There are over a hundred FUSE file systems out there for Linux, and as long as an existing FUSE file system doesn't use Linux-specific functionality, it should work with MacFUSE. Several popular file systems have in fact been tested with MacFUSE. Two of the most popular are ntfs-3g, a read/write version of NTFS, and sshfs, which lets you access a remote computer's storage as a file system, assuming you have an SSH connection to that computer.
Because MacFUSE makes file system coding much easier, you can think very creatively about what kinds of data might be represented as useful file systems. For example, Greg Miller created SpotlightFS, a file system that represents Spotlight queries as folders in a file system, with the query results appearing as files in folders. For example, you can create a folder named "Brian Wilson," and the folder contents will be all files that Spotlight finds when you send it the same "Brian Wilson" query. In fact, you can even use complex Spotlight queries, such as this:
kMDItemKind == "*PDF*" && kMDItemAuthors == "Scott"w && kMDItemNumberOfPages > 1
And because SpotlightFS is a real file system, the folders and files on a SpotlightFS volume are real as well, and can be manipulated with file-handling applications (such as the Finder) and by file I/O code that you write. SpotlightFS is just an example of what you can do with MacFUSE. You can use your own creativity to think up other clever and useful file systems that can be brought to life with FUSE. Now go out and create something great!
More Information
To keep up with the pulse of MacFUSE, or to help out on the project, see the project home page. For fun and inspiration, be sure to check out this video that shows several MacFUSE tech demos in action.
Late Breaking News
The MacFUSE project now includes an Objective-C library that supplies a template MacFUSE file system. To use it, all you have to do is implement the particular methods you're interested in, such as open, read, write, and so on. This library handles details like volume icons and resource forks as well. You can find the Objective-C library and an example here: http://macfuse.googlecode.com/svn/trunk/filesystems-objc/.
Scott Knaster is a technical writer on the Mac team at Google.
Return to the Mac DevCenter.
You must be logged in to the O'Reilly Network to post a talkback.
Showing messages 1 through 7 of 7.
-
st_nlink is the number of hard links.
2007-03-14 09:47:49 ralph@inputplus.co.uk [Reply | View]
Hi Scott,
"For a directory, the st_nlink field should be set to the total number of entries within the directory."
I think this is incorrect. stat(2) should confirm that st_nlink is the path's number of hard links. For an empty directory, this is normally two; one for . in the empty directory, and one for the directory's entry in the parent.
$ mkdir foo
$ ls -ld foo
drwxr-xr-x 2 ralph ralph 4096 2007-03-14 16:17 foo
$ stat foo | g Links
Device: 301h/769d Inode: 3107002 Links: 2
$ ls -di foo foo/.
3107002 foo 3107002 foo/.
$
For the root directory it is at least three; /, /., and /... Make a directory /foo and /foo/.. will have the same inode number as /, incrementing st_nlink.
Cheers, Ralph. -
st_nlink is the number of hard links.
2007-03-14 23:30:00 osxbook [Reply | View]
Ralph, what Scott wrote is correct.
Directories don't have hard links on Mac OS X (well, not yet), or on any typical Unix system for that matter. The rationale is that with directory hard links, it would be all too easy to create cycles.
st_nlink *would* be the number of hard links, but for files. Looks like the man page for stat(2) fails to mention that.
On Mac OS X, for directories, you can either set st_nlink to the number of objects within the directory ("." and ".." + the rest), or if the volume is such that the number is hard to calculate, you can set st_nlink to 1.
-A -
st_nlink is the number of hard links.
2007-03-15 07:33:36 ralph@inputplus.co.uk [Reply | View]
Hi osxbook,
"Directories don't have hard links on Mac OS X (well, not yet), or on any typical Unix system for that matter."
Oh, but they do, at least on Unix as my post above shows. ls -ld foo is showing that foo's inode has a reference count of 2; the second word of output. And ls -di foo foo/. is showing that both paths have the same inode number, 3107002.
It's true that originally directories could also be hard linked with a system call but this was stopped because confusion could result, including hierarchy cycles. However, the kernel never stopped using hard links on directories, for example to implement .., and if you are writing a filesystem you still set up those hard links when making a directory; I doubt FUSE alters this.
As for Mac OS X altering this aspect, perhaps it does but I doubt it. Here's a test to do on the native, non-FUSE, shipped by Apple, filesystem.
$ mkdir foo
$ cd foo
$ ls -ld .
drwxr-xr-x 2 ralph ralph 4096 2007-03-15 14:29 .
$ touch a b c d e f g h
$ ls -ld .
drwxr-xr-x 2 ralph ralph 4096 2007-03-15 14:29 .
$ mkdir 1 2 3
$ ls -ld .
drwxr-xr-x 5 ralph ralph 4096 2007-03-15 14:29 .
$
Keep an eye on the second word of ls(1)'s output. On a newly created, empty, directory it should be 2. Creating files in the directory doesn't change this. But creating subdirectories does; the link count is incremented because 1/.., 2/.., and 3/.. are all hard links to foo.
Cheers, Ralph. -
st_nlink is the number of hard links.
2007-03-16 01:35:52 osxbook [Reply | View]
Ralph
"Oh, but they do,"
First off, when I say "directories don't have hard links", I mean you can't create a hard link to a directory (through link(2)). You are interpreting "hard link" to be synonymous with the link count of a file or directory.
"As for Mac OS X altering this aspect, perhaps it does but I doubt it. Here's a test to do on the native, non-FUSE, shipped by Apple, filesystem."
The native file system on OS X is HFS+ and not UFS. Looks like your example uses UFS, and you also seem to assume that an inode-based "traditional" Unix file system is the only way to do things.
On UFS, which indeed uses "real" inodes, you see the behavior you mentioned. That's because UFS will cause st_nlink to be set to the inode's link count. A brand new directory's inode's link count is 2. The mkdir vnode operation for UFS increments the parent directory's inode's link count. This is why you see what you're seeing.
Now, HFS+ uses a bunch of B-Trees: one for the file system's hierarchical structure (the Catalog), another for attributes, another for extents, and so on. It doesn't use inodes the way you're thinking. In fact, what's reported as the inode number isn't really an inode number, but the node number in the Catalog B-Tree. There are no inodes in HFS+. In particular, the "." and ".." entries are "fake" special cases.
Try your example on HFS+:
$ cd /tmp/foo
$ ls -ld .
drwxr-xr-x 2 user wheel 68 Mar 16 00:55 .
$ touch a b c d e f g h
$ ls -ld .
drwxr-xr-x 10 user wheel 340 Mar 16 00:56 .
$ mkdir 1 2 3
$ ls -ld .
drwxr-xr-x 13 user wheel 442 Mar 16 00:56 .
$
As you see, the second word of ls(1)'s output has a different meaning here. It is the number of file system objects within the directory.
How file hard links are implemented on HFS+ is interesting too, but that's another story.
Moreover, if a Mac OS X file system tells the kernel that it doesn't support the nlink field, the kernel will substitute a value of 1.
"for example to implement .., and if you are writing a filesystem you still set up those hard links when making a directory; I doubt FUSE alters this."
MacFUSE is a file system enabler--not a file system itself (well, technically, it is a file system, just a generic one). Specific file systems are free to report whatever they like for st_nlink.
It's actually a bit more complicated on Mac OS X, since there is also the concept of file system attributes. In the man page of getattrlist(2), look for ATTR_DIR_LINKCOUNT and ATTR_FILE_LINKCOUNT.
Hopes this clarifies things. -
st_nlink is the number of hard links.
2007-03-19 11:21:06 ralph@inputplus.co.uk [Reply | View]
First off, when I say "directories don't have hard links", I mean you can't create a hard link to a directory (through link(2)). You are interpreting "hard link" to be synonymous with the link count of a file or directory.
Indeed I am since where I come from, UNIX, they are the same thing, hard links being resposible for incrementing an inode's reference count. It's even in the struct member's name: st_nlink.
Thanks for pointing out that HFS+ doesn't conform to this model. I wonder if this affects find(1)'s attempt at leaf optimisation, see -noleaf.
Cheers, Ralph. -
st_nlink is the number of hard links.
2007-03-21 14:57:17 osxbook [Reply | View]
"Indeed I am since where I come from, UNIX, they are the same thing" ... doesn't conform to this model.
I can understand that you might be surprised by the difference in behavior, but you're assuming too much if you think that the UNIX file system style behavior is a "model" that must be conformed to.
It's more of an implementation detail: incarnations of the UNIX file system and many inode-based file systems do things that way. In fact, it is common enough behavior amongst *nix file systems that POSIX/SUSv3 are subtly ambiguous aboutst_nlinkfor directories. Yet, if you read one of these standards carefully enough, it is not required forst_nlinkfor directories to be what you were expecting it to be.






http://www.netbsd.org/Changes/#puffs+refuse
refuse(3) is a way for netbsd to use FUSE filesystems that was written on top of a google summer-of-code project from 2005. (PUFFS)