ZFS: "Probably more abuse in 20 seconds than you'd see in a lifetime"

I've been reading (thanks to Brett Cannon) about the Zettabyte File System (ZFS) - an open source project from Sun Microsystems that seems to offer real opportunities to take proper advantage of both SAN and virtualization technologies. If you want to know what a zettabyte is then as usual the Wikipedia is reliable on technical matters (and no, I had never run across the term before).

Brett was simply happy that Apple announced the inclusion of ZFS into the Leopard (OS X 10.5) distribution. I was interested because Sun developers were taking an interest in advanced filesystems even back when I used to work there - they produced the Translucent File System (TFS), which was a copy-on-write mechanism to allow users to share a common source pool and keep their own changes.

One of the problems with TFS was its need for kernel integration (in those days even the windowing system was integrated into the kernel). Another issue arose when a user base their TFS on another TFS which was based on a third TFS which ... effectively it implemented an copy-on-write inheritance mechanism, but efficiency could drop quite rapidly with multiple layers.

Unlike TFS, ZFS can run in either kernel or user mode, and the nightly testing appears to twist the filesystem in every possible direction simultaneously (hence the title of this blog entry). Here's a list of what goes on in parallel in each nightly test:

? Read, write, create, and delete files and directories
? Create and destroy entire filesystems and storage pools
? Turn compression on and off (while filesystem is active)
? Change checksum algorithm (while filesystem is active)
? Add and remove devices (while pool is active)
? Change I/O caching and scheduling policies (while pool is active)
? Scribble random garbage on one side of live mirror to test self-healing data
? Force violent crashes to simulate power loss, then verify pool integrity

So, it would appear that ZFS is a worthy successor to TFS and much more besides, as you will find out if you too read the slide show. I haven't looked at the license yet, but it does appear that under Johnathan Schwartz's direction Sun might be beginning to find its way in the open source world of the 21st century, despite the amusing graphic that Eric Bangeman produced when Scott McNealy resigned. Maybe I should buy some stock again. Maybe we all should.