The core Shell storage architecture has existed in some form since Windows 95. Since then, it's been evolving and will continue to as long as Windows is shipping. There are at least three notable places (that I can think of right now) where this storage platform is carried forward in Vista. First, more of the shell core functionality can now operate over non-filesystem places in the shell namespace, which is most evident in the new copy engine API (IFileOperation) and the new common file dialog API (IFileDialog and friends). Second, the addition of a rich shell namespace extension that can reason over the Windows search platform and abstractly distribute queries to multiple providers and scopes including remote queries to other machines. And finally, the addition of the shell property system which brokers property metadata for user interaction and multiplexes properties on items from their various providers.

I will almost certainly be blogging about these three and more in the future, so I'd like as a baseline to briefly discuss some of the high-level concepts of the shell storage architecture. This topic is discussed in greater detail elsewhere (here too), but I'm going use more modern terminology and paint only a conceptual and perhaps more idealistic picture than actually exists. After all, Explorer is both a platform and an application, and sometimes platform beauty has to succumb to the realities of shipping the application with the desired feature set.

The Namespace. The shell namespace is a tree of items, shell items to be precise. You can manipulate these items by using the IShellItem interface. The namespace is rooted at the root item, called the desktop folder. Shell items can have children that are other shell items. And you can enumerate the children of a shell item by binding to its IShellFolder handler and calling EnumObjects. You can ask a shell item for other services as well: you can bind to a stream on that item, you can read metadata for the item, you can get the icon or context menu on the item, you can enumerate and invoke verbs associated with the item, you can register for change notifications on the item, etc. You could build a whole shell-like application in fact by using this API set, and indeed the shell itself uses this API to deliver its user scenarios.

One of my favorite examples of a shell item is a ZIP archive file, because it is an example of an item that behaves both like a leaf node, in that it has streaming content, and a non-leaf node, in that it has child items which themselves can have children and streaming content, etc. In fact, not many people know this, but you can use the shell's namespace object model as a sort of native API for enumerating and extracting ZIP file content.

Junctions. The ZIP file is also an example of something called a junction, which is a place where one type of namespace handler delegates to another namespace handler by exposing an entrypoint to the other handler's namespace from within its own namespace. In the case of ZIP files, the file system namespace handler, which exposes a shell item namespace over the file system, junctions control to the ZIP namespace handler, which exposes a shell item namespace over the contents of a ZIP file.

Another example of a junction is the history folder. If you go into the history folder from the command prompt by saying: 

> dir /a "%USERPROFILE%\Local Settings\History"
 Volume in drive C is CLAUDIA
 Volume Serial Number is 94C3-00F1

 Directory of C:\Documents and Settings\marcmill\Local Settings\History

08/10/2005  04:00 PM    <DIR>          .
08/10/2005  04:00 PM    <DIR>          ..
08/10/2005  04:00 PM               113 desktop.ini
10/06/2005  10:54 AM    <DIR>          History.IE5
               1 File(s)            113 bytes
               3 Dir(s)  25,113,985,024 bytes free
But if you view the same location in Explorer, you'll see something completely different:

That's because the History folder itself is another one of these junction points. And while the file system doesn't recognize the shell namespace junction points, Explorer does. That's what allows us to have content that appears only in the file system, in this case the history database itself, separated from the content that appears within Explorer.

Using this concept of junctions, the shell item namespace that ships with Windows provides extensibility points where third-party item namespace providers can plug in their own item namespaces. In fact, these same extensibility mechanisms are used internally to deliver features like the History folder, the control panel, and especially the new "virtual folders" feature in Windows Vista. In the case of virtual folders, the ".vfolder" files that you find in your profile directory are akin to ZIP files, in that they are junction points to a shell item namespace that operates over database query results. More on how this works in a later post.

File System Versus Item. Any shell item can be asked if it is really backed by a real file or folder on the file system, and if so, asked for that item's path. It is often tempting as a result to circumvent the shell storage abstraction by extracting the file system path from the shell item, and talking directly to the file system. After all, your code will work in the majority case where the underlying item is actually backed in some way by the file system. And in fact, sometimes this circumvention is required since the shell, for example, provides no abstraction over memory mapping the item's content from its storage.

However, when making a decision to only operate over file-backed items, you should be aware of the potential loss of generality. Do you want, for example, your thumbnail provider to be able to render thumbnails for items that appear inside ZIP files, or on storage devices that plug in to the shell namespace (but not the filesystem)? If you support the abstract storage model, then your code will have more reach.

More to come...