Discussion of custom development for the purpose of
catastrophic protection has thus far focused on the SharePoint object
model and types it exposes for your use in backup and restore
operations. Although these types provide a variety of ways to leverage
and extend SharePoint’s built-in capabilities, they suffer from a
significant number of constraints. The following are just a few:
The approach
that the SharePoint object model employs for backup and restore isn’t
particularly scalable. As a SharePoint farm grows, use of the types in
the Microsoft.Share-Point.Administration.Backup
namespace have a growing impact on normal farm operations. With
extremely large farms containing terabytes of content, execution of a
built-in catastrophic backup can literally bring a farm to its knees.
All
of SharePoint’s catastrophic backup operations are based on writing
backup data to a file share through standard file operations. This is
less than desirable in many circumstances, because the file operations
can’t be altered to incorporate encryption, compression, backup location
grooming, and many other expected functions.
Data
that you might consider important for backup purposes that isn’t tied
directly to the farm and recognized within SharePoint’s content
component hierarchy is difficult to protect. Although it is true that SharePoint’s catastrophic backup and recovery system can be extended using the IBackupRestore interface, the IBackupRestoreConfiguration
interface, and attributes offered by the Service Application Framework,
the implementation tasks are far from trivial. Even if significant time
and effort are invested to create new content components to integrate
with the built-in catastrophic backup system, the other limitations
already discussed still apply.
In light of these and other
limitations, most enterprise-scale backup systems that are designed to
protect SharePoint stay clear of the types in the Microsoft.SharePoint.Administration. Backup namespace and opt for a different approach. In many cases, these systems leverage VSS.
What Is VSS?
The Volume Shadow
Copy Service, or VSS, is a somewhat mysterious-sounding set of
extensible backup-related APIs that Microsoft has been developing and
evolving since they first appeared in Windows XP. VSS was designed from
the ground up to provide applications, including the Windows operating
system, with a mechanism for creating consistent point-in-time snapshots
of file system data—even if some or all of the files captured in the
snapshot are in use when the snapshot is taken. You can then use these
volume snapshots as the source for backup operations and subsequently
dispose of them if you no longer need them.
The heart of VSS is the VSS
itself. This service is implemented as a Windows service that runs in
the Local System context and is started and stopped on demand to create
volume snapshots as directed.
Types of Snapshots
VSS is aware of the underlying
file system at the block level, and the built-in system provider is
capable of creating volume snapshots in either full copy or differential
copy modes. A full copy of a file system volume is exactly what its
name implies: a clone of the data on the volume that is targeted for a
shadow copy snapshot. When the full copy operation is complete, the
result is a shadow copy volume that exists independently of the source
volume and can be used for readonly backup operations.
Differential copies are
similar to full copies in that they result in the creation of a
read-only shadow copy snapshot that is suitable for backup operations,
but the underlying execution of the snapshot process is significantly
different and typically much faster. Differential copies leverage a
differencing area and a copy-on-write system that tracks only those
blocks of data that have changed in the original volume since the
snapshot was taken.
Differencing areas start
out empty because the source volume and the snapshot volume are
identical at the time of snapshot creation. As changes are made to the
source volume during normal activities, the original file system blocks
(which are about to be overwritten by the new data) are copied from the
source volume to the snapshot’s differencing area. As changes continue,
the differencing area continues to grow to hold all the original source
volume blocks that have been overwritten since the snapshot was taken.
For
backup consumers, the snapshot volume that is exposed ends up being a
combination of the current source volume plus the blocks of the
differencing area overlaid on top. The result is a read-only volume that
appears exactly as it did when the differential copy snapshot was
taken.
VSS Components
The options you have
available to you when working with VSS depend on how you intend to
leverage it. To understand the architecture of VSS, it helps to
understand the different component types that are involved in the
creation of a shadow copy:
VSS.
As described earlier, the VSS is integrated into the Windows operating
system and is responsible for exposing, orchestrating, and coordinating
the actions of most of the other component types related to the creation
and management of volume shadow copies.
Writer. A writer
is a component that serves as a bridge between VSS and an application
or service that possesses data that can be included in a VSS operation.
Writers integrate knowledge of their application or service and work
with the VSS to ensure that applications and services are quiesced—their
pending operations are settled and operational activities suspended.
This ensures that application and service data is consistent prior to
VSS executing a snapshot. Writers also commonly provide some form of
post-restore synchronization support for their applications or services.
Writers are available for SQL Server, Hyper-V, SharePoint, Windows
Management Instrumentation (WMI), the Registry, and a multitude of other
services and applications.
Provider. A provider
is a component that actually carries out the shadow copy process and
manages the storage volumes that are associated with VSS operations.
Providers can either be software based, like the built-in system
provider that is supplied with the operating system, or they can be
hardware based and tied to their associated storage arrays. Regardless
of the actual implementation, providers do the “grunt work” of
point-in-time volume snapshot generation, management of the underlying
volumes to store the critical pieces of a snapshot (such as differencing
areas), and more.
Requestor. A requestor
is an application that consumes VSS for the purposes of backing up or
restoring data.
When Windows Server Backup needs to create a backup, it engages the VSS
to generate a snapshot of the data to be backed up. Once VSS generates
the snapshot, Windows Server Backup creates a backup from the snapshot
and then releases it.
Each of these component
types is a critical piece in the overall architecture of VSS.
Leveraging VSS for SharePoint backup operations focuses primarily on two
of the four component types mentioned, though: writers and requestors.
The Role of the SharePoint Foundation VSS Writer
After
reading the component description of a VSS writer and how writers are
associated with applications and services, you may be wondering if
SharePoint has its own writer. If so, the answer is “yes.” In fact, it
technically has two.
The SharePoint Foundation VSS
writer (or SPF-VSS writer) is installed when either SharePoint
Foundation 2010 or SharePoint Server 2010 is installed on a server. Once
registered, the writer integrates with the VSS infrastructure to
provide the application the intelligence and capabilities it needs to
generate consistent and usable shadow copies of each of the components
that make up a SharePoint farm. This includes content databases, Service
Application databases, registered third-party databases, and more. The
SPF-VSS writer also works with a secondary search index writer that is
installed at the same time as the SPF-VSS writer to provide snapshot and
restore support for SharePoint’s file-based search indexes.
An important point to note is
that the SPF-VSS writer itself doesn’t create backups or execute
restores. The VSS (in conjunction with one or more VSS providers) is
still responsible for creating shadow copies of the volumes housing
SharePoint data, and actual backup and restore operations are
orchestrated by a VSS requestor. The key advantages that the SPF-VSS
writer offers are based on its application knowledge of SharePoint:
Consistent snapshot state.
Prior to VSS snapshot creation, the SPF-VSS writer transitions the
SharePoint farm into a consistent state on disk. Databases are quiesced,
pending transactions may be flushed, search index file operations are
completed, and the entire farm is effectively paused for a brief period
to allow the VSS to generate its snapshot. This is a critical step. If a
snapshot were taken of the farm in an inconsistent state, there’s a
reasonable chance that a restore operation that is conducted using data
in the snapshot could fail.
Post-restore activities.
When a requestor or its parent application restores SharePoint farm
data that was backed up from a VSS snapshot, there are typically some
post-restore synchronization steps that must be carried out to get the
SharePoint farm back to a fully functional state. When the SPF-VSS
writer is notified by a requestor that a restore operation is taking
place, the SPF-VSS writer takes care of the farm synchronization steps
once the restore is completed.
These qualities make the
SPF-VSS writer an integral part of any backup and restore solution that
intends to leverage VSS for SharePoint farm protection.
Developing Solutions with VSS
Without a doubt, VSS brings a lot
to the table if you are seeking to develop an enterprise-ready,
scalable, robust backup and restore store system for SharePoint. The
primary component type of interest for custom backup and restore
application development is the VSS requestor. Requestors are
the ultimate users and orchestrators of VSS services, and a custom
backup and restore application falls squarely into the requestor
component type category.
Now that you’re probably
excited about leveraging the power of VSS, we need to share some
potentially bad news: Microsoft doesn’t supply any form of managed
libraries or extensions that would allow you to use VSS in your .NET
applications. Microsoft’s support for application development against
VSS is provided through a set of COM and C++ types that are packaged
into the Windows SDK. This leaves you with a couple of options:
Develop your requestor in a language such as unmanaged C++.
Write
or leverage a wrapper library around the VSS types of interest and
generate a runtime callable wrapper (RCW) that a .NET application can
consume. This approach still requires that you do some C++ or equivalent
development—just less of it.
Neither of these approaches
is trivial for the average .NET developer. In addition, the
orchestration of VSS operations is tricky and fraught with pitfalls.
Creating a snapshot isn’t as simple as making a single call such as vss.GenerateSnapshot().
Writer metadata has to be gathered and processed, writers must be
enlisted for snapshot operations, dependencies across writers must be
established, the snapshots have to be taken and released, and more. The
tasks and processes are documented in the Windows SDK, but there is a
significant amount of material to learn and incorporate.
Tip
If you are a .NET developer
and still have your heart set on working with VSS after all that’s been
said, know that there are at least a couple of resources available to
you. At a minimum, you need to download the Windows SDK and become
familiar with its contents. The quickest path to productivity lies with
the AlphaVSS project on CodePlex (http://alphavss.codeplex.com),
though. AlphaVSS is a managed library around the interfaces you need to
work with VSS. It is well documented and highly instructive. It won’t
make you instantly productive with VSS, but it dramatically reduces the
learning curve and work you have to do.
