Volume Shadow Copy Service (VSS)

Inconsistent Backup

This is probably the oldest type of backup. The process is simple, backup software starts at the beginning of a file structure and copies all of the data until it reaches the end, resulting in a backup. What can make this backup inconsistent is that if any user added/modified after a backup but before it completed that would result in an inconsistent backup; the files in the backup are not consistent. For databases these types of backups do not provide adequate protection because they only capture what is on disk. Example, with SQL Server there are a several different files, primary (MDF), log (LDF) and secondary (NDF) files and there could be I/O (transactions) that are still in memory. The inconsistent backup method only captures what is on disk and in the case of a database there may be transactions still in memory.

Crash-Consistent Backup

A crash-consistent backup is where all data is captured at the same point in time. For all things not database related this method of backup should suffice for most recovery situations. A crash-consistent backup, as with an inconsistent backup, does not capture any data that is in memory or any pending I/O transactions. The term “crash” is used because if the data is recovered using this method it would be equivalent to restoring to the exact moment that the server had crashed. Crash-consistent backups can be used but it is important to perform the correct operational tasks to ensure you are restoring properly. For example with SQL Server you may need to replay transaction logs to get the database back to a consistent transactional state.

Application-Consistent Backup

Application-consistent backups provide the highest level of protection and consistency, it captures what is on disk and memory. For Microsoft Windows this is achieved using the Volume Shadow Copy Service (VSS) which freezes I/O, flushes everything to disk and takes a block-level snapshot of the volume. VSS is provided as a framework in the Windows operating system that performs these operations by coordinating between Requestor, Writer and Provider. When a VSS request is initiated a VSS Writer (Eg. SqlServerWriter) will flush all I/O to ensure that the database is in a consistent state. Then the Pure VSS Hardware Provider will take a block-level snapshot of the volume. Once this is completed the Provider notifies the Writer to resume operations.

Today the Purity Operating Environment takes crash consistent snapshots, but now with the introduction of the Pure VSS Hardware Provider we can now provide Application-Consistent Backups for Microsoft server products like SQL Server.

The Pure VSS Hardware Provider uses Diskshadow.exe as our out-of-box VSS Requestor. Other requestors can be used as long as they adhere to the implementation guidelines of the VSS SDK.

The process by which the Volume Shadow Copy Service works is illustrated below:

The following at the steps involved in a VSS workflow: 

1. The requestor asks the Volume Shadow Copy Service to enumerate the writers, gather the writer metadata, and prepare for shadow copy creation.
2. Each writer creates an XML description of the components and data stores that need to be backed up and provides it to the Volume Shadow Copy Service. The writer also defines a restore method, which is used for all components. The Volume Shadow Copy Service provides the writer’s description to the requestor, which selects the components that will be backed up.
3. The Volume Shadow Copy Service notifies all the writers to prepare their data for making a shadow copy.
4. Each writer prepares the data as appropriate, such as completing all open transactions, rolling transaction logs, and flushing caches. When the data is ready to be shadow-copied, the writer notifies the Volume Shadow Copy Service.
5. The Volume Shadow Copy Service tells the writers to temporarily freeze application write I/O requests (read I/O requests are still possible) for the few seconds that are required to create the shadow copy of the volume or volumes. The application freeze is not allowed to take longer than 60 seconds. The Volume Shadow Copy Service flushes the file system buffers and then freezes the file system, which ensures that the file system metadata is recorded correctly and the data to be shadow-copied is written in a consistent order.
6. The Volume Shadow Copy Service tells the provider to create the shadow copy. The shadow copy creation period lasts no more than 10 seconds, during which all write I/O requests to the file system remain frozen.
7. The Volume Shadow Copy Service releases file system write I/O requests.
8. VSS tells the writers to thaw application write I/O requests. At this point applications are free to resume writing data to the disk that is being shadow-copied.
9. The requestor can retry the process (go back to step 1) or notify the administrator to retry at a later time.
10. If the shadow copy is successfully created, the Volume Shadow Copy Service returns the location information for the shadow copy to the requestor.

The  Pure VSS Hardware Provider can be downloaded and installed from here. Download the newest package and install on each individual Microsoft Window Server host that needs to take application consistent snapshots. Because this could potentially be a high-touch deployment on a per server basis it is advisable to use a software deployment tools to install on a large number of systems.

Once the Pure VSS Hardware Provider has been installed use DiskShadow from an elevated Administrator prompt to ensure the provider is visible.

PS C:\> diskshadow
Microsoft DiskShadow version 1.0
Copyright (C) 2013 Microsoft Corporation
On computer:  SERVER01,  5/26/2017 4:52:04 PM

DISKSHADOW> list providers
        * ProviderID: {781c006a-5829-4a25-81e3-d5e43bd005ab}
                Type: [3] VSS_PROV_HARDWARE
                Name: Pure Storage VSS Hardware Provider (64-bit)
                Version: 1.1.0
                CLSID: {484e0f8a-b54c-45c6-acaf-be682a421f25}
         * ProviderID: {89300202-3cec-4981-9171-19f59559e0f2}
                Type: [4] VSS_PROV_FILESHARE
                Name: Microsoft File Share Shadow Copy provider
                Version: 1.0.0.1
                CLSID: {fce59da7-7bac-40da-8d21-3e7311ba51cd}
         * ProviderID: {b5946137-7b9f-4925-af80-51abd60b20d5}
                Type: [1] VSS_PROV_SYSTEM
                Name: Microsoft Software Shadow Copy provider 1.0
                Version: 1.0.0.7
                CLSID: {65ee1dba-8ff4-4a58-ac1c-3470ee2f376a}
         
         Number of providers registered: 3

DISKSHADOW>

The Pure VSS Hardware Provider must be configured before use. The below example shows how to configure the provider using the PureProviderConfig utility.  

PS C:\Program Files\Pure Storage\VSS\Provider> .\PureProviderConfig.exe add --url https://10.21.8.202 --user pureuser M20
Pure Storage PureProviderConfig version 1.1.0.8.8 (64-bit Release)
Copyright (C) 2017 - Pure Storage. All rights reserved.
On Computer: SERVER01 - Windows 6.2 (9200)
Supported REST version is 1.1
Enter user password:
********

URL                                               Compatible  Online  Authenticated  Array Name
https://10.21.8.202                               yes         yes     yes            M20

The new change(s) will take effect the next time 'Pure Storage VSS Hardware Provider (64-bit)' starts.
To restart 'Pure Storage VSS Hardware Provider (64-bit)' now, run 'PureProviderConfig restartprovider'.

Once the provider has been configured an application consistent snapshot can be taken. The following example is creating an application consistent snapshot of the F: drive.

PS C:\> diskshadow
Microsoft DiskShadow version 1.0
Copyright (C) 2013 Microsoft Corporation
On computer:  SERVER01,  5/26/2017 5:29:42 PM
DISKSHADOW> reset
DISKSHADOW> set context persistent
DISKSHADOW> set option transportable
DISKSHADOW> set metadata "C:\Program Files\Pure Storage\VSS\ExampleMetadata.cab"
DISKSHADOW> begin backup
DISKSHADOW> add volume F:
DISKSHADOW> create
Alias VSS_SHADOW_1 for shadow ID {41e659bc-8f36-427d-86d0-811d44c040f3} set as environment variable.
Alias VSS_SHADOW_SET for shadow set ID {958e45a6-dd34-4fd4-b9b0-fe82351f98af} set as environment variable.

DISKSHADOW> end backup
DISKSHADOW>

The application consistent snapshot has a unique identifier as the name shown below for the added volume in DiskShadow.