Outline for Disaster Recovery Plan Document

The purpose of this post is to share a disaster recovery plan outline. This is an outline that I have been evolving for several years. And, although this outline was intended for SharePoint environments, it can be used for any system or application.

The reason I like this outline is because I know it works. Unlike many other plans on the Web, I think this outline provides more complete coverage of the business and technical aspects of disaster recovery. Many of the plans I have seen are merely explanations of the SharePoint product features and limitations, or advertisements for third party products. Detailing what SharePoint's strengths and weaknesses are with regards to backup and restore will not do you an ounce of good in the event you need to perform a recovery of some sort. Additionally, no matter what tools you have for backing up and restoring, you still need a good plan to address the process. Different tools only change the content of the backup and recovery procedure section of the plan, they don't change the structure of the document or eliminate the need for any of the sections of the document.

The audience for this document can include the tactical team members who execute the plan, but also the business stakeholders, sponsors, and business process owners who are affected by the availability and recovery policies and procedures. Everybody should be involved with contributing to and reviewing the plan. This review process is very healthy because it resets expectations and keeps everybody on the same page.

SharePoint Disaster Recovery Plan Outline

I. Overview
a. Explanation of the document
b. Explanation of when to use the plan

II. Business Profile
a. Application or system owners, points of contact
b. Application or system summary
c. Related business processes
d. Usage information
e. Availability requirements (link to SLA)
f. Recovery requirements (link to SLA)

III. Technical Profile
a. Hardware inventory
b. Software inventory
c. Related systems
d. Vendor contact information (contact, support agreement information, etc.)

IV. Failure Scenarios
Matrix listing all failure scenarios with description of each scenario

V. Backup Procedures
Instructions for each method of back up used to support every possible recovery scenario

VI. Recovery Procedures
Instructions for recovering from each failure scenario

VII. Verification Procedures
a. Functional testing
b. Security testing
c. Performance testing
d. Checklists

VII. Appendix
a. Glossary
b. Links to policies, procedures

VIII. References

SharePoint 3.0: Error: The Backup/Restore Job Failed

Description:


In this scenario, a full farm backup failed because the backup of the "SharedSearchIndex" failed.

Error: The backup/restore job failed. In order to rerun the backup/restore the timer job must be deleted from the Timer Job Definitions page. Failure Message: The backup job failed. For more information, see the error log that is located in the backup directory.

Central Administration > Operations (Tab) > Backup and Restore (Heading) > Backup and restore job status

Error: Object Shared Search Index failed in event OnBackupComplete. For more information, see the error log located in the backup directory. WebException: The current operation timed-out after 3600 seconds.

Central Administration > Operations (Tab) > Backup and Restore (Heading) > Backup and restore job status



Error: The Backup/Restore job has the status of "Failed"

Central Administration > Operations (Tab) > Global Configuration (Heading) > Timer job status



Solution:


This issue may be caused by a timeout. To resolve this issue, there are a few steps. First, any time a SharePoint Backup/Restore job fails, you must delete the failed timer job definition, otherwise subsequent backup attempts will also fail.

Second, take an inventory of scheduled jobs that are running on the farm, including incremental crawl schedules, SharePoint backup scheduled tasks, and Enterprise Backup jobs (e.g. Symantec BackupExec, Networker, etc..). Measure how much time incremental crawls are taking to run. Also, measure how much time the backup is taking to run. Determine what time the backup job needs to start. Keep in mind that if an enterprise backup solution is backing up the SharePoint backup files, then the SharePoint backup should COMPLETE before the tape backup job begins.

If necessary, adjust the scheduled jobs so that the crawls are not interferring with the backup job.


Steps:

1. Delete the Backup/Restore timer job definition.

Central Administration > Operations (Tab) > Global Configuration (Heading) > Timer job definition

2. Adjust the start time of the incremental crawl. Make sure the schedule is set up in a way that the incremental crawls complete at a time that is earlier than the start time of the scheduled task that runs the backup job.

For example, if the incremental crawl takes less than an hour to run, and the the backup job is scheduled to run at 8:00 PM, then schedule the last incremental crawl to occur at 7:00PM.

SSP Administration > Search (Heading) > Search Settings > Content sources and crawl schedulesContent Source > Edit > Crawl schedules


3. If necessary, adjust the start time of the scheduled task that runs the backup

Start > Settings > Control Panel > Scheduled Tasks

SharePoint V3: Recover Site Collection from SQL DB Backup

Here are the steps to follow if you need to recover a deleted site collection from a SQL database backup. For purposes of covering various scenarios, I tried to outline related IIS and AAM configurations to be aware of during the process.

* SharePoint Central Administration > Operations > Backup and Restore
Perform a full farm backup

* SharePoint Central Administration > Application Management > Create or Extend Web Application

* Create a new Web Application
It doesn't matter what you name this database, it will be deleted in a later step

* SharePoint Central Administration > Application Management > Content Databases
You will see that the number of sites is 0 because there are no site collections associated with this web application yet

* SQL Management Studio

* Locate the content database backup which contains the site collection you would like to restore

* Restore this content database, overwriting the one you created for your new web application
Here is where you want to be concious of the name you give the restored database. I like to name the first content database for a web application WSS_WebApp_WebAppName and subsequent content databases for the web application as WSS_SiteCollection_SiteCollectionName.

* Attach the restored content database to the new web application
C:\Program Files\Common Files\Microsoft Shared\web server extensions\12\BIN> Stsadm.exe –o addcontentdb –url http://servername –databasename databasename –databaseserver servername

* SharePoint Central Administration > Application Management > Content Databases
You will now see the content database you restored. The number of sites should be 1 if there was originally 1 site collection in that database when it was backed up.

* Navigate to the URL of the new web application. Verify that the site collection was restored and is
functioning properly

* IIS Management
Note all IIS settings for the original web application including host headers and ip addresses of the IIS web site

* SharePoint Central Administration > Operations > Alternate Access Mappings
Note all of the AAM's for the original web application

* SharePoint Central Administration > Application Management > Delete Web Application
Delete the original web application

* SharePoint Central Administration > Operations > Alternate Access Mappings
Add all of the necessary AAM's to the new web application, make sure the default AAM is fully qualified

* IIS Management
Update the IIS web site for the new web application with all necessary host headers. If the original web application had its own unique, internal ip address asssigned to it, then assign that ip address to the new web application

* Test the URLs for the restored site collection

SharePoint V3: Stsadm.exe Backup Script

Create Backups Directory
Create a backup directory for storing the scheduled SharePoint backups (e.g. c:\backupsharepoint)

Create Bat File Directory
Create a directory for storing the backup script file (e.g. c:\scriptssharepoint)

Create Bat File
Create a bat file using the code below, modified for your environment. Use UNC paths when using in a server farm environment.

@echo off
echo ====================================================
echo Backup Script For Office SharePoint Server 2007
echo ====================================================
cd \program files\common files\microsoft shared\web server extensions\12\bin
@echo off
stsadm.exe -o backup -url http://servername -filename c:\backupssharepoint\backupsharepoint.dat -overwrite
@echo off
echo completed

Schedule bat file
Schedule the backup to run frequently, during off peak hours.

Open Start > Programs > Accessories > System Tools > Scheduled Tasks
Add a Scheduled Task
Scheduled Task Welcome Screen, Click Next
Click the program you want Windows to run, Click Browse
Select the backupsharepoint.bat file you created (e.g. c:\scriptssharepoint\backupsharepoint.bat)
Name the task, Schedule it (e.g. Daily)
Start Time, Frequency, Start Date
Username and Password (use a domain user account that has proper permissions and non expiring password )
Finish

Verify Backups
Keep the backup directory secure. Monitor the job to ensure the job is running properly. Perform test restores from the backup files.


Click Here if you need similar instructions for SharePoint Portal Server 2003.

SharePoint V3: Basic Backup Plan Using Out of Box Tools

Backup

1.
Using the SharePoint Central Admin Backup/Restore Utility, perform a full farm backup after each time you make configuration changes to the SharePoint farm. This backup will protect most of the farm configurations.

2.
Schedule an stsadm.exe backup script to run at least daily on each site collection. These backups will protect the site collections and their content.

3.
Create a SQL Server maintenance plan to backup all databases and transaction log filres on a regular schedule. SQL database backups can be also used to recover site collections. SQL backups also protect farm configurations.

Restore

To test a full restore, you can first restore the farm using the SharePoint Central Administration Backup/Restore Utility. Next, you can restore each site collection from the latest backups using stsadm.exe. This should get you to to a healthy state, as of the time of your latest site collection backup.

You may also want to test restoring a content database by creating a new web application on the farm, restoring a content database backup in SQL, then running the stsadm.exe command for attaching the content database to the new web application. Run iisreset /noforce then try hitting the restored site collections from the browser.

SharePoint V3: Out of Box Options for Backing MOSS 2007 Farm Configurations

This post is about backing up farm configurations. Backing up web applications, site collections, and sites is a different subject.

1.
SharePoint Central Administration Backup/Restore Utility
Central Admin > Operations > Backup and Restore

Caveats:
Cannot be scheduled

2.
STSADM.EXE
Program Files > Common Files > Microsoft Shared > Web Server Extensions > 12 > Bin > stsadm.exe

a. Identify the components you would like to backup
stsadm.exe -o showtree

b. Perform Backup Operation
stsadm.exe -o backup -directory file://servername/backup_share_name -backupmethod full -item farm -backupthreads 1

Caveats:
Cannot backup the following:
Configuration Database
Central Administration web application
Shared Services Provider web application
Shared Services Provider database
User Profile Application
Shared Services Index
Shared Services Provider Search database
Search Instance
WSS Search database

Backup Script For Office SharePoint Server 2007

Create Backups Directory
Create a backup directory for storing the scheduled SharePoint backups (e.g. c:\backupsharepoint)

Create Bat File Directory
Create a directory for storing the backup script file (e.g. c:\scriptssharepoint)

Create Bat File
Create a bat file using the code below, modified for your environment. Use UNC paths when using in a server farm environment.

@echo off
echo ====================================================
echo Backup Script For Office SharePoint Server 2007
echo ====================================================
cd \program files\common files\microsoft shared\web server extensions\12\bin
@echo off
stsadm.exe -o backup -url http://servername -filename c:\backupssharepoint\backupsharepoint.dat -overwrite
@echo off
echo completed

Schedule bat file
Schedule the backup to run frequently, during off peak hours.

Open Start > Programs > Accessories > System Tools > Scheduled Tasks
Add a Scheduled Task
Scheduled Task Welcome Screen, Click Next
Click the program you want Windows to run, Click Browse
Select the backupsharepoint.bat file you created (e.g. c:\scriptssharepoint\backupsharepoint.bat)
Name the task, Schedule it (e.g. Daily)
Start Time, Frequency, Start Date
Username and Password (use a domain user account that has proper permissions and non expiring password )
Finish

Verify Backups
Keep the backup directory secure. Monitor the job to ensure the job is running properly. Perform test restores from the backup files.

MOSS 2007: The backup/restore failed because there is already another job scheduled.

If you run a backup/restore job from SharePoint V3 Central Admin and the job fails, then it may leave a job timer in the Jobs Definition page. You need to delete this timer before you can run the backup/restore utility successfully. If you don't delete the timer job then you may receive this error:

"The backup/restore failed because there is already another job scheduled. Delete the timer job from the Timer Jobs Definition page, and then restart the backup/restore job. "

In Central Admin > Operations > Global Configuration, click on Timer job definitions.

Click the timer job with the title of "backup/restore."

Delete the timer job.

Now you should be able to run the backup/restore from Central Admin.