T-SQL - wiseDATAman

Removing large backup history with sp_delete_backuphistory

Sun, 04 Feb 2024 15:04:57 +0000

Problem

In SQL Server, the sp_delete_backuphistory stored procedure is is the standard and supported method to purge old data from the backup and restore history tables in the msdb database:

backupfile
backupfilegroup
backupmediafamily
backupmediaset
backupset
restorefile
restorefilegroup
restorehistory

Recently, I encountered an issue running sp_delete_backuphistory on servers that hosted a large number of databases with frequent log backup & restore operations. The clean up task hadn’t been scheduled and the history tables had grown very large over several months. The msdb databases was also hosted on a volume with limited IOPs.

Attempting to run sp_delete_backuphistory under these conditions you will likely encounter these issues:

The operation will take a long time
Log backup and restore operations are blocked during the process
Your msdb log file will grow large if you try to remove too much data at once
If you kill off the process at any point, the whole thing will rollback.

The sp_delete_backuphistory stored procedure gets the job done in most cases but it’s not very well optimized if you have a large amount of history to clear.

Optimized Solution for Backup History Cleanup

To clear a large amount of history without resorting to schema modifications or unsupported methods we can take an incremental approach to removing data.

The script below will check how much retention we currently have. It will then call sp_delete_backuphistory in a loop, removing data 1hr (configurable) at a time until we hit the target retention. We can also specify a maximum runtime for the process.

This approach won’t necessarily speed things up but it provides the following benefits:

Data is cleared as a series of smaller transactions rather than a single large transaction
Smaller transactions allow for log truncation and prevent the log file from growing.
We can interrupt the process without having to rollback all the progress that has been made. (just the current batch)
Log backups/restores might still be blocked, but only for the duration it takes to complete a single loop - not for the entire time it takes to clear the history. This can still impact RPO/RTO though.
We can control how long the process runs for. We can keep it running outside business hours or allow for breaks to prevent log backup/restores from falling too far behind.
The script will print progress information.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76


/********************* Params *******************/
DECLARE @RetentionDays INT = 30 /* How long should we keep the data for in days */
DECLARE @Mins INT = 60 /* Delete data X minutes at a time. Small interval keeps transaction short. Less blocking and log usage*/
DECLARE @MaxRuntimeMins INT = 60 /* Stop processing if runtime exceeds this threshold */
/************************************************/

DECLARE @TargetDate DATETIME
DECLARE @oldest_date DATETIME
DECLARE @Message NVARCHAR(4000)
DECLARE @t1 DATETIME
DECLARE @Duration INT
DECLARE @TotalDuration BIGINT=0
DECLARE @IterationCount INT=0
DECLARE @StopAt DATETIME

/* The target date we want to get to based on the specified @RetentionDays */
SET @TargetDate = DATEADD(dd,-@RetentionDays,GETDATE())
/* When should we stop processing batches based on @MaxRuntimeMins */
SET @StopAt = DATEADD(mi,@MaxRuntimeMins,GETDATE())

/* Get the oldest backup finish date from table. We will start removing data older than this point + @Mins minutes */
SELECT @oldest_date = MIN(backup_finish_date)
FROM msdb.dbo.backupset

PRINT CONCAT('Oldest Date: ',@oldest_date)
PRINT CONCAT('Target Date: ', @TargetDate)
PRINT CONCAT('Max Run Time (Mins): ',@MaxRuntimeMins,', Processing will stop after ',@StopAt)

/* Process the delete operation in small chunks until we reach the target date for our retention */
 WHILE 1=1
 BEGIN
	/* Increment date by @Mins minutes */
	SET @oldest_date = DATEADD(mi,@Mins,@oldest_date)
	/* Ensure that we don't overshoot our target date */
	IF @oldest_date > @TargetDate
		SET @oldest_date = @TargetDate	
		
	SET @IterationCount = @IterationCount + 1
	/* 
	Print message using RAISERROR WITH NOWAIT so message is printed immediately. 
	For keeping track of progress 
	*/
	SET @Message = CONCAT(CONVERT(VARCHAR,GETDATE(),114),' Start iteration ',@IterationCount,': Removing backup history before ',@oldest_date)
	RAISERROR(@Message,0,1) WITH NOWAIT
	SET @t1 = SYSUTCDATETIME()
	/* Remove the history up to @oldest_date (@Mins minutes worth of history) */
	EXEC msdb.dbo.sp_delete_backuphistory @oldest_date = @oldest_date;

	/* Print how long this iteration took & keep track of total duration and iteration count */
	SET @Duration = DATEDIFF(ms,@t1,SYSUTCDATETIME())
	SET @TotalDuration = @TotalDuration + @Duration 
	SET @Message = CONCAT(CONVERT(VARCHAR,GETDATE(),114),' Iteration ', @IterationCount,' completed in (hh:mi:ss:mmm): ',CONVERT(VARCHAR,DATEADD(ms,@Duration,0),114))
	
	RAISERROR(@Message,0,1) WITH NOWAIT

	/* Stop once we hit our target date for retention or we have ran for longer than @MaxRuntimeMins */
	IF @oldest_date>=@TargetDate 
		OR GETDATE() > @StopAt
		BREAK

 END

 /* 
	Print overall stats 
	Accounting for possibility that total duration could span > 1 day
 */
SET @Message = CONCAT('Completed ',
				@IterationCount,
				' iterations in ',
				CAST(NULLIF(DATEDIFF(d,0,DATEADD(s,@TotalDuration/1000,0)),0) AS NVARCHAR(4000)) + ' Days ',
				LEFT(CONVERT(VARCHAR,DATEADD(s,@TotalDuration/1000,0),114),8),
				' (hh:mi:ss), Avg Duration: '
				,CONVERT(VARCHAR,DATEADD(ms,@TotalDuration/NULLIF(@IterationCount,0),0),114), 
				' (hh:mi:ss:mmm). Reached target date: ',CASE WHEN @oldest_date>=@TargetDate THEN 'YES' ELSE 'NO' END )

RAISERROR(@Message,0,1) WITH NOWAIT

Links:

These related links might be interesting:

https://www.brentozar.com/archive/2018/02/the-annals-of-hilariously-bad-code-part-1/
https://www.brentozar.com/archive/2018/02/annals-hilariously-bad-code-part-2/
https://dba.stackexchange.com/questions/233375/how-to-minimize-plan-ahead-the-effect-of-running-sp-delete-backuphistory-measur

Get drive data for ALL drives in T-SQL

Wed, 24 Mar 2021 21:48:49 +0000

There are a few methods you can use to get drive information using T-SQL. The problem is all these methods have some limitations. They provide free space but not drive capacity or they only provide data for volumes that contain database files. Ideally Microsoft would provide an easy way to get both drive capacity and free space for all drives. In this article I’ll demonstrate a way to do this using T-SQL.

Here are a few exiting methods that you might consider before we continue:

Get drive information using T-SQL


 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16


-- Gets size and free space but only for volumes that have SQL data files.
SELECT DISTINCT vs.volume_mount_point AS DriveLetter,
	vs.total_bytes / POWER(1024.0,3) AS SizeGB,
	vs.available_bytes / POWER(1024.0,3) AS FreeGB,
	vs.logical_volume_name as Label
FROM sys.master_files mf
CROSS APPLY sys.dm_os_volume_stats(mf.database_id, mf.file_id) vs

-- Needs SQL 2019 & free space only
SELECT fixed_drive_path, 
	drive_type_desc, 
	free_space_in_bytes / POWER(1024.0,3) FreeGB
FROM sys.dm_os_enumerate_fixed_drives;

-- Gets free space on drive only
EXEC xp_fixeddrives

If these work for your needs, great! If you need capacity and free space for ALL drives you will need a different solution. The best option might be to query that data outside of SQL Server using PowerShell or your programming/scripting language of choice. If you need to do this in T-SQL though, the script below provides a way.

The script uses xp_cmdshell to run a powershell command to get data for ALL the volumes on your system. Powershell formats this data as XML which we capture in the @output table variable - 1 row per line of XML. Then we convert this back to a single string and store as XML in variable @x. We then shred the XML into a table format. This is a bit of a hack - sys.dm_os_volume_stats is a cleaner method but doesn’t give you data for all drives. As a bonus you get more detailed data about those drives; file system, allocation unit size & more.

Drive information for ALL drives using T-SQL


 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50


/* 
	Created by David Wiseman
	https://wiseDATAman.com

	Get drive information for all drives via T-SQL.
	This requires access to xp_cmdshell
*/
IF EXISTS(SELECT * FROM sys.configurations
			WHERE name = 'xp_cmdshell'
			AND value_in_use=1
			) AND EXISTS(SELECT * FROM fn_my_permissions ( 'xp_cmdshell', 'OBJECT' ) WHERE permission_name='EXECUTE')
BEGIN
	SET NOCOUNT ON
	DECLARE @x XML
	DECLARE @output TABLE(
		ID INT IDENTITY(1,1) PRIMARY KEY,
		line NVARCHAR(MAX)
	)
	INSERT INTO @output
	EXEC xp_cmdshell 'powershell -noprofile -Command " & { ConvertTo-Xml -As "String" -NoTypeInformation -InputObject (Get-Volume | Select-Object -Property DriveLetter,Size,SizeRemaining,FileSystem,FileSystemLabel,AllocationUnitSize,DriveType,FileSystemType,HealthStatus,OperationalStatus,UniqueId )}"'

	/* Alternative option for SQL 2017+
	SELECT @x = STRING_AGG(line,CHAR(13) + CHAR(10)) WITHIN GROUP (ORDER BY ID) 
	FROM @output
	WHERE ID>1 -- skip encoding line
	*/

	SELECT @x = (SELECT line + CHAR(13) + CHAR(10)
	FROM @output 
	WHERE ID >1 -- skip encoding line
	ORDER BY ID
	FOR XML PATH(''),TYPE).value('.','NVARCHAR(MAX)')

	SELECT T.c.value('(Property[@Name="DriveLetter"])[1]','CHAR(1)') as DriveLetter,
		T.c.value('(Property[@Name="Size"])[1]','BIGINT')/POWER(1024.0,3) as SizeGB,
		T.c.value('(Property[@Name="SizeRemaining"])[1]','BIGINT')/POWER(1024.0,3) as FreeGB,
		T.c.value('(Property[@Name="FileSystem"])[1]','NVARCHAR(256)') as FileSystem,
		T.c.value('(Property[@Name="FileSystemLabel"])[1]','NVARCHAR(256)') as FileSystemLabel,
		T.c.value('(Property[@Name="AllocationUnitSize"])[1]','INT') as AllocationUnitSize,
		T.c.value('(Property[@Name="DriveType"])[1]','NVARCHAR(256)') as DriveType,
		T.c.value('(Property[@Name="FileSystemType"])[1]','NVARCHAR(256)') as FileSystemType,
		T.c.value('(Property[@Name="HealthStatus"])[1]','NVARCHAR(256)') as HealthStatus,
		T.c.value('(Property[@Name="OperationalStatus"])[1]','NVARCHAR(256)') as OperationalStatus,
		T.c.value('(Property[@Name="UniqueId"])[1]','NVARCHAR(256)') as UniqueId
	FROM @x.nodes('/Objects/Object/Property') T(c)
END
ELSE
BEGIN
	RAISERROR('xp_cmdshell required',11,1)
END

A similar technique could be used to capture the output of other powershell commands in T-SQL. It might not be a good idea but I think it is an interesting technique to demo.

How to handle SQL Agent Jobs with Always On Availability Groups?

Wed, 17 Mar 2021 22:43:52 +0000

Introduction

When you configure your SQL servers for high availability you want the failover to be as seamless as possible. If you use availability groups, mirroring or log shipping as part of your high availability strategy you will need to take care of anything that lives outside the context of your databases. That might include logins, SQL Server agent jobs and ensuring consistent server level configuration across your availability nodes. This article will discuss how to handle SQL server agent jobs.

Not so seamless failover of agent jobs

If you have any SQL agent jobs that run in the context of your user databases these will need special handling to ensure a seamless failover. Ideally Microsoft would have added some improvements to the SQL agent to help with this. Maybe given us agent jobs that can live within the context of a user database instead of msdb?

As it stands you will need to keep your agent jobs in sync between the primary and secondaries and figure out a mechanism to activate the jobs on the new primary following failover and disable them on the old primary.

A Solution

There are a few ways to handle this, but my approach is to run all the jobs in the context of the master DB. I then check if the user DB is accessible and execute the command with a 3 part name. See the script below for an example. This will work for availability groups, mirroring and log shipping.

Template T-SQL for Agent Job


 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11


DECLARE @DB SYSNAME = 'MyDB'
IF DATABASEPROPERTYEX(@DB, 'Updateability') = 'READ_WRITE'
	AND EXISTS(SELECT 1 FROM sys.databases WHERE name=@DB and state = 0)
BEGIN
	PRINT 'Run command'
	--EXEC MyDB.dbo.MyProc
END
ELSE
BEGIN
	PRINT 'Not run: Database is not writeable'
END	

With this approach you can create your jobs and have them active on your primary and secondaries. You don’t need to worry about enabling/disabling jobs on failover - the jobs are already running but they only do any work if the database is primary. This makes the failover totally seamless. The downside to this approach is that the jobs will report that they ran successfully even if they didn’t do any work which could cause confusion.

The next problem is keeping the jobs in sync between your servers. You can script out the jobs using SSMS easy enough but this could get tedious if you have a larger number of jobs. I’ve created a powershell script below to help with this which you can use below. You can use this to report if your jobs are out of sync and also for performing a synchronization. See the comments at the top of the script for some usage examples.

SyncSQLAgentJobs.ps1


 1
 2
 3
 4
 5
 6
 7
 8
 9
 10
 11
 12
 13
 14
 15
 16
 17
 18
 19
 20
 21
 22
 23
 24
 25
 26
 27
 28
 29
 30
 31
 32
 33
 34
 35
 36
 37
 38
 39
 40
 41
 42
 43
 44
 45
 46
 47
 48
 49
 50
 51
 52
 53
 54
 55
 56
 57
 58
 59
 60
 61
 62
 63
 64
 65
 66
 67
 68
 69
 70
 71
 72
 73
 74
 75
 76
 77
 78
 79
 80
 81
 82
 83
 84
 85
 86
 87
 88
 89
 90
 91
 92
 93
 94
 95
 96
 97
 98
 99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141


Param(
 [Parameter(mandatory=$true)] 
 [string]$ServerA, 
 [Parameter(mandatory=$true)] 
 [string]$ServerB, 
 [Parameter(mandatory=$false)] 
 [System.Array]$ExcludedCategories=@(), #Job categories to exclude from the synchronization process.
 [Parameter(mandatory=$false)] 
 [System.Array]$IncludedCategories=@(), #Job categories to include in the synchronization process.
 [Parameter(mandatory=$false)] 
 [switch]$DoSync #perform a two way synchronization if specified, otherwise run in reporting mode.
)
<# Created: 2017-12-06
 Author: David Wiseman
 
 Description: Sync agent jobs between two SQL Server instances or check synchronization status. This is useful if you are using a high availability technology like
 availability groups, log shipping or database mirroring that does not include SQL Agent jobs in the failover. The script can be used to validate that the jobs are in sync
 or it can synchronize the jobs. 

 Requires SqlServer powershell module:
 Install-Module -Name SqlServer -AllowClobber

 Example1: 
 ./SyncSQLAgentJobs -ServerA "SERVER1" -ServerB "SERVER2"
 Agent jobs will be checked and differences will be reported. 
 
 Example2: 
 ./SyncSQLAgentJobs -ServerA "SERVER1" -ServerB "SERVER2" -DoSync
 Any jobs that don't exist will be copied from SERVER1 to SERVER2 and from SERVER2 to SERVER1. Any jobs that are different will be copied from the server with the newer version of the job (Based on DateLastModified). 

 Example3: 
 ./SyncSQLAgentJobs -ServerA "SERVER1" -ServerB "SERVER2" -DoSync ExcludedCategories "ADHOC"
 Same as example 2 but jobs in the category "ADHOC" are excluded from synchronization.

 Note: SQL Agent loads SQLPS which isn't compatible with SqlServer module. A workaround is to run the following powershell in the SQL Agent job.
 $Error.Clear()
 $out = powershell -NoProfile -Command "C:\Scripts\SyncSQLAgentJobs.ps1" -ServerA "SERVER1" -ServerB "SERVER2"
 if ($Error.Count -gt 0){
 throw $out
 }
 else{
 $out
 }
 This will ensure the job fails if anything is out of sync. The SQL Server agent account will need appropriate access to msdb database on both instances.

#>
$ErrorActionPreference="stop"
Import-Module SqlServer -DisableNameChecking

$IncludedCategories

$jobs = Get-SqlAgentJob -ServerInstance $ServerA
$ValidationStatus=$true
$errorMsg=""

$jobs | ForEach-Object{
 $jobA = $_
 $jobB = $null
 $scriptA = ""
 $jobA.Script() | ForEach-Object {$scriptA += $_}
 $jobB = Get-SqlAgentJob -ServerInstance $ServerB -Name $jobA.Name -ErrorAction Ignore

 if ($ExcludedCategories.Contains($jobA.Category)){
 "Skipping " + $ServerA + ": " + $jobA.Name + " : Excluded category: " + $jobA.Category
 }
 elseif($jobB -ne $null -and $ExcludedCategories.Contains($jobB.Category)){
 "Skipping " + $ServerB + ": " + $jobB.Name + " : Excluded category: " + $jobB.Category
 }
 elseif($jobA -ne $null -and $IncludedCategories.Count -gt 0 -and (!$IncludedCategories.Contains($jobA.Category))){
 "Skipping " + $ServerA + ": " + $jobA.Name + " : Not included category: " + $jobA.Category
 }
 elseif($jobB -ne $null -and $IncludedCategories.Count -gt 0 -and (!$IncludedCategories.Contains($jobB.Category))){
 "Skipping " + $ServerB + ": " + $jobB.Name + " : Not included category: " + $jobB.Category
 }
 elseif ($jobB -eq $null){
 $ErrorMsg += "Missing on " + $ServerB + ":" + $jobA.Name + "`n"
 "Missing on " + $ServerB + ":" + $jobA.Name
 $ValidationStatus=$false
 if ($DoSync){
 "Creating job " + $jobA.Name + " on " + $ServerB
 Invoke-Sqlcmd -ServerInstance $ServerB -Query $scriptA -DisableVariables -DisableCommands
 }
 }
 else {
 
 $scriptB = ""
 $jobB.Script() | ForEach-Object {$scriptB += $_}
 
 if ($scriptA -eq $scriptB){
 "OK: " + $_.Name
 }
 else {
 if ($jobA.DateLastModified -lt $jobB.DateLastModified){
 $ErrorMsg += "Different (Newer on $ServerB): " + $_.Name + "`n"
 "Different (Newer on $ServerB): " + $_.Name
 $ValidationStatus=$false
 if ($DoSync){
 "Dropping job " + $jobA.Name + " on " + $ServerA
 $JobA.Drop()
 "Creating job " + $jobA.Name + " on " + $ServerA
 Invoke-Sqlcmd -ServerInstance $ServerA -Query $scriptB -DisableVariables -DisableCommands
 }
 }
 else{
 $ErrorMsg += "Different (Newer on $ServerA): " + $_.Name + "`n"
 "Different (Newer on $ServerA): " + $_.Name
 $ValidationStatus=$false
 if ($DoSync){
 "Dropping job " + $jobB.Name + " on " + $ServerB
 $JobB.Drop()
 "Creating job " + $jobB.Name + " on " + $ServerB
 Invoke-Sqlcmd -ServerInstance $ServerB -Query $scriptA -DisableVariables -DisableCommands
 }
 }
 }
 }
}

 $missingA = Get-SqlAgentJob -ServerInstance $ServerB | Where-Object {@(Get-SqlAgentJob -ServerInstance $ServerA -Name $_.Name -ErrorAction Ignore).Count -eq 0}
 $missingA | ForEach-Object{
 $jobB = $_
 if ($ExcludedCategories.Contains($jobB.Category)){
 "Skipping " + $ServerB + ": " + $jobB.Name + " : Excluded category: " + $jobB.Category
 }
 else{
 $ErrorMsg += "Missing on " + $ServerA + ":" + $jobB.Name + "`n"
 "Missing on " + $ServerA + ":" + $jobB.Name 
 $ValidationStatus=$false
 if ($DoSync){
 $scriptB = ""
 $jobB.Script() | ForEach-Object {$scriptB += $_}
 "Creating job " + $jobB.Name + " on " + $ServerA
 Invoke-Sqlcmd -ServerInstance $ServerA -Query $scriptB -DisableVariables -DisableCommands

 }
 }
 }
 if ($ValidationStatus -eq $false -and $DoSync -eq $false){
 $errorMsg = "SQL Agent jobs require synchronization`n" + $errorMsg
 throw $errorMsg
 }

Set Max Server Server Memory at Startup

Wed, 03 Mar 2021 23:28:40 +0000

Max server memory is one of the SQL Server configuration options that you should change from the default value. Brent Ozar has some good advice on this here. TLDR: You should leave some extra memory for the o/s.

A good starting point is to leave 10% (4GB minimum). If you are running other processes on your SQL server instances like SSRS, SSAS or if you are running multiple instances you might need to leave a lot more.

You can easily set the max server memory to your desired value in MB using sp_configure or using SSMS and simply forget about it:

Set Max Server Memory


1
2
3


-- Replace 12345 with your value in MB 
EXEC sys.sp_configure N'show advanced options', N'1' RECONFIGURE WITH OVERRIDE
EXEC sys.sp_configure 'max server memory (MB)', '12345'; RECONFIGURE;

What if you add/remove memory from your SQL instance? This is a setting that you need to remember to change. Gone are the days where you need to physically open your servers and install new DIMMS. With the advent of cloud and hypervisors - adding memory can be as easy as a few mouse clicks or running a script.

The point of this article is to consider doing this as a script at startup. If you scale up or down to a new instance type in the cloud - your max server memory could automatically be set to an appropriate value as a startup script. It’s something less to think about.

Show me the code!

This script will apply the 10%/4GB rule. You could adjust the formula as required.

Set Max Server Memory based on 10%/4GB


 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17


EXEC sys.sp_configure N''show advanced options'', N''1'' RECONFIGURE WITH OVERRIDE

DECLARE @MaxServerMemory INT
SELECT @MaxServerMemory=FLOOR((CASE WHEN total_physical_memory_kb <= (8*POWER(1024,2)) -- < 8GB, leave 50% for o/s
 THEN (total_physical_memory_kb * 0.5)
 WHEN total_physical_memory_kb <= (40*POWER(1024,2)) -- < 40GB, leave 4GB for o/s
 THEN total_physical_memory_kb-(4*POWER(1024,2))
 ELSE total_physical_memory_kb*0.9 -- leave 10% for o/s
 END)/1024)
FROM sys.dm_os_sys_memory

EXEC sys.sp_configure N''max server memory (MB)'', @MaxServerMemory
GO
RECONFIGURE WITH OVERRIDE
GO

EXEC sys.sp_configure N''show advanced options'', N''0'' RECONFIGURE WITH OVERRIDE

You just need to run this as a SQL Agent job at startup which you can do using the script below:

SET MAX SERVER MEMORY


 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83


USE [msdb]
GO

/****** Object: Job [SET MAX SERVER MEMORY] Script Date: 03/03/2021 22:22:46 ******/
BEGIN TRANSACTION
DECLARE @ReturnCode INT
SELECT @ReturnCode = 0
/****** Object: JobCategory [[Uncategorized (Local)]] Script Date: 03/03/2021 22:22:46 ******/
IF NOT EXISTS (SELECT name FROM msdb.dbo.syscategories WHERE name=N'[Uncategorized (Local)]' AND category_class=1)
BEGIN
EXEC @ReturnCode = msdb.dbo.sp_add_category @class=N'JOB', @type=N'LOCAL', @name=N'[Uncategorized (Local)]'
IF (@@ERROR <> 0 OR @ReturnCode <> 0) GOTO QuitWithRollback

END

DECLARE @jobId BINARY(16)
EXEC @ReturnCode = msdb.dbo.sp_add_job @job_name=N'SET MAX SERVER MEMORY',
@enabled=1,
@notify_level_eventlog=0,
@notify_level_email=2,
@notify_level_netsend=0,
@notify_level_page=0,
@delete_level=0,
@description=N'No description available.',
@category_name=N'[Uncategorized (Local)]',
@owner_login_name=N'sa',
@notify_email_operator_name=N'DBA', @job_id = @jobId OUTPUT
IF (@@ERROR <> 0 OR @ReturnCode <> 0) GOTO QuitWithRollback
/****** Object: Step [SET MAX SERVER MEMORY] Script Date: 03/03/2021 22:22:46 ******/
EXEC @ReturnCode = msdb.dbo.sp_add_jobstep @job_id=@jobId, @step_name=N'SET MAX SERVER MEMORY',
@step_id=1,
@cmdexec_success_code=0,
@on_success_action=1,
@on_success_step_id=0,
@on_fail_action=2,
@on_fail_step_id=0,
@retry_attempts=0,
@retry_interval=0,
@os_run_priority=0, @subsystem=N'TSQL',
@command=N'
EXEC sys.sp_configure N''show advanced options'', N''1'' RECONFIGURE WITH OVERRIDE

DECLARE @MaxServerMemory INT
SELECT @MaxServerMemory=FLOOR((CASE WHEN total_physical_memory_kb <= (8*POWER(1024,2)) -- < 8GB, leave 50% for o/s
 THEN (total_physical_memory_kb * 0.5)
 WHEN total_physical_memory_kb <= (40*POWER(1024,2)) -- < 40GB, leave 4GB for o/s
 THEN total_physical_memory_kb-(4*POWER(1024,2))
 ELSE total_physical_memory_kb*0.9 -- leave 10% for o/s
 END)/1024)
FROM sys.dm_os_sys_memory

EXEC sys.sp_configure N''max server memory (MB)'', @MaxServerMemory
GO
RECONFIGURE WITH OVERRIDE
GO

EXEC sys.sp_configure N''show advanced options'', N''0'' RECONFIGURE WITH OVERRIDE',
@database_name=N'master',
@flags=0
IF (@@ERROR <> 0 OR @ReturnCode <> 0) GOTO QuitWithRollback
EXEC @ReturnCode = msdb.dbo.sp_update_job @job_id = @jobId, @start_step_id = 1
IF (@@ERROR <> 0 OR @ReturnCode <> 0) GOTO QuitWithRollback
EXEC @ReturnCode = msdb.dbo.sp_add_jobschedule @job_id=@jobId, @name=N'AgentStart',
@enabled=1,
@freq_type=64,
@freq_interval=0,
@freq_subday_type=0,
@freq_subday_interval=0,
@freq_relative_interval=0,
@freq_recurrence_factor=0,
@active_start_date=20171101,
@active_end_date=99991231,
@active_start_time=0,
@active_end_time=235959
IF (@@ERROR <> 0 OR @ReturnCode <> 0) GOTO QuitWithRollback
EXEC @ReturnCode = msdb.dbo.sp_add_jobserver @job_id = @jobId, @server_name = N'(local)'
IF (@@ERROR <> 0 OR @ReturnCode <> 0) GOTO QuitWithRollback
COMMIT TRANSACTION
GOTO EndSave
QuitWithRollback:
 IF (@@TRANCOUNT > 0) ROLLBACK TRANSACTION
EndSave:
GO

DBA Daily Checks Email Report

Mon, 13 Jul 2009 12:00:37 +0000

Update: This was my original solution to automate the DBA Checks process. I don’t recommend people use this solution as it doesn’t scale well to a larger number of servers and the CLR function used to get drive space requires the TRUSTWORTHY property to be enabled which can be a potential security risk.

I recommend people use DBA Dash for daily DBA Checks. This gives you a central dashboard so you can see the status of all your SQL instances on a single page with drill downs for more detailed information.

Introduction

In addition to alerts I recommend running daily checks to validate that backups are running as expected, servers have sufficient disk space and agent jobs are running successfully. Although alerts provide immediate notification that there is an issue you can’t assume that everything is OK because you haven’t received an email notification.

For example, an email alert might get caught by a spam filter or matched by a rule in your email system that moves it to a different folder. It’s also possible there was a problem sending the notification email. If you are not careful with how you configure alerts you can easily get spammed by a large number of alert emails and important notifications could be lost in this flood.

You might not get a failed backup job notification if your backup job never ran - maybe the job got disabled or the SQL Agent wasn’t running. These are some of the reasons I recommend daily checks.

The solution in this post has each SQL Server send a report via email that runs various checks and highlights any issues. It’s a solution I used when I managed a smaller number of servers but as the number of instances I managed increased it became unwieldy. Depending on the number of serves you manage you might find it useful.

I now use a different solution that collects data from multiple servers in a central repository. This allows me to have a single dashboard that highlights problems across multiple servers instead of having to check individual reports for each server. I hope to share my new method at some point.

Report Sample

The image below is a sample report that was generated from a test instance of SQL Server.

Compatibility

SQL Server 2005 (SP3)
There seems to be a bug in earlier builds related to attachments in email messages. You might get an error similar to this one:
_Exception happened when running extended stored procedure ‘xp_sysmail_format_query’ in the library ‘xpstar90.dll’. SQL Server is terminating process 62. Exception type: Win32 exception; Exception code: 0xc0000005.
_You can fix this by upgrading to the latest service pack or removing the code that adds the error log attachment to the email. Restarting the SQL Server Agent can also provide a temporary fix for this issue, but the problem is likely to re-occur.
SQL Server 2008
SQL Server 2008 R2

Installation Instructions

Create a database for DBA related tasks if one doesn’t already exist.
Install SQLIO_fnGetDrives() CLR function. See here.
Install DBA Checks stored procedures. (Script below)
Make sure you have configured database mail on the SQL Server instance.
Create a SQL Server Agent Job to run the dbo.DBAChecks stored procedure.

SET ANSI_NULLS ON
GO
SET QUOTED_IDENTIFIER ON
GO
CREATE PROC [dbo].[DBAChecks_Backups](
	@HTML varchar(max) OUT,
	@FullWarningThresholdDays int,
	@DiffWarningThresholdDays int,
	@TranWarningThresHoldHours int
)
AS
SET NOCOUNT ON
/* 
	Returns HTML for "Backups" section of DBA Checks Report
*/ 
declare @Server varchar(40)
set @server = CONVERT(varchar(35), SERVERPROPERTY('machinename'))+ COALESCE('\'+ CONVERT(varchar(35), SERVERPROPERTY('instancename')),'')

DECLARE @backuplog TABLE(
	DBServer sysname,
	DBName sysname,
	LastFullBackup datetime,
	LastDiffBackup datetime,
	LastTranBackup datetime,
	DiffDays int,
	FullDays int,
	TranHours int
)

INSERT INTO @backuplog
SELECT fullrec.server_name [DBServer],
	 fullrec.database_name, 
	 fullrec.backup_finish_date LastFullBackup,
	 diffrec.backup_finish_date LastDiffBackup,
	 tranrec.backup_finish_date LastTranBackup,
	 datediff(dd,diffrec.backup_finish_date,getdate()) DiffDays,
	 datediff(dd,fullrec.backup_finish_date,getdate()) FullDays,
	 datediff(hh,tranrec.backup_finish_date,getdate()) TranHours
FROM msdb..backupset as fullrec
left outer join msdb..backupset as tranrec 
	on tranrec.database_name = fullrec.database_name
			and tranrec.server_name = fullrec.server_name
			and tranrec.type = 'L'
			and tranrec.backup_finish_date =
			 ((select max(backup_finish_date) 
				 from msdb..backupset b2 
				 where b2.database_name = fullrec.database_name
				 and b2.server_name = fullrec.server_name 
				 and b2.type = 'L'))
left outer join msdb..backupset as diffrec
	on diffrec.database_name = fullrec.database_name
		and diffrec.server_name = fullrec.server_name
		and diffrec.type = 'I'
		and diffrec.backup_finish_date =
		 ((select max(backup_finish_date) 
			 from msdb..backupset b2 
			where b2.database_name = fullrec.database_name 
			 and b2.server_name = fullrec.server_name 
			 and b2.type = 'I'))
where fullrec.type = 'D' -- full backups only
	and fullrec.backup_finish_date = 
	 (select max(backup_finish_date) 
		 from msdb..backupset b2 
		 where b2.database_name = fullrec.database_name 
		 and b2.server_name = fullrec.server_name 
		 and b2.type = 'D')
	and fullrec.database_name in (select name from master..sysdatabases) 
	and fullrec.database_name <> 'tempdb'
Union all
-- never backed up
select @server
		,name 
		,null
		,NULL
		,NULL 
		,NULL
		,NULL
		,NULL
from sys.databases as record
where not exists (select * from msdb..backupset where record.name = database_name and server_name = @server)
and name <> 'tempdb'
and source_database_id is null --exclude snapshots

SET @HTML = '<h2>Backups</h2>
			<table>
			<tr>
			<th>DB Server</th>
			<th>DB Name</th>
			<th>Last Full Backup</th>
			<th>Last Diff Backup</th>
			<th>Last Tran Backup</th>
			<th>Full Days</th>
			<th>Diff Days</th>
			<th>Tran Hours</th>
			<th>Recovery Model</th>
			</tr>' 
			+(SELECT DBServer td,
					DBName td,
					CAST(CASE WHEN FullDays IS NULL THEN '<div class="Critical">None/Unknown</div>'
						WHEN FullDays > @FullWarningThresholdDays THEN '<div class="Warning">' + LEFT(CONVERT(varchar,LastFullBackup,113),17) + '</div>'
						ELSE '<div class="Healthy">' + LEFT(CONVERT(varchar,LastFullBackup,113),17) + '</div>' END as XML) td,
					CAST(CASE WHEN DiffDays IS NULL THEN '<div class="Critical">None/Unknown</div>'
						WHEN DiffDays > @DiffWarningThresholdDays THEN '<div class="Warning">' + LEFT(CONVERT(varchar,LastDiffBackup,113),17) + '</div>'
						ELSE '<div class="Healthy">' + LEFT(CONVERT(varchar,LastDiffBackup,113),17) + '</div>' END as XML) td,
					CAST(CASE WHEN TranHours IS NULL THEN COALESCE(LEFT(NULLIF(recovery_model_desc,'SIMPLE'),0) + '<div class="Critical">None/Unknown</div>','N/A')
						WHEN TranHours > @TranWarningThresholdHours THEN '<div class="Warning">' + LEFT(CONVERT(varchar,LastTranBackup,113),17) + '</div>'
						ELSE '<div class="Healthy">' + LEFT(CONVERT(varchar,LastTranBackup,113),17) + '</div>' END as XML) td,
					FullDays td,
					DiffDays td,
					TranHours td,
					recovery_model_desc td
			FROM @backuplog bl
			LEFT JOIN sys.databases sdb on bl.DBName = sdb.name COLLATE SQL_Latin1_General_CP1_CI_AS AND bl.DBServer = @Server
			WHERE DBServer = @@SERVERNAME
			ORDER BY LastFullBackup,LastDiffBackup,LastTranBackup,DBName
			FOR XML RAW('tr'),ELEMENTS XSINIL
			)
			+ '</table>'
GO
SET ANSI_NULLS ON
GO
SET QUOTED_IDENTIFIER ON
GO
CREATE PROC [dbo].[DBAChecks_ErrorLog](@NumDays int)
AS
/* 
	Returns HTML for "ErrorLog.htm" attachment of DBA Checks Report
*/ 
SET NOCOUNT ON
CREATE TABLE #ErrorLog(
	LogDate datetime,
	ErrorSource nvarchar(max),
	ErrorMessage nvarchar(max)
)

CREATE TABLE #ErrorLogs(
	ID INT primary key not null,
	LogDate DateTime NOT NULL,
	LogFileSize bigint
)
DECLARE @MinDate datetime
SET @MinDate = CONVERT(datetime,CONVERT(varchar,DATEADD(d,-@NumDays,GetDate()),112),112)

--Get a list of available error logs
INSERT INTO #ErrorLogs(ID,LogDate,LogFileSize)
EXEC master.dbo.xp_enumerrorlogs

DECLARE @ErrorLogID int

DECLARE cErrorLogs CURSOR FOR
	SELECT ID
	FROM #ErrorLogs
	WHERE LogDate >= @MinDate

OPEN cErrorLogs
FETCH NEXT FROM cErrorLogs INTO @ErrorLogID
-- Read applicable error logs into the #errorlog table
WHILE @@FETCH_STATUS = 0
BEGIN
	INSERT INTO #ErrorLog(LogDate,ErrorSource,ErrorMessage)
	exec sp_readerrorlog @ErrorLogID
	FETCH NEXT FROM cErrorLogs INTO @ErrorLogID
END

CLOSE cErrorLogs
DEALLOCATE cErrorLogs

SELECT '<HTML>
<HEAD>
<style type="text/css">
table {
/*width:100%;*/
font:8pt tahoma,arial,sans-serif;
border-collapse:collapse;
}
th {
color:#FFFFFF;
font:bold 8pt tahoma,arial,sans-serif;
background-color:#204c7d;
padding-left:5px;
padding-right:5px;
}
td {
color:#000000;
font:8pt tahoma,arial,sans-serif;
border:1px solid #DCDCDC;
border-collapse:collapse;
}
.Warning {
background-color:#FFFF00; 
color:#2E2E2E;
}
.Critical {
background-color:#FF0000;
color:#FFFFFF;
}
.Healthy {
background-color:#458B00;
color:#FFFFFF;
}
</style>
</HEAD>
<BODY>
<table><tr><th>Log Date</th><th>Source</th><th>Message</th></tr>' + 
(SELECT CONVERT(varchar,LogDate,120) td,
	CAST('<div><![CDATA[' + ErrorSource + N']]></div>' as XML) td,
	CAST('<div' + 
		CASE WHEN (ErrorMessage LIKE '%error%' OR ErrorMessage LIKE '%exception%' 
					OR ErrorMessage LIKE '%stack dump%' OR ErrorMessage LIKE '%fail%') 
				AND ErrorMessage NOT LIKE '%DBCC%' THEN ' Class="Critical"' 
		WHEN ErrorMessage LIKE '%warning%' THEN ' Class="Warning"'
		ELSE '' END 
		+ '><![CDATA[' + ErrorMessage + N']]></div>' as XML) td
FROM #ErrorLog
WHERE LogDate >= @MinDate
/*	Remove any error log records that we are not interested in
	ammend the where clause as appropriate
*/
AND ErrorMessage NOT LIKE '%This is an informational message%'
AND ErrorMessage NOT LIKE 'Authentication mode is%'
AND ErrorMessage NOT LIKE 'System Manufacturer%'
AND ErrorMessage NOT LIKE 'All rights reserved.'
AND ErrorMessage NOT LIKE 'Server Process ID is%'
AND ErrorMessage NOT LIKE 'Starting up database%'
AND ErrorMessage NOT LIKE 'Registry startup parameters%'
AND ErrorMessage NOT LIKE '(c) 2005 Microsoft%'
AND ErrorMessage NOT LIKE 'Server is listening on%'
AND ErrorMessage NOT LIKE 'Server local connection provider is ready to accept connection on%'
AND ErrorMessage NOT LIKE 'Logging SQL Server messages in file%'
AND ErrorMessage <> 'Clearing tempdb database.'
AND ErrorMessage <> 'Using locked pages for buffer pool.'
AND ErrorMessage <> 'Service Broker manager has started.'
ORDER BY LogDate DESC
FOR XML RAW('tr'),ELEMENTS XSINIL)
+ '</table></HEAD></BODY>' as HTML

DROP TABLE #ErrorLog
DROP TABLE #ErrorLogs
GO
SET ANSI_NULLS ON
GO
SET QUOTED_IDENTIFIER ON
GO
CREATE PROC [dbo].[DBAChecks_DBFiles](
	@IncludeDBs varchar(max)=NULL,
	@ExcludeDBs varchar(max)='master,model,msdb,tempdb',
	@WarningThresholdPCT int=90,
	@CriticalThresholdPCT int=95,
	@HTML varchar(max) output
)
AS
/* 
	Returns HTML for "Database Files" section of DBA Checks Report
*/ 
CREATE TABLE #FileStats(
	[db] sysname not null,
	[name] [sysname] not null,
	[file_group] [sysname] null,
	[physical_name] [nvarchar](260) NOT NULL,
	[type_desc] [nvarchar](60) NOT NULL,
	[size] [varchar](33) NOT NULL,
	[space_used] [varchar](33) NULL,
	[free_space] [varchar](33) NULL,
	[pct_used] [float] NULL,
	[max_size] [varchar](33) NOT NULL,
	[growth] [varchar](33) NOT NULL
) 
DECLARE @IncludeXML XML
DECLARE @ExcludeXML XML
DECLARE @DB sysname

IF @IncludeDBs = ''
BEGIN
	SET @HTML = ''
	DROP TABLE #FileStats
	RETURN
END

SELECT @IncludeXML = '<a>' + REPLACE(@IncludeDBs,',','</a><a>') + '</a>'
SELECT @ExcludeXML = '<a>' + REPLACE(@ExcludeDBs,',','</a><a>') + '</a>'

DECLARE cDBs CURSOR FOR
			SELECT name FROM sys.databases
			WHERE (name IN(SELECT n.value('.','sysname')
						FROM @IncludeXML.nodes('/a') T(n))
						OR @IncludeXML IS NULL)
				AND (name NOT IN(SELECT n.value('.','sysname')
						FROM @ExcludeXML.nodes('/a') T(n))
						OR @ExcludeXML IS NULL)
			AND source_database_id IS NULL
			AND state = 0 --ONLINE
			ORDER BY name
			
OPEN cDBs
FETCH NEXT FROM cDBs INTO @DB
WHILE @@FETCH_STATUS = 0
BEGIN
	DECLARE @SQL nvarchar(max)
	SET @SQL =		 N'USE ' + QUOTENAME(@DB) + ';
					INSERT INTO #FileStats(db,name,file_group,physical_name,type_desc,size,space_used,free_space,[pct_used],max_size,growth)
					select DB_NAME() db,
					f.name,
					fg.name as file_group,
					f.physical_name,
					f.type_desc,
					CASE WHEN (f.size/128) < 1024 THEN CAST(f.size/128 as varchar) + '' MB'' 
						ELSE CAST(CAST(ROUND(f.size/(128*1024.0),1) as float) as varchar) + '' GB'' 
						END as size,
					CASE WHEN FILEPROPERTY(f.name,''spaceused'')/128 < 1024 THEN CAST(FILEPROPERTY(f.name,''spaceused'')/128 as varchar) + '' MB''
						ELSE CAST(CAST(ROUND(FILEPROPERTY(f.name,''spaceused'')/(128*1024.0),1) as float) as varchar) + '' GB'' 
						END space_used,
					CASE WHEN (f.size - FILEPROPERTY(f.name,''spaceused''))/128 < 1024 THEN CAST((f.size - FILEPROPERTY(f.name,''spaceused''))/128 as varchar) + '' MB''
						ELSE CAST(CAST(ROUND((f.size - FILEPROPERTY(f.name,''spaceused''))/(128*1024.0),1) as float) as varchar) + '' GB''
						END free_space,
					ROUND((FILEPROPERTY(f.name,''spaceused''))/CAST(size as float)*100,2) as [pct_used],
					CASE WHEN f.max_size =-1 THEN ''unlimited'' 
						WHEN f.max_size/128 < 1024 THEN CAST(f.max_size/128 as varchar) + '' MB'' 
						ELSE CAST(f.max_size/(128*1024) as varchar) + '' GB''
						END as max_size,
					CASE WHEN f.is_percent_growth=1 THEN CAST(f.growth as varchar) + ''%''
						WHEN f.growth = 0 THEN ''none''
						WHEN f.growth/128 < 1024 THEN CAST(f.growth/128 as varchar) + '' MB'' 
						ELSE CAST(CAST(ROUND(f.growth/(128*1024.0),1) as float) as varchar) + '' GB''
						END growth
					from sys.database_files f
					LEFT JOIN sys.filegroups fg on f.data_space_id = fg.data_space_id
					where f.type_desc <> ''FULLTEXT'''
	exec sp_executesql @SQL	
					
	FETCH NEXT FROM cDBs INTO @DB
END
CLOSE cDBs
DEALLOCATE cDBs

SELECT @HTML = '<h2>Database Files</h2><table>' + 
		(SELECT 'Database' th,
		'Name' th,
		'File Group' th,
		'File Path' th,
		'Type' th,
		'Size' th,
		'Used' th,
		'Free' th,
		'Used %' th,
		'Max Size' th,
		'Growth' th
		FOR XML RAW('tr'),ELEMENTS ) +		
		(SELECT db td,
					name td,
					file_group td,
					physical_name td,
					type_desc td,
					size td,
					space_used td,
					free_space td,
					CAST(CASE WHEN pct_used > @CriticalThresholdPCT 
						THEN '<div class="Critical">' + CAST(pct_used as varchar) + '</div>'
						WHEN pct_used > @WarningThresholdPCT 
						THEN '<div class="Warning">' + CAST(pct_used as varchar) + '</div>'
						ELSE '<div class="Healthy">' + CAST(pct_used as varchar) + '</div>'
						END as XML) td,
					max_size td,
					CAST(CASE WHEN growth='none' THEN '<div class="Warning">' + growth + '</div>'
					ELSE growth END as XML) td
				FROM #FileStats
				ORDER BY db,type_desc DESC,file_group,name
				FOR XML RAW('tr'),ELEMENTS XSINIL) + '</table>'
				
DROP TABLE #FileStats
GO
SET ANSI_NULLS ON
GO
SET QUOTED_IDENTIFIER ON
GO
CREATE PROC [dbo].[DBAChecks_JobStats](@NumDays int,@HTML nvarchar(max) out)
AS
/* 
	Returns HTML for "Agent Jobs Stats in the last 'X' days" section of DBA Checks Report
*/ 
SET ANSI_WARNINGS OFF
DECLARE @FromDate char(8)
SET @FromDate = CONVERT(char(8), (select dateadd (day,(-1*@NumDays), getdate())), 112);

WITH nextRun as (
	SELECT js.job_id, 
		MAX(CONVERT(datetime,CONVERT(CHAR(8), NULLIF(next_run_date,0), 112) 
			+ ' ' 
			+ STUFF(STUFF(RIGHT('000000' 
			+ CONVERT(VARCHAR(8), next_run_time), 6), 5, 0, ':'), 3, 0, ':') )
			) as next_run_time
	FROM msdb..sysjobschedules js
	GROUP BY js.job_id
),
lastRun as (
	SELECT jh.job_id,CONVERT(datetime,CONVERT(CHAR(8), run_date, 112) 
		+ ' ' 
		+ STUFF(STUFF(RIGHT('000000' 
		+ CONVERT(VARCHAR(8), run_time), 6), 5, 0, ':'), 3, 0, ':') ) as last_run_time,
		run_status as last_run_status,
		CAST(message as nvarchar(max)) as last_result,
		ROW_NUMBER() OVER(PARTITION BY job_id ORDER BY run_date DESC,run_time DESC) rnum
	FROM msdb..sysjobhistory jh
	WHERE run_status IN(0,1,3) --Succeeded/Failed/Cancelled
)
,JobStats AS (
	select name,
			MAX(enabled) enabled,
			SUM(CASE WHEN run_status = 1 THEN 1 ELSE 0 END) as SucceededCount,
			SUM(CASE WHEN run_status = 0 THEN 1 ELSE 0 END) as FailedCount,
			SUM(CASE WHEN run_status = 3 THEN 1 ELSE 0 END) as CancelledCount,
			MAX(last_run_time) last_run_time,
			MAX(next_run_time) next_run_time,
			MAX(last_run_status) last_run_status,
			COALESCE(MAX(last_result),'Unknown') last_result
	from msdb..sysjobs j
	LEFT JOIN msdb..sysjobhistory jh ON j.job_id = jh.job_id AND jh.run_date >= @FromDate and jh.step_id = 0
	LEFT JOIN nextrun ON j.job_id = nextrun.job_id
	LEFT JOIN lastRun ON j.job_id = lastRun.job_id AND rnum=1
	GROUP BY name
)
SELECT @HTML =N'<h2>Agent Job Stats in the last ' + CAST(@NumDays as varchar) + N' days</h2>
	<table>' +
	(SELECT 'Name' th,
	'Enabled' th,
	'Succeeded' th,
	'Failed' th,
	'Cancelled' th,
	'Last Run Time' th,
	'Next Run Time' th,
	'Last Result' th
	FOR XML RAW('tr'),ELEMENTS) 
	+ (SELECT name td,
			CAST(CASE WHEN enabled = 1 THEN N'<div class="Healthy">Yes</div>'
					ELSE N'<div class="Warning">No</div>' END as XML) td,
			CAST(CASE WHEN SucceededCount = 0 THEN N'<div class="Warning">'
					ELSE N'<div>' END
					+ CAST(SucceededCount as varchar) + '</div>' as XML) td,
			CAST(CASE WHEN FailedCount >0 THEN N'<div class="Critical">'
					ELSE N'<div class="Healthy">' END
					+ CAST(FailedCount as varchar) + N'</div>' as XML) td,
			CAST(CASE WHEN CancelledCount >0 THEN N'<div class="Critical">'
					ELSE N'<div class="Healthy">' END
					+ CAST(CancelledCount as varchar) + N'</div>' as XML) td,
			LEFT(CONVERT(varchar,last_run_time,13),17) td,
			LEFT(CONVERT(varchar,next_run_time,13),17) td,
			CAST(CASE WHEN last_run_status = 1 THEN N'<span class="Healthy"><![CDATA[' + last_result + N']]></span>' 
					ELSE N'<span class="Critical"><![CDATA[' + last_result + N']]></span>' END AS XML) td 
		FROM JobStats
		ORDER BY last_run_time DESC
		FOR XML RAW('tr'),ELEMENTS XSINIL
	) + N'</table>'
GO
SET ANSI_NULLS ON
GO
SET QUOTED_IDENTIFIER ON
GO
CREATE PROC [dbo].[DBAChecks_FailedAgentJobs](@HTML varchar(max) out,@NumDays int)
AS
/* 
	Returns HTML for "Failed jobs in the last 'X' days" section of DBA Checks Report
*/ 
DECLARE @FromDate char(8)
DECLARE @SucceededCount int
SET @FromDate = CONVERT(char(8),dateadd (day,-@NumDays, getdate()), 112)

IF EXISTS(	
	SELECT *
	FROM msdb..sysjobhistory jh
	JOIN msdb..sysjobs j ON jh.job_id = j.job_id
	WHERE jh.run_status IN(0,3) -- Failed/Cancelled
		AND jh.step_id <> 0
		AND	run_date >= @FromDate)
BEGIN
SET @HTML = '<h2>Failed Jobs in the last ' + CAST(@NumDays as varchar) + ' days</h2>
	<table>
	<tr>
	<th>Date</th>
	<th>Job Name</th>
	<th>Job Status</th>
	<th>Step ID</th>
	<th>Step Name</th>
	<th>Message</th>
	<th>Run Duration</th>
	</tr>'
	+
	(SELECT CONVERT(datetime,CAST(jh.run_date AS char(8)) + ' ' + STUFF(STUFF(RIGHT('000000' + CONVERT(VARCHAR,jh.run_time),6),5,0,':'),3,0,':'),101) AS td,
		j.name AS td,
		CAST(CASE jh.run_status WHEN 0 THEN '<div class="Critical">Failed</div>'
			WHEN 3 THEN '<div class="Warning">Cancelled</div>'
			ELSE NULL END as XML) as td,
		jh.step_id as td,
		jh.step_name as td,
		jh.message as td,
				RIGHT('00' +CAST(run_duration/10000 as varchar),2) + ':' +
				RIGHT('00' + CAST(run_duration/100%100 as varchar),2) + ':' +
				RIGHT('00' + CAST(run_duration%100 as varchar),2) as td
	FROM	msdb..sysjobhistory jh
	JOIN	msdb..sysjobs j
		ON jh.job_id = j.job_id
	WHERE	jh.run_status IN(0,3) -- Failed/Cancelled
		AND jh.step_id <> 0
		AND	run_date >= @FromDate
	ORDER BY CONVERT(datetime,CAST(jh.run_date AS char(8)) + ' ' + STUFF(STUFF(RIGHT('000000' + CONVERT(VARCHAR,jh.run_time),6),5,0,':'),3,0,':'),101) DESC
	FOR XML RAW('tr'),ELEMENTS XSINIL
	)
	+ '</table><br/>'
END
ELSE
BEGIN
	SET @HTML = '<h2>Failed Jobs in the last ' + CAST(@NumDays as varchar) + ' days</h2>
				<span class="Healthy">No failed jobs</span><br/>'	
END
GO
SET ANSI_NULLS ON
GO
SET QUOTED_IDENTIFIER ON
GO
CREATE PROC [dbo].[DBAChecks_DiskDrives](@PCTFreeWarningThreshold int,@PCTFreeCriticalThreshold int,@HTML varchar(max) out)
AS
/* 
	Returns HTML for "Disk Drives" section of DBA Checks Report
*/ 
DECLARE @BytesToTB float,@BytesToGB float,@BytesToMB float
SELECT @BytesToTB = 1099511627776,@BytesToGB = 1073741824,@BytesToMB=1048576.0;

WITH DriveInfo AS (
	SELECT drive,
		volume_label,
		CASE WHEN total_size > @BytesToTB THEN CAST(ROUND(total_size /@BytesToTB,1) as varchar) + 'TB'
			WHEN total_size > @BytesToGB THEN CAST(ROUND(total_size /@BytesToGB,1) as varchar) + 'GB'
			ELSE CAST(ROUND(total_size / @BytesToMB,1) as varchar) + 'MB' END as Size,
		CASE WHEN free_space > @BytesToTB THEN CAST(ROUND(free_space /@BytesToTB,1) as varchar) + 'TB'
			WHEN free_space > @BytesToGB THEN CAST(ROUND(free_space /@BytesToGB,1) as varchar) + 'GB'
			ELSE CAST(ROUND(free_space / @BytesToMB,1) as varchar) + 'MB' END as Free,
		ROUND(free_space/cast(total_size as float)*100,2) as PercentFree
	FROM dbo.SQLIO_fnGetDrives()
	WHERE drive_type=3 --Fixed
)
SELECT @HTML = 
	'<h2>Disk Drives</h2>
	<table>' +
	(SELECT 'Drive' th,
			'Label' th,
			'Size' th,
			'Free' th,
			'Free %' th
	FOR XML RAW('tr'),ELEMENTS) 
	+
	(SELECT drive td,
			volume_label td,
			Size td,
			Free td,
			CAST(CASE WHEN PercentFree < @PCTFreeCriticalThreshold THEN '<div class="Critical">' + CAST(PercentFree as varchar) + '</div>'
			WHEN PercentFree < @PCTFreeWarningThreshold THEN '<div class="Warning">' + CAST(PercentFree as varchar) + '</div>'
			ELSE '<div class="Healthy">' + CAST(PercentFree as varchar) + '</div>' END as XML) td
	FROM DriveInfo 
	FOR XML RAW('tr'),ELEMENTS XSINIL)
	+ '</table>'
GO
SET ANSI_NULLS ON
GO
SET QUOTED_IDENTIFIER ON
GO
CREATE PROC [dbo].[DBAChecks](
	/* Agent Jobs related Params */
	@AgentJobsNumDays int=3, --Num of days to report failed jobs over
	/* Database Files related Params */
	@FileStatsIncludedDatabases varchar(max)=NULL, -- Comma sep list of databases to report filestats for. NULL=All, '' = None
	@FileStatsExcludedDatabases varchar(max)=NULL, -- Comma sep list of databases to report filestats for. NULL=No Exclusions
	@FileStatsPctUsedWarning int=90, -- Warn (Yellow) if free space in a database file is less than this threshold (Just for database specified in @FileStatsDatabases)
	@FileStatsPctUsedCritical int=95, -- Warn (Red) if free space in a database file is less than this threshold (Just for database specified in @FileStatsDatabases)
	/* Backup related Params */
	@DiffWarningThresholdDays int=3, -- Backup warning if no diff backup for "x" days
	@FullWarningThresholdDays int=7, -- Backup warning if no full backup for "x" days
	@TranWarningThresholdHours int=4, -- Backup warning if no tran backup for "x" hours
	/* Disc Drive related params*/
	@FreeDiskSpacePercentWarningThreshold int=15, -- Warn (Yellow) if free space is less than this threshold
	@FreeDiskSpacePercentCriticalThreshold int=10, -- Warn (Red) if free space is less than this threshold
	/* General related params */
	@UptimeCritical int = 1440, -- Critical/Red if system uptime (in minutes) is less than this value
	@UptimeWarning int = 2880, -- Warn/Yellow if system uptime (in minutes) is less than this value,
	/* Error Log Params */
	@ErrorLogDays int = 3,
	/* Email/Profile params */
	@Recipients nvarchar(max), -- Email list
	@MailProfile sysname=NULL
)
AS
/*	Created By David Wiseman
	Version: 1.0
	http://www.wisesoft.co.uk
	Generates a DBA Checks HTML email report
*/
SET NOCOUNT ON
DECLARE @AgentJobsHTML varchar(max)
DECLARE @AgentJobStatsHTML varchar(max)
DECLARE @FileStatsHTML varchar(max)
DECLARE @DisksHTML varchar(max)
DECLARE @BackupsHTML varchar(max)
DECLARE @HTML varchar(max)
DECLARE @Uptime varchar(max)

SELECT @Uptime = 
	CASE WHEN DATEDIFF(mi,create_date,GetDate()) < @UptimeCritical THEN '<span class="Critical">'
	WHEN DATEDIFF(mi,create_date,GetDate()) < @Uptimewarning THEN '<span class="Warning">'
	ELSE '<span class="Healthy">' END + 
	-- get system uptime
	COALESCE(NULLIF(CAST((DATEDIFF(mi,create_date,GetDate())/1440 ) as varchar),'0') + ' day(s), ','')
	+ COALESCE(NULLIF(CAST(((DATEDIFF(mi,create_date,GetDate())%1440)/60) as varchar),'0') + ' hour(s), ','')
	+ CAST((DATEDIFF(mi,create_date,GetDate())%60) as varchar) + 'min'
	--
	+ '</span>'
FROM sys.databases 
WHERE NAME='tempdb'

exec dbo.DBAChecks_FailedAgentJobs @HTML=@AgentJobsHTML out,@NumDays=@AgentJobsNumDays

exec dbo.DBAChecks_JobStats @HTML=@AgentJobStatsHTML out,@NumDays=@AgentJobsNumDays

exec dbo.DBAChecks_DBFiles 
	@IncludeDBs=@FileStatsIncludedDatabases,
	@ExcludeDBs=@FileStatsExcludedDatabases,
	@WarningThresholdPCT=@FileStatsPctUsedWarning,
	@CriticalThresholdPCT=@FileStatsPctUsedCritical,
	@HTML=@FileStatsHTML out

exec dbo.DBAChecks_DiskDrives @HTML=@DisksHTML out,@PCTFreeWarningThreshold=@FreeDiskSpacePercentWarningThreshold,@PCTFreeCriticalThreshold=@FreeDiskSpacePercentCriticalThreshold 

exec dbo.DBAChecks_Backups @HTML=@BackupsHTML OUT,@DiffWarningThresHoldDays=@DiffWarningThresHoldDays,
	@FullWarningThresholdDays=@FullWarningThresholdDays ,@TranWarningThresholdHours=@FullWarningThresholdDays

SET @HTML = 
'<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN">
<html>
<head>
<style type="text/css">
table {
font:8pt tahoma,arial,sans-serif;
}
th {
color:#FFFFFF;
font:bold 8pt tahoma,arial,sans-serif;
background-color:#204c7d;
padding-left:5px;
padding-right:5px;
}
td {
color:#000000;
font:8pt tahoma,arial,sans-serif;
border:1px solid #DCDCDC;
border-collapse:collapse;
padding-left:3px;
padding-right:3px;
}
.Warning {
background-color:#FFFF00; 
color:#2E2E2E;
}
.Critical {
background-color:#FF0000;
color:#FFFFFF;
}
.Healthy {
background-color:#458B00;
color:#FFFFFF;
}
h1 {
color:#FFFFFF;
font:bold 16pt arial,sans-serif;
background-color:#204c7d;
text-align:center;
}
h2 {
color:#204c7d;
font:bold 14pt arial,sans-serif;
}
h3 {
color:#204c7d;
font:bold 12pt arial,sans-serif;
}
body {
color:#000000;
font:8pt tahoma,arial,sans-serif;
margin:0px;
padding:0px;
}
</style>
</head>
<body>
<h1>DBA Checks Report for ' + @@SERVERNAME + '</h1>
<h2>General Health</h2>
<b>System Uptime (SQL Server): ' + @Uptime + '</b><br/>
<b>Version: </b>' + CAST(SERVERPROPERTY('productversion') as nvarchar(max)) + ' ' 
	+ CAST(SERVERPROPERTY ('productlevel') as nvarchar(max))
 + ' ' + CAST(SERVERPROPERTY ('edition') as nvarchar(max)) 
+ COALESCE(@DisksHTML,'<div class="Critical">Error collecting Disk Info</div>')
+ COALESCE(@BackupsHTML,'<div class="Critical">Error collecting Backup Info</div>')
+ COALESCE(@AgentJobStatsHTML,'<div class="Critical">Error collecting Agent Jobs Stats</div>')
+ COALESCE(@AgentJobsHTML,'<div class="Critical">Error collecting Agent Jobs Info</div>')
+ COALESCE(@FileStatsHTML,'<div class="Critical">Error collecting File Stats Info</div>')
+ '</body></html>'

declare @subject varchar(50)
set @subject = 'DBA Checks (' + @@SERVERNAME + ')'

DECLARE @ErrorLogSQL nvarchar(max)
DECLARE @ExecuteQueryDB sysname
SET @ErrorLogSQL = 'exec DBAChecks_ErrorLog @NumDays=' + CAST(@ErrorLogDays as varchar)
SET @ExecuteQueryDB = DB_NAME()

EXEC msdb.dbo.sp_send_dbmail
	@query=@ErrorLogSQL,
	@attach_query_result_as_file = 1,
	@query_attachment_filename = 'ErrorLog.htm',
	@query_result_header = 0,
	@query_no_truncate = 1,
	@query_result_width=32767,
	@recipients =@Recipients,
	@body = @HTML,
	@body_format ='HTML',
	@subject = @subject,
	@execute_query_database=@ExecuteQueryDB,
	@profile_name = @MailProfile
GO

Version History

Date	Version	Notes
03-Apr-2012	Version 1.2 BETA	Fix to DBAChecks_JobStats stored procedure. Added CDATA section.
14-Dec-2010	Version 1.1 BETA	Fixed datetime conversion bug in DBAChecks_JobStats procedure.
13-Jul-2009	Version 1.0 BETA	Beta release

Running the report

The DBA Checks report is run by executing the dbo.DBAChecks stored procedure. This stored procedure takes a number of parameters, but only one is required:exec dbo.DBAChecks @recipients=‘dummyemail@wisesoft.co.uk’

The code below shows a call to the DBAChecks stored procedure with all parameters specified:EXEC dbo.DBAChecks @AgentJobsNumDays=3,
@FileStatsIncludedDatabases=NULL,
@FileStatsExcludedDatabases=NULL,
@FileStatsPctUsedWarning=90,
@FileStatsPctUsedCritical=95,
@DiffWarningThresholdDays=3,
@FullWarningThresholdDays=7,
@TranWarningThresholdHours=4,
@FreeDiskSpacePercentWarningThreshold=15,
@FreeDiskSpacePercentCriticalThreshold=10,
@UptimeCritical=1440 ,
@UptimeWarning=2880,
@ErrorLogDays=3,
@Recipients=‘dummyemail@wisesoft.co.uk’,
@MailProfile=NULL

A full explanation of these parameters is available here:

@AgentJobsNumDays	The number of days SQL SQL Server jobs are reported over.
@FileStatsIncludedDatabases	A list of databases (comma-separated) to display file stats for. Default value is NULL (All databases).
@FileStatsExcludedDatabases	A list of databases (comma-separated) that are excluded from database file stats. Default values is NULL (No excluded databases)
@FileStatsPctUsedWarning	If the percent used space in the database file is larger than this value (but less than critical threshold) it will be highlighted in yellow.
@FileStatsPctUsedCritical	If the percent used space in the database file is larger than this value it will be highlighted in red.
@DiffWarningThresholdDays	Highlights differential backups that have not been completed for over "X" number of days
@FullWarningThresholdDays	Highlights full backups that have not been completed for over "X" number of days
@TranWarningThresholdHours	Highlights transaction log backups that have not been completed for over "X" number of hours.
@FreeDiskSpacePercentWarningThreshold	Used to highlight disk drives with low disk space in yellow, where the free disk space percent is less than the value specified.
@FreeDiskSpacePercentCriticalThreshold	Used to highlight disk drives with low disk space in red, where the free disk space percent is less than the value specified.
@UptimeCritical	The uptime in minutes threshold that causes the uptime to be highlighted in red.
@UptimeWarning	The uptime in minutes threshold that causes the uptime to be highlighted in yellow.
@ErrorLogDays	The number of days worth of events included in the attached error log html file.
@Recipients	The email addresses where the report will be sent.
@MailProfile	The mail profile used to send the email. NULL = default profile.

Database Code

Name	Info
dbo.DBAChecks	This is the stored procedure you run to generate the email report. The stored procedure collates the information from the other stored procedures into a single email report. The parameters are described in the previous section.
dbo.DBAChecks_Backups	Produces HTML for the "Backups" section of the report.
dbo.DBAChecks_DBFiles	Produces HTML for the "Database Files" section of the report.
dbo.DBAChecks_DiskDrives	Produces HTML for the "Disk Drives" section of the report.
dbo.DBAChecks_ErrorLog	Produces HTML for the "ErrorLog.htm" report attachment. Review and ammend the filter applied to the error log as appropriate.
dbo.DBAChecks_FailedAgentJobs	Produces HTML for the "Failed Jobs" section of the report.
dbo.DBAChecks_JobStats	Produces HTML for the "Agent Job Stats" section of the report
dbo.SQLIO_fnGetDrives()	This CLR function is used by the dbo.DBAChecks_DiskDrives stored procedure to produce the report on disk drives and their available free space. This function can be obtained here.

Conclusion

Daily checks are an important part of any DBA’s routine, giving you confidence that your SQL instances are running problem free and allowing you to take corrective actions when required. Alerts are also important, but these should be used to compliment daily checks, rather than to replace them. If you rely on by exception reporting only, your servers will be at risk if something happens to the alert process.

I’m sure there are a wide variety of methods people currently use to perform their daily checks. The process you use is not important as long as it enables you to effectively monitor your SQL Server instances. If you’ve already got a routine that works for you - great! If not, hopefully this article will provide you with a better solution.

SQL Server CLR IO Utility

Sun, 12 Jul 2009 00:00:00 +0000

Warning: the use of TRUSTWORTHY described in this article can be a potential security risk so you might want to consider alternative options.

Introduction

Using CLR (.NET Framework) code in SQL Server is a very powerful way of extending the built-in features and functionality. The .NET framework provides access to a library of useful code, allowing you to accomplish useful tasks with a minimal amount of code. The SQL Server IO project does just that; providing access to some useful IO utilities from within SQL Server by leveraging the power of the .NET Framework.

Installation Instructions

You need to ensure that CLR is enabled for the server and the trustworthy database setting is turned on. The trustworthy database setting is required to allow CLR code to have external access.

To enable CLR, run the following code:

Enable CLR
sp_configure 'clr enabled', 1
GO
RECONFIGURE
GO

To enable the trustworthy setting for the database, run the following code:

SET TRUSTWORTHY ON
ALTER DATABASE MyDatabase SET TRUSTWORTHY ON;

Warning: the use of TRUSTWORTHY can be a potential security risk.

Next, download the “T-SQL Code” and run it in SQL Server Management Studio to install the CLR functions and stored procedures in the current database.

Note: Consider the security implecations before running any of the code in this article.

Code Download

T-SQL Code

Visual Studio Source Code

CLR Functions

SQLIO_fnGetDrives

Returns information about the drives installed on your computer/server. SELECT * FROM dbo.SQLIO_fnGetDrives()

SQLIO_fnGetFiles

Returns a list of files in the specified folder. The function takes 3 parameters:

@path - The folder from which to start the search
@searchPattern - The search string to match against the names of files in path
@includeSubfolders - Specifies whether the search operation should include all subdirectories or only the current directory.

fnGetFiles Example
SELECT * FROM dbo.SQLIO_fnGetFiles('C:\Program Files (x86)\Microsoft SQL Server\MSSQL.4\MSSQL\LOG','',0)

SQLIO_fnGetFolders

Returns a list of subfolders within the specified folder. The function takes 3 parameters:

fnGetFolders
SELECT * FROM dbo.SQLIO_fnGetFolders('C:\Program Files (x86)\Microsoft SQL Server\MSSQL.4\MSSQL','',0)

SQLIO_fnFileExists

Takes a file path as a parameter and returns the bit value “1” if the file exists.

fnFileExists example
SELECT dbo.SQLIO_fnFileExists('C:\test.txt')

SQLIO_fnFolderExists

Takes a folder path as a parameter and returns the bit value “1” if the folder exists.

fnFileExists
SELECT dbo.SQLIO_fnFileExists('C:\MyFolder')

SQLIO_fnReadFile

Takes the path of a file as a parameter and returns the file data in varbinary format.

fnReadFile
SELECT dbo.SQLIO_fnReadFile('C:\Test.txt')

SQLIO_fnReadTextFile

Takes the path of a file as an parameter and returns the file data in nvarchar format.

fnReadTextFile
SELECT dbo.SQLIO_fnReadTextFile('C:\test.txt')

CLR Stored Procedures

SQLIO_DeleteFile

Takes the path of a file as an parameter and deletes the specified file.

DeleteFile
EXEC dbo.SQLIO_DeleteFile 'C:\test.txt'

SQLIO_DeleteFolder

Takes the path of a folder as a parameter and deletes the specified folder.

DeleteFolder
EXEC dbo.SQLIO_DeleteFolder 'C:\MyFolder'

SQLIO_DeleteFiles

Takes the path of a folder, a search pattern and an include subfolders parameter, similar to the dbo.SQLIO_fnGetFiles function and deletes all the files matched from the search.

DeleteFiles
EXEC dbo.SQLIO_DeleteFiles 'C:\Temp','*.tmp',1

SQLIO_DeleteFilesOlderThan

This stored procedure is the same as the dbo.SQLIO_DeleteFiles procedure, but with an additional maxAge date parameter. Only files older than the maxAge date specified will be deleted.

DeleteFilesOlderThan
EXEC dbo.SQLIO_DeleteFilesOlderThan 'C:\Temp','*.tmp',1,'20090101'

SQLIO_CreateFolder

Takes a path as a parameter and creates the new folder on the file system.

CreateFolder
EXEC dbo.SQLIO_CreateFolder 'C:\Temp\New Folder'

SQLIO_CreateFile

Creates a new file, by taking a file path and varbinary data value.

CreateFile
EXEC dbo.SQLIO_CreateFile 'C:\Temp\NewFile.txt',0x68007400740070003A002F002F007700770077002E00770069007300650073006F00660074002E0063006F002E0075006B00

SQLIO_CreateTextFile

Creates a new text file, by specifying a file path and string value.

CreateTextFile
EXEC dbo.SQLIO_CreateTextFile 'C:\Temp\NewFile.txt','Text Here'

SQLIO_AppendTextFile

Appends text to the end of a file, by specifying a file path and text to append.

AppendTextFile
EXEC dbo.SQLIO_AppendTextFile 'C:\Temp\myfile.txt','My Appended Text'

SQLIO_MoveFile

Takes source and destination parameters, moving a file from one location to the other.

MoveFile
EXEC dbo.SQLIO_MoveFile 'C:\Old Folder\MyFile.txt','C:\New Folder\MyFile.txt'

SQLIO_CopyFile

This stored procedure takes source and destination parameters, copying a file from one location to the other.

CopyFile
EXEC dbo.SQLIO_CopyFile 'C:\temp\MyFile.txt','C:\temp\Copy Of MyFile.txt'

SQLIO_MoveFolder

Takes source and destination parameters, copying a folder from one location to the other.

MoveFolder
EXEC dbo.SQLIO_MoveFolder 'C:\Old Folder\MyFolder','C:\New Folder\MyFolder

SQL Server Database Restore Utility

Thu, 11 Jun 2009 00:00:00 +0000

This script is a simple utility to allow users to restore SQL Server databases. It’s easier to use for non-DBAs and it provides a method of restoring databases when the management tools are not installed. It’s a good demonstration of a HTA application, but it’s not a intended as a replacement for SQL Server Management Studio. You can only use this utility for simple restore scenarios.A simple utility that allows you to restore SQL Server databases from a backup file.

SQL Database Restore HTA Script
<html><head>
<STYLE TYPE="text/css">
	BODY {font-family: "Verdana, Arial, Helvetica, sans-serif";
		font-size:x-small;
		background-color:#dfe0e5;
	}
	
	table {font-family: "Verdana, Arial, Helvetica, sans-serif";
		font-size:x-small;}

	#RestoreCompleted {
		font-weight:bold;
		color:green;
	}

	#RestoreInProgress {
		font-weight:bold;
		color:blue;
	}

	#RestoreError {
		font-weight:bold;
		color:red;
	}
	#Footer {
	font-weight:bold;
	font-size:10px;
}

</STYLE>

<script language="vbscript" OnLoad="Main()">
	option explicit
	Const HKEY_LOCAL_MACHINE = &H80000002
	Const adVarChar = 200
	Const adParamInput = 1 
	Const intAppWidth = 600
	Const intAppHeight = 280 ' 255

	' Set app size and centre screen on load
	Sub Window_Onload()
		Dim objWMIService,objItem,intHorizontal,intVertical
		Dim intLeft, intTop
 	 	Set objWMIService = GetObject("winmgmts:\\.\root\cimv2")
 
 		For Each objItem in objWMIService.ExecQuery("Select * From Win32_DesktopMonitor")
 		intHorizontal = objItem.ScreenWidth
 		intVertical = objItem.ScreenHeight
 	Next
 	If IsNull(intHorizontal) = False And IsNull(intVertical) = False Then
 	 intLeft = (intHorizontal - intAppWidth) / 2
		 	intTop = (intVertical - intAppHeight) / 2
		 	 	window.moveTo intLeft, intTop
 	 	End If
 	 	window.resizeTo intAppWidth,intAppHeight
 	

		LoadSQLServerInstances()
	End Sub
	
	' Populate list box with local SQL Server instances
	Sub LoadSQLServerInstances()
		Dim strComputer,strKeyPath,strValueName,strValue
		Dim oReg,arrValues, d
		
		strComputer = "."

		Set oReg=GetObject("winmgmts:{impersonationLevel=impersonate}!\\" & _ 
		 strComputer & "\root\default:StdRegProv")
		 
		strKeyPath = "SOFTWARE\Microsoft\Microsoft SQL Server\"
		strValueName = "InstalledInstances"
		oReg.GetMultiStringValue HKEY_LOCAL_MACHINE,strKeyPath, _
		 strValueName,arrValues
		If ISArray(arrValues) = false then
			msgbox "No local SQL Server instances found",vbCritical
			Close()
		end If	
		set d = createobject("Scripting.Dictionary")
		For Each strValue In arrValues
			Dim objOption
		 Set objOption = Document.createElement("OPTION")
			if strValue = "LOCAL" OR strValue="MSSQLSERVER" then
				strValue = "(local)"
			else
				strValue = ".\" & strValue
			end if
			objOption.Text = strValue
			objOption.Value = strValue
			
			if d.Exists(strValue) = false then
				if selInstances.Contains(objOption) = false then
					selInstances.Add(objOption)
				end if
				d.Add strValue,strValue
			end if
		Next
	End Sub
	
	' Prompt user to select backup file
	Sub SelectBackupFile()
		Dim oDLG
		Set oDLG=CreateObject("MSComDlg.CommonDialog") 
		With oDLG
			.DialogTitle="Open"
			.Filter="Backup Files|*.bak|All files|*.*"
			.MaxFileSize=255
			.Flags=.Flags Or &H1000	'FileMustExist (OFN_FILEMUSTEXIST)
			.ShowOpen
			If .FileName<>"" Then
				txtFileName.Value =.FileName
			End If
		End With
		Set oDLG=Nothing
	End Sub
	
	' Prompt user for location of database restore
	Sub SelectRestoreFolder()
		Const MY_COMPUTER = &H11&
		Const WINDOW_HANDLE = 0
		Const OPTIONS = 0
		Dim objShell,objFolder,objFolderItem,strPath

		Set objShell = CreateObject("Shell.Application")
		Set objFolder = objShell.Namespace(MY_COMPUTER)
		Set objFolderItem = objFolder.Self
		strPath = objFolderItem.Path

		Set objShell = CreateObject("Shell.Application")
		Set objFolder = objShell.BrowseForFolder _
				(WINDOW_HANDLE, "Select a folder:", OPTIONS, strPath) 
			 
		If not objFolder Is Nothing Then
			Set objFolderItem = objFolder.Self
			txtRestoreFolder.Value = objFolderItem.Path
		End If

	End Sub
	
	' Show/Hide username/password textbox depending on integrated security option
	Sub ShowAuth()
		if chkIntegrated.Checked = true then
			trUserName.Style.Display = "none"
			trPassword.Style.Display = "none"
			window.resizeTo intAppWidth,intAppHeight
		else
			trUserName.Style.Display = "block"
			trPassword.Style.Display = "block"
			window.resizeTo intAppWidth,intAppHeight + 50
		end if
	End Sub
	
	' Generate connection string for the local database
	Private Function GetConnectionString
		Dim strInstance,strUserName,strPassword
		strInstance = selInstances.Value
		strUserName = txtUserName.Value
		strPassword = txtPassword.Value
		if chkIntegrated.Checked = true then
			GetConnectionString = "Provider=SQLOLEDB.1;Data Source=" & strInstance & ";Integrated Security=SSPI;"
		else
			GetConnectionString = "Provider=SQLOLEDB.1;Data Source=" & strInstance & ";UID=" & strUserName & ";PWD=" & strPassword
		end if
	End Function

	' Check if the specified database name already exists
	Function DatabaseExists(byval dbName)
		
		Dim strSQL,objCn,objCmd,strCn, objRs,pName
		strCn = GetConnectionString()
		strSQL = "SELECT COUNT(*) FROM sysdatabases where name = ?"
		set objCn = CREATEOBJECT("ADODB.Connection")
		set objCmd = CREATEOBJECT("ADODB.Command")
		set pName = objCmd.CreateParameter("@name",advarchar,adParamInput,128,dbname)

		objCmd.Parameters.Append pName
		objCn.Open strCn
		objCmd.ActiveConnection = objCn
		objCmd.CommandText = strSQL
		set objRs = objCmd.Execute
		if objRs(0) = 0 then
			DatabaseExists = false
		else
			DatabaseExists = true
		end if
		objRs.Close
		objCn.Close
	end function
	
	' Perform validations before the restore
	Private Function Validate()
		dim objFSO, strCn, objCn,dbName
		set objFSO=createobject("Scripting.FileSystemObject")
		
		' Check backup file exists
		if NOT objFSO.FileExists(txtFileName.Value) then
			RestoreError.InnerHTML = "File specified does not exist"
			Validate = false
			exit function
		end if
		' Check restore folder exists
		if not objFSO.FolderExists(txtRestoreFolder.Value) then
			RestoreError.InnerHTML = "Folder specified does not exist"
			Validate = false
			exit function
		end if
		err.clear
		on error resume next
		' Check connection to database
		strCn = GetConnectionString
		set objCn = CREATEOBJECT("ADODB.Connection")
		objCn.Open strCn
		objCn.Close
		if err.number <> 0 then
			RestoreError.InnerHTML= "Connection error:" & err.number
			Validate = false
			exit function
		end if
		' Get database name from backup file.
		' Just a quick check that backup file is readable
		GetDatabaseNameFromBackupFile()
		if err.number <> 0 then
			RestoreError.InnerHTML = "Error:" & err.number & " - " & err.description
			Validate = false
			exit function
		end if
		on error goto 0
		' Get database name. 
		dbName = GetDatabaseName()
		' Check if database with same name already exists
		if DatabaseExists(dbName) then
			RestoreError.InnerHTML = "Error: Database '" & dbname & "' already exists"
			Validate = false
			exit function	
		end if
	
		Validate = True
	End Function
	
	' Return database name for restore (either from backup file or user specified name)
	Private Function GetDatabaseName
		if chkDBNameDefault.Checked = True then
			GetDatabaseName = GetDatabaseNameFromBackupFile()
		Else
			GetDatabaseName = txtDBName.Value
		end if
	End Function

	' Retreive database name from backup file
	Private Function GetDatabaseNameFromBackupFile
		Dim strSQL,objCn,objCmd,strCn, objRs,pRestoreFile
		strCn = GetConnectionString()
		strSQL = "RESTORE HEADERONLY FROM DISK = ?"
		set objCn = CREATEOBJECT("ADODB.Connection")
		set objCmd = CREATEOBJECT("ADODB.Command")
		set pRestoreFile = objCmd.CreateParameter("@RestoreFile",advarchar,adParamInput,255,txtFileName.Value)

		objCmd.Parameters.Append pRestoreFile
		objCn.Open strCn
		objCmd.ActiveConnection = objCn
		objCmd.CommandText = strSQL
		set objRs = objCmd.Execute
		GetDatabaseNameFromBackupFile = objRs("DatabaseName")
		objCn.Close
	end function

	' Similar to SQL Server QUOTENAME function
	' Wraps quotes around string and performs escapes
	Private Function QuoteName(byval Value, quoteChar)
		if quotechar = "[" or quotechar = "]" then
			QuoteName = "[" & REPLACE(Value,"]","]]") & "]"
		elseif quoteChar = "'" then
			QuoteName = "'" & REPLACE(Value,"'","''") & "'"
		else
			QuoteName = ""
		end if
	End Function
	
	' Function to get the SQL statement used to perform the DB restore
	Private Function GetRestoreSQL
		Dim strSQL,objCn,objCmd,strCn, objRs,pRestoreFile
		Dim dbName
		dbName = GetDatabaseName()
		strCn = GetConnectionString()
		strSQL = "RESTORE FILELISTONLY FROM DISK = ?"
		set objCn = CREATEOBJECT("ADODB.Connection")
		set objCmd = CREATEOBJECT("ADODB.Command")
		set pRestoreFile = objCmd.CreateParameter("@RestoreFile",advarchar,adParamInput,255,txtFileName.Value)

		objCmd.Parameters.Append pRestoreFile
		objCn.Open strCn
		objCmd.ActiveConnection = objCn
		objCmd.CommandText = strSQL
		on error resume next
		set objRs = objCmd.Execute
		if err.number <> 0 then
			msgbox(err.description & err.number)
			exit function
		end if
		
		Dim strFolder
		strFolder = txtRestoreFolder.Value
		if not RIGHT(strFolder,1) = "\" then
			strFolder = strFolder & "\"
		end if
		
		Dim strRestore,strPath
		while objRs.EOF <> True AND objRs.BOF <> True
			strPath = strFolder + dbname & "_" & objRs("LogicalName")
			if objRs("Type") = "L" then 'LOG
				strPath = strPath & ".ldf"
			elseif objRs("Type") = "S" then 'FILESTREAM
				' Folder don't add extension
			else 'DATA
				if objRs("FileID") = "1" then 'Primary File
					strPath = strPath & ".mdf"
				else
					strPath = strPath & ".ndf"
				end if
			end if
			if LEN(strRestore)> 0 then
				strRestore = strRestore & ", " 
			end if
			strRestore = strRestore & "MOVE N" & QuoteName(objRs("LogicalName"),"'") & " TO " & QuoteName(strPath,"'")
			objRs.MoveNext
		wend
		objCn.Close
		
		strRestore = "RESTORE DATABASE " & QuoteName(dbName,"]") & " FROM DISK=N" & QuoteName(txtFileName.Value,"'") & " WITH FILE=1," & strRestore
		
		GetRestoreSQL = strRestore
	End Function
	
	' Perform database restore
	Sub RestoreDatabase()
		Dim strSQL,strCn, objCn, objCmd

		' Clear any existing error/completed notifications
		RestoreError.InnerHTML = ""
		RestoreCompleted.Style.Display = "none"

		' Perform validation checks prior to restore
		if Validate() = false then
			exit sub
		end if
		' Set restore notification
		RestoreInProgress.Style.Display = "block"
		UpdateScreen 

		' Get SQL for Restore
		strSQL = GetRestoreSQL()
		strCn = GetConnectionString()
	
		set objCn = CREATEOBJECT("ADODB.Connection")
		set objCmd = CREATEOBJECT("ADODB.Command")
		objCn.Open strCn
		objCmd.ActiveConnection = objCn
		objCmd.CommandText = strSQL
		objCmd.CommandTimeout = 0
		on error resume next
		objCmd.Execute
		objCn.Close
		if err.number <> 0 then
			RestoreError.InnerHTML = "Error: " & err.Number & " - " & err.description
		else
			RestoreCompleted.Style.Display = "block"
		end if

		RestoreInProgress.Style.Display = "none"
		
	End Sub

	' Causes a refresh
	Sub UpdateScreen() 
		' this is used to allow screen updates 
		dim CMD, WSHShell 
	
		Set WSHShell = CreateObject("WScript.Shell") 
		CMD = WSHShell.ExpandEnvironmentStrings("%comspec%") 
		WSHShell.run CMD + " /c ", 0, true 
		Set WSHShell = Nothing 
	End Sub 

	' Enables/Disables database name textbox 
	Sub DefaultOptionChanged()
		if chkDBNameDefault.Checked = True then
			txtDBName.Disabled = "Disabled"
			txtDBName.style.backgroundcolor = "#dfe0e5"
		else
			txtDBName.Disabled = ""
			txtDBName.style.backgroundcolor = "#ffffff"
		end if
	End Sub
	
</script>
</head>
<HTA:APPLICATION
	APPLICATIONNAME="Restore Database Utility" ID="RestoreDBHTA" INNERBORDER="no" >
<body leftmargin=30 topmargin=20 rightmargin=30>
<table>
<tr>
<td>Backup File:</td><td><input id="txtFileName" style="width=300px;" type="Text" Value=""/><input type="submit" value="Open" OnClick="SelectBackupFile()" /></td>
</tr>
<tr>
<td>Restore Folder:</td><td><input id="txtRestoreFolder" style="width=300px;" type="Text" Value="" /><input type="submit" value="Open" OnClick="SelectRestoreFolder()" /></td>
</tr>
<tr>
<td>SQL Server Instance:</td>
<td>
<select name="selInstances">
</select>
<input id="chkIntegrated" type="checkbox" checked="true" OnClick="ShowAuth()">Integrated Security</input>
</td>
<tr id="trUserName" style="display:none">
<td>UserName:</td><td><input id="txtUserName" style="width=200px;" type="Text" /></td>
</tr>
<tr id="trPassword" style="display:none">
<td>Password</td><td><input id="txtPassword" style="width=200px;" type="password" /></td>
</tr>
<tr>
<td>Database Name:</td><td><input id="txtDBName" style="width=200px;background-color:#dfe0e5" disabled="disabled" type="Text" /><input id="chkDBNameDefault" type="checkbox" checked="true" OnClick="DefaultOptionChanged">Use Default</input></td>
</tr>
</table>
<br/>
<input type="submit" value="Restore Database" OnClick="RestoreDatabase()" />&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<input type="submit" value="Close" OnClick="Close()" />
<br/>
<span id="RestoreInProgress" style="display:none">Database is been restored - this might take a while depending on the size of the database. Please wait...</span>
<span id="RestoreCompleted" style="display:none">Restore Completed.</span>
<span id="RestoreError"></span>
<div id="Footer">
<hr/>
 <div style="float:left;">Version 1.0</div>
 <div style="float:right;">By David Wiseman<br />
 </div>
 <div style="clear:both;"></div>
 <div style="text-align:center;font-size:12px;font-weight:bold"><a href="http://www.wisesoft.co.uk">www.wisesoft.co.uk</a></div>
</div>

</body></html>

Backup/Restore Progress script

Fri, 12 Sep 2008 00:00:00 +0000

Show running time and estimated time to go for SQL backup and restore operations.

Backup/Restore Progress script


 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14


SELECT command,
 s.text,
 start_time,
 percent_complete,
 CAST(((DATEDIFF(s,start_time,GetDate()))/3600) as varchar) + ' hour(s), '
 + CAST((DATEDIFF(s,start_time,GetDate())%3600)/60 as varchar) + 'min, '
 + CAST((DATEDIFF(s,start_time,GetDate())%60) as varchar) + ' sec' as running_time,
 CAST((estimated_completion_time/3600000) as varchar) + ' hour(s), '
 + CAST((estimated_completion_time %3600000)/60000 as varchar) + 'min, '
 + CAST((estimated_completion_time %60000)/1000 as varchar) + ' sec' as est_time_to_go,
 dateadd(second,estimated_completion_time/1000, getdate()) as est_completion_time
FROM sys.dm_exec_requests r
CROSS APPLY sys.dm_exec_sql_text(r.sql_handle) s
WHERE r.command in ('RESTORE DATABASE', 'BACKUP DATABASE', 'RESTORE LOG', 'BACKUP LOG')

T-SQL Proper Case Function

Sun, 30 Sep 2007 09:00:00 +0000

This function converts a string to proper case.

e.g.
Convert this: A LONG TIME AGO IN A GALAXY FAR, FAR AWAY
To this: A Long Time Ago In A Galaxy Far, Far Away

Note: It might be better to do this in the application rather than the database and it could be better written as a CLR function. This is useful if you need a T-SQL solution for a one off purpose.

Proper Case T-SQL Function


 1
 2
 3
 4
 5
 6
 7
 8
 9
 10
 11
 12
 13
 14
 15
 16
 17
 18
 19
 20
 21
 22
 23
 24
 25
 26
 27
 28
 29
 30
 31
 32
 33
 34
 35
 36
 37
 38
 39
 40
 41
 42
 43
 44
 45
 46
 47
 48
 49
 50
 51
 52
 53
 54
 55
 56
 57
 58
 59
 60
 61
 62
 63
 64
 65
 66
 67
 68
 69
 70
 71
 72
 73
 74
 75
 76
 77
 78
 79
 80
 81
 82
 83
 84
 85
 86
 87
 88
 89
 90
 91
 92
 93
 94
 95
 96
 97
 98
 99
100
101
102
103
104
105
106
107
108
109
110


CREATE FUNCTION [dbo].[fProperCase](@Value varchar(8000), @Exceptions varchar(8000),@UCASEWordLength tinyint)
returns varchar(8000)
as
/* ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Function Purpose: To convert text to Proper Case.
Created By: David Wiseman
Website: wisedataman.com
Created: 2005-10-03
Updated: 2006-06-22
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
INPUTS:
 
@Value : This is the text to be converted to Proper Case
@Exceptions: A list of exceptions to the default Proper Case rules. e.g. |RAM|CPU|HDD|TFT|
 Without exception list they would display as Ram, Cpu, Hdd and Tft
 Note the use of the Pipe "|" symbol to separate exceptions.
 (You can change the @sep variable to something else if you prefer)
@UCASEWordLength: You can specify that words less than a certain length are automatically displayed in UPPERCASE
 
USAGE1:
 
Convert text to ProperCase, without any exceptions
 
select dbo.fProperCase('THIS FUNCTION WAS CREATED BY DAVID WISEMAN',null,null)
>> This Function Was Created By David Wiseman
 
USAGE2:
 
Convert text to Proper Case, with exception for WiseSoft
 
select dbo.fProperCase('THIS FUNCTION WAS CREATED BY DAVID WISEMAN @ WISESOFT','|WiseSoft|',null)
>> This Function Was Created By David Wiseman @ WiseSoft
 
USAGE3:
 
Convert text to Proper Case and default words less than 3 chars to UPPERCASE
 
select dbo.fProperCase('SIMPSON, HJ',null,3)
>> Simpson, HJ
 
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ */
begin
 declare @sep char(1) -- Seperator character for exceptions
 declare @i int -- counter
 declare @ProperCaseText varchar(5000) -- Used to build our Proper Case string for Function return
 declare @Word varchar(1000) -- Temporary storage for each word
 declare @IsWhiteSpace as bit -- Used to indicate whitespace character/start of new word
 declare @c char(1) -- Temp storage location for each character
 
 set @Word = ''
 set @i = 1
 set @IsWhiteSpace = 1
 set @ProperCaseText = ''
 set @sep = '|'
 
 -- Set default UPPERCASEWord Length
 if @UCASEWordLength is null set @UCASEWordLength = 1
 -- Convert user input to lower case (This function will UPPERCASE words as required)
 set @Value = LOWER(@Value)
 
 -- Loop while counter is less than text lenth (for each character in...)
 while (@i <= len(@Value)+1)
 begin
 
 -- Get the current character
 set @c = SUBSTRING(@Value,@i,1)
 
 -- If start of new word, UPPERCASE character
 if @IsWhiteSpace = 1 set @c = UPPER(@c)
 
 -- Check if character is white space/symbol (using ascii values)
 set @IsWhiteSpace = case when (ASCII(@c) between 48 and 58) then 0
 when (ASCII(@c) between 64 and 90) then 0
 when (ASCII(@c) between 96 and 123) then 0
 else 1 end
 
 if @IsWhiteSpace = 0
 begin
 -- Append character to temp @Word variable if not whitespace
 set @Word = @Word + @c
 end
 else
 begin
 -- Character is white space/punctuation/symbol which marks the end of our current word.
 -- If word length is less than or equal to the UPPERCASE word length, convert to upper case.
 -- e.g. you can specify a @UCASEWordLength of 3 to automatically UPPERCASE all 3 letter words.
 set @Word = case when len(@Word) <= @UCASEWordLength then UPPER(@Word) else @Word end
 
 -- Check word against user exceptions list. If exception is found, use the case specified in the exception.
 -- e.g. WiseSoft, RAM, CPU.
 -- If word isn't in user exceptions list, check for "known" exceptions.
 set @Word = case when charindex(@sep + @Word + @sep,@exceptions collate Latin1_General_CI_AS) > 0
 then substring(@exceptions,charindex(@sep + @Word + @sep,@exceptions collate Latin1_General_CI_AS)+1,len(@Word))
 when @Word = 's' and substring(@Value,@i-2,1) = '''' then 's' -- e.g. Who's
 when @Word = 't' and substring(@Value,@i-2,1) = '''' then 't' -- e.g. Don't
 when @Word = 'm' and substring(@Value,@i-2,1) = '''' then 'm' -- e.g. I'm
 when @Word = 'll' and substring(@Value,@i-3,1) = '''' then 'll' -- e.g. He'll
 when @Word = 've' and substring(@Value,@i-3,1) = '''' then 've' -- e.g. Could've
 else @Word end
 
 -- Append the word to the @ProperCaseText along with the whitespace character
 set @ProperCaseText = @ProperCaseText + @Word + @c
 -- Reset the Temp @Word variable, ready for a new word
 set @Word = ''
 end
 -- Increment the counter
 set @i = @i + 1
 end
 return @ProperCaseText
end