Sql server performance tuning

By:
Jugal Shah
Jugal.shah@sqldbpool.com
https://sqldbpool.com

I have 14 plus years of IT experience and I have worked on different kinds of
roles like Manager, Solution Architect, DBA, Developer and Software engineer.
I have worked on many database technologies which includes
SQL Server, MySQL, PostgreSQL, AWS Redshift, AWS Aurora, MariaDB, AWS RDS
and cloud technologies.
I received the MVP award in SQL Server (2010, 2011, 2012 and 2013) by
Microsoft and trend setter award from MSSQLTips.com

Most common cause of performance problems
 SQL Server/Database Configuration Issues
 Database/table/schema Design Issues
 Wait Statistics
 CPU/IO/Memory Bottleneck
 Blocking Bottleneck
 Network Bottleneck
 Wait Statistics
 Poor Indexing Strategy
 Out-of-date/missing statistics
 Application Code
 T-SQL Code

 DMVs: Dynamic management views and functions return server state information
that can be used to monitor the health of a server instance, diagnose problems, and
tune performance.
 Performance Monitor: This tool is available as part of the Windows®
operating system.
 SQL Server Profiler: See SQL Server Profiler in the Performance Tools group
in the SQL Server 2008 program group.
 DBCC commands: DBCC statements that act as Database Console Commands
for SQL Server
 SSMS: SQL Server Management Studio

 Separate AD Service account with the least privilege for SQL Server Services
 Configure antivirus software to skip LDF/MDF/NDF files
 Configure the minimum, max memory and lock pages in memory local policy
 Tweak the model database to setup best practice across all the databases
 TempDB configuration
 Configure AWE and PAE for 32-bit system
 Set the Page File in C: 2GB
 Configure the default SQL Server Settings

 Normalization
 Use the Constraints while DB Design
 Auto Update/Create Statistics
 Auto Shrink should be turned off
 Set the Auto Growth option
 Files and File groups
 Database Maintenance
 Excessive Sorting Operations
 Excessive RID Lookups performed on heap tables
 Key lookups against the clustering keys look like joins however they are
marked as “lookups” only in the XML showplan.

One of the first items that I check, when troubleshooting performance problems on a SQL
Server, is the wait statistics, which are tracked by the SQLOS during normal operations of any
SQL Server
SELECT TOP 10
wait_type ,
max_wait_time_ms wait_time_ms ,
signal_wait_time_ms ,
wait_time_ms - signal_wait_time_ms AS resource_wait_time_ms ,
100.0 * wait_time_ms / SUM(wait_time_ms) OVER ( )
AS percent_total_waits ,
100.0 * signal_wait_time_ms / SUM(signal_wait_time_ms) OVER ( )
AS percent_total_signal_waits ,
100.0 * ( wait_time_ms - signal_wait_time_ms )
/ SUM(wait_time_ms) OVER ( ) AS percent_total_resource_waits
FROM sys.dm_os_wait_stats
WHERE wait_time_ms > 0 -- remove zero wait_time
AND wait_type NOT IN -- filter out additional irrelevant waits
( 'SLEEP_TASK', 'BROKER_TASK_STOP', 'BROKER_TO_FLUSH',
'SQLTRACE_BUFFER_FLUSH','CLR_AUTO_EVENT', 'CLR_MANUAL_EVENT',
'LAZYWRITER_SLEEP', 'SLEEP_SYSTEMTASK', 'SLEEP_BPOOL_FLUSH',
'BROKER_EVENTHANDLER', 'XE_DISPATCHER_WAIT', 'FT_IFTSHC_MUTEX',
'CHECKPOINT_QUEUE', 'FT_IFTS_SCHEDULER_IDLE_WAIT',
'BROKER_TRANSMITTER', 'FT_IFTSHC_MUTEX', 'KSOURCE_WAKEUP',
'LAZYWRITER_SLEEP', 'LOGMGR_QUEUE', 'ONDEMAND_TASK_QUEUE',
'REQUEST_FOR_DEADLOCK_SEARCH', 'XE_TIMER_EVENT', 'BAD_PAGE_PROCESS',
'DBMIRROR_EVENTS_QUEUE', 'BROKER_RECEIVE_WAITFOR',
'PREEMPTIVE_OS_GETPROCADDRESS', 'PREEMPTIVE_OS_AUTHENTICATIONOPS',
'WAITFOR', 'DISPATCHER_QUEUE_SEMAPHORE', 'XE_DISPATCHER_JOIN',
'RESOURCE_QUEUE' )
ORDER BY wait_time_ms DESC

CXPACKET :Often indicates nothing more than that certain queries are executing with parallelism;
CXPACKET waits in the server are not an immediate sign of problems, although they may be the
symptom of another problem, associated with one of the other high value wait types in the instance.
SOS_SCHEDULER_YIELD :The tasks executing in the system are yielding the scheduler, having
exceeded their quantum, and are having to wait in the runnable queue for other tasks to execute. This
may indicate that the server is under CPU pressure.
THREADPOOL :A task had to wait to have a worker bound to it, in order to execute. This could be a
sign of worker thread starvation, requiring an increase in the number of CPUs in the server, to handle a
highly concurrent workload, or it can be a sign of blocking, resulting in a large number of parallel tasks
consuming the worker threads for long periods.
LCK_* :These wait types signify that blocking is occurring in the system and that sessions have had to
wait to acquire a lock of a specific type, which was being held by another database session. This
problem can be investigated further using, for example, the information in the
sys.dm_db_index_operational_stats.
PAGEIOLATCH_*, IO_COMPLETION, WRITELOG :These waits are commonly associated with disk
I/O bottlenecks, though the root cause of the problem may be, and commonly is, a poorly performing
query that is consuming excessive amounts of memory in the server. PAGEIOLATCH_* waits are
specifically associated with delays in being able to read or write data from the database files.
WRITELOG waits are related to issues with writing to log files. These waits should be evaluated in
conjunction with the virtual file statistics as well as Physical Disk performance counters, to determine if
the problem is specific to a single database, file, or disk, or is instance wide.

PAGELATCH_* :Non-I/O waits for latches on data pages in the buffer pool. A lot of times
PAGELATCH_* waits are associated with allocation contention issues. One of the best-known
allocations issues associated with PAGELATCH_* waits occurs in tempdb when the a large number
of objects are being created and destroyed in tempdb and the system experiences contention on
the Shared Global Allocation Map (SGAM), Global Allocation Map (GAM), and Page Free Space
(PFS) pages in the tempdb database.
LATCH_* :These waits are associated with lightweight short-term synchronization objects that
are used to protect access to internal caches, but not the buffer cache. These waits can indicate a
range of problems, depending on the latch type. Determining the specific latch class that has the
most accumulated wait time associated with it can be found by querying the
sys.dm_os_latch_stats DMV.
ASYNC_NETWORK_IO :This wait is often incorrectly attributed to a network bottleneck. In fact,
the most common cause of this wait is a client application that is performing row-by-row
processing of the data being streamed from SQL Server as a result set (client accepts one row,
processes, accepts next row, and so on). Correcting this wait type generally requires changing
the client-side code so that it reads the result set as fast as possible, and then performs
processing.

 A CPU bottleneck can be caused by hardware resources that are insufficient for the
load. However, excessive CPU utilization can commonly be reduced by query tuning
(especially if there was a sudden increase without additional load or different queries
on the server), addressing any application design factors, and optimizing the system
configuration.
 Investigate when:
◦ Processor:% Processor Time for each CPU consistently above 80%
 Look to see if:
◦ Process:% Processor Time for sqlservr is high
◦ Value of runnable_tasks_count is non-zero
SELECT scheduler_id, current_tasks_count, runnable_tasks_count
FROM sys.dm_os_schedulers
WHERE scheduler_id < 255
 What to do next?
◦ These values only indicate that your server is CPU bound.
◦ However, first, find the largest consumers of CPU bandwidth and fine tune them.
◦ Only consider buying additional CPU power, if the these consumers cannot be fine tuned.

◦ Inefficient Query Execution Plan
◦ Excessive Compilations and Recompilations
◦ Intra-Query Parallelism
◦ Inefficient Use of Server Side Cursors
◦ User-Defined Functions
Background
 SQL Server optimizer attempts to find a plan with best
response time based on internal costing algorithms
 However, due to several factors (missing indexes, out of
date statistics), it might end up using a sub-optimal
execution plan.
 Certain elements of such sub-optimal execution plan (Hash,
Sort etc.) could drive CPU high.

 Using DMVs
◦ sys.dm_exec_query_stats
◦ Run the following query to get the TOP 25 completed
queries that are consuming the most cumulative CPU
SELECT highest_cpu_queries.plan_handle,
(highest_cpu_queries.total_worker_time/highest_cpu_queries.execution_count) AS
AverageCPU, highest_cpu_queries.execution_count,
highest_cpu_queries.total_worker_time, highest_cpu_queries.plan_generation_num,
highest_cpu_queries.creation_time, highest_cpu_queries.last_execution_time,
highest_cpu_queries.last_physical_reads, highest_cpu_queries.min_physical_reads,
q.dbid, q.objectid, q.number, q.encrypted, q.[text]
FROM (SELECT TOP 25 qs.plan_handle, qs.total_worker_time, qs.last_execution_time,
qs.plan_generation_num, qs.creation_time, qs.execution_count, qs.last_physical_reads,
qs.min_physical_reads FROM sys.dm_exec_query_stats qs
ORDER BY qs.total_worker_time DESC) AS highest_cpu_queries
CROSS APPLY sys.dm_exec_sql_text(plan_handle) AS q
ORDER BY AverageCPU DESC

◦ Plan_Handle
◦ Total_worker_time
◦ Plan_generation_number
◦ Creation_time
◦ Last_execution_time
◦ Execution_count
◦ Last_physical_reads
◦ Min_physical reads
Run each query identified in Management Studio
 To get the actual execution plan with SET STATISTICS PROFILE ON or
SET STATISTICS XML ON
 Use sys.dm_exec_query_plan to find out the execution plan and
missing indexes
 Compare the actual vs (estimate rows x estimated executions). A huge
difference of the order of 10x could indicate inaccurate cardinality
estimates
 Look at the execution plan for any inefficient operators – index scans vs
seeks, hash or sort operators, table spools, etc.
What to do with the Output?

◦ UPDATE STATISTICS
If cardinality estimates are off, update the statistics.
◦ Check for missing Indexes
Use Database Engine Tuning Advisor to find out any missing indexes that could
enhance query performance
◦ Modify Query
Try to modify the query to make better use (restrictive WHERE clause etc.)
◦ Use new features in SQL 2005 to force execution plan
You can use the Plan Guides in SQL 2005 to force a particular execution plan
for a query. Refer to the BOL section on Understanding Plan Guides for more
details on how to use plan guides, the associated overheads and the scenarios
in which they are most suited.

 Background
◦ Query optimizer picks the plan that gives the fastest
response time based on internal SQL Server costing
algorithms.
◦ If the query’s estimated cost is > cost threshold for
parallelism option (default is 5), optimizer may use
parallelism in the query plan.
◦ In most scenarios, parallelism should enhance query
performance.
◦ However, the response time for a given query must be
weighed against the overall throughput and
responsiveness of the rest of the queries on the system.

 Using DMVs
◦ Run this query to determine all active queries
running in parallel
SELECT r.session_id, r.request_id, r.start_time, r.status, r.command, r.database_id,
r.user_id, r.blocking_session_id, r.wait_type, r.wait_time, r.last_wait_type,
r.wait_resource, max(isnull(exec_context_id, 0)) as number_of_workers, r.sql_handle,
r.statement_start_offset, r.statement_end_offset, r.plan_handle
FROM sys.dm_exec_requests r JOIN sys.dm_os_tasks t
ON r.session_id = t.session_id
JOIN sys.dm_exec_sessions s ON r.session_id = s.session_id
WHERE s.is_user_process = 0x1
GROUP BY r.session_id, r.request_id, r.sql_handle, r.plan_handle, r.statement_start_offset,
r.statement_end_offset, r.start_time, r.status, r.command, r.database_id, r.user_id,
r.blocking_session_id, r.wait_type, r.wait_time, r.last_wait_type, r.wait_resource
HAVING max(isnull(exec_context_id, 0)) > 0

 Looking at the output
◦ Session_id
◦ Start_time
◦ Status
◦ Database_id
◦ User_id
◦ Blocking_session_id, wait_type, wait_time,
last_wait_type, wait_resource
 Use sys.dm_exec_query_plan DMF to
find the query text

 Common Resolutions
◦ MAXDOP = 1
For the queries identified as running in parallel, use MAXDOP = 1 to see if there is
any improvement.
◦ Cardinality Estimates
Compare the differences in the actual versus the estimated cardinality. If they are
significant, then update statistics on underlying tables to see if there is an
improvement in performance
◦ Analyze missing indexes
Use Database Engine Tuning Advisor to check for any missing indexes which could
improve performance.
◦ Modify the query
Investigate the possibilities of re-writing the query to take advantage of other T-
SQL constructs

Background
Recompilations - before SQL executes a query, it checks for the
validity and correctness of the query plan. If one of these checks
fails, the batch may have to be compiled again to produce a
different query plan.
However, in SQL 2005/2008, only the statement that caused the
recompile is compiled as opposed to the entire SP (as in SQL
2000)
Recompilations are CPU intensive and a lot of them can drive CPU
high.

◦ Using DMVs
 Run the following query to find out the top 25 SPs
that have been recompiled the most
 The text column will give you the exact text of the SP
that has been recompiled.
SELECT TOP 25 sql_text.text, sql_handle, plan_generation_num,
last_execution_time, execution_count, dbid, objectid
FROM sys.dm_exec_query_stats a
CROSS APPLY sys.dm_exec_sql_text(sql_handle) as sql_text
WHERE plan_generation_num >1
ORDER BY plan_generation_num DESC

◦ Recompilations due to SET
Using profiler, you will be able to find out the statement that caused the recompile.
If it was a SET statement, you might want to take it out and set at the connection
level, to avoid recompilations.
◦ Hint Usage
Make sure that the SP was not created using WITH RECOMPILE option or
RECOMPILE query hint.
◦ Use DTA
Use Database Engine Tuning Advisor to see if any index changes improve the
compile time and execution time of the query.
◦ Workarounds
 KEEP FIXED PLAN  ensures that the same plan is used even though the statistics have been
updated. This plan will be recompiled only when the schema of the underlying tables change
 Disable Auto Update Statistics  This should be the last resort, as it could lead to inefficient
query execution plans

 About Cursor Usage
◦ Earlier versions of SQL allowed only one command per connection.
◦ Another command couldn’t be issued on the same connection, if the query
was still executing or results were pending to be sent to the client
◦ In scenarios where another command needs to be run based on the row just
read, customers resorted to server side cursors.
◦ sp_cursorfetch controls the number of rows to be returned by the server to
the client. This allows ODBC or OLEDB driver to cache the rows.
◦ The same connection can then be used to issue another command.

 Using DMVs
◦ Run the following query to find out the API
cursors which have a fetch buffer size of 1.
◦ From the output, you can find the session_id
and the sql_handle. Use the DMV
sys.dm_exec_sql_text to find the details
about the sql_handle.
SELECT cur.*
FROM sys.dm_exec_connections con
CROSS APPLY sys.dm_exec_cursors(con.session_id) as cur
WHERE cur.fetch_buffer_size = 1

◦ Can you avoid server side cursors?
You should evaluate other options if possible.
◦ Consider using MARS
MARS allows multiple commands to be issued on the same
connection in parallel.
◦ Try using a bigger fetch buffer size
Instead of fetching 1 result, check to see if you could return a
larger buffer from the server

 UDFs causing High CPU
◦ Sometimes the query that you identify is using one
or more UDFs which account for most of the CPU
usage.
◦ Identify the UDFs via query review, along with UDF
source code.

 Alternatives to UDF usage
◦ Can the query be re-written to use more efficient
methods?
◦ Can the UDF code be moved into a stored procedure
being called?
◦ Can the UDF be rewritten in SQL CLR as a User-
Defined Function or User-Defined Aggregate?

 Introduction
◦ I/O is a very expensive activity. SQL Server’s
performance depends upon the performance of
your disk subsystem.
◦ Not all the data pages of all your databases can
fit into memory at all times, thereby causing I/O
to disk.
◦ Also, SQL Server depends on Tempdb (row
versions, sort results etc.), so optimizing
TempDB performance is also critical for overall
SQL Server performance.

 When to investigate
Counters to identify I/O bottlenecks:
Counter Values
PhysicalDisk Object:
Avg. Disk Queue Length
Consistently > 2
Physical Disk: %Disk Time > 50%
Avg. Disk Sec/Read Less than 10 ms - very good
Between 10 - 20 ms - okay
Between 20 - 50 ms - slow, needs
attention
Greater than 50 ms – Serious I/O
bottleneck
Avg. Disk Sec/Write
Avg. Disk Reads/Sec
> 85% of disk capacity
Avg. Disk Writes/Sec

Identifying queries causing high IO
Using DMVs
Run this query to find out the top queries causing the maximum I/O.
For each row in the output, use the sql_handle with sys.dm_exec_sql_text() to
find the actual text of the query
Use plan_handle with sys.dm_exec_query_plan to find the execution plan for the
query.
Analyze the execution plan to locate the steps causing maximum I/O.
Use Database Engine Tuning Advisor to find out missing Physical Design
Structures that could reduce I/O.
SELECT TOP 10 (total_logical_reads/execution_count) as avg_logical_reads,
min_logical_reads, last_logical_reads, (total_logical_writes/execution_count) as
avg_logical_writes, min_logical_writes, last_logical_writes,
(total_physical_reads/execution_count) as avg_phys_reads, min_physical_reads,
last_physical_reads, Execution_count, last_execution_time, statement_start_offset as
stmt_start_offset, sql_handle, plan_handle
FROM sys.dm_exec_query_stats
ORDER BY (total_logical_reads + total_logical_writes) DESC

◦ Analyze the execution plans for missing PDS
Use Database Engine Tuning Advisor to find out any missing physical design
structures (i.e. Indexes) that could enhance performance and reduce I/O overhead.
For example, the right index could reduce the I/O, thereby causing lesser wait times
for other queries to execute on the same disk sub-system.
◦ Analyze Memory Consumption
Insufficient memory and its configuration could cause more I/Os for your disk (as
not all the data pages are in memory and need to be fetched from the disk).
◦ Implement best practices for SQL
 Log files should be on a separate physical disk than the data files
 Use RAID for data files and log files
 Optimize tempdb for performance.
◦ Increase I/O bandwidth
If none of the above yield any significant improvements, you might want to consider
getting a better disk subsystem

 Common Causes for High Disk I/O
◦ SQL Server Related
 Bad Query Plan
 Poorly Designed Queries
 SQL Server MDF, LDF, and Backup Locations
◦ Hardware Related
 Firmware out of Date
 Faulty or Degraded Hardware
 RAID configurations
 Hardware not sufficient for IO Demands

 One of the most important routes to high performance in a SQL Server database is
the index. Indexes speed up the querying process by providing swift access to rows
in the data tables, similarly to the way a book’s index helps you find information
quickly within that book.
 Index Types: In addition to an index being clustered or non clustered, it can be
configured in other ways:
 Clustered Index : A clustered index sorts and stores the data rows of the table or view in
order based on the clustered index key. The clustered index is implemented as a B-tree
index structure that supports fast retrieval of the rows, based on their clustered index key
values.
 Non-Clustered Index :A nonclustered index can be defined on a table or view with a
clustered index or on a heap. Each index row in the nonclustered index contains the
nonclustered key value and a row locator. This locator points to the data row in the
clustered index or heap having the key value. The rows in the index are stored in the order
of the index key values, but the data rows are not guaranteed to be in any particular order
unless a clustered index is created on the table.
 Composite Index: An index that contains more than one column
 Unique Index: An index that ensures the uniqueness of each value in the indexed
column.
 Cover Index: A type of index that includes all the columns that are needed to
process a particular query

 Index fragmentation actually comes in two different forms: external fragmentation and
internal fragmentation. Each of these two forms of fragmentation basically is the inefficient
use of pages within an index. This inefficient use may be because the logical order of the
pages are wrong (external fragmentation) or because the amount of data stored within
each page is less than the data page can contain (internal fragmentation). Whichever type
of fragmentation occurs in your index, you could face performance issues with your queries
because of the fragmentation.
Fragmentation % Value Action
avg_fragmentation_in_percent > 5 AND < 30 Reorganize Index
avg_fragmentation_in_percent > 30 Rebuild Index
How do you know when a table is fragmented?
Poor query performance over time.
More disk activity.
Poor cache utilization.
Verify the I/O of a query.
Verify scan density in SQL 2000 using DBCC SHOWCONTIG and in SQL 2005/2008 using the
dynamic management view sys.dm_db_index_physical_stats.
SELECT OBJECT_NAME(i.object_id) AS TableName ,
i.name AS TableIndexName ,phystat.avg_fragmentation_in_percent
FROM sys.dm_db_index_physical_stats(DB_ID(), NULL, NULL, NULL, ‘DETAILED’) phystat inner JOIN sys.indexes i
ON i.object_id = phystat.object_id AND i.index_id = phystat.index_id WHERE
phystat.avg_fragmentation_in_percent > 5
AND phystat.avg_fragmentation_in_percent < 30

 SQL Server collects statistics about individual columns (single-column
statistics) or sets of columns (multicolumn statistics). Statistics are used
by the query optimizer to estimate the selectivity of expressions, and thus
the size of intermediate and final query results. Good statistics allow the
optimizer to accurately assess the cost of different query plans and then
choose a high-quality plan. All information about a single statistics object
is stored in several columns of a single row in the sysindexes table, and
in a statistics binary large object (statblob) kept in an internal-only table.
information about statistics can be found in the metadata views sys.stats
and sys.indexes.

 How old are your statistics?
 How many rows sampled in the statistics (out of the total rows)?
 How many steps in the statistics (up to 201 steps in SQL Server
2008)?
 Steps = number of rows in your histogram.
 What is the density of that key?
 Density * # of Rows = average rows returned for a given value
 You also get the average length of the column.
 For every step: actual value (high for step), total rows, count of
unique rows.
 Data could be sampled (not based on every row, but just a
subset).

Sql server performance tuning

Recommended

More Related Content

What's hot (20)

Similar to Sql server performance tuning (20)

Recently uploaded (20)

Sql server performance tuning

Editor's Notes