Sql Server & PowerShell

Aaron Shilo www.dbconsultant.co.il aaron@dbconsultant.co.il

Who am I ?

• Married + 2.5 children
• A DBA for 10 years.
• Oracle And Sql-Server Certified Professional.
• Used to be CTO@johnbryce training.
• Lead my own consulting business.
• Advisor to Tapuz.co.il / BezeqInt / Lavie Time-tec and more

More about me @ Www.dbconsultant.co.il

Topics

 Introduction to PowerShell

 Using PowerShell to automate administrative tasks

 Governing Your Enterprise with PolicyBased Management

 Extend Policy Based Management to the Enterprise Scaling PowerShell
Evaluation

 Enhance your usage of DMVs

 Developing maintenance tasks using SSIS Multi Server Administration
in SQL Server 2008 R2

 Deploying apps using the Data Tier

Powershell

New Command-line shell & Scripting Language
Improves productivity & control
Accelerates automation of system admin
Easy-to-use
Works with existing scripts

More
Control

TechNet ScriptCenter
Exchange Server 2007
Terminal Server
Hundreds of Scripts WMI, Registry, Hardware, etc.
Community-Submitted scripts
MyITForum.com

Books & Training Manning Publications
O’Reilly Media
Materials Sapien Press & others…

MS MVPs
PowerShell Team Blog
Community Support Active Newsgroup
Channel 9: DFO Show
IIS.net

What is Powershell?
Revolutionary new interactive shell and scripting language
•Based on .NET
•New set of built-in tools (+120)
•New language to take advantage of .NET
•A new “object-pipeline” system view
•Can continue to use current tools
•Can continue to use current automation (COM)

Benefits

•Automate administration of multiple servers through a task-oriented scripting
language
•Accelerate script authoring, testing and debugging and write customer tools in a
new command shell environment
•Utilize new scripts and Cmdlets
•Manage command-line services, processes, registry, and WMI data
•Manage and/or automate administration tasks for server roles such as IIS and
Active Directory
•Automate Terminal Server configuration changes by means of PowerShell scripts,
and examine configuration similarities and differences across a Terminal Server
farm.
•Manage an Internet Information Services 7.0 environment.
•Remotely manage servers.

No Fear, Uncertainty or Doubt

• Do I need to learn .NET before I can use PowerShell?
– No - you can continue to use existing tools.
• Do I need to rewrite all my existing tools?
– No - existing tools will run just fine.
• Do I need to learn the new language?
– No - You can easily run existing commands without modification.
• Learning the new
– Online help is full of examples that are ready to use
– The new language elements make interacting with .NET a snap.
– Using .NET can help you where new tools don’t exist and opens a vast space
of productivity
– Learn at your own pace, PowerShell will be ready for you

10 reasons why you should learn to use
PowerShell
1: It’s not going away any time soon
Microsoft has made it clear that PowerShell is here to stay. In fact,
PowerShell version 2 is not only included in Windows Server 2008 R2
and in Windows 7, it is enabled by default. Part of the reason why
Microsoft has done this is that going forward, many add-on products
will be based on PowerShell.
2: Most Microsoft products will eventually use it
Virtually all of the server products Microsoft is producing right now can
be managed through PowerShell. From an administrative standpoint,
this means that if you become proficient in PowerShell, you will have
the skill set necessary for managing most of Microsoft‘s newer
products. The basic built-in PowerShell commands are used in every
product that supports PowerShell. However, some server products
extend PowerShell to include additional cmdlets.

3: You can’t do everything from the GUI any more
When Microsoft created Exchange 2007, it designed the GUI so that it
could be used only for the most common administrative functions.
Any obscure functions or anything potentially destructive has to be
performed using PowerShell. I expect this design philosophy to carry
over to other Microsoft products.
4: It can make your life easier
Believe it or not, using the command line can make your life easier.
Suppose for a moment that you need to update an Active Directory
attribute for a thousand users. Performing the task manually would
likely take hours to complete. Using PowerShell, though, you can
complete the task using a single line of code.

5: Many GUIs are PowerShell front ends
Many of the GUI interfaces that Microsoft has been designing for its
various products are actually front end interfaces to PowerShell.
Probably the best known example of this is the Exchange
Management Console. Although this utility looks like a standard
management tool, it is built entirely on top of PowerShell. Any
function you perform through the GUI actually generates PowerShell
code that completes the requested task. In many cases, the console
even shows you the PowerShell command that was used at the
completion of the task.
6: Microsoft certification exams contain PowerShell questions
Microsoft has been adding PowerShell-specific questions to many of its
new certification exams. My experience with these exams has been
that you don‘t necessarily have to know the full command syntax, but
you do need to know which command you should be using in a
given situation.

7: You can use PowerShell commands to manage your domains
If you have domain controllers running Windows Server 2003 with
Service Pack 2 or higher, you can install the Active Directory Web
Services on at least one domain controller. After doing so, you will be
able to use the Windows 7 RSAT Suite to manage Windows 2003 and
Windows 2008 domains.
8: It enables interactivity between products
PowerShell is the common thread between all the new server products
Microsoft is creating, so I expect to start seeing PowerShell used as a
mechanism for providing interactivity between server products. I
have yet to see a real world example of this interactivity, but
eventually I would expect to be able to use a PowerShell script to
work seamlessly between products such as IIS, SQL Server, and
Exchange.

9: Microsoft says it’s important
Just because someone at Microsoft says that something is important,
that doesn‘t mean I take it as gospel. However, In the October 2009
issue of TechNet Magazine, Microsoft says, ―It‘s safe to say that the
single most important skill a Windows administrator will need in the
coming years is proficiency with Windows PowerShell.‖
Such a bold statement is hard to ignore. This is especially true given the
fact that this statement mirrors what I‘ve been hearing from various
people at Microsoft every time I have made a trip to Redmond lately.
10: If you don’t learn it, someone else will
As we all know, the economy is in a slump, and many companies are
downsizing. Needless to say, there is a lot of competition for the few
IT jobs that are available. Therefore, if you suddenly find yourself
looking for another job, your odds of finding one may be better if
you can list PowerShell among your skill set.

CMDlets

•A cmdlet (pronounced "command-let")
•is a single-feature command that manipulates objects in Windows
PowerShell.
•You can recognize CMDlets by their name format –
•a verb and noun separated by a dash (-),
•Get-Help
•Get-Process
•Start-Service.

Powershell - Recommendations

Start using Windows PowerShell
immediately!
Don‘t throw away any existing scripts or
batch files – they can still be used!
Don‘t forget the power of the wildcard, such
as ―*‖
Don‘t deploy Windows PowerShell on any
machine where it is not actually needed
Centrally-Control Windows PowerShell
security settings through GPOs – do it now!

PowerShell: variable syntax

 Basic syntax similar to all scripting languages
 $a = ―value‖
 $a = 1 + 2
 $a += 137
 Built-in support for types
 $a = [xml]―<test><a>avalue</a></test>‖
 [int]$a = 42
 or cast to arbitrary .NET object

PowerShell: parsing

 Two contexts:
 command context
 starts with regular character
 tries to execute command
 expression context
 starts with num, var, or quoted string
 executes as expression
 Can reset mode with ―( )‖
 (Get-Date).day + 2

PowerShell: array and hash

 $a = 1,2,3,4
 $a += 5..10
 + adds elements to array
 .. is range operator. What is $a[1..3]?
 $a[1], $a[2], $a[3]
 $a = @{ one=1; val = get-childitem }
 $a[‗val‘]
 $a.one

PowerShell: if/switch

 if { <test> } { <true> } else { <false> }
 elseif { <test> } { <else> }
 switch (<var>) { <val> { <case>; break } … }
 default case
 switch –regex (<var>) { word.* { <case> } … }
 dropping break gives multiple matches
 $_: variable referring to current <var>
 where { <test> }

PowerShell: looping

 while($a –lt 137) { <block> }
 foreach($var in get-process) { <block> }
 IEnumerable support
 Shorten foreach to %
 ls | %{ $_.Length }
 receive piped objects
 $_ as before
 for { $i = 0; $i –lt 10; $i += 2} { $i }
 regular for loop

PowerShell: useful operators

 &, .
 call operator
 $a = ―Get-childitem‖; &$a # calls get-childitem
 . used for executing scripts in current context
 -as, -is
 is/as in C#
 $a –as [int]
 if { $a –is [System.DateTime] } { … }

PowerShell: functions

 function [(args)] { <body> }
 if Param is first statement, then gives parameters
 arguments passed in $args
 input passed in $input
 parameter passing on cmd line
 add –x 2 –y 3
 add 2 3

Useful commands

 get-member
 return the members of an object
 eg. get-member –MemberType property
 or, method
 Authenticode signing
 can use certificates to verify scripts
 checks the hash and returns

PowerShell: errors

 throw
 throw ―error‖
 same as ThrowTerminatingError in Cmdlets
 trap
 catch exceptions
 trap [DivideByZeroException] { <do something> }
 break/continue semantics

PowerShell: surprises

 Drives
 C, D
 Env, Alias, HKLM, variable, function
 mount lets you create others
 variable scoping
 private, local, script, global


 import/export –CSV
 get-unique
 $(foreach ($line in get-content C:Test1File1.txt)
{$line.tolower().split(" ")}) | sort | get-unique
 get-item
 group
 $x = new-object –COM <ProgID>
 eg. Outlook.Application


 calc
 $calc = get-process calc
 $calc.add_exited({write-host $this.Processname has exited})
 $calc.HasExited
 $calc.kill()
 $calc.HasExited

Scriptblocks

 a chunk of script
 a type in PowerShell
 $x = [scriptblock]{$y = 137}
&$x;
 Can be used as an EventHandler delegate
 this is what‘s happening in the last example
 object parameter -> $this
 EventArgs parameter -> $_

Cmdlet creation

 Subclass of System.Management. Automation.Cmdlet
 must have an attribute
 gives ―verb‖ and ―noun‖ components of name
 overrides at least one of
 BeginProcessing, ProcessRecord, EndProcessing
 use make-shell command to include it
 adds dll to set of cmdlets
 requires some registry manipulation

Cmdlet creation

 Cmdlet parameters
 add [Parameter] attribute
 Mandatory=true|false
 Position=<index>
 … others for in-pipeline action
 Return value: any object
 native object facilities allow inspection
 can use arbitrary code

Example in script

 function MyWhere {
param ( [scriptblock]$expression )
begin { $matches = 0 }
process {
if ( &$expression ) { $_; $matches++ }
}
end { "Found $matches matches" }
}

Execution Policy

 AllSigned
 require a digital signature and prompt
 user must agree to run script
 RemoteSigned
 only files from internet need to be signed
 default setting
 Unrestricted
 no signing required
 Attacks on AllSigned?

Using PowerShell to Manage And automate
administrative tasks

 SQLPS.exe – The SQL Server Mini-Shell
 SQL Server Management Objects – SMO
 Using ADO.NET for Queries

 To make things easier for administrators using PowerShell for SQL
Server, Microsoft created an executable - a mini-shell - which
automatically loads the SQL Server PowerShell snap-ins. The
program is called sqlps.exe and is included when you install SQL
Server 2008. Besides automatically loading the snap-ins the
sqlps.exe environment enables script execution automatically by
setting the execution policy to RemoteSigned.

 This mini-shell can be invoked in a number of ways. You can use the
Windows Start, Run menu item and type in sqlps and start it that
way. SQL Server Agent jobs can use the SQL Server Agent
PowerShell subsystem in a step to run a script by selecting the step
type of PowerShell. Each time a step runs the PowerShell step it
loads a separate copy of sqlps.exe into memory, each taking about
40MB, so caution is advised when using this feature.

 The most interesting method is within SQL Server Management
Studio. In Object Explorer you can right-click on any object in the
―tree‖ under an instance and select ―Start PowerShell‖, and sqlps.exe
will start up, and the Set-Location cmdlet is used to set the current
location in the shell window to the object you pointed to. This
provides a very nice ability to do some ad-hoc browsing of SQL
Server objects as necessary.

SQL Server Management Objects - SMO

 SQL Server Management Objects (SMO) is the object library
provided by Microsoft to perform administrative actions against SQL
Server. The library is built on top of the .NET Framework (2.0) so
applications written using SMO are using managed code and benefit
from the features available within the Framework, including the
security and cleanup features. SMO was introduced with SQL Server
2005 and allows you to manage servers running SQL Server 2000,
SQL Server 2005, and SQL Server 2008.

Using ADO.NET for Queries

 ADO.NET has two sets of objects, a connected set that allows you to
connect with a data source, run queries and return result sets, and a
disconnected set of objects that allow you to work with the data
after it‘s been collected. Here‘s a list of the objects.

ADO.NET Object Description
Connection
Object A connection to the data source

Can represent a query against a database, a call to a stored procedure, or a direct
Command Object request to return the contents of a specific table
DataReader
Object Designed to return query results as quickly as possible
Transaction
Object Groups a number of changes to a database and treats them as a single unit of work

The Connection object has a BeginTransaction method that can be used to create
Transaction objects
Parameter Object Allows the specification of parameters for stored procedures or parameterized queries

DataAdapter Acts as a bridge between the database and the disconnected objects in the ADO.NET
Object object model

ADO.NET Object Description

Allows the examination of data through collections of rows and
DataTable Object columns
DataColumn
Object Corresponds to a column in a table

Constraint Object Defines and enforces column constraints
DataRow Object Provides access to the DataTable's Rows collection
DataSet Object The container for a number of DataTable objects
DataRelation
Object Defines the relations between DataTables in the DataSet object
DataView Object Allows the examination of DataTable data in different ways

Using the Set-ExecutionPolicy Cmdlet

 The Set-ExecutionPolicy cmdlet enables you to determine which
Windows PowerShell scripts (if any) will be allowed to run on your
computer. Windows PowerShell has four different execution policies:

 Restricted - No scripts can be run. Windows PowerShell can be
used only in interactive mode.
 AllSigned - Only scripts signed by a trusted publisher can be
run.
 RemoteSigned - Downloaded scripts must be signed by a
trusted publisher before they can be run.
 Unrestricted - No restrictions; all Windows PowerShell scripts
can be run.

Using PowerShell to Manage And automate
administrative tasks

Conclusion

 Scripting has been a powerful tool for Unix administrators for a long
time. Windows administrators have had fewer and less capable
options for automating administrative processes until the
introduction of PowerShell.
 There are a great many sites providing quality information on using
PowerShell to automate administrative tasks. The addition of
PowerShell support to SQL Server 2008 tool set adds an extra
dimension of manageability to the Windows Server
environment. You should review the available books and online
material to develop your skills in using this powerful tool, and by
doing so automate and streamline the processes in your own
environment.

Governing Your Enterprise with Policy-Based
Management

Enterprise Policy Management

• Provide auditors with assurance that SQL Server
Compliance complies with all security and business guidelines
• Complement All Actions Audited

Consistency
• Ensure peak performance
• High levels of security & reliability

• Drive strategic management initiative to control
Costs costs
• More efficient and proactive management

 Policy-Based Management Framework
 Introduction to the Framework
 Evaluation Modes
 Extending Policy-Based Management to the Enterprise
 Solution Architecture
 Considerations for Large-Scale Environments
 Centralized Reporting

Policy-Based Management

Defines the evaluation mode, target filters, and schedule of the conditions.

Policy

Specifies a set of allowed states of a managed target with regard to a facet

Condition

Set of related logical properties

Facet

Server
Restriction

Category Policy Target

Evaluation
Mode

Policy Evaluation Modes

On Demand On Schedule
• Evaluate a policy when specified by user • SQL Server 2008 only
• Available through SSMS or Windows • SQL Server Agent job periodically evaluates
PowerShell™ a policy
• Option to force certain conditions to
comply with policy
• Supports down-level evaluation
(depends on properties exposed)
Evaluation modes

On Change: Prevent On Change: Log Only
• SQL Server 2008 only • SQL Server 2008 only
• DDL triggers prevent policy violations • Event notification evaluates a policy when a
relevant change is made

On Demand Evaluation
Windows PowerShell™ Integration
 Windows PowerShell™ is a framework and runtime for executing
management commands
 Cmdlets are instances of .NET classes that process input objects from the
pipeline
 SQL Server Provider for Windows PowerShell™ encompasses SMO

Invoke-PolicyEvaluation –Policy
DatabaseStatus.xml,
Trustworthy.xml -TargetServerName inst1

Invoke-SQLCMD –Query ”SELECT name FROM
sys.Databases;” –ServerInstance
“MyServerInstance”

Extend you enterprise- Bringing it all together

policy
results policy
results

policy
results policy
results

policy
results policy
results

Extend to the Enterprise
Bringing It All Together

policy
results
policy
results

policy
results policy
results

policy
results

policy
results

Extend to the Enterprise- Central Management
Server
 Logically group instances based on
business function(s)
 Centrally publish policies to groups of
SQL Server 2008 instances
 Evaluate policies on-demand against
a group of servers
 Filter by logical groups in Windows
PowerShell™ scripts

Extend to the Enterprise- Scaling PowerShell
Evaluation
 Add Intelligence to Policies
 Place each policy in a category
 Define server restrictions for
versions and editions where
appropriate

Extend to the Enterprise- Scaling PowerShell
Evaluation
 Create Custom Server Groups in the
CMS
 Run specific policies against a list
of servers
 Examples: Production,
Development, PCI

 Define Concurrent Jobs
 Define multiple concurrent
executions based on Policy
Category and/or logical Central
Management Server group

Extend to the Enterprise
Benefits of Upgrading to SQL Server 2008
 Real-Time Enforcement and
Reporting
 Monitor the event log through
Alerting integration
 Advanced functionality and
integration with SSMS
 Dependency, health states,
subscriptions, history
 Scale
 Security
 Access to other rich features in SQL
Server 2008

Centralizing Policy History
SQL Server 2008

policy
results

policy
results

policy
results

syspolicy_policy_execution_history policy
results
syspolicy_policy_execution_history_details

Additional Resources
Policy Based Management
 Enterprise Policy Management Framework
 http://www.codeplex.com/EPMFramework
 Policy Based Management Blog
 http://blogs.msdn.com/sqlpbm/default.aspx
 Lara Rubbelke‘s Blog
 http://sqlblog.com/blogs/lara_rubbelke/default.aspx
 Dan Jones‘ Blog
 http://blogs.msdn.com/dtjones/default.aspx
 Buck Woody‘s Blog
 http://blogs.msdn.com/buckwoody/default.aspx

Additional Resources
Windows PowerShell™
 To learn more about the Windows PowerShell™ scripting Language
 http://www.microsoft.com/downloads/details.aspx?FamilyID=b4720b
00-9a66-430f-bd56-ec48bfca154f&DisplayLang=en
 Windows PowerShell™ Blog
 http://blogs.msdn.com/powershell/
 Allen White‘s Blog
 http://sqlblog.com/blogs/allen_white/
 SQL Server PowerShell Overview
 http://msdn.microsoft.com/en-us/library/cc281954.aspx

Extend Policy Based Management to the
Enterprise Scaling PowerShell Evaluation

The Enterprise Policy Management Framework (EPM) is a solution to
extend SQL Server 2008 Policy-Based Management to all versions of
SQL Server in an enterprise, including SQL Server 2000 and SQL
Server 2005. The EPM Framework will report the state of specified
SQL Server instances against policies that define intent, desired
configuration, and deployment standards.

When the Enterprise Policy Management Framework (EPM) is
implemented, policies will be evaluated against specified instances of
SQL Server through PowerShell. This solution will require at least one
instance of SQL Server 2008. The PowerShell script will run from this
instance through a SQL Server Agent job or manually through the
PowerShell interface. The PowerShell script will capture the policy
evaluation output and insert the output to a SQL Server table. SQL
Server 2008 Reporting Services reports will deliver information from
the centralized table.

This solution requires the following components to be configured in
your environment. All SQL Server 2008 requirements listed below
may be executed from and managed on the same instance:

§ SQL Server 2008 instance to store policies
§ SQL Server 2008 instance to act as the Central Management Server
§ SQL Server 2008 instance to execute the PowerShell script
§ SQL Server management database to archive policy evaluation results
§ SQL Server 2008 Reporting Services to render and deliver policy
history reports

Extend Policy Based Management to the
Enterprise Scaling PowerShell Evaluation

Overview
 SQL Server 2000 provided
 ~11 PerfMon objects and counters
 Access to some system tables
 A few table-valued functions
 SQL Server 2005 introduced numerous long-awaited performance
measurement metrics
 Compared to SQL 2000, SQL 2005
 Approximately tripled SQL-specific PerfMon objects and counters
 Created numerous views and functions (~80 Dynamic Management Views
and Functions) that provide very useful and previously unavailable
performance information
 SQL 2008
 Added four SQL-specific PerfMon objects and counters
 Added ~40 Dynamic Management Views and Functions

67

Overview
 Sheer number of performance views and new metrics can be dizzying
 New metrics documented in
 Books Online
 Numerous articles and books
 However, much of the literature simply defines the DMVs and metrics,
so knowing where to begin is extremely difficult
 http://www.microsoft.com/downloads/details.aspx?FamilyID=531c53e7
-8a2a-4375-8f2f-5d799aa67b5c&displaylang=en

68

Overview

 Performance analyst must understand
 Which PerfMon objects and DMVs are pertinent to a particular
problem
 How to interpret numerous metrics provided by new objects and
views
 Most valuable views and metrics

69

Emphasis of Presentation

 Highlight most useful of the newly available
 SQL Server-specific PerfMon objects and counters
 DMVs
 How to convert the numeric DMV identifiers into understandable text
 Properly use and interpret most important
 SQL Server-specific PerfMon objects and counters
 DMV metrics

70

SQL Server PerfMon Objects

 Many more useful metrics, e.g.,
 Wait statistics
 Tempdb (General Statistics)
 Cursors
 Transactions
 .Net Data Provider for SQL Server
 Deprecated Features (2008)

71

Wait Statistics Object

 Subset of this information was previously available only via complex
queries from several disparate sources
 Small overlap with Locks object
 Instances where Counters would normally be
 Most useful instances
 Cumulative wait time (ms)
 Average wait time (ms)

72


 Cumulative wait time (ms) instance extremely useful
 Unfortunately, still only counts values within the last second
 Sampling every 15-30 seconds usually provides an adequate
representation
 Use sys.dm_os_wait_stats to determine
 Total wait times by specific type of wait without sampling problems
 Discussed later in presentation

73

 Summarizes very useful counters including
 Lock waits (total for all lock resources)
 Log buffer waits
 Log write waits
 Memory grant queue waits
 Network IO waits
 Non-Page latch waits
 Page IO latch waits
 Page latch waits
 Thread-safe memory object waits
 Transaction ownership waits

74

Lockable PerfMon Resources

Resource Description
AllocUnit (2005/2008) Allocation unit
Application Application-specified resource
(2005/2008)
Database Entire database
Extent Contiguous group of 8 data or index pages
File (2005/2008) Database file
HoBT (2005/2008) Heap or B-Tree index
Key Row lock within an index that protects range of keys in
serializable transactions
Metadata (2005/2008) Meta data or catalog information
Object (2005/2008) Job flow synchronization object, table, stored procedure, or
view
Page 8-kilobyte (KB) data or index page
RID Row ID. Used to lock a single row within a table
Table (replaced by Entire table, including all data and indices
Object on 2005/2008) 75

Deprecated Features Object
 Simplifies detection of deprecated feature usage since last SQL Server
instance restart
 Examples of deprecated features/counters
 '::' function calling syntax
 Data types: text ntext or image
 DATABASEPROPERTY
 DATABASEPROPERTYEX('IsFullTextEnabled')
 DBCC DBREINDEX
 DBCC INDEXDEFRAG
 DBCC SHOWCONTIG
 NOLOCK or READUNCOMMITTED in UPDATE or DELETE
 sp_adduser
 sysindexes
 sysobjects
 Use Deprecation Announcement & Deprecation Final Support trace
events to determine exact causes 76

Critical General Views
 Required for converting numeric DMV identifiers into understandable text
 Many were available under 2000 with slightly different names
 2000 sysobjects -> 2005 sys.objects
 See ―Mapping System Tables to System Views (Transact-SQL)‖ topic
in BOL for further info
 sys.databases
 Lists all databases and their IDs so proper associations can be made
 sys.partitions
 Only way to decode HOBT IDs returned by lock-specific information,
e.g. blocked process records
 sys.configurations
 Provides information regarding OS and SQL Server configurations
 sys.dm_os_sys_info
 Provides information regarding system (can be sampled once or
repeatedly)
 Useful for determining whether hyperthreading is active and accessible77

sys.dm_os_sys_info Useful Columns

 No parameters required
 Number of logical CPUs on the system
 Ratio of number of logical or physical cores exposed by one physical
processor package (hyperthread_ratio)
 Amount of physical memory available
 Amount of virtual memory available to the process in user mode.
This can be used to determine whether SQL Server was started by
using a 3-GB switch
 Date and time SQL Server last started

78

Database-Specific Views

 Query for each database separately
 sys.objects
 Lists all database objects such as tables, views, stored procedures,
etc.
 sys.indexes
 Lists all indices and their associated table IDs
 Does not provide row counts as sysindexes does
 sys.filegroups
 Lists all file groups and their IDs
 sys.database_files
 Lists all physical database files and their IDs

79

Dynamic Management Views and Functions

 “Designed to give you a window into what's going on inside SQL
Server”
 Two types
 DMV - Pure view, i.e., no parameters required
 DMF - Table-valued function, i.e., parameters required
 Parameters usually specify database, table, index, partition, etc.
 Provide significant amount of information regarding
 System
 Databases
 Internal workings of SQL Server
 Performance

80


 Some were possible with complex queries on 2000
 When possible
 Very clumsy to use
 Required temp tables
 Required significant understanding of the underlying tables
 Most DMVs are very simple to use
 Some still require joins with other DMVs and DMFs
 Some may lessen need for using SQL Trace

81


 Contain values since last SQL Server instance restart, unless manually
reset
 Perfect for periodic or intermittent sampling
 Therefore, must compute differences between adjacent or distant
samples for same database, file, table, index, etc.
 Many DMVs can be reset manually using command similar to
 DBCC SQLPERF ('sys.dm_os_wait_stats', CLEAR)

82

Dynamic Management Views and Functions –
General Guidelines
 Most performance DMVs begin with sys.dm_
 Must reconcile numeric IDs with static views that contain textual
names
 Values accumulated from last SQL Server instance restart
 Sample rates can range from once per minute to a few times per
day
 Must calculate differences between individual sample records
 Be sure to diff records with same database ID, object ID, and index
ID

83

General Usage Scenario
 Method 1
 Capture before monitored activities begin and store results into a
SQL table or a spreadsheet
 Execute workload (whether natural production or test)
 Capture again and compare value differences
 Load into spreadsheets for further analysis, if necessary
 Method 2
 Capture every <n> minutes and store results in a flat file
 After workload and capture process complete, load data into SQL
tables
 Use SQL to compare incremental value differences and store results
 Extract interval data to spreadsheets for further analysis

84

Usage Warnings
 Microsoft assures us these views DO NOT BLOCK (single exception
cited below) as usage of older system tables could
 However, be careful of execution frequency when cross apply or
several joins involved because resource consumption could be
high
 Beware of the following two
 sys.dm_db_index_physical_stats, which examines the data within
the physical files and provides extremely valuable data, can block
depending upon the option used, but not nearly as badly as
DBCC SHOWCONTIG
 sys.dm_tran_locks can consume large amounts of CPU and can
generate huge output when # of locks large
 Use only if absolutely necessary

85

Category List
View Category View Category
Change Data Capture Related Dynamic Query Notifications Related Dynamic
Management Views Management Views
Common Language Runtime Related Replication Related Dynamic Management
Dynamic Management Views Views
Database Mirroring Related Dynamic Resource Governor Dynamic Management
Management Views Views
Database Related Dynamic Management Service Broker Related Dynamic
Views Management Views
Execution Related Dynamic Management SQL Server Extended Events Dynamic
Views and Functions Management Views
Full-Text Search Related Dynamic SQL Server Operating System Related
Management Views Dynamic Management Views
Index Related Dynamic Management Transaction Related Dynamic Management
Views and Functions Views and Functions
I/O Related Dynamic Management Views Security Related Dynamic Management
and Functions Views
Object Related Dynamic Management
86
Views and Functions

Execution-Related Dynamic Management Views
and Functions
 Most commonly used ones
 sys.dm_exec_query_stats
 sys.dm_exec_requests
 sys.dm_exec_sessions
 sys.dm_exec_sql_text
 Other less commonly used ones
 sys.dm_exec_cached_plans
 sys.dm_exec_connections
 sys.dm_exec_cursors

87

sys.dm_exec_query_stats DMV

 Returns aggregate performance statistics for cached and completed
query plans
 Top <n> can be returned for any category
 Execution count, CPU time, CLR time, and elapsed time
 Physical and logical reads, Logical writes
 Activity Monitor on 2008 appears to use this DMV primarily
 Currently active long-running queries NOT shown via this view
 View contains one row per query plan
 Lifetime of a row is tied to the plan itself
 Can reduce dependence upon SQL statement tracing, but
creation_time, last_execution_time, and accumulated values MUST be
accounted for

88

sys.dm_exec_query_stats Columns

 ―Starting and ending positions of the query‖… ―within the text of its
batch or persisted object‖
 Pointer to plan, which can be passed to the sys.dm_exec_sql_text
DMF
 Times at which plan was compiled and last executed
 # of times plan executed since last compilation
 Total, minimum, maximum, and last amounts consumed by executions
since compilation
 CPU, CLR, and elapsed times, all in microseconds

89

sys.dm_exec_query_stats Example

 Returns top 20 query statements that performed most logical reads
select top 20 [sql_handle], creation_time, execution_count,
last_execution_time, total_worker_time, total_physical_reads,
last_physical_reads, max_physical_reads, total_logical_reads,
last_logical_reads, max_logical_reads, total_logical_writes,
last_logical_writes, max_logical_writes, total_clr_time, last_clr_time,
max_clr_time, total_elapsed_time, last_elapsed_time,
max_elapsed_time,
SUBSTRING(st.text, (qs.statement_start_offset/2) + 1,
((CASE statement_end_offset
WHEN -1 THEN DATALENGTH(st.text)
ELSE qs.statement_end_offset END
- qs.statement_start_offset)/2) + 1) as statement_text
FROM sys.dm_exec_query_stats as qs
CROSS APPLY sys.dm_exec_sql_text(qs.sql_handle) as st
ORDER BY total_physical_reads DESC
Note how other DMVs and DMFs are used
90

sys.dm_exec_requests Columns

 Timestamp when request arrived
 Status of the request
 Identifies current type of command being processed, e.g., select,
insert, update, etc.
 Plan handles and offsets similar to sys.dm_exec_query_stats
 If blocked, ID of session blocking request
 If request currently (or previously) blocked, returns wait type and wait
time (ms)
 # of transactions and result sets open for request
 CPU, total elapsed times in (ms)
 # of rows that have been returned to the client by this request

91

sys.dm_exec_requests Example

 Returns batches that contain top 20 currently active query statements
that consumed the most CPU
select top 20 *
from sys.dm_exec_requests ExecReq
OUTER APPLY sys.dm_exec_sql_text
(ExecReq.sql_handle) ExecSQLText
where (session_id > 50 and session_id <>
@@spid) and
command not like 'WAITFOR%'
order by cpu_time desc

92

Index Related Dynamic Management Views and
Functions
 sys.dm_db_index_usage_stats
 Returns information regarding query code usage of tables and
indices, e.g., inserts, updates, deletes, random accesses, and
sequential accesses
 sys.dm_db_index_operational_stats
 ―Returns current low-level I/O, locking, latching, and access method
activity for each partition of a table or index in the database‖
 Most commonly useful metric tracks actual forwarded record
fetches
 sys.dm_db_index_physical_stats
 Execute infrequently because physical files are interrogated and
causes some blocking to occur
 Replaces DBCC SHOWCONTIG

93

sys.dm_db_index_usage_stats DMV

 Returns
 One row per index
 Number of seeks, scans, lookups, and updates for user and system
queries
 Seeks  random
 Scans  sequential
 Time of last seek, scan, lookup, and update for user and system
queries
 System category
 Maintenance, e.g., statistics updates
 User category
 Insert, update, delete, and select operations
 Reports user statements, not record counts
 One insert statement can result in millions of rows being inserted
 View will report one, not millions
 Helps determine index and table usage patterns
 Particularly useful for identifying indices that are never used by 94
queries

sys.dm_db_index_usage_stats Example

List Rarely-Used Indices For Specific DB With Resolved Names
declare @dbid int = db_id()
select objectname=object_name(inxusage.object_id),
inxusage.object_id,
indexname=sysinx.name, sysinx.index_id, user_seeks, user_scans,
user_lookups, user_updates
from sys.dm_db_index_usage_stats inxusage,
sys.indexes sysinx
where database_id = @dbid and
objectproperty(inxusage.object_id,'IsUserTable') = 1 and
sysinx.object_id = inxusage.object_id and
sysinx.index_id = inxusage.index_id
order by (user_seeks + user_scans + user_lookups +
user_updates) asc
95

sys.dm_db_index_operational_ stats DMF

 One row per index
 Helpful for
 Determining how indices are used
 Identifying contention areas
 Four parameters required
 { database_id | NULL | 0 | DEFAULT } (use db_id() for current db)
 { object_id | NULL | 0 | DEFAULT }
 { index_id | NULL | -1 | DEFAULT }
 { partition_number | NULL | 0 | DEFAULT }
 Use NULL parameters to obtain information for all databases and files
select * from sys.dm_db_index_operational_stats (NULL, NULL,
NULL, NULL)

96

sys.dm_db_index_operational_ stats Columns Pt 1

 Cumulative counts
 Range and table scans started on index or heap
 Single row retrievals from index or heap
 Rows that were fetched through forwarding record
 Row locks requested
 Page locks requested
 Cumulative counts and elapsed times database engine
 Waited on row lock
 Waited on page lock
 Waited because of latch contention
 Waited on I/O page latch

97

sys.dm_db_index_operational_ stats Columns Pt 2

 Cumulative counts
 Leaf-level insert, delete, update, delayed delete operations
 Above leaf-level insert, delete, update, delayed delete operations
 Zeroes  heap operations
 Leaf-level page allocations in the index or heap
 For an index, page allocation corresponds to a page split
 Page allocations caused by page splits above leaf level
 Page compression attempt and success counts (2008)

98

sys.dm_db_index_operational_ stats Usage Pt 1

 Reports actual operation counts, not user statements
 One insert statement can result in millions of rows being inserted
 View will report millions, not one
 To analyze a common access pattern to the table or index partition
 leaf_insert_count
 leaf_delete_count
 leaf_update_count
 leaf_ghost_count
 range_scan_count
 singleton_lookup_count

99

sys.dm_db_index_operational_ stats Usage Pt 2

 To identify latching and locking contention
 page_latch_wait_count and page_latch_wait_in_ms
 Indicate whether latch contention exists on index or heap, and
significance of contention
 row_lock_count and page_lock_count
 Indicate how many times the Database Engine tried to acquire
row and page locks
 row_lock_wait_in_ms and page_lock_wait_in_ms
 Indicate whether lock contention exists on index or heap, and
significance of contention
 To analyze statistics of physical I/Os on an index or heap partition
 page_io_latch_wait_count and page_io_latch_wait_in_ms
 Indicate how many physical I/Os were issued to bring index or
heap pages into memory and how much waiting was involved

100

sys.dm_db_index_physical_stats DMF

 Five parameters required
 { database_id| NULL | 0 | DEFAULT } (use db_id() for current db)
 { object_id| NULL | 0 | DEFAULT }
 { index_id| NULL | -1 | DEFAULT }
 { partition_number| NULL | 0 | DEFAULT }
 { LIMITED | SAMPLED | DETAILED | NULL | DEFAULT }
 LIMITED mainly provides fragmentation information
 Strict locking scheme not used for LIMITED or SAMPLED, so DETAILED
mode primary cause of any blocking
 DETAILED provides much useful information
 Record count, forwarded record count, min-max-avg record
lengths

101

sys.dm_db_index_physical_stats

select * from sys.dm_db_index_physical_stats (NULL, NULL, NULL,
NULL, 'DETAILED')
 DETAILED mode
 Used instead of SAMPLED when index or heap has < 10,000 pages
 Provides information about non-leaf levels

102

sys.dm_db_index_physical_stats Columns

 Index_type_desc
 Index type
 Index_id, index_level
 0  heap
 Index_depth
 # of index levels
 Avg_fragmentation_in_percent
 Logical fragmentation for indexes or extent fragmentation for
heaps
 Indices with values > 30 are candidates for rebuild/reorganization
 Fragment_count
 Number of fragments in the leaf level
 Avg_fragment_size_in_pages
 Average number of pages per fragment in the leaf level
 Larger is better
 Forwarded_record_count
 # of heap records having forward pointers to another data location 103

I/O Related Dynamic Management Views and
Functions
 sys.dm_io_pending_io_requests
 sys.dm_io_virtual_file_stats

104

sys.dm_io_virtual_file_stats DMF

 Returns I/O statistics for data and log files, one row per file
select * from sys.dm_io_virtual_file_stats (NULL, NULL)
 Equivalent to select * from ::fn_virtualfilestats(-1,-1) on SQL 2000

105

sys.dm_io_virtual_file_stats Columns

 ID of database
 ID of file
 Number of milliseconds since SQL Server instance started (useful for
detecting restarts)
 Number of reads and writes issued against this file
 Total number of bytes read from and written to this file
 Total time, in milliseconds, users waited for I/O completions overall, as
well as reads and writes specifically
 Number of disk bytes used by this file

106

SQL Server Operating System-Related Dynamic
Management Views
 Abundant views
 Most frequently used views
 sys.dm_os_wait_stats
 sys.dm_os_waiting_tasks
 sys.dm_os_latch_stats

107

sys.dm_os_wait_stats DMV

 Useful for diagnosing performance issues with
 SQL Server
 Specific queries and batches
 Expands upon PerfMon Wait Statistics object
 Returns information about waits encountered by threads in execution
 226 possible wait types
 Several of these are innocuous because they are accumulated as
part of normal SQL Server operation
 Very important to eliminate normal wait types from
consideration

108

sys.dm_os_wait_stats

 Very useful to categorize and group individual wait types to simplify
understanding and analysis, e.g.,
 Combine LOGMGR, IMPPROV_IOWAIT, IO_AUDIT_MUTEX,
IO_COMPLETION, DISKIO_SUSPEND, IO_AUDIT_MUTEX,
IO_COMPLETION, ASYNC_DISKPOOL_LOCK,
ASYNC_IO_COMPLETION, REQUEST_DISPENSER_PAUSE,
PAGEIOLATCH_DT, PAGEIOLATCH_EX, PAGEIOLATCH_KP,
PAGEIOLATCH_NL, PAGEIOLATCH_SH, PAGEIOLATCH_UP into I/O
 Combine PAGELATCH_DT, PAGELATCH_EX, PAGELATCH_KP,
PAGELATCH_NL, PAGELATCH_SH, PAGELATCH_UP into Latches

109

sys.dm_os_waiting_tasks DMV

 Reports information about currently waiting tasks
 Can be used to construct blocker/waiter lists
 Important Fields
 Total wait time (ms) for wait type
 Name of wait type
 Task currently holding this resource
 Blocking task session ID
 Description of resource being consumed

110

sys.dm_os_latch_stats DMV

 SQL Server uses numerous kinds of latches (118 documented as of
2008 SP1)
 Latch times can be helpful in verifying I/O subsystem performance
 Large latch wait times often indicate poor I/O performance
 View returns information regarding all latch types
 Very useful to categorize and group individual wait types to
simplify understanding, e.g.,
 BUFFER_POOL_GROW and BUFFER into Buffer
 ALLOC_CACHE_MANAGER, ALLOC_CREATE_FREESPACE_CACHE,
ALLOC_CREATE_RINGBUF, ALLOC_EXTENT_CACHE, and
ALLOC_FREESPACE_CACHE into Allocation

111

Transaction Related Dynamic Management Views
and Functions
 Numerous views
 sys.dm_tran_locks is useful, but can distort system depending upon #
of locks being held

112

sys.dm_tran_locks DMV

 Each row represents currently active request to lock manager for lock
that has been granted or is waiting to be granted
 Result set columns are divided into two main groups — resource and
request
 Resource group describes resource on which the lock request is
being made
 Resource type same as lockable resource
 Request group describes the lock request

113

Conclusions

 SQL Server 2008 provides a wealth of performance information
 Numerous Dynamic Views are essential for uncovering and resolving
performance bottlenecks
 Dynamic Views and Functions are
 Easy to use
 Well documented
 Low overhead with very few exceptions

114

New sys.dm_ DMVs and DMFs in SQL 2008
View Name View Name View Name
dm_audit_actions dm_fts_fdhosts dm_server_audit_status
dm_audit_class_type_map dm_fts_outstanding_batches dm_tran_commit_table
dm_cdc_errors dm_os_dispatcher_pools dm_xe_map_values
dm_cdc_log_scan_sessions dm_os_dispatchers dm_xe_object_columns
dm_cryptographic_provider_pro
perties dm_os_memory_brokers dm_xe_objects
dm_database_encryption_keys dm_os_memory_node_access_stats dm_xe_packages
dm_db_mirroring_auto_page_rep dm_xe_session_event_actio
air dm_os_memory_nodes ns
dm_db_mirroring_past_actions dm_os_nodes dm_xe_session_events
dm_xe_session_object_colu
dm_db_persisted_sku_features dm_os_process_memory mns
dm_db_script_level dm_os_spinlock_stats dm_xe_session_targets
dm_exec_procedure_stats dm_os_sys_memory dm_xe_sessions
dm_resource_governor_configurati
dm_exec_trigger_stats on
dm_resource_governor_resource_p
dm_filestream_file_io_handles ools
dm_resource_governor_workload_g 115
dm_filestream_file_io_requests roups

Developing maintenance tasks using SSIS Multi
Server Administration in SQL Server 2008 R2

Deploying apps using the Data Tier

 Data-Tier Applications, abbreviated DAC

 R2 makes it easier for DBAs to move databases around from server
to server in much the same way virtualization admins move guest
OS‘s around between physical hosts.

 No more need for backup and restore.

 Easily deploy databases.

The following steps illustrate a simple
deployment process:

A database developer completes the development of a Finance version
1.0 DAC, and then builds a FinanceVersion1.dacpac DAC package.

A database administrator deploys the Finance 1.0 DAC to production.
The production instance of the database engine now has a deployed
DAC that has an application name of Finance, a version of 1.0, and an
associated database named Finance.

The database developer then starts work on the next version, and when
development completes builds a FinanceVersion2.dacpac.

The database administrator then runs the Upgrade Data-tier Application
Wizard specifying both the current deployed version 1.0 DAC and the
version 2.0 DAC package. The wizard performs the following steps:

 Verifies that the deployed DAC and the DAC package both have
their application names set to Finance. It also evaluates whether
the instance of the Database Engine meets the requirements
specified in the server selection policy (if defined) in the version
2.0 DAC.

 Deploys the Finance version 2.0 DAC from the DAC package. This
creates a new database with a temporary name.

 Sets the original database to read-only if it is not already in read-
only mode, and copies the data to the new database.

 If the original database was in read-only mode, the new database
is set to read-only.

 The original database is renamed by appending a string to the
end of the database name.

 The new database is assigned the original database name:
Finance.

After the database administrator has confirmed that the new database is
operating correctly, the original database can be archived.

 The DAC does not include the data inside your database. For
deployment best practices, you should have any necessary data
(configuration tables, basic lookup tables) already scripted out as
part of your deployment strategy. With the DAC approach, it makes
sense to put these scripts inside the database as objects.

For example, you might have a stored procedure called usp_deploy
that populates all of the necessary configuration tables via insert
statements.

Deploying apps using the Data Tier

So We Got Some Time Left ….

 Filtered indices.
 Resource Governor.
 CDC – change data capture.
 T-sql New Features.

 Aaron@dbconsultant.co.il
 All PPT & samples @
Www.dbconsultant.co.il

Sql Server & PowerShell

Recommended

More Related Content

What's hot (20)

Viewers also liked (8)

Similar to Sql Server & PowerShell (20)

Sql Server & PowerShell