![Physical Server
Once you have the I/Os you will also need to understand what
type of RAID volume is hosting your data. The type of RAID will
add additional I/Os.
RAID 1/10 – [Reads + (2*Writes)]/ # of Disks
RAID 5 – [Reads + (4*Writes)]/# of Disks
Generally speaking todays fiber channel SAS disks can support
approximately 150 I/Os per second. Check the manufacturers
specifications for exact details. We tune for 100 to 120 I/Os.](https://image.slidesharecdn.com/hasbngclic530-240715073957-90dacf06/85/hasbngclicvhhhhhjgzitxgkxhcjcjb-I-u-I-u530-pptx-7-320.jpg)
![Virtual Log Fragmentation – How do you find them?
You can use the DBCC Loginfo command. Each line
returned represents a VLF.
Syntax Command
DBCC Loginfo(‘[DB Name]’)
Physical Server](https://image.slidesharecdn.com/hasbngclic530-240715073957-90dacf06/85/hasbngclicvhhhhhjgzitxgkxhcjcjb-I-u-I-u530-pptx-25-320.jpg)
![Virtual File Stats – Getting the 411
Here is a basic query
SELECT mf.name, vf.num_of_reads, vf.io_stall_read_ms,
vf.io_stall_read_ms/vf.num_of_reads AS [read_latency_ms],
vf.num_of_writes, vf.io_stall_write_ms,
vf.io_stall_write_ms/vf.num_of_writes AS [write_latency_ms]
FROM sys.dm_io_virtual_file_stats (NULL,NULL) as vf
JOIN sys.master_files as mf
ON vf.database_id = mf.database_id AND
vf.[file_id] = mf.[file_id]
Physical Server](https://image.slidesharecdn.com/hasbngclic530-240715073957-90dacf06/85/hasbngclicvhhhhhjgzitxgkxhcjcjb-I-u-I-u530-pptx-34-320.jpg)
hasbngclicvhhhhhjgzitxgkxhcjcjb. I u I u530.pptx
- 1. SQL Server 2008 24X7 Maintenance Considerations Drew Flint Plex Systems, Inc.
- 2. Database Maintenance What is the purpose of database maintenance? To ensure consistent, high performance of databases while protecting the integrity of the schema, end user data, and the SQL Server.
- 3. Database Maintenance Physical Server • Disk Layout • Virtual Log Fragmentation • Wait Stats • Virtual File Stats Consistency • Indexes • Statistics Protection • Integrity Checks • Backups
- 4. Physical Server Physical Disk Layout Considerations How many spindles are enough to support the I/O workload? How to distribute the data files onto a disk? Selecting the correct block size. Should you sector align the disk?
- 5. Physical Server How many spindles are enough to support the I/O workload? Ideally, you should be able to answer the following questions: How many I/Os are going to the disk? How long is it taking?
- 6. Physical Server To answer the How Many I/Os question use disk performance counters from the last several weeks. I prefer to use the physical disk because that captures I/O information for all partitions on the drive. Physical Disk Disk Reads/Sec Disk Writes/Sec Disk Transfers/Sec
- 7. Physical Server Once you have the I/Os you will also need to understand what type of RAID volume is hosting your data. The type of RAID will add additional I/Os. RAID 1/10 – [Reads + (2*Writes)]/ # of Disks RAID 5 – [Reads + (4*Writes)]/# of Disks Generally speaking todays fiber channel SAS disks can support approximately 150 I/Os per second. Check the manufacturers specifications for exact details. We tune for 100 to 120 I/Os.
- 8. Physical Server To answer the How Long question once again use the disk performance counters from the last several weeks. I prefer to use the physical disk because that captures I/O information for all partitions on the drive. Physical Disk Average Disk Sec/Reads Average Disk Sec/Writes Average Disk Sec/Transfers Ideally all transfers to .mdf/.ndf files should be less than 10ms and all transfers to .ldf files should be 1 to 3ms.
- 9. Physical Server Where to locate database files? Microsoft’s best practices recommend separating the main data file from the log data file onto physically separate spindles.
- 10. Physical Server What block size should the disk be formatted with? The official answer is it depends on the type of workload running on the server. In general a 64K block size has been found to provide a consistently high level of throughput for the majority of workloads. SQL Server scales the size of the read or write to the workload it’s processing. For example backups can be written in larger chunks than a log file write.
- 11. Physical Server Should you sector align your disk? Yes, you should. This will only increase your performance to the back end disk. There is some debate as to how much it will help. Here is an article written by the SQLCAT team on alignment. http://msdn.microsoft.com/en-us/library/dd758814(v=sql.100).aspx
- 12. Physical Server Here is a graph of performance improvement from Microsoft’s white paper:
- 13. Physical Server As a contrast, on the next slide are some results from an experiment performed by Linchi Shea.
- 15. Physical Server Be aware the impact of a configuration change varies with the type of workload. MAINTENANCE FIRST PRINCIPLE You should only make changes if you can show they make improvements in your environment.
- 16. Physical Server Sector Alignment Most likely, if you’re using Windows 2008 server OS or higher you are most likely sector aligned. The setting is configurable and can be controlled through the registry at: HKLMSYSTEMCurrentControlSetServicesVDSAlignment
- 17. Physical Server Sector Alignment If you’re running on an OS earlier than 2008 your disk configuration isn’t aligned by default. If you’ve moved disks from an earlier OS to Windows 2008 they won’t automatically be fixed.
- 18. Physical Server Sector Alignment You’ll have to take different approaches to find out if your disk is sector aligned. If you have basic partitions then you can use the WMI console. If you have dynamic partitions, you need to use dmddiag.exe (Windows 2003) or diskdiag.exe (Windows 2008). If you have a basic, MBR partition you can use diskpart.
- 19. Physical Server Sector Alignment Basic MBR Partition Example Commands Diskpart Select disk 0 List partition
- 20. Physical Server Sector Alignment Basic MBR Partition Example Sample Output
- 21. Virtual Log Fragmentation This can cause slowing of inserts, updates, and deletes. Further, it can cause your recovery times to skyrocket. There is a case of a log file having 1.6 million fragments taking about 30 hours to complete recovery. Physical Server
- 22. Virtual Log Fragmentation Here is the breakdown on how the log file grows. • < 64MB = 4 VLFs • > = 64MB and < 1GB = 8 VLFs • > = 1GB = 16 VLFs Physical Server
- 23. Virtual Log Fragmentation – Example growth scenario Log file assumptions: The base log file size is 1MB with a default file growth of 10%. • The first growth would be 102KB and contain 4 VLFs. The log file size would be 1.1MB. • The next file growth would be 110KB and contain 4 VLFs. The log file size would be 1.2MB. Physical Server
- 24. Virtual Log Fragmentation – How many? “It depends” is the answer. You will have to experiment to find the right number for your environment. Generally speaking you should strive for no more than 200. Physical Server
- 25. Virtual Log Fragmentation – How do you find them? You can use the DBCC Loginfo command. Each line returned represents a VLF. Syntax Command DBCC Loginfo(‘[DB Name]’) Physical Server
- 26. Virtual Log Fragmentation – How do you fix it? • Backup the log file. • Shrink the log file to the smallest size possible. • Grow the log file to the right size for your environment in chunks. One big growth could be just as bad as too many small ones. Physical Server
- 27. Wait Stats – What are they? SQL Server’s method of letting us know what a given SQL Server instance is waiting on when trying to complete its workload. This information is invaluable when trying to determine where best to invest your troubleshooting time. Physical Server
- 28. Wait Stats – How do I see them? They are stored in the SQL DMV called sys.dm_os_wait_stats. A very basic syntax to retrieve what SQL Server is waiting on at that exact point in time is: SELECT * FROM sys.dm_os_wait_stats ORDER BY wait_time_ms DESC Physical Server
- 29. Wait Stats – How do I use them? Wait stats are compiled from the time the SQL Server instance is started. In order to get meaningful point-in-time information, you will need to gather two point in time images and calculate the difference. Physical Server
- 30. Wait Stats – How do I use them? When wait stats are gathered over time they provide a starting point for continual improvement for your SQL Server. The list of wait stats and what they are decoded as can be found in books online at: http://msdn.microsoft.com/en- us/library/ms179984.aspx Physical Server
- 31. Virtual File Stats Virtual file stats measure I/O information perspective from SQL Server’s side. It details such information as how many reads, writes, how much data was read or written, and whether or not it encountered any stalls while completing them. Physical Server
- 32. Virtual File Stats Armed with this information, you can determine the latency, from SQL Server’s perspective, to access a given database file. Using these, combined with perfmon physical disk counters, will enable you to know which files to separate when disk performance issues are encountered. Physical Server
- 33. Virtual File Stats – Getting the 411 The data contained in this DMV is a point in time snapshot compiled from when the SQL Server instance is restarted. The best way to get performance data over time is to take the delta between two point-in-time snapshots. There are many samples of how to do this on the internet. Below is a link to T-SQL scripts for 2000, and 2005/2008. http://www.sql-server-pro.com/virtual-file-stats.html Physical Server
- 34. Virtual File Stats – Getting the 411 Here is a basic query SELECT mf.name, vf.num_of_reads, vf.io_stall_read_ms, vf.io_stall_read_ms/vf.num_of_reads AS [read_latency_ms], vf.num_of_writes, vf.io_stall_write_ms, vf.io_stall_write_ms/vf.num_of_writes AS [write_latency_ms] FROM sys.dm_io_virtual_file_stats (NULL,NULL) as vf JOIN sys.master_files as mf ON vf.database_id = mf.database_id AND vf.[file_id] = mf.[file_id] Physical Server
- 35. Virtual File Stats – Getting the 411 Here is some sample output Physical Server name num_of_reads io_stall_read_ms read_latency_ms num_of_writes io_stall_write_ms write_latency_ms master 49 620 12 77 628 8 mastlog 10 42 4 115 948 8 tempdev 26 266 10 34 106 3 templog 8 6 0 69 358 5 modeldev 61 465 7 4 5 1 modellog 6 7 1 14 31 2 MSDBData 164 1507 9 3499 23955 6 MSDBLog 24 12 0 5265 9365 1
- 36. These are maintenance tasks that help SQL Server keep performance at a predictably high level. Index Maintenance Statistics Maintenance Consistency
- 37. Index Maintenance We all know that indexes are a way to access data more efficiently. Over time normal use of the system will cause fragmentation in the indexes and makes them less efficient. If we don’t maintain them, system performance will degrade. Consistency
- 38. Index Maintenance We also know that there are two approaches: ALTER INDEX REORGANIZE or REBUILD. Let’s discuss the right time to use each one. Consistency
- 39. Index Maintenance The REORGANIZE option is a fully online process that offers a best effort optimization process. If a page is in use, it will be skipped by the optimization process. This means that after this process is complete, the index might still have some fragmentation. There is some overhead incurred while this process is running but for the most part you can run it safely anytime. Consistency
- 40. Index Maintenance Something else to keep in mind when using the REORGANIZE option of ALTER INDEX is that the statistics for the index aren’t rebuilt. You will have to put your own process into place to maintain those. The REORGANIZE process is partition aware so you can choose to maintain only the partitions necessary or the whole index. Consistency
- 41. Index Maintenance The REBUILD process is a process that removes all fragmentation possible from the index being maintained. This is an intrusive process for those of us not cool enough to have Enterprise Edition of SQL. The index will be inaccessible for the duration of time necessary for the index maintenance to complete. However, if you do have Enterprise Edition, you can use the magical WITH ONLINE parameter. Consistency
- 42. Index Maintenance When using the WITH ONLINE parameter there is only a brief lock at the beginning and end of the maintenance process. However, the WITH ONLINE parameter can only be used on the whole index not on an individual partition. The REBUILD process can be focused to a single partition. The partition will be locked for the duration of the maintenance. Consistency
- 43. Index Maintenance The REBUILD process should be scheduled for periods of lower utilization as it is more intrusive. The REBUILD process will also update the statistics for the index being maintained. Consistency
- 44. Index Maintenance Both types of index maintenance can cause log file growth. Index maintenance can and should be automated. There is software for sale you can implement for this. It’s also straightforward to build your own. However, it will take much more time. Consistency
- 45. Index Maintenance Here are some key points to consider when building your own process: • Use the maintenance process to enforce your maintenance schedule. • How long the maintenance process can run. • What types of maintenance can run on which day. • Keep historical fragmentation information. • This allows you to measure the effectiveness of the maintenance process. Consistency
- 46. Index Maintenance Here are some key points to consider when building your own process: • The maintenance process is a living program. There are going to be changes that will have to be made over the lifecycle of the process. • If the analysis process starts to impact performance of the production server run the analysis on another server. Consistency
- 47. Why are statistics important? Statistics are used by the query optimizer to determine the most efficient method to update or retrieve data from a database. Consistency
- 48. When are statistics created? • As part of the CREATE INDEX process. • When you manually create a statistic with the CREATE STATISTIC command. • If automatic creation/updating of statistics are enabled SQL Server can create single column statistics when the query optimizer determines they are needed. Consistency
- 49. Find information about statistics sys.stats – This has information about the statistic in general. • How it was created Automatically vs. User • If it should be automatically updated no_recompute = 0 Sample syntax: select * from sys.stats Consistency
- 50. Find information about statistics sp_helpstats – This helps you know the columns (statistic_key) that make up a statistic (statistic_key). Statistics that begin _WA are created automatically by SQL Server. Sample syntax: exec sp_helpstats ‘customer’, ALL Consistency
- 51. When are statistics updated? Automatically when: • When the row count becomes > 0. • When the row count becomes > 500. • When the colmodctr has changed by more than 500 + 20% of the rows in the table. Consistency
- 52. When are statistics updated? Manually when: • UPDATE STATISTICS is executed • Used to manually update statistics for a single index or a whole table, depending on how it’s executed. • sp_updatestats • Automatically updates all statistics for all user defined and system tables. Consistency
- 53. Statistic Tips • Leave Auto Create and Auto Update Statistics enabled. This will ensure the statistics are updated periodically. • Strategically change the sample size used by UPDATE STATISTICS. A larger sample size gives the query optimizer more accurate data to use when creating a new plan. Consistency
- 54. Statistic Tips Update statistics more frequently for ascending keys. This is because all new values lie outside the currently generated statistics. Consistency
- 55. These are things that database administrators should be doing to guarantee their databases survive problems. Protection
- 56. There are two different major things database administrators should be thinking about: Backups Integrity Checks Protection
- 57. Backups We’re going to take a different approach to talking about backups. Instead of talking about what the different type of backups are and how you should mesh them together to form a complete backup plan, tonight we’re going to talk about what comes after the backup plan. Protection
- 58. Backups So what comes after the backup plan? What, as administrators, are we obligated to do? • Test restoring the backups? • Managing your backup history? • Getting the backups off site? Protection
- 59. Backups Test restores This operation is time consuming and takes a lot of disk space to complete. In order to alleviate this burden there is a “lite” option that you can execute. It’s an option for the restore command called VERIFYONLY. It checks to make sure the backup set is complete and the entire backup is readable. Protection
- 60. Backups Backup History The backup history table needs to be manually maintained. It’s added to every time a backup is completed. The command is called sp_delete_backuphistory. Protection
- 61. Backups Offsite Backups We need to constantly be thinking about how we can go further to protect our end user data. We need to consider the steps we need to take to protect our data should something happen to our primary facility. Protection
- 62. Integrity Checks These give us a point in time guarantee that the state of our database is good. It says as of this point in time everything about me is good. As with anything else you’re only as good as your last success. Nirvana would be to continually run integrity checks. However, the overhead for running these checks will impact normal business operations. Protection
- 63. Integrity Checks How to execute them? An integrity check can be performed by executing the following command: DBCC CHECKDB ( Database Name, Options) Protection
- 64. Integrity Checks What does it do? An integrity check verifies the following information about the database being checked: • DBCC CHECKALLOC (Disk space allocation) • DBCC CHECKTABLE (On every table) • DBCC CHECKCATALOG (Verifies MetaData) Protection
- 65. Integrity Checks Operational tricks Run the checks on another server with recently restored backups. Obviously, you need to have the disk space to do this. If you don’t have disk space but have a SAN try taking a snapshot and attaching it to another server to facilitate the integrity checks. This does create some overhead on the source volume and might impact normal business operations. Protection
- 66. Integrity Checks Operational tricks If you only have a very limited amount of time to run your checks, you can use the option WITH PHYSICAL_ONLY. This only checks the integrity of the physical structure of the page and record headers. It doesn’t perform any logical checking. Protection
- 67. Integrity Checks Operational tricks Another option that may reduce execution time is NOINDEX. This tells the DBCC engine to not check the non-clustered indexes. Protection
- 68. Integrity Checks Operational tricks Run a combination of checks if you can’t consistently run a full DBCC check. The idea is it’s better to be checking something than to miss something potentially career altering like data corruption. Protection
- 69. • Statistics used by the Query Optimizer in Microsoft SQL Server 2008. By: Lubor Kollar, February 2009. • SQL Server 2008 Internals. By: Kalen Delaney • SQL Skills web site: www.sqlskills.com • SQLCAT Team web site: www.sqlcat.com • Linchi Shea’s Blog: http://sqlblog.com/blogs/linchi_shea/default.aspx Technical Credits