Operational foundation for the sql server dba
- 1. Operational Foundation for the SQL Server DBA Peter Shore
- 2. About Me ▪ SQL Server Database Administrator – ThirtyOne Gifts ▪ Intentionally Accidental DBA ▪ Over 25 years IT experience – Server Engineer – Desktop Engineer – Network Infrastructure – Desk side support ▪ President CBusPASS ▪ Co-Organizer SQL Saturday Columbus ▪ How to find me – Twitter: @pshore73 – E-mail: [email protected]
- 4. What is Microsoft SQL Server? ▪ SQL Server is Microsoft’s Relational Database Management System (RDBMS) ▪ Relational database – digital database based on the relational model proposed by E.F. Codd in 1970* ▪ SQL is ANSI/ISO standard ▪ Transact SQL (T-SQL) Microsoft Adaptation of the standard ▪ *https://en.wikipedia.org/wiki/Relational_database ▪ https://www.techopedia.com/definition/24559/relational-model-database
- 5. A.C.I.D. ▪ Atomicity – No partial changes, all or nothing – Any part of transaction fails all parts fail ▪ Consistent – Only valid data is written to the database – Transactions violating consistency rules will be entirely rolled back – Completed transaction takes database from one consistent state to another ▪ Isolation – Multiple simultaneous transactions do not impact each other – Transactions will not read intermediate data – Not ordinal just isolated ▪ Durability – Committed transactions will not be lost – Backups and transactions logs facilitate restoration regardless of software or hardware failure
- 6. SQL Server System Databases System database Description master Database Records all the system-level information for an instance of SQL Server. msdb Database Is used by SQL Server Agent for scheduling alerts and jobs. model Database Is used as the template for all databases created on the instance of SQL Server. Modifications made to the model database, such as database size, collation, recovery model, and other database options, are applied to any databases created afterward. Resource Database Is a read-only database that contains system objects that are included with SQL Server. System objects are physically persisted in the Resource database, but they logically appear in the sys schema of every database. tempdb Database Is a workspace for holding temporary objects or intermediate result sets.
- 7. SQL Server System Databases ▪ Master – Instance wide metadata ▪ Login accounts ▪ Endpoints ▪ Linked Servers ▪ System Configuration Settings ▪ Existence of other databases ▪ File location for other databases – Cannot ▪ Add files or file groups ▪ Change collation ▪ Drop database https://docs.microsoft.com/en-us/sql/relational- databases/databases/master-database?view=sql-server-2017 ▪ MSDB – Used by SQL Agent ▪ Jobs schedules ▪ Alerts ▪ Maintains backup history – Cannot ▪ Change collation ▪ Drop database ▪ Mirror ▪ Remove primary file/filegroup https://docs.microsoft.com/en-us/sql/relational- databases/databases/msdb-database?view=sql-server-2017
- 8. SQL Server System Databases ▪ Model ▪ Template for all new databases ▪ Used if no additional information is specified in CREATE ▪ Some settings used forTempDB on startup ▪ Must exist – Cannot ▪ Change collation ▪ Drop database ▪ Mirror ▪ Remove primary file/filegroup https://docs.microsoft.com/en-us/sql/relational- databases/databases/model-database?view=sql-server-2017 ▪ Resource – Not visible in SQL Server Management Studio (SSMS) – Read-Only – Holds all system objects – Cannot be backed up via SQL – Backup can be taken as you would a file – Should not be moved https://docs.microsoft.com/en-us/sql/relational- databases/databases/resource-database?view=sql-server-2017
- 9. SQL Server System Databases ▪ TempDB – Hold temporary objects that are explicitly created ▪ TempTables ▪ Temp Indexes ▪ Temp stored procs ▪ Temp variables ▪ Tables returned in table valued functions or cursors – Holds internal system objects ▪ Work tables to store intermediate results for spools, cursors, sorts, and temporary large object (LOB) storage. ▪ Work files for hash join or hash aggregate operations. ▪ Intermediate sort results for operations such as creating or rebuilding indexes (if SORT_IN_TEMPDB is specified), or certain GROUP BY, ORDER BY, or UNION queries.
- 10. SQL Server System Databases ▪ TempDB – Hold sVersion Stores ▪ Collection of data pages holding data rows required to support features which use row versioning ▪ Common version store and on-line index build version store ▪ Version stores contain – Row versions that are generated by data modification transactions in a database that uses read-committed using row versioning isolation or snapshot isolation transactions. – Row versions that are generated by data modification transactions for features, such as: online index operations, Multiple Active Result Sets (MARS), and AFTER triggers. https://docs.microsoft.com/en-us/sql/relational-databases/databases/tempdb-database?view=sql-server-2017
- 11. SQL Server System Databases ▪ TempDB – Best practices ▪ For every logical processor have one TempDB file up to eight (8) ▪ When logical processor is greater than eight start with eight files ▪ Add files beyond eight if there is contention ▪ Add files in groups of four (4) ▪ Size files equally ▪ Set growth to the same increment https://support.microsoft.com/en-us/help/2154845/recommendations-to-reduce-allocation-contention-in-sql-server-tempdb-d
- 12. Database Files ▪ Data files – Storage for the data – Tend to be random access – Extension MDF or NDF ▪ MDF – Main Data File ▪ NDF – SecoNdary Data File – MDF is required – NDF optional – File Groups – Zero writing not required ▪ Transaction Log – ACID compliance – Sequential Read/Write – Extension LDF ▪ Log Data File – Must be zero written on-premises
- 13. Pages and Extents ▪ SQL Server data on disk and in RAM is stored in 8KB pages ▪ Pages are managed in extents – Extents are 8 physically contiguous pages – Extents are 64KB – Uniform extents are owned by the same object and only can be used by that object – Mixed extents are share by up to 8 objects https://docs.microsoft.com/en-us/sql/relational-databases/pages-and-extents-architecture-guide?view=sql-server-2017
- 18. Virtual Log Files VLF1 ACTIVE VLF2 FREE VLF3 FREE VLF4 FREE Transaction 1 Transaction 2
- 19. Virtual Log Files https://social.technet.microsoft.com/wiki/contents/articles/51379.sql-server-transaction-log-internal-architecture-facts-every-sql-server- database-professional-should-know.aspx VLF1 ACTIVE VLF2 FREE VLF3 FREE VLF4 FREE Transaction 1 Transaction 2 Transaction 3
- 20. SQL Agent ▪ SQL Server’s task scheduler ▪ Jobs – Tasks ▪ TSQL ▪ Command line ▪ SSIS ▪ Maintenance plans – Schedules ▪ Alerts – Operators https://docs.microsoft.com/en-us/sql/ssms/agent/sql-server-agent?view=sql-server-2017
- 21. Database Mail ▪ Reliable, Scalable, Secure, Supportable ▪ Send e-mail from within SQL Server https://docs.microsoft.com/en-us/sql/relational- databases/database-mail/database-mail?view=sql-server-2017
- 22. Restore Strategy
- 23. Why a recovery strategy? ▪ On-line nature of SQL Server ▪ Focus on RecoveryTime and Recovery Point ▪ RecoveryTime Objective (RTO) – “How long can we be down?” – Controlled by the business/application owner ▪ Recovery Point Objective (RPO) – “How much data can we lose?” ▪ Time not size – Controlled by the business/application owner
- 24. Recovery Models ▪ Simple – No log backups – No point in time recovery – Operations which require log backups cannot be used on databases in Simple ▪ Log Shipping ▪ Availability Groups ▪ Database Mirroring – Only recover to last full/differential backup – SQL Server manages log truncation
- 25. Recovery Models ▪ Bulk logged – Requires log backups – Adjunct to full recovery but permits high performance bulk copy operations – Most bulk operations are minimally logged and use less disk space https://docs.microsoft.com/en-us/sql/relational-databases/import-export/prerequisites-for-minimal-logging-in-bulk-import?view=sql- server-2017
- 26. Backup Types ▪ Full – Complete data backup – Contains enough of the log to allow for recovery of that data ▪ Differential – Backup of all data changed since the last FULL backup ▪ Log – Backup of the transaction log – Includes all records not previously backed up ▪ Copy Only – Full backup that does not affect log sequence https://docs.microsoft.com/en-us/sql/relational-databases/backup-restore/backup-overview-sql-server?view=sql-server-2017
- 27. Backup Types – Keeping Files in Order ▪ Full – Complete data backup – Contains enough of the log to allow for recovery of that data ▪ Differential – Backup of all data changed since the last FULL backup ▪ Log – Backup of the transaction log – Includes all records not previously backed up ▪ Copy Only – Full backup that does not affect log sequence
- 28. The How of Backups ▪ Schedule ▪ Maintenance Plans ▪ Roll your own ▪ FreeTools ▪ PaidTools ▪ Who owns backup? ▪ Test Restores – GivesTiming for restore – Not done without Integrity Check
- 30. Preparing to install SQL Server ▪ Windows Configuration – Give SQL Server Engine account permissions ▪ Lock Pages In Memory ▪ PerformVolume Maintenance (Instant File Initialization aka IFI) – ConfirmTCP Chimney Offload is set to off ▪ Storage Configuration – Follow vendor documentation for SQL Server – Allocation Unit Size 64Kb ▪ Service Accounts – Windows accounts – Separate for each service & server ▪ VM configuration – VMware follow document https://www.vmware.com/content/dam/digitalmarketing/vmware/en/pdf/solutions/sql-server-on-vmware-best-practices-guide.pdf
- 31. Beyond Next Next Next ▪ Screenshots
- 32. Beyond Next Next Next ▪ Post install configuration ▪ Max Memory ▪ Ad Hoc ▪ MaxDOP ▪ CTP ▪ Alerts
- 34. What to monitor? ▪ CPU – Overall – By socket – By Core ▪ Memory – Page Life Expectency – Memory Grants Pending ▪ Disk – IOPs – Queue depth ▪ Queries – Long running – Blocking – Deadlocking – WAITS ▪ Misc. – Up/Down of server and services – Job failures
- 35. Where to monitor from? ▪ Windows Performance Counters – Instance specific – Grouped by function – Can be collected ▪ Dynamic ManagementViews and Dynamic Management Functions (DMV and DMF) – Collectively called DMVs – https://www.sqlskills.com/blogs/gl enn/category/dmv-queries/
- 36. How to monitor? ▪ Build – Exactly what you want ▪ Details ▪ Reports – You must maintain – You must test with new SQL versions ▪ Buy – Vendor handles the maintaining and version updates – May or may not give exactly what you want ▪ Hybrid – Vendor solution primary – Use DMVs/roll your own process for deeper dive or specific details
- 37. Baseline ▪ Shows what normal is ▪ Easy to see anomalies ▪ Easy to see results of changes ▪ Critical for monitoring and tuning
- 38. Details on bad things ▪ SQL Server Logs ▪ Windows Event Logs ▪ Extended Events
- 41. ZEN and the Art of SQL Maintenance
- 42. Maintenance ▪ Backups ▪ Integrity Check – DBCC CHECKDB – Explain what it does – Link to Paul’s article ▪ Index