The roadmap for sql server 2019
Download as PPTX, PDF•2 likes•459 views
This document discusses SQL Server 2019 and provides the following information: 1. It introduces Javier Villegas, a technical speaker and SQL Server expert. 2. It outlines several new capabilities in SQL Server 2019 including artificial intelligence, container support, and big data analytics capabilities using Apache Spark. 3. It compares editions and capabilities of SQL Server on Windows and Linux and notes they are largely the same.
The roadmap for sql server 2019
- 1. Inside SQL Server 2019 The Roadmap for SQL Server
- 2. Technical Speaker NetConf , SQL PASS, 24 HOP, SQL Saturdays and PASS Virtual Groups GroupBy and DataPlatformGeeks @sqlargentina Javier Villegas DBA Manager at Mediterranean Shipping Company Involved with the Microsoft SQL Server since early versions Specialization in SQL Server Administration, Performance Tuning and High Availability Microsoft MVP Data Platform MCP and MCTS Blogger and MSDN Forums contributor @javier_vill /javiervillegas sql-javier-villegas.blogspot.com.ar [email protected]
- 3. Insights in minutes and rich reports Choice of platform and language Most secure over the last 8 years5 0 20 40 60 80 100 120 140 160 180 200 Vulnerabilities(2010-2017) The best of Power BI and SQL Server Reporting Services in Power BI Report Server Industry-leading performance #1 OLTP performance1 #1 DW performance on 1TB2, 10TB3, and 30TB4 Private cloud Public cloud Most consistent data platformIn-memory across all workloads 1/10th the cost of Oracle T-SQL Java C/C++ SCALA Node.js C#/VB.NET Python Ruby .NET core All TPC Claims as of 1/19/2018. 1 http://www.tpc.org/4081; 2 http://www.tpc.org/3331; 3 http://www.tpc.org/3326; 4 http://www.tpc.org/3321; 5 National Institute of Standards and Technology Comprehensive Vulnerability Database Intelligence over any data Analytics over structured and unstructured data with the power of SQL and Apache Spark Now with big data clusters Modernize on premises SQL
- 4. Build intelligent apps and AI with all your data Analyzing all data Easily and securely manage data big and small Managing all data Simplified management and analysis through a unified deployment, governance, and tooling Unified access to all your data with unparalleled performance Integrating all data
- 5. Including Relational, noSQL, Hadoop Using Apache Spark and SQL With Azure Data Studio
- 6. Managed SQL Server, Spark, and data lake Store high volume data in a data lake and access it easily using either SQL or Spark Management services, admin portal, and integrated security make it all easy to manage SQL Server Data virtualization Combine data from many sources without moving or replicating it Scale out compute and caching to boost performance T-SQL Analytics Apps Open database connectivity NoSQL Relational databases HDFS Complete AI platform Easily feed integrated data from many sources to your model training Ingest and prep data and then train, store, and operationalize your models all in one system SQL Server External Tables Compute pools and data pools Spark Scalable, shared storage (HDFS) External data sources Admin portal and management services Integrated AD-based security SQL Server ML Services Spark & Spark ML HDFS REST API containers for models
- 7. Compute pool SQL Compute Node SQL Compute Node SQL Compute Node … Compute pool SQL Compute Node IoT data Directly read from HDFS Persistent storage … Storage pool SQL Server Spark HDFS Data Node SQL Server Spark HDFS Data Node SQL Server Spark HDFS Data Node Kubernetes pod Analytics Custom apps BI SQL Server master instance Node Node Node Node Node Node Node SQL Data mart SQL Data Node SQL Data Node Compute pool SQL Compute Node Storage Storage
- 8. Security AvailabilityPerformance High availability for business critical workloads Data protected at rest and in motion Breakthrough performance and scalability
- 9. The intelligent database The Intelligent Query Processing feature family Intelligent QP Adaptive QP Adaptive Joins Batch Mode Interleaved Execution Memory Grant Feedback Row ModeBatch Mode Table Variable Deferred Compilation Approximate QP Approximate Count Distinct Batch Mode for Row Store Scalar UDF inlining
- 10. SQL Server 2016 SP1/2017 • Trace flag 7412 • XEvent query_thread_profile SQL Server 2019 • On by default. No knobs required dm_exec_query_statistics_xml
- 12. Solution • Create multiple files to partition allocation pages • SQL Server 2017+ creates multiple tempdb files during setup • Start with 8 and add by 4 until concurrency alleviated • But…what about system tables? The SQL 2019 Solution • Key tempdb system tables become SCHEMA_ONLY memory optimized tables • Latch and lock free • Turn on with ALTER SERVER CONFIGURATION • This is NOT user data just metadata so memory requirements small
- 14. Confidential computing plaintext ciphertext Enhanced client driver Enclave plaintext Always Encrypted with secure enclaves
- 15. ADD SENSITIVITY CLASSIFICATION TO dbo.sales.price, dbo.sales.discount WITH ( LABEL='Highly Confidential', INFORMATION_TYPE='Financial' ) data_sensitivity_information Who, what, and when accessed my classified data?
- 18. Availability groups on Kubernetes Keep SQL Server running AG Pod Operator Pod SQL Server primary AG agent Load balancer Pod SQL Server secondary AG agent Pod SQL Server secondary AG agent SQL Server primary SQL Server secondary Load balancer
- 19. Online index create and rebuild This has been around for a while Resumable online rebuild SQL Server 2017 Pause and Resume Incremental rebuild Resumable online index create SQL Server 2019 Online Non-clustered Columnstore create and rebuild SQL Server 2017 Online Clustered Columnstore create and rebuild SQL Server 2019
- 21. No clustering software required Replica support built into the Engine Load balancing of readable secondaries HA and DR for Linux and Windows Manual failover
- 22. Parallel redo can help Undo dependent on oldest active transaction “Long” recovery usually due to long undo
- 24. Speed app development and admin with new enhancements Develop on your choice of language and tooling
- 25. SQL Server R
- 27. SQL Server on the platform of your choice Support for RedHat Enterprise Linux (RHEL), Ubuntu, and SUSE Enterprise Linux (SLES) Linux and Windows Docker containers Windows Server / Windows 10 Package-based installation: Yum Install, Apt-Get, and Zypper Database Compatibility on all Platforms
- 28. 11M+ Docker Pulls Thousands of instances and CPUs using SQL Server on Linux Is it the same as Windows? Is the performance the same as on Windows? Is the licensing the same as on Windows? Do I need to know Linux? Is SQL Server different in a container? Need multi-instance? Use containers
- 29. Windows Linux Editions Developer, Express, Web, Standard, Enterprise Services Database Engine, Integration Services, SQL Server Agent Analysis Services, Reporting Services, MDS, DQS Mission critical performance and HADR Maximum number of cores Unlimited Unlimited Maximum memory utilized per instance 24 TB 12 TB Maximum database size 524 PB 524 PB Basic OLTP (Basic In-Memory OLTP, Basic operational analytics) Advanced OLTP (Advanced In-Memory OLTP, Advanced operational analytics, adaptive query processing) SQL Server Replication Basic high availability (2-node single database failover, non-readable secondary) Advanced HA (Always On - multi-node, multi-db failover, readable secondaries) Security Basic security (Basic auditing, Row-level security, Data masking, Always Encrypted, Active Directory Authentication) Advanced security (Transparent Data Encryption) Data warehousing PolyBase Planned Basic data warehousing/data marts (Basic In-Memory ColumnStore, Partitioning, Compression) Advanced data warehousing (Advanced In-Memory ColumnStore) Advanced data integration (Fuzzy grouping and look ups) Tools Windows ecosystem: Full-fidelity Management & Dev Tool (SSMS & SSDT), command line tools Linux/OSX/Windows ecosystem: Dev tools (VS Code), DB Admin GUI tool, command line tools Developer Programmability (T-SQL, CLR, Data Types, JSON, Graph) Distributed Transactions Machine Learning Services NEW NEW NEW
- 30. SQL Platform Abstraction Layer (SQLPAL) RDBMS IS AS RS Windows Linux Windows Host Ext. Linux Host Extension SQL Platform Abstraction Layer (SQLPAL) Host extension mapping to OS system calls (IO, Memory, CPU scheduling) Win32-like APIsSQL OS API SQL OS v2 All other systems System resource & latency sensitive code paths Choice across OS and containers
- 31. Light and fast using package managers RHEL: RPM and yum. Ubuntu: Debian and apt-get SUSE: RPM and zypper. Dependency management built-in Install fast configure later Break up the product into packages Offline, unattended, update, rollback, remove mssql-server mssql-tools mssql-server- fts mssql-server- ha mssql-server-is “extensibility, Java, and ML Services”
- 32. Inside the Engine ALTER SERVER CONFIGURATION sp_configure Outside the Engine Environment variables during setup mssql-conf script after setup Package for each CU and GDR Packager managers support update and rollback No more service packs
- 33. Azure Data Studio Visual Studio Code mssql extension Native OS command line tools • sqlcmd, bcp, sqlpackage mssql-cli mssql-scripter Windows SQL Server Management Studio (SSMS) Windows SQL Server Data Tools (SSDT) 3rd party tools continue to work Existing drivers/frameworks supported Tools and programmability
- 34. Pacemaker and Corosync Single SQL Server instance SQL resource agent (mssql-server-ha) Open-source resource agent is available on GitHub Repository1 Shared Storage ISCI, NFS, SMB Failover Cluster Instance CLUSTER_TYPE = EXTERNAL Auto failover Pacemaker SQL resource agent (mssql-server-ha) 3 replicas required Configuration only replica for metadata Full AG capabilities High Availability Group CLUSTER_TYPE=NONE No clustering required Manual or forced failover Sync or async replicas Read scale routing Cross-platform Read-Scale Availability Group Industry-leading solutions available from HA/DR partners2
- 36. String or binary data would be truncated String or binary data would be truncated in table '%.*ls', column '%.*ls’. Truncated value: '%.*ls' SELECT page_info.* FROM sys.dm_exec_requests AS d CROSS APPLY sys.fn_PageResCracker(d.page_resource) AS r CROSS APPLY sys.dm_db_page_info(r.db_id, r.file_id, r.page_id,'DETAILED') AS page_info;
- 37. Azure Data Studio is a lightweight, open source, cross-platform graphical management tool and code editor Enable a modern DevOps experience for database developers and DBAs on their platform of choice Simplify development, configuration, management, monitoring and troubleshooting for SQL databases on-premises and in the cloud Use SQL Server Management Studio 18.0 Preview to access, configure, manage, and administer all SQL Server components
- 39. Questions ? Inside SQL Server 2019 @javier_vill /javiervillegas sql-javier-villegas.blogspot.com.ar [email protected]
- 40. Thank you !!
- 41. Current support level End mainstream End extended SQL Server 2014 Currently supporting all versions July 9, 2019 July 9, 2024 SQL Server 2012 SQL Server 2012 SP2+ is in mainstream support until CY 2017 July 11, 2017 July 12, 2022 SQL Server 2008 and SQL Server 2008 R2 SQL Server 2008 and 2008 R2 are in extended support which includes security updates, paid support, and requires purchasing non-security hotfix support July 8, 2014 July 9, 2019 SQL Server 2005 SQL Server 2005 support ended on April 12, 2016 April 12, 2011 April 12, 2016 Learn more about the SQL Server support lifecycle: support.microsoft.com/lifecycle/ SQL Server 2008 and 2008 R2 will no longer be supported starting on July 9, 2019. S Q L S E R V E R E N D O F S U P P O RT
- 42. Azure Data Modernization Choices SQL SQL
Editor's Notes
- #4: That’s not all it can do. SQL Server 2017 continues to deliver industry-leading capabilities: Our latest performance benchmarks on Windows and Linux blow away our old records. OLTP – We have #1 OLTP TPC-E performance DW: And, we have the fastest performing DW. With best price/performance. We offer the most secure database. According to US National Institute of Standards and Technology (NSIT), we have had fewer vulnerabilities over the last 7 years than Oracle or IBM Fewer vulnerabilities mean less patching for you! SQL Server is the first commercial database with Advanced Analytics using R and Python built-in. Why does this matter to you? Now you can use SQL Server to operationalize your data science models in a secure and performant way Use native T-SQL commands to score data in near real-time And unlike our competitors, mobile BI on every device comes built-in. Or add access to powerful, self-service BI visualizations through Power BI - at a fraction of the cost of our competitors. SQL Server 2017 gives you your choice of platform and language, and the most consistent on-prem to cloud environment. And it does all this for 1/10th the cost of Oracle
- #5: Asad starts here
- #6: Data is the new currency. We are constantly seeing a proliferation of data into all segments of our lives in various form factors. Data plays a key part in our lives whether it is hosted in a private data center or a public cloud and even in the devices at the Edge. I currently have various forms of data being exchanged between my cell phone, my home automation devices and even the fitbit that I am wearing. This data EXPLOSION that we are experiencing in every aspect of our lives requires us to be able to converge disparate data sets from different sources whether it is relational, non-relational or even unstructured into a single data hub to gain meaningful insights. As you saw from our announcements yesterday, we are introducing a unified data platform that will be able to talk to all data within your data to allow you to derive valuable business insights from your data estate. This allows you to create a data virtualization layer without the need to move data from one service to the other for the purpose of insights. Not only do you get to manage your data from a single place but you also have the ability to query using Apache Spark and T-SQL.
- #8: Amit starts here
- #9: Increase analytics and apps performance with scale out data marts
- #10: Amit: Polybase Oracle Demo
- #11: Bob takes over
- #16: 2 Demos IQP – Deferred Temp Table Compilation LQP – Lightweight Query Profiler
- #17: Amit takes over
- #21: Deploy an Operator The Operator will deploy SQL Server primary and secondary instances with AG Agents for failover health detection You add our database to the AG and we seed it to the secondary replicas Your Primary app connects to a Load Balancer which connects to the Primary replica A reporting app can connect read-only to secondary replicas If the primary goes down, AG agents help auto failover to one of the secondaries The primary load balancer is redirected to the new primary The reporting app is also connected to the new secondary
- #27: Bob takes over
- #30: Bob presents this Choose your platform based on the platform not SQL Server running on that platform.
- #32: Presenter: Bob
- #33: The Platform Abstraction Layer (“PAL”) is what enables SQL Server to run on Linux and Docker. The PAL is used to consolidate OS/platform specific code to enable SQL Server code to become OS agnostic. The SQL Server team set strict requirements to ensure that functionality, performance, and scale were not compromised when deployed to Linux. Part of what makes this possible is the integration of certain parts of MSR’s project Drawbridge. Drawbridge provided an abstraction between the underlying operating system and the application for the purposes of secure containers. Drawbridge was combined with SQL Server OS, which provided memory management, thread scheduling, and IO services, to create SQLPAL. In short, the creation of the PAL allows the same, time-proven core code base for SQL Server to run on new environments such as Docker and Linux – as opposed to porting the Windows code base into multiple operating environments. SQL Server 2017 is not a re-write or a port – it is the same performant, scalable product Microsoft customers have relied upon for years. Speaker note: for more detail, see https://blogs.technet.microsoft.com/dataplatforminsider/2016/12/16/sql-server-on-linux-how-introduction/
- #35: TODO: What will be the upgrade path to SQL 2019 from SQL 2017 on Linux?
- #37: Technically we only support Pacemaker for Linux but we have other partners building HADR solutions for Linux. In addition, we have created an open source project for our resource agent so other vendors can build their own solutions based on it.
- #39: 1 min
- #40: Optional Demo: Page latch info with temp table stress
- #42: Asad takes over