Whats new in Columnstore Indexes for SQL Server 2017
1 like1,100 views
Columnstore indexes in SQL Server 2017 include: 1) Support for LOB columns allowing values up to 2GB to be stored and compressed on or off-row. 2) Ability to define computed columns in tables for the first time, improving data warehouse scenarios. 3) Online rebuilding of nonclustered columnstore indexes for high availability during maintenance. 4) Enhancements to the query optimizer including adaptive memory grants, adaptive join selection, and removal of trivial plans that could prevent batch mode processing.
Whats new in Columnstore Indexes for SQL Server 2017
- 1. Whats new in Columnstore 2017 Niko Neugebauer
- 2. Niko Neugebauer Microsoft Data Platform Professional OH22 (http://www.oh22.net) Data Platform MVP Community Guy Blog: http://www.nikoport.com (over 100 Columnstore blog posts) Twitter: @NikoNeugebauer LinkedIn: http://pt.linkedin.com/in/webcaravela
- 3. Agenda, part I Storage Engine - LOB Support - Computed Columns Index Building - Online Rebuilds Query Optimiser: - Adaptive Query Processing - Query Processing Construction - NCCI Updates - Trivial Plans Query Optimiser Something new
- 4. Agenda, part II Machine Learning Services - Much faster interaction Query Optimiser Something new
- 6. Storage Engine LOB Support One of the less important features for the DWH on the first sight/thought, but in the real life, people do put LOBs in the Relational DB Another possibility is to open Log Tables for the Columnstore Compression. Computed Columns The most missing feature for the DWH, the one which definitely prevented projects migrating to Columnstore Indexes. Was asked/begged/demanded before 2012 release. Unfortunately still incomplete in 2017.
- 7. LOBs - Large objects that require more than 2 MB of the space and can grow a single entry up to 2 GB. - We can have 3 datatypes for the LOBs: - Varchar(MAX) - Nvarchar(MAX) - Varbinary(MAX)
- 8. LOBs - For Clustered Columnstore Indexes there are 3 categories of compression: - 1. Inline, within the Row Group – compressing all strings with the total length below 8000 bytes. All values in this case are stored within the dictionaries. - 2. Compressed Off-Row – compressing values between 8000 bytes and 16 Megabytes. The strings are individually block-compressed. Only the headers/pointers to these off-row LOBs are stored in dictionaries. - 3. Pass-through – containing values above 16 MB, these are simply uncompressed LOBs with headers/pointers stored in the dictionaries.
- 9. Demo!
- 10. Computed Columns - In SQL Server 2017, we can have (finally, finally, finally) the computed columns within our table (Fact Table, for example)! - The other news: the non-persisted computed columns are not supported.
- 11. Demo!
- 12. Index Operations
- 13. Online NCCI Builds/Rebuilds - In SQL Server 2017, we finally get the ONLINE operations for the Nonclustered Columnstore Indexes Build/Rebuild processes. This is the final major blocker for the HTAP scenarios that is removed. Notice: NCCI only
- 16. Query Processor
- 17. Storage Engine Adaptive Query Processing The biggest improvement in QA since forever! Query Processing Construction Small optimisations bringing some major results NCCI Updates Updating your Delta-Stores will be much more efficient Trivial Plans All plans are created equal, but some are more trivial than the others.
- 19. Adaptive Query Optimiser Memory Grant Feedback Did SQL Server granted more memory than the query really needs ? Did SQL Server granted too few memory and your queries are spilling on TempDB. Memory Grant Feedback is here to save the day! Adaptive Joins Sometimes your queries are being executed near the tipping point of choosing between NLP (Loop Joins) and HM (Hash Match). Would it be nice, if we could choose at the run-time?
- 20. Adaptive Memory Grant Feedback After first memory grant the consequitive ones will be adjust UP or DOWN, according to the current needs
- 21. Adaptive Joins The execution plan will provide 2 ways of running the very same execution plan – with HASH MATCH or INNER LOOP JOIN and this will be decided during the pre-execution phase of the query execution. This way adapting to the number of the rows flowing into the Adaptive Join the execution plan can run either way.
- 22. Trivial Plans
- 23. Trivial Plans Trivial Plans can be “not so trivial” for the end-user. If you can’t get the speed you are expecting from the Columnstore Indexes Batch Mode, verify if you have your iterators being executed with the Batch Mode. And for the sake of the sanity, check if you are having the “Trivial Plan” optimisation.
- 24. Trivial Plans In SQL Server 2017 there are no trivial plans for Columnstore Indexes.
- 26. Demo
- 27. Thank you!