Query handlingbytheserver

Nov 10, 2012Download as PPTX, PDF0 likes782 views

sqlserver.co.il

The document discusses the role of the SQL query optimizer in generating efficient query plans. It describes the optimizer's multi-stage process of parsing the SQL statement, binding objects, optimizing through different search levels, applying logical and physical properties and over 350 rules to simplify and optimize the query tree, and selecting the cheapest plan. It notes challenges like a large number of possible join orders and timeouts during complex optimizations.

OPTIMIZER’S ROLE

• The optimizer should generate an efficient
plan to access the data that we need to
work with.

WHY DO WE NEED IT?

• SQL Statement defines what we are looking
for. It doesn’t define how to get the data.
• The optimizer has to generate a query plan
which tells the server how to get the data.

OPTIMIZER

• The optimizer doesn't necessarily look for
the best query plan.
• The optimizer searches for a good enough
plan.

WORK STAGES (HIGH LEVEL)

• Create a logical plan
• Apply transformation rules
• Apply heuristics
• Select the cheapest one

PARSING

• Checks that the syntax is correct
• Creates a logical tree that represents the
query.

DEMO 1

• See logical tree that was produced by parse
step.

BINDIND

• Gets the logical tree that was produced by
the parsing step
• Makes sure that all objects referenced by
the logical tree exist and that the user can
see them.

OPTIMIZATION

• The most important step. Generates the
query plan that will be executed

TRIVIAL PLAN

• Very simple queries with very simple logical
tree won’t get full optimization
• The optimizer will generate a plan that is
called a trivial plan
• The trivial plan is very cheap to generate
and it won’t be inserted into the plan cache

TRIVIAL PLAN

• If the trivial plan’s cost is more then the
value that is configured as “cost threshold
for parallelism”, the query will get full
optimization
• You can disable the trivial plan by using
trace flag 8757

SIMPLIFICATION

• The OPTIMIZER rewrites the tree to make it
more simple. There are few simplification
methods:
– Constant folding
– Domain simplification
– Contradiction detection

DEMO 2

• Simplification and trivial plan

OPTIMIZATION LEVELS

• If no trivial plan was found, SQL Server
starts optimizing the query.
• SQL Server has 3 stages that are called:
– Search 0 (Transaction processing)
– Search 1 (Quick plan)
– Search 2 (Full optimization)

OPTIMIZATION LEVELS

• Each level has an entry and termination
condition.
• Termination condition can be good enough
plan was found, or to much time passed
• Optimization can begin at a low search step
and get to a higher search step.

PROPERTIES

• Each node in the logical tree has properties
attached to it.
• There are 2 types of properties
– Logical properties (Node cost, output columns,
Type information and nullability, etc)
– Physical properties (Sort order, partition
information etc’)

RULES

• The optimizer has set of more then 350
rules that is using to optimize the query.
• The rules help the optimizer to modify the
logical tree in a way that doesn’t effect the
query’s results
• The rules also dictate the physical
implementation of the logical tree

RULES

• There are four types of rules that can be
used:
– Simplification rules
– Exploration rules
– Implementation rules
– Physical properties enforcement rules

MEMO

• All the trees are stored in memory in a
structure that is called Memo
• Each optimization has its own memo
• A single memo can get to the size of 1.6 GB

OPTIMIZATION PROBLEMS

• On rare occasions there can be a timeout
for the optimization process
• Most times when it happens, it will be on
search 2 stage, and the server will use the
query plan that was produced on search 1
stage.
• Sometime the server stops optimization
because of memory pressure

DMVs

• There are 2 DMVs that can give us
information about the optimizer:
– sys.dm_exec_query_optimizer_info, shows
information about optimization process
– sys.dm_exec_query_transformation_stats, sho
ws information about rules usage

QUSTIONS?

• Additional reading:
– http://sqlblog.com/blogs/paul_white/archive/
2012/04/28/query-optimizer-deep-dive-part-
1.aspx
– http://www.red-
gate.com/community/books/inside-sql-server-
query-optimizer

The document discusses the SQL Server plan cache, which stores and reuses query execution plans. It describes what is stored in the plan cache, how plans are looked up and aged out, and dynamic management views for exploring the plan cache. Methods covered include identifying unoptimized ad hoc queries, parameterizing queries for better plan reuse, and handling cases where plan reuse is not possible like with local variables.

Products.intro.forum versionsqlserver.co.il

Introducing 3 FREE Smart solutions for SQL Server (Adi Sapir, Docco Labs) As Database experts, we work with SQL Server Databases on a daily basis. We face the same problems every SQL Administrator and/or developer does. And – we spend our time writing solutions for these problems! In this session Adi will introduce the following 3, totally FREE solutions: · ClipTable – A revolutionary new *anything* to SQL Table importer · Database File Explorer – a much easier way to explore our database->filegroups->files->storage mapping · Log Table Viewer – a complete client/server logger solution for SQL Server

Migrating Oracle database to CassandraUmair Mansoob

This document discusses migrating Oracle databases to Cassandra. Cassandra offers lower costs, supports more data types, and can scale to handle large volumes of data across multiple data centers. It also allows for more flexible data modeling and built-in compression. The document compares Cassandra and Oracle on features, provides examples of companies using Cassandra, and outlines best practices for data modeling in Cassandra. It also discusses strategies for migrating data from Oracle to Cassandra including using loaders, Sqoop, and Spark.

Inside SQL Server In-Memory OLTPBob Ward

Hekaton is the original project name for In-Memory OLTP and just sounds cooler for a title name. Keeping up the tradition of deep technical “Inside” sessions at PASS, this half-day talk will take you behind the scenes and under the covers on how the In-Memory OLTP functionality works with SQL Server. We will cover “everything Hekaton”, including how it is integrated with the SQL Server Engine Architecture. We will explore how data is stored in memory and on disk, how I/O works, how native complied procedures are built and executed. We will also look at how Hekaton integrates with the rest of the engine, including Backup, Restore, Recovery, High-Availability, Transaction Logging, and Troubleshooting. Demos are a must for a half-day session like this and what would an inside session be if we didn’t bring out the Windows Debugger. As with previous “Inside…” talks I’ve presented at PASS, this session is level 500 and not for the faint of heart. So read through the docs on In-Memory OLTP and bring some extra pain reliever as we move fast and go deep. This session will appear as two sessions in the program guide but is not a Part I and II. It is one complete session with a small break so you should plan to attend it all to get the maximum benefit.

Migrating Oracle database to PostgreSQLUmair Mansoob

This document discusses migrating an Oracle database to PostgreSQL. It covers initial discovery of the Oracle database features and data types used. A migration assessment would analyze data type mapping, additional PostgreSQL features, and testing requirements. Challenges include porting PL/SQL code, minimizing downtime during migration, and comprehensive testing of applications on the new PostgreSQL platform. Migrating large data sets and ensuring performance for critical applications are also challenges.

Making Postgres Central in Your Data CenterEDB

Postgres has the unique ability to act as a powerful data aggregator or information hub in many IT centers bringing together data from different databases and in different formats. This presentation reviews Postgres' extensibility, foreign data wrappers, and ability to work with structured relational and unstructured NoSQL-like information such as documents and key-value data. The Postgres capabilities are unrivaled in enabling a complete view of customers or businesses, analyzing disparate data together, and breaking down data silos within the enterprise. If you would like to listen to the recording please visit EnterpriseDB > Resources > Webcasts > Ondemand Webcasts. To speak to someone about EnterpriseDB's solutions and services please email [email protected].

2016 may-countdown-to-postgres-v96-parallel-queryAshnikbiz

It has just been a few months since the PostgreSQL9.5 is released. We have got some of our customers excited about great new features and performance enhancements in v9.5. But here we are already taking a peak into the next version, and we find it awesome! One of the most awaited features – parallelism makes it to Postgres. The infrastructure for parallelism has been added over last few releases but the first parallel operation in query execution will be seen only in v9.6.

Log miner in oracle.pptHitesh Kumar Markam

Log Miner in Oracle allows users to analyze redo log files to perform auditing, determine causes of errors, and aid in performance tuning and capacity planning. It requires a source and mining database running the same Oracle version with a LogMiner dictionary and redo log files. Users direct Log Miner operations using PL/SQL packages to build the dictionary, add log files, start the session, query results, and end the session.

Introduction to Structured StreamingKnoldus Inc.

State transfer With GaleraMydbops

This document summarizes information about state transfers in Galera, an open source replication plugin for MySQL. It discusses when state transfers are needed, such as during network partitions or node failures. It describes the two types of state transfers - incremental state transfers (IST), which sync nodes incrementally, and state snapshot transfers (SST), which transfer a full data copy. The document compares different methods for SST, such as mysqldump, RSYNC, Xtrabackup, and Clone, and discusses their speeds, impact on donor nodes, and other characteristics. Overall, the document provides an overview of state transfers in Galera synchronous replication.

MySQL Query Optimization (Basics)Karthik .P.R

This document discusses various techniques for optimizing queries in MySQL databases. It covers storage engines like InnoDB and MyISAM, indexing strategies including different index types and usage examples, using explain plans to analyze query performance, and rewriting queries to improve efficiency by leveraging indexes and removing unnecessary functions. The goal of these optimization techniques is to reduce load on database servers and improve query response times as data volumes increase.

SQL Explore 2012: P&T Part 3sqlserver.co.il

Read committed snapshot isolation (RCSI) allows readers to see committed data without blocking writers or other readers. It can greatly reduce locking and deadlocking. SQL Server supports partition-level lock escalation, allowing concurrent access to different partitions. Filtered indexes improve performance by indexing a subset of table data. Optimize for ad hoc workloads improves plan caching for queries that are run infrequently. Enabling data compression reduces database size and storage costs without requiring application changes.

Online index rebuild automationCarlos Sierra

Oracle to PostgreSQL migrationstrikr .

StrikrSystemsLLP investigated various freeware tools for migrating databases ranging in size from 10Mb to 80Gb from Oracle to PostgreSQL and MariaDB. The goal was to migrate the schema, tablespace, table, view, index and some constraints with high fidelity. CSV dumps of data from Oracle were successful. Tables, views and partial indexes were migrated but stored procedures and user-defined functions did not migrate. Challenges included different data type storage sizes and encoding between databases. After customizations, schema, tables, views and partial indexes were successfully migrated but stored procedures and functions require parsing SQL grammar and object representations. Going forward it is recommended to use PostgreSQL for new projects, refactor existing applications before migration, and

SQL Server In-Memory OLTP: What Every SQL Professional Should KnowBob Ward

Perhaps you have heard the term “In-Memory” but not sure what it means. If you are a SQL Server Professional then you will want to know. Even if you are new to SQL Server, you will want to learn more about this topic. Come learn the basics of how In-Memory OLTP technology in SQL Server 2016 and Azure SQL Database can boost your OLTP application by 30X. We will compare how In-Memory OTLP works vs “normal” disk-based tables. We will discuss what is required to migrate your existing data into memory optimized tables or how to build a new set of data and applications to take advantage of this technology. This presentation will cover the fundamentals of what, how, and why this technology is something every SQL Server Professional should know

Ingesting Data from Kafka to JDBC with Transformation and EnrichmentApache Apex

Presenter - Dr Sandeep Deshmukh, Committer Apache Apex, DataTorrent engineer Abstract: Ingesting and extracting data from Hadoop can be a frustrating, time consuming activity for many enterprises. Apache Apex Data Ingestion is a standalone big data application that simplifies the collection, aggregation and movement of large amounts of data to and from Hadoop for a more efficient data processing pipeline. Apache Apex Data Ingestion makes configuring and running Hadoop data ingestion and data extraction a point and click process enabling a smooth, easy path to your Hadoop-based big data project. In this series of talks, we would cover how Hadoop Ingestion is made easy using Apache Apex. The third talk in this series would focus on ingesting unbounded data from Kafka to JDBC with couple of processing operators -Transform and enrichment.

Salesforce Batch processing - Atlanta SFUGvraopolisetti

The document discusses using batch Apex jobs to improve the user experience by offloading long-running or complex processes asynchronously. It provides examples of scenarios where batch Apex would be useful, including updating 324k accounts, nulling a field on 142k opportunities, and changing record types on opportunities. The document also includes sample batch Apex code to implement these scenarios and discusses testing and monitoring batch jobs.

SQL Server It Just Runs FasterBob Ward

Based on the popular blog series, join me in taking a deep dive and a behind the scenes look at how SQL Server 2016 “It Just Runs Faster”, focused on scalability and performance enhancements. This talk will discuss the improvements, not only for awareness, but expose design and internal change details. The beauty behind ‘It Just Runs Faster’ is your ability to just upgrade, in place, and take advantage without lengthy and costly application or infrastructure changes. If you are looking at why SQL Server 2016 makes sense for your business you won’t want to miss this session.

CosmosDB for DBAs & DevelopersNiko Neugebauer

This document provides an overview and introduction to Cosmos DB. It discusses what Cosmos DB is, its data models, APIs, partitioning, and global distribution. It explains why Cosmos DB was created to address limitations of traditional databases. Key aspects covered include throughput and consistency levels, indexing, backups, failovers, and using Cosmos DB for developers and database administrators. The document also discusses migration tools, limitations, and integrations with PowerBI and geospatial data.

Presto Testing Tools: Benchto & Tempto (Presto Boston Meetup 10062015)Matt Fuller

Tempto is a product test framework that allows developers to write and execute tests for SQL databases running on Hadoop. Individual test requirements such as data generation, HDFS file copy/storage of generated data and schema creation are expressed declaratively and are automatically fulfilled by the framework. Developers can write tests using Java (using a TestNG like paradigm and AssertJ style assertion) or by providing query files with expected results. We will show how we use it for presto product tests. Benchto is a benchmark framework that provides an easy and manageable way to define, run and analyze macro benchmarks in clustered environment. Understanding behavior of distributed systems is hard and requires good visibility intostate of the cluster and internals of tested system. This project was developed for repeatable benchmarking ofHadoop SQL engines, most importantly Presto.

Hadoop spark online demoTripti Jha

Experience sql server on l inux and dockerBob Ward

Microsoft SQL Server provides a full-featured database for Linux that offers high performance, security and flexibility across languages and platforms at a lower cost compared to other commercial databases. It has the most consistent data platform with industry-leading performance on Linux and Windows and supports machine learning and artificial intelligence capabilities. SQL Server on Linux allows customers to deploy the database on their choice of Linux distribution for both traditional and container-based workloads.

MySQL HA Percona cluster @ MySQL meetup MumbaiRemote MySQL DBA

Redshift deep diveAmazon Web Services LATAM

Intro to Apache Apex - Next Gen Platform for Ingest and TransformApache Apex

Introduction to Apache Apex - The next generation native Hadoop platform. This talk will cover details about how Apache Apex can be used as a powerful and versatile platform for big data processing. Common usage of Apache Apex includes big data ingestion, streaming analytics, ETL, fast batch alerts, real-time actions, threat detection, etc. Bio: Pramod Immaneni is Apache Apex PMC member and senior architect at DataTorrent, where he works on Apache Apex and specializes in big data platform and applications. Prior to DataTorrent, he was a co-founder and CTO of Leaf Networks LLC, eventually acquired by Netgear Inc, where he built products in core networking space and was granted patents in peer-to-peer VPNs.

Sql server 2016 it just runs faster sql bits 2017 editionBob Ward

SQL Server 2016 includes several performance improvements that help it run faster than previous versions: 1. Automatic Soft NUMA partitions workloads across NUMA nodes when there are more than 8 CPUs per node to avoid bottlenecks. 2. Dynamic memory objects are now partitioned by CPU to avoid contention on global memory objects. 3. Redo operations can now be parallelized across multiple tasks to improve performance during database recovery.

Drill architecture 20120913jasonfrantz

Apache Drill is a data analytics system with a flexible architecture that allows for pluggable components. It includes a driver, parser, compiler/optimizer, execution engine, and storage handlers. The parser converts queries to an intermediate representation, which is optimized and then executed across a cluster by the execution engine. Drill supports various data formats and sources through its extensible storage interfaces and scanner operators. Its design focuses on flexibility, ease of use, dependability, and high performance.

Inside sql server in memory oltp sql sat nyc 2017Bob Ward

This document provides a high-level summary of In-Memory OLTP in SQL Server: - In-Memory OLTP stores and processes transactional data entirely in memory using natively compiled stored procedures to avoid concurrency bottlenecks like locks and latches. - Data is stored in memory-optimized tables using either a hash index or range index for fast lookup. Transactions are logged and written to checkpoint files for durability. - The Hekaton engine handles all transaction processing in memory without locks by using techniques like multi-version concurrency control and lock-free data structures. Checkpoint files are used to reconstruct the database after a restart. - Natively compiled stored procedures provide improved performance by

Optimising Queries - Series 1 Query Optimiser ArchitectureDAGEOP LTD

Query optimizationZunera Bukhari

More Related Content

What's hot (20)

Introduction to Structured StreamingKnoldus Inc.

State transfer With GaleraMydbops

MySQL Query Optimization (Basics)Karthik .P.R

SQL Explore 2012: P&T Part 3sqlserver.co.il

Online index rebuild automationCarlos Sierra

Oracle to PostgreSQL migrationstrikr .

SQL Server In-Memory OLTP: What Every SQL Professional Should KnowBob Ward

Ingesting Data from Kafka to JDBC with Transformation and EnrichmentApache Apex

Salesforce Batch processing - Atlanta SFUGvraopolisetti

SQL Server It Just Runs FasterBob Ward

CosmosDB for DBAs & DevelopersNiko Neugebauer

Presto Testing Tools: Benchto & Tempto (Presto Boston Meetup 10062015)Matt Fuller

Hadoop spark online demoTripti Jha

Experience sql server on l inux and dockerBob Ward

MySQL HA Percona cluster @ MySQL meetup MumbaiRemote MySQL DBA

Redshift deep diveAmazon Web Services LATAM

Intro to Apache Apex - Next Gen Platform for Ingest and TransformApache Apex

Sql server 2016 it just runs faster sql bits 2017 editionBob Ward

Drill architecture 20120913jasonfrantz

Inside sql server in memory oltp sql sat nyc 2017Bob Ward

Introduction to Structured StreamingKnoldus Inc.

State transfer With GaleraMydbops

MySQL Query Optimization (Basics)Karthik .P.R

SQL Explore 2012: P&T Part 3sqlserver.co.il

Online index rebuild automationCarlos Sierra

Oracle to PostgreSQL migrationstrikr .

SQL Server In-Memory OLTP: What Every SQL Professional Should KnowBob Ward

Ingesting Data from Kafka to JDBC with Transformation and EnrichmentApache Apex

Salesforce Batch processing - Atlanta SFUGvraopolisetti

SQL Server It Just Runs FasterBob Ward

CosmosDB for DBAs & DevelopersNiko Neugebauer

Presto Testing Tools: Benchto & Tempto (Presto Boston Meetup 10062015)Matt Fuller

Hadoop spark online demoTripti Jha

Experience sql server on l inux and dockerBob Ward

MySQL HA Percona cluster @ MySQL meetup MumbaiRemote MySQL DBA

Redshift deep diveAmazon Web Services LATAM

Intro to Apache Apex - Next Gen Platform for Ingest and TransformApache Apex

Sql server 2016 it just runs faster sql bits 2017 editionBob Ward

Drill architecture 20120913jasonfrantz

Inside sql server in memory oltp sql sat nyc 2017Bob Ward

Similar to Query handlingbytheserver (20)

Optimising Queries - Series 1 Query Optimiser ArchitectureDAGEOP LTD

Query optimizationZunera Bukhari

Oracle Enteprise Manager Cloud Control 12c - Setting Up Metrics and Monitorin...Leighton Nelson

Query processing and optimization (updated)Ravinder Kamboj

The document discusses techniques used by a database management system (DBMS) to process, optimize, and execute high-level queries. It describes the phases of query processing which include syntax checking, translating the SQL query into an algebraic expression, optimization to choose an efficient execution plan, and running the optimized plan. Query optimization aims to minimize resources like disk I/O and CPU time by selecting the best execution strategy. Techniques for optimization include heuristic rules, cost-based methods, and semantic query optimization using constraints.

Evolving The Optimal Relevancy Scoring Model at Dice.com: Presented by Simon ...Lucidworks

1) The document discusses using black box optimization algorithms to automate the tuning of a search engine's configuration parameters to improve search relevancy. 2) It describes using a test collection of queries and relevance judgments, or search logs, to evaluate how changes to parameters impact relevancy metrics. An optimization algorithm would intelligently search the parameter space. 3) Care must be taken to validate any improved parameters on a separate test set to avoid overfitting and ensure gains generalize to new data. The approach holds promise for automating what can otherwise be a slow manual tuning process.

A data driven etl test framework sqlsat madisonTerry Bunio

This document provides an overview and summary of a SQL Saturday event on automated database testing. It discusses: 1. The presenter's background and their company Protegra which focuses on Agile and Lean practices. 2. The learning objectives of the presentation which are around why and how to automate database testing using tools like tSQLt and SQLtest. 3. A comparison of Waterfall and Agile methodologies with a focus on how Agile lends itself better to test automation. 4. A demonstration of setting up and running simple tests using tSQLt to showcase how it can automate database testing and make it easier compared to traditional methods.

Breaking dataTerry Bunio

This document summarizes Terry Bunio's presentation on breaking and fixing broken data. It begins by thanking sponsors and providing information about Terry Bunio and upcoming SQL events. It then discusses the three types of broken data: inconsistent, incoherent, and ineffectual data. For each type, it provides an example and suggestions on how to identify and fix the issues. It demonstrates how to use tools like Oracle Data Modeler, execution plans, SQL Profiler, and OStress to diagnose problems to make data more consistent, coherent and effective.

Evolving the Optimal Relevancy Ranking Model at Dice.comSimon Hughes

1. The document summarizes Simon Hughes' presentation on evolving the optimal relevancy scoring model at Dice.com. It discusses approaches to automated relevancy tuning using black box optimization algorithms and reinforcement learning. 2. A key challenge is preventing positive feedback loops when the machine learning model's predictions can influence user behavior and future training data. 3. Techniques to address this include isolating a subset of data from the model for training, and using reinforcement learning models that balance exploring different hypotheses with exploiting learned knowledge.

Optimizing Application Performance - 2022.pptxJasonTuran2

When it comes to user experience a snappy application beat a glamorous one. Nothing frustrates an end user more than a slow application. Did you know that any wait time greater than one second will break a user's concentration and cause them to feel frustration? How can we create applications to meet user expectations? This class will cover all things performance from design to delivery. We will go over application design, user interface guidelines, caching guidelines, code optimizations, and query optimizations.

Query optimizationZunera Bukhari

SQLDay2013_MarcinSzeliga_StoredProceduresPolish SQL Server User Group

This document discusses stored procedures in SQL Server. It notes that plan caching and reuse is generally good as it saves time and resources compared to recompiling plans. However, sometimes plan reuse can lead to parameter sniffing issues. It provides recommendations to avoid unnecessary recompilations like using temporary tables and hints, as well as techniques to address parameter sniffing like plan forcing. The presentation aims to separate facts from myths regarding stored procedures and optimization in SQL Server.

Lucene Bootcamp - 2GokulD

This document provides an overview and schedule for a Lucene Boot Camp training session. The training will cover topics like indexing, searching, performance tuning, and class projects. Participants will learn about indexing documents, querying with different query types, using filters and sorting, analyzing performance, and accessing term information. Hands-on exercises are included, as well as opportunities for participants to work on a class project to further apply and extend their Lucene knowledge. Resources and support are also mentioned for continuing to learn about Lucene after the training.

Performance tuning Grails applicationsGR8Conf

The document discusses performance tuning for Grails applications. It outlines that performance aspects include latency, throughput, and quality of operations. Performance tuning optimizes costs and ensures systems meet requirements under high load. Amdahl's law states that parallelization cannot speed up non-parallelizable tasks. The document recommends measuring and profiling, making single changes in iterations, and setting up feedback cycles for development and production environments. Common pitfalls in profiling Grails applications are also discussed.

Chromatography Data system: Process your Data Chromatography & Mass Spectrometry Solutions

Performance Tuning with Execution PlansGrant Fritchey

MySQL Optimization from a Developer's point of viewSachin Khosla

dd presentation.pdfAnSHiKa187943

This document provides an overview of database security concepts including confidentiality, integrity, and availability. It defines database security as protecting the confidentiality, integrity, and availability of data. Key concepts discussed include authentication, authorization, access control, data encryption, data privacy, auditing, and logging. The document also outlines security problems such as non-fraudulent threats from errors or disasters and fraudulent threats from authorized users abusing privileges or hostile agents attacking the system.

Remote DBA Experts SQL Server 2008 New FeaturesRemote DBA Experts

Presentación Oracle Database Migración consideraciones 10g/11g/12cRonald Francisco Vargas Quesada

Collaborate 2011-tuning-ebusiness-416502kaziul Islam Bulbul

Tuning the Applications Tier, Concurrent Manager, Client/Network, and Database Tier are discussed to provide an overview of performance methodology for optimizing the E-Business Suite. The presentation outlines best practices for tuning each layer including the applications tier, concurrent manager, database tier, and applications. Specific techniques are provided for optimizing forms, the Java stack, concurrent processing, network traffic, database configuration, I/O, statistics gathering, and performance monitoring using tools like AWR.

Optimising Queries - Series 1 Query Optimiser ArchitectureDAGEOP LTD

Query optimizationZunera Bukhari

Oracle Enteprise Manager Cloud Control 12c - Setting Up Metrics and Monitorin...Leighton Nelson

Query processing and optimization (updated)Ravinder Kamboj

Evolving The Optimal Relevancy Scoring Model at Dice.com: Presented by Simon ...Lucidworks

A data driven etl test framework sqlsat madisonTerry Bunio

Breaking dataTerry Bunio

Evolving the Optimal Relevancy Ranking Model at Dice.comSimon Hughes

Optimizing Application Performance - 2022.pptxJasonTuran2

Query optimizationZunera Bukhari

SQLDay2013_MarcinSzeliga_StoredProceduresPolish SQL Server User Group

Lucene Bootcamp - 2GokulD

Performance tuning Grails applicationsGR8Conf

Chromatography Data system: Process your Data Chromatography & Mass Spectrometry Solutions

Performance Tuning with Execution PlansGrant Fritchey

MySQL Optimization from a Developer's point of viewSachin Khosla

dd presentation.pdfAnSHiKa187943

Remote DBA Experts SQL Server 2008 New FeaturesRemote DBA Experts

Presentación Oracle Database Migración consideraciones 10g/11g/12cRonald Francisco Vargas Quesada

Collaborate 2011-tuning-ebusiness-416502kaziul Islam Bulbul

More from sqlserver.co.il (20)

Windows azure sql_database_security_isug012013sqlserver.co.il

This document discusses security for Microsoft SQL Azure (now called Windows Azure SQL Database). It provides an overview of SQL Database and its security capabilities, best practices for securing SQL Database like using encryption and configuring firewall rules, and limitations compared to on-premises SQL Server. It also introduces GreenSQL as a software-based database proxy that can provide additional security functionality for SQL Database like preventing SQL injection, auditing, and data masking. GreenSQL aims to offer a more complete solution for security, compliance, and hybrid application support compared to the native capabilities in SQL Database.

Sql server user group news january 2013sqlserver.co.il

This document summarizes upcoming SQL Server events in January 2013 in Israel, including multi-day training sessions, user group meetings, and virtual chapter meetings on SQL Server administration and development topics. It also lists current job openings and previews the tips and presentations for an upcoming user group meeting, including how Database Administration Console improvements in SQL Server 2012 will impact administrators and real-world query tuning case studies.

DAC 2012sqlserver.co.il

The document discusses the evolution of data-tier applications (DACs) and database administration over time. It notes that in the past, database administrators (DBAs) and developers worked in silos, making development, deployment, and management challenging. Recent investments in tools like SQL Server Data-tier Applications (DACs) now allow for more integrated development workflows and easier deployment and management of databases at scale. The document outlines some key concepts and enhancements in DACs over multiple versions to improve collaboration and move developers to a more declarative model.

Adi Sapir ISUG 123 11/10/2012sqlserver.co.il

The document describes several SQL server monitoring and management tools from Docco Labs, including ClipTable for data import, Database File Explorer for file usage monitoring, Log Table Viewer for logging, and SQL Stripes for server monitoring and control. SQL Stripes is a T-SQL based solution that monitors metrics like CPU, disk usage, locks, and pings and notifies users of issues by email, web, audio or visual alerts. It aims to fill the gap between generic monitoring tools and agent-specific alerts.

SQL Explore 2012: P&T Part 2sqlserver.co.il

- Distributed Replay allows replaying a captured workload from multiple client computers to better simulate production loads. - A controller coordinates the replay across clients to reproduce the original query rates or run in stress test mode faster than original rates. - It improves on SQL Server Profiler for application compatibility testing, performance debugging, capacity planning, and benchmarking. - Events are replayed in synchronization mode to match original order, or unsynchronized to stress test without timing constraints.

SQL Explore 2012: P&T Part 1sqlserver.co.il

Predicates allow filtering events based on: - Event properties (fields) - Session properties - System properties They are evaluated synchronously when the event fires. This allows filtering events and reducing overhead compared to capturing all events. Common predicates: - event_name = 'sql_statement_completed' - database_id = 5 - cpu_time > 1000 Predicates give granular control over what events are captured.

SQL Explore 2012 - Tzahi Hakikat and Keren Bartal: Extended Eventssqlserver.co.il

SQL Explore 2012 - Michael Zilberstein: ColumnStoresqlserver.co.il

This document discusses how columnstore indexes in Microsoft SQL Server can boost query performance. It provides an overview of column-oriented databases and columnstore technology. Columnstore indexes store data by column rather than by row, allowing for improved compression techniques like dictionary encoding. This reduces storage space and I/O. It also allows for more efficient query processing through batch execution and new execution plan elements. The document outlines best practices, limitations, issues and workarounds, as well as how to load data with columnstore indexes. It provides references for further reading on columnstore performance tuning.

SQL Explore 2012 - Meir Dudai: DACsqlserver.co.il

- The document discusses the challenges of database development and deployment processes and how the Data-Tier Application Framework (DAC Fx) addresses these challenges. - DAC Fx allows developers to model the database in a declarative way and generates deployment scripts and files (.dacpac) that simplify the deployment process for both developers and DBAs. - It also enables cross-platform deployment of databases to both on-premises and cloud platforms like SQL Azure.

SQL Explore 2012 - Aviad Deri: Spatialsqlserver.co.il

SQL Server 2012 introduced new spatial data types like circular arcs and support for full globe objects. It also enhanced spatial indexing with automatic grid generation and index hints. Other improvements included additional methods for spatial aggregations and reorienting objects on the globe. Overall, the new capabilities expanded support for advanced geospatial analysis and improved performance of spatial queries.

מיכאלsqlserver.co.il

Hadoop is an open-source framework for storing and processing large datasets in a distributed computing environment. It allows for the storage and analysis of datasets that are too large for single servers. The document discusses several key Hadoop components including HDFS for storage, MapReduce for processing, HBase for column-oriented storage, Hive for SQL-like queries, Pig for data flows, and Sqoop for data transfer between Hadoop and relational databases. It provides examples of how each component can be used and notes that Hadoop is well-suited for large-scale batch processing of data.

נועםsqlserver.co.il

SQL Server allows for parallel processing to improve query performance. It uses multiple threads or workers that can run tasks such as reading, writing, and calculating in parallel across multiple CPU cores. The degree of parallelism (DOP) or number of threads used can be configured through settings like MAXDOP. Determining the optimal DOP requires testing different values and monitoring performance and waits to find the best setting for each workload type.

עדיsqlserver.co.il

Common Errors That Effect Performance (Adi Cohen, Naya-Tech) There are a few common errors that have a negative effect on performance. In this session we will review some of them, see why they impact performance and provide alternative solutions. Among the issues we will cover are: · Misunderstanding of the query plan when using procedures · Query plan differences between procedures and ad-hoc batches · The differences between a temporary table and a table variable · And many more…

מיכאלsqlserver.co.il

Who is afraid of Columnstore Indexes? (Michael Zilberstein, DB-Art) This talk describes new SQL Server 2012 feature called "columnstore index". In this session we will learn about the differences between columnstore indexes and B-Tree indexes we are used to work with. We will see when it is best to use and when not to use this new index. We will cover limitations that columnstore index imposes on the tables that use it and how to live with those limitations. Like in all my sessions, I won't let you go without some internals – how columnstore index is organized on a physical level and how Query Processor works this new type of index. And of course Demos, Demos, Demos…

Bi303 data warehousing with fast track and pdw - Assaf Fraenkelsqlserver.co.il

HP offers two solutions for data warehousing - Fast Track and Parallel Data Warehouse. Fast Track provides reference architectures using SQL Server for balanced deployments scaling from 8-80TB. Parallel Data Warehouse uses a massively parallel processing architecture to scale SQL Server deployments to petabytes of data. It uses a scale-out architecture with distributed data and query processing. This provides very high performance for large and complex workloads.

DBCC - Dubi Lebelsqlserver.co.il

Trace flags are used to temporarily change SQL Server's behavior for debugging or diagnosing issues. This document discusses several trace flags including: TF 652, 661, 834, 836 which disable certain SQL Server processes or enable large page allocations. TF 1211, 1224 which avoid lock escalation. TF 1117 forces data files to auto grow equally. TF 1204, 1205, 1222 provide more information on deadlocks. TF 1118 addresses tempdb contention. TFs 3226, 3014, 3004 provide more backup/restore details. TF 4199 enables query processor fixes. TF 3502 prints checkpoint messages. The document provides explanations of these trace flags

Fast transition to sql server 2012 from mssql 2005 2008 for developers - Dav...sqlserver.co.il

Rafael Advanced Defense Systems is an Israeli defense technology company that designs, develops, manufactures and supplies defense systems for air, land, sea and space applications. It has annual sales exceeding $1.85 billion and employs about 7,000 people. The document discusses using analytic functions in SQL Server 2012, including new functions added in 2012 and enhanced window function support, and provides examples of how various analytic functions can be used to analyze sales data.

ISUG 113: File streamsqlserver.co.il

The document discusses Microsoft SQL Server 2008 R2's FILESTREAM feature. FILESTREAM allows storing unstructured binary large objects (BLOBs) such as documents, images, and videos on the file system rather than in a database. It integrates access to this file data into the database for transactional consistency. Some key points covered include FILESTREAM storage in NTFS directories, its security model using standard SQL Server permissions, and accessing BLOBs using both T-SQL queries and file streaming APIs.

Extreme performance - IDF UGsqlserver.co.il

This document summarizes Valinor's services as the leading company in Israel for professional database services. It lists that Valinor has: - An expert consulting team specialized in all database layers - Partnerships with Microsoft and Oracle - Over 300 customers across all sectors The document then discusses Valinor's agenda for an upcoming event which includes discussing the utility control point, detecting bottlenecks, and new performance monitoring features in Windows 2008 R2.

3 extreme performance - databases acceleration using ssd sqlserver.co.il

Windows azure sql_database_security_isug012013sqlserver.co.il

Sql server user group news january 2013sqlserver.co.il

DAC 2012sqlserver.co.il

Adi Sapir ISUG 123 11/10/2012sqlserver.co.il

SQL Explore 2012: P&T Part 2sqlserver.co.il

SQL Explore 2012: P&T Part 1sqlserver.co.il

SQL Explore 2012 - Tzahi Hakikat and Keren Bartal: Extended Eventssqlserver.co.il

SQL Explore 2012 - Michael Zilberstein: ColumnStoresqlserver.co.il

SQL Explore 2012 - Meir Dudai: DACsqlserver.co.il

SQL Explore 2012 - Aviad Deri: Spatialsqlserver.co.il

מיכאלsqlserver.co.il

נועםsqlserver.co.il

עדיsqlserver.co.il

מיכאלsqlserver.co.il

Bi303 data warehousing with fast track and pdw - Assaf Fraenkelsqlserver.co.il

DBCC - Dubi Lebelsqlserver.co.il

Fast transition to sql server 2012 from mssql 2005 2008 for developers - Dav...sqlserver.co.il

ISUG 113: File streamsqlserver.co.il

Extreme performance - IDF UGsqlserver.co.il

3 extreme performance - databases acceleration using ssd sqlserver.co.il

Query handlingbytheserver

1. QUERY HANDLING

2. OPTIMIZER’S ROLE • The optimizer should generate an efficient plan to access the data that we need to work with.

3. WHY DO WE NEED IT? • SQL Statement defines what we are looking for. It doesn’t define how to get the data. • The optimizer has to generate a query plan which tells the server how to get the data.

4. OPTIMIZER CHALENGES

5. NUMBER OF JOIN ORDERS

6. OPTIMIZER • The optimizer doesn't necessarily look for the best query plan. • The optimizer searches for a good enough plan.

7. WORK STAGES (HIGH LEVEL) • Create a logical plan • Apply transformation rules • Apply heuristics • Select the cheapest one

8. STEPS OF QUERY PROCESSING

9. PARSING • Checks that the syntax is correct • Creates a logical tree that represents the query.

10. DEMO 1 • See logical tree that was produced by parse step.

11. BINDIND • Gets the logical tree that was produced by the parsing step • Makes sure that all objects referenced by the logical tree exist and that the user can see them.

12. OPTIMIZATION • The most important step. Generates the query plan that will be executed

13. TRIVIAL PLAN • Very simple queries with very simple logical tree won’t get full optimization • The optimizer will generate a plan that is called a trivial plan • The trivial plan is very cheap to generate and it won’t be inserted into the plan cache

14. TRIVIAL PLAN • If the trivial plan’s cost is more then the value that is configured as “cost threshold for parallelism”, the query will get full optimization • You can disable the trivial plan by using trace flag 8757

15. SIMPLIFICATION • The OPTIMIZER rewrites the tree to make it more simple. There are few simplification methods: – Constant folding – Domain simplification – Contradiction detection

16. DEMO 2 • Simplification and trivial plan

17. OPTIMIZATION LEVELS • If no trivial plan was found, SQL Server starts optimizing the query. • SQL Server has 3 stages that are called: – Search 0 (Transaction processing) – Search 1 (Quick plan) – Search 2 (Full optimization)

18. OPTIMIZATION LEVELS • Each level has an entry and termination condition. • Termination condition can be good enough plan was found, or to much time passed • Optimization can begin at a low search step and get to a higher search step.

19. PROPERTIES • Each node in the logical tree has properties attached to it. • There are 2 types of properties – Logical properties (Node cost, output columns, Type information and nullability, etc) – Physical properties (Sort order, partition information etc’)

20. RULES • The optimizer has set of more then 350 rules that is using to optimize the query. • The rules help the optimizer to modify the logical tree in a way that doesn’t effect the query’s results • The rules also dictate the physical implementation of the logical tree

21. RULES • There are four types of rules that can be used: – Simplification rules – Exploration rules – Implementation rules – Physical properties enforcement rules

22. MEMO • All the trees are stored in memory in a structure that is called Memo • Each optimization has its own memo • A single memo can get to the size of 1.6 GB

23. OPTIMIZATION PROBLEMS • On rare occasions there can be a timeout for the optimization process • Most times when it happens, it will be on search 2 stage, and the server will use the query plan that was produced on search 1 stage. • Sometime the server stops optimization because of memory pressure

24. DMVs • There are 2 DMVs that can give us information about the optimizer: – sys.dm_exec_query_optimizer_info, shows information about optimization process – sys.dm_exec_query_transformation_stats, sho ws information about rules usage

25. Demo 3

26. QUSTIONS? • Additional reading: – http://sqlblog.com/blogs/paul_white/archive/ 2012/04/28/query-optimizer-deep-dive-part- 1.aspx – http://www.red- gate.com/community/books/inside-sql-server- query-optimizer

Query handlingbytheserver

Recommended

More Related Content

What's hot (20)

Similar to Query handlingbytheserver (20)

More from sqlserver.co.il (20)

Query handlingbytheserver