Beginners guide to_optimizer

Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |
Beginners Guide to Oracle Optimizer
Maria Colgan
Master Product Manager
Oracle Database Server Technologies
December 2018

Safe Harbor Statement
The following is intended to outline our general product direction. It is intended for
information purposes only, and may not be incorporated into any contract. It is not a
commitment to deliver any material, code, or functionality, and should not be relied upon
in making purchasing decisions. The development, release, timing and price of any
features or functionality described for Oracle’s products may change and remains at the
sole discretion of Oracle corporation. Fees apply for new database product offerings.

Expectations
• This is a short beginners guide to the different aspects of the Optimizer
• This material will not instantly make you an Optimizer expert!

What happens when a SQL statement is issued?
User
Library Cache
Shared SQL Area
Shared Pool
CnC1 C2 …
3
Optimizer
Oracle Database
Code Generator
1
4
SQL Execution
Syntax Check
Semantic Check
Shared Pool check
2
Parsing

Oracle Optimizer

Cost Based Optimizer
• Initial design based on IBM research paper
– Access Path Selection in a Relational Database Management System (1979)
• Approach outlined in the paper was
– Multiple execution plans generated for a statement
– Estimated cost is computed for each plan
– Optimizer selects the plan with the lowest estimated cost
A Brief History

Understanding how the Optimizer works
Query Transformation
Rewrite query text to allow it to be processed
more efficiently
Plan Generator
Multiple plans are generated for
each SQL, using different access
paths and join types. Each plan is
costed and plan with the lowest
cost is used.
Cost Estimator
Cost is an estimate of the amount of
CPU and the number of disk I/Os, used
to perform an operation
Optimizer
Statistics
Schema definitions
Controlled by OPTIMIZER_FEATURES_ENABLED

Optimizer Transformations
• Translate statements into semantically equivalent SQL that can be processed more
efficiently
• Initial transformations were heuristic based
– Applied to SQL statements based on their structural properties only
• Predominately cost based now
• Transformations include
– Subquery Unnesting
– View Merging
– OR Expansion
– Star transformation

• Transforms queries that contain OR
predicates into the form of a
UNION ALL query of two or more
branches
• Without the transformation
Optimizer treats OR predicates as a
single unit
• Can’t use index on either column
SELECT *
FROM products p
WHERE prod_category ='Photo'
OR prod_subcategory ='Camera Media';
Or Expansion

• The transformation adds an
LNNVL() function to the second
branch in order to avoid duplicates
being generated across branches
• The LNNVL function returns TRUE,
if the predicate evaluates to FALSE
or if the predicate involved is NULL;
otherwise it will return FALSE
• lnnvl(true) is FALSE,
lnnvl(false||null) is TRUE
SELECT *
FROM products p
WHERE prod_subcategory ='Camera Media’
UNION ALL
SELECT *
FROM products p
WHERE prod_category ='Photo’
AND lnnvl(prod_subcategory =
'Camera Media')
;
Or Expansion

OR Expansion
• Transformation allows an index access to be considered for each branch of
the UNION ALL

Understanding how the Optimizer works
Query Transformation
Rewrite query text to allow it to be processed
more efficiently
Plan Generator
Multiple plans are generated for
each SQL, using different access
paths and join types. Each plan is
costed and plan with the lowest
cost is used.
Cost Estimator
Cost is an estimate of the amount of
CPU and the number of disk I/Os, used
to perform an operation
Optimizer
Statistics
Schema definitions

DATA DICTIONARY
OPTIMIZER STATISTICS
Index Table Column System
PROMO_PK Index
PROMOTIONS Table
Execution
plan
Optimizer
CPU & IO
Optimizer Statistics
PROMO_I
D
PROMO_NAME … PROMO_DATE
1 Promo_1 … 15-NOV-98
2 Promo_1 … 31-DEC-98

Basic table statistics collected by Oracle
• By default the following basic table & column statistic are collected either
during a CTAS or IAS command* or via a DBMS_STATS.GATHER_*_STATS op:
– Number of Rows
– Number of blocks
– Average row length
– Number of distinct values
– Number of nulls in column
– Minimum and maximum values for a column
– Data distribution via histograms++
*Online stats available since 12cR1 ++ Histograms are not gathered online

Basic index statistics collected by Oracle
• Index statistics are automatically gathered during creation and maintained by
DBMS_STATS.GATHER_*_STATS commands
• Index statistics include:
– Number of leaf blocks
– Branch Levels
– Clustering factor

How statistics are automatically gathered
• Oracle automatically collect statistics for all database objects, which are
missing statistics or have stale statistics
• AutoTask runs during a predefined maintenance window
• Internally prioritizes the database objects
– Both user schema and dictionary tables
– Objects that need updated statistics most are processed first
• Controlled by DBMS_AUTO_TASK_ADMIN package or Enterprise Manager
• Monitor using DBA_AUTOTASK_* views
and fixed object statistics in 12c

How you can manually gather statistics
• Use DBMS_STATS Package to manually gather statistics
• Your gather statistics commands should be this simple
BEGIN
dbms_stats.Gather_database_stats();
END; /
BEGIN
dbms_stats.Gather_schema_stats('SH');
END; /
BEGIN
dbms_stats.Gather_table_stats('SH', 'SALES');
END; /

How to change default parameter values for gathering stats
• Occasionally default parameter values may need to change
• For example - features not automatically on by default
– Incremental Statistics
• Ability to accurate generate global statistics from partition level statistics
BEGIN
dbms_stats.Set_global_prefs('INCREMENTAL', 'TRUE');
END;
/
BEGIN
dbms_stats.Set_table_prefs(‘SH’,’SALES’'INCREMENTAL', 'TRUE');
END;
/

What is an execution plan?
• Execution plans show the detailed steps necessary to execute a SQL statement
• These steps are expressed as a set of database operators that consumes and
produces rows
• The order of the operators and their implementation is decided by the
optimizer using a combination of query transformations and physical
optimization techniques
• The display is commonly shown in a tabular format, but a plan is in fact tree-
shaped

What is an execution plan?
Query:
SELECT prod_category, avg(amount_sold)
FROM sales s, products p
WHERE p.prod_id = s.prod_id
GROUP BY prod_category;
HASH JOIN
TABLE ACCESS
SALES
Tree-shaped representation of plan
TABLE ACCESS
PRODUCTS
GROUP BY
Tabular representation of plan

• Autotrace • SQL Monitor
22
• SQL Developer • TKPROF
Many Ways to View an Execution Plan

Identifying elements of an execution plan - ID
ID COLUMN MAKES IT EASIER TO IDENTIFY EACH STEP
* INDICATES THAT STEP IS BASED OFF A PREDICATED
LISTED BELOW THE PLAN TABLE

Identifying elements of an execution plan - OPERATION
OPERATION COLUMN SHOWS
ACTUAL STEPS TAKEN TO EXECUTE
THE PLAN INCLUDING:
• DATA ACCESS METHODS
• JOIN METHODS & TYPES
• AGGREGATIONS

Access paths – Getting the data
Access Path Explanation
Full table scan Reads all rows from table & filters out those that do not meet the where clause predicates. Used when no index,
DOP set etc
Table access by Rowid Rowid specifies the datafile & data block containing the row and the location of the row in that block. Used if
rowid supplied by index or in where clause
Index unique scan Only one row will be returned. Used when stmt contains a UNIQUE or a PRIMARY KEY constraint that guarantees
that only a single row is accessed
Index range scan Accesses adjacent index entries returns ROWID values Used with equality on non-unique indexes or range
predicate on unique index (<.>, between etc)
Index skip scan Skips the leading edge of the index & uses the rest Advantageous if there are few distinct values in the leading
column and many distinct values in the non-leading column
Full index scan Processes all leaf blocks of an index, but only enough branch blocks to find 1st leaf block. Used when all necessary
columns are in index & order by clause matches index struct or if sort merge join is done
Fast full index scan Scans all blocks in index used to replace a FTS when all necessary columns are in the index. Using multi-block IO
& can going parallel
Index joins Hash join of several indexes that together contain all the table columns that are referenced in the query. Won’t
eliminate a sort operation
Bitmap indexes Uses a bitmap for key values and a mapping function that converts each bit position to a rowid. Can efficiently
merge indexes that correspond to several conditions in a WHERE clause

Access path example 1
Table products contains 10K rows & has a primary key on product_id
SELECT product_id, name
FROM products
WHERE products_id IN (‘P123’,’P456’,’P789‘);
What plan would you expect for this query?
PRODUCTS
PRODUCT_PK

Access path example 2
Table products contains 10K rows & has a primary key on product_id
SELECT product_id, name
FROM products
WHERE product_id BETWEEN ‘P123' AND ‘P789';
What plan would you expect for this query?
PRODUCTS
PRODUCT_PK

Join methods
Join Methods Explanation
Nested Loops joins For every row in the outer table, Oracle accesses all the rows in the inner
table Useful when joining small subsets of data and there is an efficient
way to access the second table (index look up)
Hash Joins The smaller of two tables is scan and resulting rows are used to build a hash
table on the join key in memory. The larger table is then scan, join column
of the resulting rows are hashed and the values used to probing the hash
table to find the matching rows. Useful for larger tables & if equality
predicate
Sort Merge joins Consists of two steps:
1. Sort join operation: Both the inputs are sorted on the join key.
2. Merge join operation: The sorted lists are merged together.
Useful when the join condition between two tables is an inequality condition

Join types
Join Type Explanation
Cartesian Joins Joins every row from one data source with every row from the other data
source, creating the Cartesian Product of the two sets. Only good if
tables are very small. Only choice if there is no join condition specified in
query
Outer Joins Returns all rows that satisfy the join condition and also returns all of the rows
from the table without the (+) for which no rows from the other table satisfy
the join condition

Join method example 1
SELECT e.last_name, e.salary, d.department_name
FROM hr.employees e, hr.departments d
WHERE d.departments_name IN ('Marketing‘,'Sales')
AND e.department_id = d.department_id;
Employees has 107 rows
Departments has 27 rows
Foreign key relationship between Employees and Departments on dept_id
What join method would you expect for this query?

Join method example 1
SELECT e.last_name, e.salary, d.department_name
FROM hr.employees e, hr.departments d
WHERE d.departments_name IN ('Marketing‘,'Sales')
AND e.department_id = d.department_id;
What join method would you expect for this query?

Identifying elements of an execution plan - CARDINALITY
CARDINALITY IS THE
ESTIMATED NUMBER OF
ROWS RETURN BY EACH
OPERATION IN THE PLAN
AFTER PREDICATES HAVE
BEEN APPLIED
Determine correct cardinality
using a SELECT COUNT(*) from
each table applying any WHERE
Clause predicates belonging to
that table

Checking cardinality estimates
SELECT /*+ gather_plan_statistics */
p.prod_name, SUM(s.quantity_sold)
FROM sales s, products p
WHERE s.prod_id =p.prod_id
GROUP BY p.prod_name ;
SELECT * FROM table (
DBMS_XPLAN.DISPLAY_CURSOR(FORMAT=>'ALLSTATS LAST'));

Checking cardinality estimates
Compare estimated number of rows returned with actual rows returned

Checking cardinality estimates for Parallel Execution
Note: a lot of the data is zero in the
A-rows column because we only
show last executed cursor which is
the QC. Need to use ALLSTATS ALL to
see info on all parallel server cursors

Easiest way to compare the estimated number of rows returned with actual rows returned
Check cardinality using SQL Monitor

AMOUNT OF
DATA TO BE
PROCESSED IN
BYTES BY EACH
OPERATION

COST REPRESENTS UNITS OF WORK OR
RESOURCES (CPU & IO) USED TO PERFORM
EACH OPERATION IN THE PLAN
COST IS AN INTERNAL ORACLE
MEASUREMENT AND SHOULD BE USED
FOR COMPARISION PUROPOSES ONLY

Best way to influence the Optimizer
• Statistics
• SQL Plan Management
• SQL Patch
• SQL Profile*
• Hints
* Requires Diagnostics Pack

What are hints?
• Hints allow you to influence the Optimizer when it has to choose between
several possibilities
• A hint is a directive that will be followed when applicable
– Hints are only evaluated when they apply to a decision that has to be made
• Can influence everything from the Optimizer mode used to each operation
in the execution
• Automatically means the Cost Based Optimizer will be used
– Only exception is the RULE hint but it must be used alone
• Use OPTIMIZER_IGNORE_HINTS parameter to test query performance
without hints

Hints can influence all aspects of a plan
• Most hints have corresponding negative hint preceded by word ‘NO_’
• More information on hints can be found in chapter 3 of SQL Reference Guide
 Hints to influence cardinality
• DYNAMIC_SAMPLING
• CARDINALITY
 Hints to influence join methods
• USE_NL_WITH_INDEX
• USE_HASH
 Hints to influence access paths
• FULL
• INDEX
 Hints to influence join order
• LEADING
• ORDERED

How to use Optimizer hints
• Hints are inserted in a SQL statement in the form of a comment with an
additional + sign
• They go immediately after the keyword (SELECT, INSERT, etc)
SELECT /* this is a comment */ count(*) FROM sales;
SELECT /*+ this is a comment */ count(*) FROM sales;
Overview
Note: Hint syntax is correct but it is not a valid hint so it is treated as comment

If you can hint it, baseline it
• Its not always possible to add hints to third party applications
• Hints can be extremely difficult to manage over time
• Once added never removed
Alternative approach to hints
Solution
 Use SQL Plan Management (SPM)
 Influence the execution plan without adding hints directly to queries
 SPM available in both SE and EE, no additional options required

Summary
• Learn to work with the Optimizer
and not against it
• Enjoy the challenge, knowing its
only as smart as the folks
that feed it information!
• More information can be found on
the Optimizer blog or Oracle.com
– https://blogs.oracle.com/optimizer
– https://www.oracle.com/technetwork/database/database-technologies/query-
optimization/overview/index.html

Beginners guide to_optimizer

More Related Content

What's hot (20)

Similar to Beginners guide to_optimizer (20)

More from Maria Colgan (14)

Recently uploaded (20)

Beginners guide to_optimizer

Editor's Notes