SlideShare a Scribd company logo
A deeper dive
into EXPLAIN
Michael Christofides
Hi, I’m Michael
Half of the team behind pgMustard
Spent a lot of time looking into EXPLAIN
Background: product management, database tools
pgmustard.com/docs/explain
michael@pgmustard.com
michristofides
Picking up from other EXPLAIN talks
Not the basics*
1) Some of the less intuitive arithmetic
2) Some less well covered issues
* postgresql.org/docs/current/performance-tips
thoughtbot: reading EXPLAIN ANALYZE
YouTube: Josh Berkus Explaining EXPLAIN
Picking up from other EXPLAIN talks
Not the basics*
1) Arithmetic: why is this query slow?
2) Issues: what can we do about it?
* postgresql.org/docs/current/performance-tips
thoughtbot: reading EXPLAIN ANALYZE
YouTube: Josh Berkus Explaining EXPLAIN
Arithmetic: loops
Many of the stats are a per-loop average
This includes costs, rows, timings
Watch out for rounding, especially to 0 rows
Disclaimer: heavily doctored
plans ahead, inaccuracies likely.
Nested Loop
(cost=0.84..209.82 rows=16 width=11)
(actual time=0.076..0.368 rows=86 loops=1)
-> Index Only Scan using a on b
(cost=0.42..4.58 rows=9 width=4)
(actual time=0.013..0.019 rows=9 loops=1)
-> Index Scan using x on y
(cost=0.42..22.73 rows=7 width=15)
(actual time=0.012..0.030 rows=10 loops=9)
Nested Loop: 86 rows
Index Scan: 9 * 10 = 90 rows
(Rounding not too bad here)
Arithmetic: threads
Costs, rows, and timings are also per-thread
Shown as loops
Threads = workers + 1
Tip: use VERBOSE
<- the leader
Parallel Seq Scan on table
(cost=0.00..6772.21 rows=79521 width=22)
(actual time=0.090..71.866 rows=63617 loops=3)
Output: column1, column2, column3
Worker 0: actual time=0.111..66.325 rows=56225 loops=1
Worker 1: actual time=0.138..66.027 rows=58792 loops=1
Seq Scan: 63617 * 3 = 190851 rows
Leader: 190851 - 58792 - 56225
= 75834 rows
Arithmetic: buffers
Buffer stats are a total, not per-loop
They are inclusive of children
Nested Loop (... loops=1)
Buffers: shared hit=105
-> Index Only Scan using a on b (... loops=1)
Buffers: shared hit=4
-> Index Scan using x on y (... loops=9)
Buffers: shared hit=101
Nested Loop buffers:
105 - (101 + 4) = 0 blocks
Arithmetic: timings
Per-loop, per-thread
Inclusive of children
Calculating per-node (exclusive) times can get
tricky, even for tools
Nested Loop
(cost=0.84..209.82 rows=16 width=11)
(actual time=0.076..0.368 rows=86 loops=1)
-> Index Only Scan using a on b
(cost=0.42..4.58 rows=9 width=4)
(actual time=0.013..0.019 rows=9 loops=1)
-> Index Scan using x on y
(cost=0.42..22.73 rows=7 width=15)
(actual time=0.012..0.030 rows=10 loops=9)
Index Scan: 0.030 * 9 = 0.27 ms
Nested Loop: 0.368 - 0.27 - 0.019
= 0.079 ms
WITH init AS (
SELECT * FROM pg_sleep_for('100ms')
UNION ALL
SELECT * FROM pg_sleep_for('200ms')
)
(SELECT * FROM init LIMIT 1)
UNION ALL
(SELECT * FROM init);
Credit @felixge
Append (actual time=100.359..
300.688 … )
CTE init
-> Append (actual time=100.334..
300.652 … )
-> Function Scan (actual time=100.333..
100.335 … )
-> Function Scan (actual time=200.310..
200.312 … )
-> Limit (actual time=100.358..
100.359 … )
-> CTE Scan a (actual time=100.355..100.356 … )
-> CTE Scan b (actual time=0.001..
200.322 … )
Execution Time: 300.789 ms
Further reading:
flame-explain.com/docs/general/quirk-correction
Some double-counting in this case.
Arithmetic: tools can help
eg explain.depesz.com
explain.dalibo.com
flame-explain.com
pgmustard.com
<- fellow calculations nerd
<- 👋
Issues: let’s skip the basics
Seq Scans with large filters
Bad row estimates
Sorts and Hashes on disk
Issues: inefficient index scans
Looks out for lots of rows being filtered
Filters are per-loop
So again, watch out for rounding
-> Index Scan using x on y
(cost=0.42..302502.05 rows=1708602 width=125)
(actual time=172810.219..173876.540 rows=1000 loops=1)
Index Cond: (id = another_id)
Filter: (status = 1)
Rows Removed by Filter: 3125626
Index efficiency: 1000/(1000+3125626) = 0.03%
Watch out for high loops
Issues: lossy bitmap scans
When bitmap would otherwise exceed work_mem
Point to a block rather than a row (Tuple Id)
Lossy blocks are a total (ie not per-loop)
-> Bitmap Heap Scan on table
(cost=49153.29..4069724.27 rows=3105598 width=1106)
(actual time=591.928..56472.895 rows=3853272 loops=1)
Recheck Cond: (something > something_else)
Rows Removed by Index Recheck: 5905323
Heap Blocks: exact=14280 lossy=1951048
Lossy blocks: 1951048/(1951048+14280) = 99%
Extra rows read: 5.9 million
Issues: lots of data read
Requires BUFFERS
Lots of data being read for the amount returned
Can be a sign of bloat
Default block size: 8kB
-> Index Scan using x on y
(cost=0.57..2.57 rows=1 width=8)
(actual time=0.064..0.064 rows=1 loops=256753)
Index Cond: (id = another_id)
Filter: (status = 1)
Buffers: shared hit=1146405 read=110636
Caveats: width estimated, rows rounded
Data read: (1146405 + 110636) * 8kB = 10GB
Data returned: 1 * 256753 * 8 bytes = 2MB
Issues: planning time
At the end of the query plan
Can be planning related: eg joins, partitions
But other things too: eg extensions, locks
Warning: not included in auto_explain
(...)
Planning Time: 27.844 ms
Execution Time: 11.162 ms
Planning proportion:
27.844/(27.844 + 11.162) = 71%
Issues: Just In Time compilation
At the end of the query plan
Included in execution time
On by default in PostgreSQL 12 and 13
Start-up time can be a tell-tale
Planning Time: 9.138 ms
JIT:
Functions: 277
Options: Inlining true, Optimization true, Expressions true,
Deforming true
Timing: Generation 31.602 ms, Inlining 253.114 ms, Optimization
1498.268 ms, Emission 913.945 ms, Total 2696.929 ms
Execution Time: 5194.851 ms
JIT proportion:
2696.929/(9.138 + 5194.851) = 52%
Very suspicious actual start-up time
from a JIT dominated plan.
-> Seq Scan on table
(cost=0.00..3.57 rows=72 width=8)
(actual time=2262.312..2262.343 rows=54 loops=1)
Buffers: shared hit=3
Issues: triggers
At the end of the query plan
Total time across calls
Check foreign keys indexed
Before triggers vs after triggers
Planning Time: 0.227 ms
Trigger: RI_ConstraintTrigger_a_12345 on table
time=83129.491 calls=2222623
Execution Time: 87645.739 ms
Trigger proportion:
83129.491/(0.227 + 87645.739) = 95%
Summary: check the arithmetic
Watch out for loops and threads
Watch out for CTEs
Tools can help, if in doubt check two
Summary: keep rarer issues in mind
Check the end section first
Also look out for filters, rechecks, lossy
blocks, amount of data
Tools, mailing lists, and communities can help
Thank you! Any questions?
michael@pgmustard.com
michristofides
Further reading:
* flame-explain.com/docs/general/quirk-correction
* pgmustard.com/docs/explain
* wiki.postgresql.org/wiki/Slow_Query_Questions
Ad

More Related Content

What's hot (20)

Creating and Using the Flux SQL Datasource | Katy Farmer | InfluxData
Creating and Using the Flux SQL Datasource | Katy Farmer | InfluxData Creating and Using the Flux SQL Datasource | Katy Farmer | InfluxData
Creating and Using the Flux SQL Datasource | Katy Farmer | InfluxData
InfluxData
 
A Fast Intro to Fast Query with ClickHouse, by Robert Hodges
A Fast Intro to Fast Query with ClickHouse, by Robert HodgesA Fast Intro to Fast Query with ClickHouse, by Robert Hodges
A Fast Intro to Fast Query with ClickHouse, by Robert Hodges
Altinity Ltd
 
Taming GC Pauses for Humongous Java Heaps in Spark Graph Computing-(Eric Kacz...
Taming GC Pauses for Humongous Java Heaps in Spark Graph Computing-(Eric Kacz...Taming GC Pauses for Humongous Java Heaps in Spark Graph Computing-(Eric Kacz...
Taming GC Pauses for Humongous Java Heaps in Spark Graph Computing-(Eric Kacz...
Spark Summit
 
Our Story With ClickHouse at seo.do
Our Story With ClickHouse at seo.doOur Story With ClickHouse at seo.do
Our Story With ClickHouse at seo.do
Metehan Çetinkaya
 
Obtaining the Perfect Smoke By Monitoring Your BBQ with InfluxDB and Telegraf
Obtaining the Perfect Smoke By Monitoring Your BBQ with InfluxDB and TelegrafObtaining the Perfect Smoke By Monitoring Your BBQ with InfluxDB and Telegraf
Obtaining the Perfect Smoke By Monitoring Your BBQ with InfluxDB and Telegraf
InfluxData
 
Accumulo Summit 2015: Performance Models for Apache Accumulo: The Heavy Tail ...
Accumulo Summit 2015: Performance Models for Apache Accumulo: The Heavy Tail ...Accumulo Summit 2015: Performance Models for Apache Accumulo: The Heavy Tail ...
Accumulo Summit 2015: Performance Models for Apache Accumulo: The Heavy Tail ...
Accumulo Summit
 
Kapacitor - Real Time Data Processing Engine
Kapacitor - Real Time Data Processing EngineKapacitor - Real Time Data Processing Engine
Kapacitor - Real Time Data Processing Engine
Prashant Vats
 
On heap cache vs off-heap cache
On heap cache vs off-heap cacheOn heap cache vs off-heap cache
On heap cache vs off-heap cache
rgrebski
 
Postgres Performance for Humans
Postgres Performance for HumansPostgres Performance for Humans
Postgres Performance for Humans
Citus Data
 
Supercharge your Analytics with ClickHouse, v.2. By Vadim Tkachenko
Supercharge your Analytics with ClickHouse, v.2. By Vadim TkachenkoSupercharge your Analytics with ClickHouse, v.2. By Vadim Tkachenko
Supercharge your Analytics with ClickHouse, v.2. By Vadim Tkachenko
Altinity Ltd
 
InfluxData Platform Future and Vision
InfluxData Platform Future and VisionInfluxData Platform Future and Vision
InfluxData Platform Future and Vision
InfluxData
 
[241]large scale search with polysemous codes
[241]large scale search with polysemous codes[241]large scale search with polysemous codes
[241]large scale search with polysemous codes
NAVER D2
 
codecentric AG: Using Cassandra and Clojure for Data Crunching backends
codecentric AG: Using Cassandra and Clojure for Data Crunching backendscodecentric AG: Using Cassandra and Clojure for Data Crunching backends
codecentric AG: Using Cassandra and Clojure for Data Crunching backends
DataStax Academy
 
Distributed Computing on PostgreSQL | PGConf EU 2017 | Marco Slot
Distributed Computing on PostgreSQL | PGConf EU 2017 | Marco SlotDistributed Computing on PostgreSQL | PGConf EU 2017 | Marco Slot
Distributed Computing on PostgreSQL | PGConf EU 2017 | Marco Slot
Citus Data
 
ClickHouse Introduction by Alexander Zaitsev, Altinity CTO
ClickHouse Introduction by Alexander Zaitsev, Altinity CTOClickHouse Introduction by Alexander Zaitsev, Altinity CTO
ClickHouse Introduction by Alexander Zaitsev, Altinity CTO
Altinity Ltd
 
Adventures in Observability: How in-house ClickHouse deployment enabled Inst...
 Adventures in Observability: How in-house ClickHouse deployment enabled Inst... Adventures in Observability: How in-house ClickHouse deployment enabled Inst...
Adventures in Observability: How in-house ClickHouse deployment enabled Inst...
Altinity Ltd
 
Hadoop Query Performance Smackdown
Hadoop Query Performance SmackdownHadoop Query Performance Smackdown
Hadoop Query Performance Smackdown
DataWorks Summit
 
Data correlation using PySpark and HDFS
Data correlation using PySpark and HDFSData correlation using PySpark and HDFS
Data correlation using PySpark and HDFS
John Conley
 
R and cpp
R and cppR and cpp
R and cpp
Romain Francois
 
Accumulo Summit 2015: Ferrari on a Bumpy Road: Shock Absorbers to Smooth Out ...
Accumulo Summit 2015: Ferrari on a Bumpy Road: Shock Absorbers to Smooth Out ...Accumulo Summit 2015: Ferrari on a Bumpy Road: Shock Absorbers to Smooth Out ...
Accumulo Summit 2015: Ferrari on a Bumpy Road: Shock Absorbers to Smooth Out ...
Accumulo Summit
 
Creating and Using the Flux SQL Datasource | Katy Farmer | InfluxData
Creating and Using the Flux SQL Datasource | Katy Farmer | InfluxData Creating and Using the Flux SQL Datasource | Katy Farmer | InfluxData
Creating and Using the Flux SQL Datasource | Katy Farmer | InfluxData
InfluxData
 
A Fast Intro to Fast Query with ClickHouse, by Robert Hodges
A Fast Intro to Fast Query with ClickHouse, by Robert HodgesA Fast Intro to Fast Query with ClickHouse, by Robert Hodges
A Fast Intro to Fast Query with ClickHouse, by Robert Hodges
Altinity Ltd
 
Taming GC Pauses for Humongous Java Heaps in Spark Graph Computing-(Eric Kacz...
Taming GC Pauses for Humongous Java Heaps in Spark Graph Computing-(Eric Kacz...Taming GC Pauses for Humongous Java Heaps in Spark Graph Computing-(Eric Kacz...
Taming GC Pauses for Humongous Java Heaps in Spark Graph Computing-(Eric Kacz...
Spark Summit
 
Our Story With ClickHouse at seo.do
Our Story With ClickHouse at seo.doOur Story With ClickHouse at seo.do
Our Story With ClickHouse at seo.do
Metehan Çetinkaya
 
Obtaining the Perfect Smoke By Monitoring Your BBQ with InfluxDB and Telegraf
Obtaining the Perfect Smoke By Monitoring Your BBQ with InfluxDB and TelegrafObtaining the Perfect Smoke By Monitoring Your BBQ with InfluxDB and Telegraf
Obtaining the Perfect Smoke By Monitoring Your BBQ with InfluxDB and Telegraf
InfluxData
 
Accumulo Summit 2015: Performance Models for Apache Accumulo: The Heavy Tail ...
Accumulo Summit 2015: Performance Models for Apache Accumulo: The Heavy Tail ...Accumulo Summit 2015: Performance Models for Apache Accumulo: The Heavy Tail ...
Accumulo Summit 2015: Performance Models for Apache Accumulo: The Heavy Tail ...
Accumulo Summit
 
Kapacitor - Real Time Data Processing Engine
Kapacitor - Real Time Data Processing EngineKapacitor - Real Time Data Processing Engine
Kapacitor - Real Time Data Processing Engine
Prashant Vats
 
On heap cache vs off-heap cache
On heap cache vs off-heap cacheOn heap cache vs off-heap cache
On heap cache vs off-heap cache
rgrebski
 
Postgres Performance for Humans
Postgres Performance for HumansPostgres Performance for Humans
Postgres Performance for Humans
Citus Data
 
Supercharge your Analytics with ClickHouse, v.2. By Vadim Tkachenko
Supercharge your Analytics with ClickHouse, v.2. By Vadim TkachenkoSupercharge your Analytics with ClickHouse, v.2. By Vadim Tkachenko
Supercharge your Analytics with ClickHouse, v.2. By Vadim Tkachenko
Altinity Ltd
 
InfluxData Platform Future and Vision
InfluxData Platform Future and VisionInfluxData Platform Future and Vision
InfluxData Platform Future and Vision
InfluxData
 
[241]large scale search with polysemous codes
[241]large scale search with polysemous codes[241]large scale search with polysemous codes
[241]large scale search with polysemous codes
NAVER D2
 
codecentric AG: Using Cassandra and Clojure for Data Crunching backends
codecentric AG: Using Cassandra and Clojure for Data Crunching backendscodecentric AG: Using Cassandra and Clojure for Data Crunching backends
codecentric AG: Using Cassandra and Clojure for Data Crunching backends
DataStax Academy
 
Distributed Computing on PostgreSQL | PGConf EU 2017 | Marco Slot
Distributed Computing on PostgreSQL | PGConf EU 2017 | Marco SlotDistributed Computing on PostgreSQL | PGConf EU 2017 | Marco Slot
Distributed Computing on PostgreSQL | PGConf EU 2017 | Marco Slot
Citus Data
 
ClickHouse Introduction by Alexander Zaitsev, Altinity CTO
ClickHouse Introduction by Alexander Zaitsev, Altinity CTOClickHouse Introduction by Alexander Zaitsev, Altinity CTO
ClickHouse Introduction by Alexander Zaitsev, Altinity CTO
Altinity Ltd
 
Adventures in Observability: How in-house ClickHouse deployment enabled Inst...
 Adventures in Observability: How in-house ClickHouse deployment enabled Inst... Adventures in Observability: How in-house ClickHouse deployment enabled Inst...
Adventures in Observability: How in-house ClickHouse deployment enabled Inst...
Altinity Ltd
 
Hadoop Query Performance Smackdown
Hadoop Query Performance SmackdownHadoop Query Performance Smackdown
Hadoop Query Performance Smackdown
DataWorks Summit
 
Data correlation using PySpark and HDFS
Data correlation using PySpark and HDFSData correlation using PySpark and HDFS
Data correlation using PySpark and HDFS
John Conley
 
Accumulo Summit 2015: Ferrari on a Bumpy Road: Shock Absorbers to Smooth Out ...
Accumulo Summit 2015: Ferrari on a Bumpy Road: Shock Absorbers to Smooth Out ...Accumulo Summit 2015: Ferrari on a Bumpy Road: Shock Absorbers to Smooth Out ...
Accumulo Summit 2015: Ferrari on a Bumpy Road: Shock Absorbers to Smooth Out ...
Accumulo Summit
 

Similar to A Deeper Dive into EXPLAIN (20)

Dive into EXPLAIN - PostgreSql
Dive into EXPLAIN  - PostgreSqlDive into EXPLAIN  - PostgreSql
Dive into EXPLAIN - PostgreSql
Dmytro Shylovskyi
 
Why learn Internals?
Why learn Internals?Why learn Internals?
Why learn Internals?
Shaul Rosenzwieg
 
SQL: Query optimization in practice
SQL: Query optimization in practiceSQL: Query optimization in practice
SQL: Query optimization in practice
Jano Suchal
 
Becoming a better developer with EXPLAIN
Becoming a better developer with EXPLAINBecoming a better developer with EXPLAIN
Becoming a better developer with EXPLAIN
Louise Grandjonc
 
Tiling matrix-matrix multiply, code tuning
Tiling matrix-matrix multiply, code tuningTiling matrix-matrix multiply, code tuning
Tiling matrix-matrix multiply, code tuning
mukhi265
 
Gpu programming with java
Gpu programming with javaGpu programming with java
Gpu programming with java
Gary Sieling
 
JDD 2016 - Tomasz Borek - DB for next project? Why, Postgres, of course
JDD 2016 - Tomasz Borek - DB for next project? Why, Postgres, of course JDD 2016 - Tomasz Borek - DB for next project? Why, Postgres, of course
JDD 2016 - Tomasz Borek - DB for next project? Why, Postgres, of course
PROIDEA
 
Quick Wins
Quick WinsQuick Wins
Quick Wins
HighLoad2009
 
Three steps to untangle data traffic jams
Three steps to untangle data traffic jamsThree steps to untangle data traffic jams
Three steps to untangle data traffic jams
Bol.com Techlab
 
Golang in TiDB (GopherChina 2017)
Golang in TiDB  (GopherChina 2017)Golang in TiDB  (GopherChina 2017)
Golang in TiDB (GopherChina 2017)
PingCAP
 
Overview of the Hive Stinger Initiative
Overview of the Hive Stinger InitiativeOverview of the Hive Stinger Initiative
Overview of the Hive Stinger Initiative
Modern Data Stack France
 
Writing Faster Python 3
Writing Faster Python 3Writing Faster Python 3
Writing Faster Python 3
Sebastian Witowski
 
GALE: Geometric active learning for Search-Based Software Engineering
GALE: Geometric active learning for Search-Based Software EngineeringGALE: Geometric active learning for Search-Based Software Engineering
GALE: Geometric active learning for Search-Based Software Engineering
CS, NcState
 
Java/Scala Lab: Анатолий Кметюк - Scala SubScript: Алгебра для реактивного пр...
Java/Scala Lab: Анатолий Кметюк - Scala SubScript: Алгебра для реактивного пр...Java/Scala Lab: Анатолий Кметюк - Scala SubScript: Алгебра для реактивного пр...
Java/Scala Lab: Анатолий Кметюк - Scala SubScript: Алгебра для реактивного пр...
GeeksLab Odessa
 
Conf orm - explain
Conf orm - explainConf orm - explain
Conf orm - explain
Louise Grandjonc
 
AA_Unit 1_part-I.pptx
AA_Unit 1_part-I.pptxAA_Unit 1_part-I.pptx
AA_Unit 1_part-I.pptx
swapnilslide2019
 
Mathemetics module
Mathemetics moduleMathemetics module
Mathemetics module
manikanta361
 
Performance Tuning and Optimization
Performance Tuning and OptimizationPerformance Tuning and Optimization
Performance Tuning and Optimization
MongoDB
 
PostgreSQL: Advanced indexing
PostgreSQL: Advanced indexingPostgreSQL: Advanced indexing
PostgreSQL: Advanced indexing
Hans-Jürgen Schönig
 
Parallel Computing - Lec 4
Parallel Computing - Lec 4Parallel Computing - Lec 4
Parallel Computing - Lec 4
Shah Zaib
 
Dive into EXPLAIN - PostgreSql
Dive into EXPLAIN  - PostgreSqlDive into EXPLAIN  - PostgreSql
Dive into EXPLAIN - PostgreSql
Dmytro Shylovskyi
 
SQL: Query optimization in practice
SQL: Query optimization in practiceSQL: Query optimization in practice
SQL: Query optimization in practice
Jano Suchal
 
Becoming a better developer with EXPLAIN
Becoming a better developer with EXPLAINBecoming a better developer with EXPLAIN
Becoming a better developer with EXPLAIN
Louise Grandjonc
 
Tiling matrix-matrix multiply, code tuning
Tiling matrix-matrix multiply, code tuningTiling matrix-matrix multiply, code tuning
Tiling matrix-matrix multiply, code tuning
mukhi265
 
Gpu programming with java
Gpu programming with javaGpu programming with java
Gpu programming with java
Gary Sieling
 
JDD 2016 - Tomasz Borek - DB for next project? Why, Postgres, of course
JDD 2016 - Tomasz Borek - DB for next project? Why, Postgres, of course JDD 2016 - Tomasz Borek - DB for next project? Why, Postgres, of course
JDD 2016 - Tomasz Borek - DB for next project? Why, Postgres, of course
PROIDEA
 
Three steps to untangle data traffic jams
Three steps to untangle data traffic jamsThree steps to untangle data traffic jams
Three steps to untangle data traffic jams
Bol.com Techlab
 
Golang in TiDB (GopherChina 2017)
Golang in TiDB  (GopherChina 2017)Golang in TiDB  (GopherChina 2017)
Golang in TiDB (GopherChina 2017)
PingCAP
 
GALE: Geometric active learning for Search-Based Software Engineering
GALE: Geometric active learning for Search-Based Software EngineeringGALE: Geometric active learning for Search-Based Software Engineering
GALE: Geometric active learning for Search-Based Software Engineering
CS, NcState
 
Java/Scala Lab: Анатолий Кметюк - Scala SubScript: Алгебра для реактивного пр...
Java/Scala Lab: Анатолий Кметюк - Scala SubScript: Алгебра для реактивного пр...Java/Scala Lab: Анатолий Кметюк - Scala SubScript: Алгебра для реактивного пр...
Java/Scala Lab: Анатолий Кметюк - Scala SubScript: Алгебра для реактивного пр...
GeeksLab Odessa
 
Mathemetics module
Mathemetics moduleMathemetics module
Mathemetics module
manikanta361
 
Performance Tuning and Optimization
Performance Tuning and OptimizationPerformance Tuning and Optimization
Performance Tuning and Optimization
MongoDB
 
Parallel Computing - Lec 4
Parallel Computing - Lec 4Parallel Computing - Lec 4
Parallel Computing - Lec 4
Shah Zaib
 
Ad

More from EDB (20)

Cloud Migration Paths: Kubernetes, IaaS, or DBaaS
Cloud Migration Paths: Kubernetes, IaaS, or DBaaSCloud Migration Paths: Kubernetes, IaaS, or DBaaS
Cloud Migration Paths: Kubernetes, IaaS, or DBaaS
EDB
 
Die 10 besten PostgreSQL-Replikationsstrategien für Ihr Unternehmen
Die 10 besten PostgreSQL-Replikationsstrategien für Ihr UnternehmenDie 10 besten PostgreSQL-Replikationsstrategien für Ihr Unternehmen
Die 10 besten PostgreSQL-Replikationsstrategien für Ihr Unternehmen
EDB
 
Migre sus bases de datos Oracle a la nube
Migre sus bases de datos Oracle a la nube Migre sus bases de datos Oracle a la nube
Migre sus bases de datos Oracle a la nube
EDB
 
EFM Office Hours - APJ - July 29, 2021
EFM Office Hours - APJ - July 29, 2021EFM Office Hours - APJ - July 29, 2021
EFM Office Hours - APJ - July 29, 2021
EDB
 
Benchmarking Cloud Native PostgreSQL
Benchmarking Cloud Native PostgreSQLBenchmarking Cloud Native PostgreSQL
Benchmarking Cloud Native PostgreSQL
EDB
 
Las Variaciones de la Replicación de PostgreSQL
Las Variaciones de la Replicación de PostgreSQLLas Variaciones de la Replicación de PostgreSQL
Las Variaciones de la Replicación de PostgreSQL
EDB
 
NoSQL and Spatial Database Capabilities using PostgreSQL
NoSQL and Spatial Database Capabilities using PostgreSQLNoSQL and Spatial Database Capabilities using PostgreSQL
NoSQL and Spatial Database Capabilities using PostgreSQL
EDB
 
Is There Anything PgBouncer Can’t Do?
Is There Anything PgBouncer Can’t Do?Is There Anything PgBouncer Can’t Do?
Is There Anything PgBouncer Can’t Do?
EDB
 
Data Analysis with TensorFlow in PostgreSQL
Data Analysis with TensorFlow in PostgreSQLData Analysis with TensorFlow in PostgreSQL
Data Analysis with TensorFlow in PostgreSQL
EDB
 
Practical Partitioning in Production with Postgres
Practical Partitioning in Production with PostgresPractical Partitioning in Production with Postgres
Practical Partitioning in Production with Postgres
EDB
 
IOT with PostgreSQL
IOT with PostgreSQLIOT with PostgreSQL
IOT with PostgreSQL
EDB
 
A Journey from Oracle to PostgreSQL
A Journey from Oracle to PostgreSQLA Journey from Oracle to PostgreSQL
A Journey from Oracle to PostgreSQL
EDB
 
Psql is awesome!
Psql is awesome!Psql is awesome!
Psql is awesome!
EDB
 
EDB 13 - New Enhancements for Security and Usability - APJ
EDB 13 - New Enhancements for Security and Usability - APJEDB 13 - New Enhancements for Security and Usability - APJ
EDB 13 - New Enhancements for Security and Usability - APJ
EDB
 
Comment sauvegarder correctement vos données
Comment sauvegarder correctement vos donnéesComment sauvegarder correctement vos données
Comment sauvegarder correctement vos données
EDB
 
Cloud Native PostgreSQL - Italiano
Cloud Native PostgreSQL - ItalianoCloud Native PostgreSQL - Italiano
Cloud Native PostgreSQL - Italiano
EDB
 
New enhancements for security and usability in EDB 13
New enhancements for security and usability in EDB 13New enhancements for security and usability in EDB 13
New enhancements for security and usability in EDB 13
EDB
 
Best Practices in Security with PostgreSQL
Best Practices in Security with PostgreSQLBest Practices in Security with PostgreSQL
Best Practices in Security with PostgreSQL
EDB
 
Cloud Native PostgreSQL - APJ
Cloud Native PostgreSQL - APJCloud Native PostgreSQL - APJ
Cloud Native PostgreSQL - APJ
EDB
 
Best Practices in Security with PostgreSQL
Best Practices in Security with PostgreSQLBest Practices in Security with PostgreSQL
Best Practices in Security with PostgreSQL
EDB
 
Cloud Migration Paths: Kubernetes, IaaS, or DBaaS
Cloud Migration Paths: Kubernetes, IaaS, or DBaaSCloud Migration Paths: Kubernetes, IaaS, or DBaaS
Cloud Migration Paths: Kubernetes, IaaS, or DBaaS
EDB
 
Die 10 besten PostgreSQL-Replikationsstrategien für Ihr Unternehmen
Die 10 besten PostgreSQL-Replikationsstrategien für Ihr UnternehmenDie 10 besten PostgreSQL-Replikationsstrategien für Ihr Unternehmen
Die 10 besten PostgreSQL-Replikationsstrategien für Ihr Unternehmen
EDB
 
Migre sus bases de datos Oracle a la nube
Migre sus bases de datos Oracle a la nube Migre sus bases de datos Oracle a la nube
Migre sus bases de datos Oracle a la nube
EDB
 
EFM Office Hours - APJ - July 29, 2021
EFM Office Hours - APJ - July 29, 2021EFM Office Hours - APJ - July 29, 2021
EFM Office Hours - APJ - July 29, 2021
EDB
 
Benchmarking Cloud Native PostgreSQL
Benchmarking Cloud Native PostgreSQLBenchmarking Cloud Native PostgreSQL
Benchmarking Cloud Native PostgreSQL
EDB
 
Las Variaciones de la Replicación de PostgreSQL
Las Variaciones de la Replicación de PostgreSQLLas Variaciones de la Replicación de PostgreSQL
Las Variaciones de la Replicación de PostgreSQL
EDB
 
NoSQL and Spatial Database Capabilities using PostgreSQL
NoSQL and Spatial Database Capabilities using PostgreSQLNoSQL and Spatial Database Capabilities using PostgreSQL
NoSQL and Spatial Database Capabilities using PostgreSQL
EDB
 
Is There Anything PgBouncer Can’t Do?
Is There Anything PgBouncer Can’t Do?Is There Anything PgBouncer Can’t Do?
Is There Anything PgBouncer Can’t Do?
EDB
 
Data Analysis with TensorFlow in PostgreSQL
Data Analysis with TensorFlow in PostgreSQLData Analysis with TensorFlow in PostgreSQL
Data Analysis with TensorFlow in PostgreSQL
EDB
 
Practical Partitioning in Production with Postgres
Practical Partitioning in Production with PostgresPractical Partitioning in Production with Postgres
Practical Partitioning in Production with Postgres
EDB
 
IOT with PostgreSQL
IOT with PostgreSQLIOT with PostgreSQL
IOT with PostgreSQL
EDB
 
A Journey from Oracle to PostgreSQL
A Journey from Oracle to PostgreSQLA Journey from Oracle to PostgreSQL
A Journey from Oracle to PostgreSQL
EDB
 
Psql is awesome!
Psql is awesome!Psql is awesome!
Psql is awesome!
EDB
 
EDB 13 - New Enhancements for Security and Usability - APJ
EDB 13 - New Enhancements for Security and Usability - APJEDB 13 - New Enhancements for Security and Usability - APJ
EDB 13 - New Enhancements for Security and Usability - APJ
EDB
 
Comment sauvegarder correctement vos données
Comment sauvegarder correctement vos donnéesComment sauvegarder correctement vos données
Comment sauvegarder correctement vos données
EDB
 
Cloud Native PostgreSQL - Italiano
Cloud Native PostgreSQL - ItalianoCloud Native PostgreSQL - Italiano
Cloud Native PostgreSQL - Italiano
EDB
 
New enhancements for security and usability in EDB 13
New enhancements for security and usability in EDB 13New enhancements for security and usability in EDB 13
New enhancements for security and usability in EDB 13
EDB
 
Best Practices in Security with PostgreSQL
Best Practices in Security with PostgreSQLBest Practices in Security with PostgreSQL
Best Practices in Security with PostgreSQL
EDB
 
Cloud Native PostgreSQL - APJ
Cloud Native PostgreSQL - APJCloud Native PostgreSQL - APJ
Cloud Native PostgreSQL - APJ
EDB
 
Best Practices in Security with PostgreSQL
Best Practices in Security with PostgreSQLBest Practices in Security with PostgreSQL
Best Practices in Security with PostgreSQL
EDB
 
Ad

Recently uploaded (20)

Heap, Types of Heap, Insertion and Deletion
Heap, Types of Heap, Insertion and DeletionHeap, Types of Heap, Insertion and Deletion
Heap, Types of Heap, Insertion and Deletion
Jaydeep Kale
 
Semantic Cultivators : The Critical Future Role to Enable AI
Semantic Cultivators : The Critical Future Role to Enable AISemantic Cultivators : The Critical Future Role to Enable AI
Semantic Cultivators : The Critical Future Role to Enable AI
artmondano
 
Enhancing ICU Intelligence: How Our Functional Testing Enabled a Healthcare I...
Enhancing ICU Intelligence: How Our Functional Testing Enabled a Healthcare I...Enhancing ICU Intelligence: How Our Functional Testing Enabled a Healthcare I...
Enhancing ICU Intelligence: How Our Functional Testing Enabled a Healthcare I...
Impelsys Inc.
 
Procurement Insights Cost To Value Guide.pptx
Procurement Insights Cost To Value Guide.pptxProcurement Insights Cost To Value Guide.pptx
Procurement Insights Cost To Value Guide.pptx
Jon Hansen
 
Splunk Security Update | Public Sector Summit Germany 2025
Splunk Security Update | Public Sector Summit Germany 2025Splunk Security Update | Public Sector Summit Germany 2025
Splunk Security Update | Public Sector Summit Germany 2025
Splunk
 
Cyber Awareness overview for 2025 month of security
Cyber Awareness overview for 2025 month of securityCyber Awareness overview for 2025 month of security
Cyber Awareness overview for 2025 month of security
riccardosl1
 
Massive Power Outage Hits Spain, Portugal, and France: Causes, Impact, and On...
Massive Power Outage Hits Spain, Portugal, and France: Causes, Impact, and On...Massive Power Outage Hits Spain, Portugal, and France: Causes, Impact, and On...
Massive Power Outage Hits Spain, Portugal, and France: Causes, Impact, and On...
Aqusag Technologies
 
How analogue intelligence complements AI
How analogue intelligence complements AIHow analogue intelligence complements AI
How analogue intelligence complements AI
Paul Rowe
 
What is Model Context Protocol(MCP) - The new technology for communication bw...
What is Model Context Protocol(MCP) - The new technology for communication bw...What is Model Context Protocol(MCP) - The new technology for communication bw...
What is Model Context Protocol(MCP) - The new technology for communication bw...
Vishnu Singh Chundawat
 
SAP Modernization: Maximizing the Value of Your SAP S/4HANA Migration.pdf
SAP Modernization: Maximizing the Value of Your SAP S/4HANA Migration.pdfSAP Modernization: Maximizing the Value of Your SAP S/4HANA Migration.pdf
SAP Modernization: Maximizing the Value of Your SAP S/4HANA Migration.pdf
Precisely
 
Andrew Marnell: Transforming Business Strategy Through Data-Driven Insights
Andrew Marnell: Transforming Business Strategy Through Data-Driven InsightsAndrew Marnell: Transforming Business Strategy Through Data-Driven Insights
Andrew Marnell: Transforming Business Strategy Through Data-Driven Insights
Andrew Marnell
 
Technology Trends in 2025: AI and Big Data Analytics
Technology Trends in 2025: AI and Big Data AnalyticsTechnology Trends in 2025: AI and Big Data Analytics
Technology Trends in 2025: AI and Big Data Analytics
InData Labs
 
Manifest Pre-Seed Update | A Humanoid OEM Deeptech In France
Manifest Pre-Seed Update | A Humanoid OEM Deeptech In FranceManifest Pre-Seed Update | A Humanoid OEM Deeptech In France
Manifest Pre-Seed Update | A Humanoid OEM Deeptech In France
chb3
 
Complete Guide to Advanced Logistics Management Software in Riyadh.pdf
Complete Guide to Advanced Logistics Management Software in Riyadh.pdfComplete Guide to Advanced Logistics Management Software in Riyadh.pdf
Complete Guide to Advanced Logistics Management Software in Riyadh.pdf
Software Company
 
Increasing Retail Store Efficiency How can Planograms Save Time and Money.pptx
Increasing Retail Store Efficiency How can Planograms Save Time and Money.pptxIncreasing Retail Store Efficiency How can Planograms Save Time and Money.pptx
Increasing Retail Store Efficiency How can Planograms Save Time and Money.pptx
Anoop Ashok
 
Cybersecurity Identity and Access Solutions using Azure AD
Cybersecurity Identity and Access Solutions using Azure ADCybersecurity Identity and Access Solutions using Azure AD
Cybersecurity Identity and Access Solutions using Azure AD
VICTOR MAESTRE RAMIREZ
 
ThousandEyes Partner Innovation Updates for May 2025
ThousandEyes Partner Innovation Updates for May 2025ThousandEyes Partner Innovation Updates for May 2025
ThousandEyes Partner Innovation Updates for May 2025
ThousandEyes
 
tecnologias de las primeras civilizaciones.pdf
tecnologias de las primeras civilizaciones.pdftecnologias de las primeras civilizaciones.pdf
tecnologias de las primeras civilizaciones.pdf
fjgm517
 
Dev Dives: Automate and orchestrate your processes with UiPath Maestro
Dev Dives: Automate and orchestrate your processes with UiPath MaestroDev Dives: Automate and orchestrate your processes with UiPath Maestro
Dev Dives: Automate and orchestrate your processes with UiPath Maestro
UiPathCommunity
 
Big Data Analytics Quick Research Guide by Arthur Morgan
Big Data Analytics Quick Research Guide by Arthur MorganBig Data Analytics Quick Research Guide by Arthur Morgan
Big Data Analytics Quick Research Guide by Arthur Morgan
Arthur Morgan
 
Heap, Types of Heap, Insertion and Deletion
Heap, Types of Heap, Insertion and DeletionHeap, Types of Heap, Insertion and Deletion
Heap, Types of Heap, Insertion and Deletion
Jaydeep Kale
 
Semantic Cultivators : The Critical Future Role to Enable AI
Semantic Cultivators : The Critical Future Role to Enable AISemantic Cultivators : The Critical Future Role to Enable AI
Semantic Cultivators : The Critical Future Role to Enable AI
artmondano
 
Enhancing ICU Intelligence: How Our Functional Testing Enabled a Healthcare I...
Enhancing ICU Intelligence: How Our Functional Testing Enabled a Healthcare I...Enhancing ICU Intelligence: How Our Functional Testing Enabled a Healthcare I...
Enhancing ICU Intelligence: How Our Functional Testing Enabled a Healthcare I...
Impelsys Inc.
 
Procurement Insights Cost To Value Guide.pptx
Procurement Insights Cost To Value Guide.pptxProcurement Insights Cost To Value Guide.pptx
Procurement Insights Cost To Value Guide.pptx
Jon Hansen
 
Splunk Security Update | Public Sector Summit Germany 2025
Splunk Security Update | Public Sector Summit Germany 2025Splunk Security Update | Public Sector Summit Germany 2025
Splunk Security Update | Public Sector Summit Germany 2025
Splunk
 
Cyber Awareness overview for 2025 month of security
Cyber Awareness overview for 2025 month of securityCyber Awareness overview for 2025 month of security
Cyber Awareness overview for 2025 month of security
riccardosl1
 
Massive Power Outage Hits Spain, Portugal, and France: Causes, Impact, and On...
Massive Power Outage Hits Spain, Portugal, and France: Causes, Impact, and On...Massive Power Outage Hits Spain, Portugal, and France: Causes, Impact, and On...
Massive Power Outage Hits Spain, Portugal, and France: Causes, Impact, and On...
Aqusag Technologies
 
How analogue intelligence complements AI
How analogue intelligence complements AIHow analogue intelligence complements AI
How analogue intelligence complements AI
Paul Rowe
 
What is Model Context Protocol(MCP) - The new technology for communication bw...
What is Model Context Protocol(MCP) - The new technology for communication bw...What is Model Context Protocol(MCP) - The new technology for communication bw...
What is Model Context Protocol(MCP) - The new technology for communication bw...
Vishnu Singh Chundawat
 
SAP Modernization: Maximizing the Value of Your SAP S/4HANA Migration.pdf
SAP Modernization: Maximizing the Value of Your SAP S/4HANA Migration.pdfSAP Modernization: Maximizing the Value of Your SAP S/4HANA Migration.pdf
SAP Modernization: Maximizing the Value of Your SAP S/4HANA Migration.pdf
Precisely
 
Andrew Marnell: Transforming Business Strategy Through Data-Driven Insights
Andrew Marnell: Transforming Business Strategy Through Data-Driven InsightsAndrew Marnell: Transforming Business Strategy Through Data-Driven Insights
Andrew Marnell: Transforming Business Strategy Through Data-Driven Insights
Andrew Marnell
 
Technology Trends in 2025: AI and Big Data Analytics
Technology Trends in 2025: AI and Big Data AnalyticsTechnology Trends in 2025: AI and Big Data Analytics
Technology Trends in 2025: AI and Big Data Analytics
InData Labs
 
Manifest Pre-Seed Update | A Humanoid OEM Deeptech In France
Manifest Pre-Seed Update | A Humanoid OEM Deeptech In FranceManifest Pre-Seed Update | A Humanoid OEM Deeptech In France
Manifest Pre-Seed Update | A Humanoid OEM Deeptech In France
chb3
 
Complete Guide to Advanced Logistics Management Software in Riyadh.pdf
Complete Guide to Advanced Logistics Management Software in Riyadh.pdfComplete Guide to Advanced Logistics Management Software in Riyadh.pdf
Complete Guide to Advanced Logistics Management Software in Riyadh.pdf
Software Company
 
Increasing Retail Store Efficiency How can Planograms Save Time and Money.pptx
Increasing Retail Store Efficiency How can Planograms Save Time and Money.pptxIncreasing Retail Store Efficiency How can Planograms Save Time and Money.pptx
Increasing Retail Store Efficiency How can Planograms Save Time and Money.pptx
Anoop Ashok
 
Cybersecurity Identity and Access Solutions using Azure AD
Cybersecurity Identity and Access Solutions using Azure ADCybersecurity Identity and Access Solutions using Azure AD
Cybersecurity Identity and Access Solutions using Azure AD
VICTOR MAESTRE RAMIREZ
 
ThousandEyes Partner Innovation Updates for May 2025
ThousandEyes Partner Innovation Updates for May 2025ThousandEyes Partner Innovation Updates for May 2025
ThousandEyes Partner Innovation Updates for May 2025
ThousandEyes
 
tecnologias de las primeras civilizaciones.pdf
tecnologias de las primeras civilizaciones.pdftecnologias de las primeras civilizaciones.pdf
tecnologias de las primeras civilizaciones.pdf
fjgm517
 
Dev Dives: Automate and orchestrate your processes with UiPath Maestro
Dev Dives: Automate and orchestrate your processes with UiPath MaestroDev Dives: Automate and orchestrate your processes with UiPath Maestro
Dev Dives: Automate and orchestrate your processes with UiPath Maestro
UiPathCommunity
 
Big Data Analytics Quick Research Guide by Arthur Morgan
Big Data Analytics Quick Research Guide by Arthur MorganBig Data Analytics Quick Research Guide by Arthur Morgan
Big Data Analytics Quick Research Guide by Arthur Morgan
Arthur Morgan
 

A Deeper Dive into EXPLAIN

  • 1. A deeper dive into EXPLAIN Michael Christofides
  • 2. Hi, I’m Michael Half of the team behind pgMustard Spent a lot of time looking into EXPLAIN Background: product management, database tools pgmustard.com/docs/explain [email protected] michristofides
  • 3. Picking up from other EXPLAIN talks Not the basics* 1) Some of the less intuitive arithmetic 2) Some less well covered issues * postgresql.org/docs/current/performance-tips thoughtbot: reading EXPLAIN ANALYZE YouTube: Josh Berkus Explaining EXPLAIN
  • 4. Picking up from other EXPLAIN talks Not the basics* 1) Arithmetic: why is this query slow? 2) Issues: what can we do about it? * postgresql.org/docs/current/performance-tips thoughtbot: reading EXPLAIN ANALYZE YouTube: Josh Berkus Explaining EXPLAIN
  • 5. Arithmetic: loops Many of the stats are a per-loop average This includes costs, rows, timings Watch out for rounding, especially to 0 rows
  • 6. Disclaimer: heavily doctored plans ahead, inaccuracies likely.
  • 7. Nested Loop (cost=0.84..209.82 rows=16 width=11) (actual time=0.076..0.368 rows=86 loops=1) -> Index Only Scan using a on b (cost=0.42..4.58 rows=9 width=4) (actual time=0.013..0.019 rows=9 loops=1) -> Index Scan using x on y (cost=0.42..22.73 rows=7 width=15) (actual time=0.012..0.030 rows=10 loops=9) Nested Loop: 86 rows Index Scan: 9 * 10 = 90 rows (Rounding not too bad here)
  • 8. Arithmetic: threads Costs, rows, and timings are also per-thread Shown as loops Threads = workers + 1 Tip: use VERBOSE <- the leader
  • 9. Parallel Seq Scan on table (cost=0.00..6772.21 rows=79521 width=22) (actual time=0.090..71.866 rows=63617 loops=3) Output: column1, column2, column3 Worker 0: actual time=0.111..66.325 rows=56225 loops=1 Worker 1: actual time=0.138..66.027 rows=58792 loops=1 Seq Scan: 63617 * 3 = 190851 rows Leader: 190851 - 58792 - 56225 = 75834 rows
  • 10. Arithmetic: buffers Buffer stats are a total, not per-loop They are inclusive of children
  • 11. Nested Loop (... loops=1) Buffers: shared hit=105 -> Index Only Scan using a on b (... loops=1) Buffers: shared hit=4 -> Index Scan using x on y (... loops=9) Buffers: shared hit=101 Nested Loop buffers: 105 - (101 + 4) = 0 blocks
  • 12. Arithmetic: timings Per-loop, per-thread Inclusive of children Calculating per-node (exclusive) times can get tricky, even for tools
  • 13. Nested Loop (cost=0.84..209.82 rows=16 width=11) (actual time=0.076..0.368 rows=86 loops=1) -> Index Only Scan using a on b (cost=0.42..4.58 rows=9 width=4) (actual time=0.013..0.019 rows=9 loops=1) -> Index Scan using x on y (cost=0.42..22.73 rows=7 width=15) (actual time=0.012..0.030 rows=10 loops=9) Index Scan: 0.030 * 9 = 0.27 ms Nested Loop: 0.368 - 0.27 - 0.019 = 0.079 ms
  • 14. WITH init AS ( SELECT * FROM pg_sleep_for('100ms') UNION ALL SELECT * FROM pg_sleep_for('200ms') ) (SELECT * FROM init LIMIT 1) UNION ALL (SELECT * FROM init); Credit @felixge
  • 15. Append (actual time=100.359.. 300.688 … ) CTE init -> Append (actual time=100.334.. 300.652 … ) -> Function Scan (actual time=100.333.. 100.335 … ) -> Function Scan (actual time=200.310.. 200.312 … ) -> Limit (actual time=100.358.. 100.359 … ) -> CTE Scan a (actual time=100.355..100.356 … ) -> CTE Scan b (actual time=0.001.. 200.322 … ) Execution Time: 300.789 ms Further reading: flame-explain.com/docs/general/quirk-correction Some double-counting in this case.
  • 16. Arithmetic: tools can help eg explain.depesz.com explain.dalibo.com flame-explain.com pgmustard.com <- fellow calculations nerd <- 👋
  • 17. Issues: let’s skip the basics Seq Scans with large filters Bad row estimates Sorts and Hashes on disk
  • 18. Issues: inefficient index scans Looks out for lots of rows being filtered Filters are per-loop So again, watch out for rounding
  • 19. -> Index Scan using x on y (cost=0.42..302502.05 rows=1708602 width=125) (actual time=172810.219..173876.540 rows=1000 loops=1) Index Cond: (id = another_id) Filter: (status = 1) Rows Removed by Filter: 3125626 Index efficiency: 1000/(1000+3125626) = 0.03% Watch out for high loops
  • 20. Issues: lossy bitmap scans When bitmap would otherwise exceed work_mem Point to a block rather than a row (Tuple Id) Lossy blocks are a total (ie not per-loop)
  • 21. -> Bitmap Heap Scan on table (cost=49153.29..4069724.27 rows=3105598 width=1106) (actual time=591.928..56472.895 rows=3853272 loops=1) Recheck Cond: (something > something_else) Rows Removed by Index Recheck: 5905323 Heap Blocks: exact=14280 lossy=1951048 Lossy blocks: 1951048/(1951048+14280) = 99% Extra rows read: 5.9 million
  • 22. Issues: lots of data read Requires BUFFERS Lots of data being read for the amount returned Can be a sign of bloat Default block size: 8kB
  • 23. -> Index Scan using x on y (cost=0.57..2.57 rows=1 width=8) (actual time=0.064..0.064 rows=1 loops=256753) Index Cond: (id = another_id) Filter: (status = 1) Buffers: shared hit=1146405 read=110636 Caveats: width estimated, rows rounded Data read: (1146405 + 110636) * 8kB = 10GB Data returned: 1 * 256753 * 8 bytes = 2MB
  • 24. Issues: planning time At the end of the query plan Can be planning related: eg joins, partitions But other things too: eg extensions, locks Warning: not included in auto_explain
  • 25. (...) Planning Time: 27.844 ms Execution Time: 11.162 ms Planning proportion: 27.844/(27.844 + 11.162) = 71%
  • 26. Issues: Just In Time compilation At the end of the query plan Included in execution time On by default in PostgreSQL 12 and 13 Start-up time can be a tell-tale
  • 27. Planning Time: 9.138 ms JIT: Functions: 277 Options: Inlining true, Optimization true, Expressions true, Deforming true Timing: Generation 31.602 ms, Inlining 253.114 ms, Optimization 1498.268 ms, Emission 913.945 ms, Total 2696.929 ms Execution Time: 5194.851 ms JIT proportion: 2696.929/(9.138 + 5194.851) = 52%
  • 28. Very suspicious actual start-up time from a JIT dominated plan. -> Seq Scan on table (cost=0.00..3.57 rows=72 width=8) (actual time=2262.312..2262.343 rows=54 loops=1) Buffers: shared hit=3
  • 29. Issues: triggers At the end of the query plan Total time across calls Check foreign keys indexed Before triggers vs after triggers
  • 30. Planning Time: 0.227 ms Trigger: RI_ConstraintTrigger_a_12345 on table time=83129.491 calls=2222623 Execution Time: 87645.739 ms Trigger proportion: 83129.491/(0.227 + 87645.739) = 95%
  • 31. Summary: check the arithmetic Watch out for loops and threads Watch out for CTEs Tools can help, if in doubt check two
  • 32. Summary: keep rarer issues in mind Check the end section first Also look out for filters, rechecks, lossy blocks, amount of data Tools, mailing lists, and communities can help
  • 33. Thank you! Any questions? [email protected] michristofides Further reading: * flame-explain.com/docs/general/quirk-correction * pgmustard.com/docs/explain * wiki.postgresql.org/wiki/Slow_Query_Questions