Tuning Autovacuum in Postgresql
0 likes479 views
The document discusses tuning autovacuum in PostgreSQL. It provides an overview of autovacuum, how it helps prevent database bloat, and best practices for configuring autovacuum parameters like autovacuum_vacuum_threshold, autovacuum_analyze_threshold, autovacuum_naptime, and autovacuum_max_workers. It emphasizes regularly monitoring for bloat, configuring autovacuum appropriately based on table sizes and usage, and avoiding manual vacuuming when not needed.
![syntax :
VACUUM [ FULL ] [ FREEZE ] [ VERBOSE ] [ ANALYZE ] [ table_and_columns [, ...] ]
Vacuum](https://image.slidesharecdn.com/tuning-autovacuum-in-postgresql-220924081418-80e7f489/85/Tuning-Autovacuum-in-Postgresql-22-320.jpg)
Tuning Autovacuum in Postgresql
- 1. Tuning Autovacuum in PostgreSQL Mohammad Zaid Patel Database Consultant Mydbops Mydbops 16th MyWebinar
- 2. • Database Consultant for PostgreSQL • Active Learner • Technophile • Intrigued in PostgreSQL and open source software • Likes Cricket & Music About Me
- 3. • Founded in 2016 • Services on top open source databases • 80+ Member team • AWS Partner • PCI & ISO Certified Organisation About Mydbops
- 4. Database Consulting Services Managed Database Services Focuses on Top Opensource database MySQL,MariaDB, MongoDB and PostgreSQL both ON Premises and Cloud Mydbops Services
- 5. 500 + Clients In 6 Yrs. of Operations Our Clients
- 6. Follow us on LinkedIN !
- 7. Agenda • Database Bloat • What is Vacuum • What is Autovacuum • Why Autovacuum • Tuning Autovacuum • Best Practices
- 8. ID NAME 1 TONY 2 STEVE 3 THOR AVENGERS >> UPDATE AVENGERS SET NAME='BRUCE' WHERE ID=3 ; ID NAME 1 TONY 2 STEVE 3 BRUCE ID NAME 1 TONY 2 STEVE 3 THOR 3 BRUCE • SELECTS, INSERTS, UPDATES and DELETES are some of the common actions in any database • UPDATE = Deletion of Old value + Insertion of New Value User A Introduction
- 9. • Rows marked invisible or as 'dead tuple' • UNDO log maintained inside the table • No space reclaimed • Dead tuples accumulation Introduction
- 10. PostgreSQL's MVCC model !
- 11. ID NAME 1 TONY 2 STEVE 3 THOR 3 BRUCE ID NAME 1 TONY 2 STEVE 3 BRUCE User A User B >> SELECT NAME FROM AVENGERS WHERE ID=3 ; ID NAME 1 TONY 2 STEVE 3 THOR ID NAME 3 THOR User B ID NAME 1 TONY 2 STEVE 3 THOR MVCC model
- 12. • Data consistency is maintained • Past image and latest image • Every row of PostgreSQL table has a version number • Transaction isolation for each database session • Locks acquired for querying data don't conflict with locks acquired for writing MVCC model
- 13. Database Bloat
- 14. Query to check bloat : postgres=# SELECT pg_size_pretty(pg_relation_size('test')) as table_size,pgstattuple('test')).dead_tuple_percent; • Bloating of the Database is caused due to dead tuples • Increased Dead Tuples = Increased Bloat Database bloat
- 15. Why is bloat bad ? • Slow Sequential scans • Transaction id wraparound • Eats up Storage • Dead tuples > Live tuple = No more updates on the table
- 16. ID NAME 3 THOR Xmin Xmax 1 10 Postgres uses 32 bit Transaction ID i.e 2^32 transaction IDs Transaction ID wraparound
- 18. • Limited Transaction IDs • Once upper limit is reached , ids get wraparound • Difficult for Postgres to figure out if data is transaction is in past or future • Postgres shuts down daatabase to protect the data • Warnings by Postgres • Very Tedious to bring the database back Transaction ID wraparound
- 19. How to remove bloat ? VACUUM IT !
- 20. What is Vacuum ?
- 21. • An important PostgreSQL feature • Removes the dead tuples • Reclaims storage • Various options Vacuum
- 22. syntax : VACUUM [ FULL ] [ FREEZE ] [ VERBOSE ] [ ANALYZE ] [ table_and_columns [, ...] ] Vacuum
- 23. VACUUM FULL : • Reclaimed storage is returned back to the OS • High on resource consumption • Holds up lock • Slower than Vacuum • Requires extra disk space Vacuum
- 24. Vacuum VACUUM FREEZE : • Marks a table's contents with a very special transaction timestamp • Freezes committed transaction IDs • Next update will unfreeze it.
- 25. VERBOSE : • Prints a detailed vacuum activity report for each table Vacuum
- 26. Vacuum ANALYZE : • Updates statistics used by the planner • Determines the most efficient way to execute a query
- 27. Is VACUUM perfect solution ?
- 28. • Manual • High in Cost • Employs Locking • Needs extra disk space Downsides of Vacuum
- 29. We've got Autovacuum !
- 30. What is Autovacuum ?
- 31. • Released with Postgresql version 8.3 • Background utility • Automate the execution of VACUUM and ANALYZE • Not required to be triggered manually • Does not free up the space • Reallocates deleted blocks Autovacuum
- 32. The autovacuum daemon consists of multiple processes : • Autovacuum launcher • Distributes the work across time, attempting to start one worker within each database every 'autovacuum_nap' seconds • Maximum of 'autovacuum_max_worker' worker processes are allowed to run at the same time • Checks each table within its database and execute VACUUM and/or ANALYZE as needed • Functionality of Autovacuum depends on the settings available inside postgresql.conf file. Autovacuum
- 33. Disabling AUTOVACUUM on a Table: ALTER TABLE table_name SET (autovacuum_enabled = false); Autovacuum
- 34. Why Autovacuum ?
- 35. • Table bloating is prevented • No locking on Tables • Not resource-intensive • Better Storage space utilization • Improves FSM Advantages of Autovacuum
- 36. • Prevents transaction id wraparound • Faster query execution • Faster sequential scans • Better performance of Database Advantages of Autovacuum
- 39. autovacuum = ON autovacuum = OFF track_counts = on Tuning Autovacuum
- 40. autovacuum_vacumm_threshold : Specifies the minimum number of updated or deleted tuples needed to trigger a VACUUM in any one table Tuning Autovacuum
- 41. Tuning Autovacuum autovacuum_analyze_threshold : Specifies the minimum number of inserted, updated or deleted tuples needed to trigger an ANALYZE in any one table
- 42. autovacuum_vacuum_scale_factor: Specifies a fraction of the table size to add to autovacumm_vacuum_threshold when deciding whether to trigger a VACUUM Tuning Autovacuum
- 43. Tuning Autovacuum autovacuum_analyze_scalefactor: Specifies a fraction of the table size to add to autovacuum_analyze_threshold when deciding whether to trigger an ANALYZE
- 44. autovacuum_naptime: Minimum time that postgresql wait between each auto vacuum. Tuning Autovacuum
- 45. Tuning Autovacuum autovacuum_max_workers: Specifies the maximum number of autovacuum processes
- 46. autovacuum_vacuum_cost_limit: Specifies the cost delay value that will be used in automatic VACUUM operations after cost limit is reached. Tuning Autovacuum
- 47. Tuning Autovacuum autovacuum_vacuum_cost_delay: Specifies the cost limit value that will be used in automatic VACUUM operations
- 48. Tuning Autovacuum max_parallel_maintenance_workers: Sets the maximum number of parallel workers that can be started by a single utility command
- 49. Tuning Autovacuum hot_standby_feedback: Specifies whether or not a hot standby will send feedback to the primary or upstream standby about queries currently executing on the standby
- 50. • autovacuum = on • autovacuum_vacuum_threshold = 50 , default • autovacuum_analyze_threshold = 50 , default • autovacuum_vacuum_scale_factor = 0.20 , default • autovacuum_analyze_scale_factor = 0.20 , default • autovacuum_naptime = 60 , default Parameters Overview
- 51. • autovacuum_max_workers = 3, default • autovacuum_vacuum_cost_limit = -1 • autovacuum_vacuum_cost_delay = 2ms • max_parallel_maintenance_workers = 2 • hot_standby_feedback = on Parameters Overview
- 52. Best practises
- 53. • Detecting Bloat • Configuring Autovacuum suitable for most of the tables • Maintenance windows • Removing bloat for Indexes • Avoid running manual vacuum/analyze for no reason • Considering important and critical queries • Performance after vacuum/analyze should be studied Best Practises
- 54. Reach Us : [email protected] Thank You