Quality Assurance in PostgreSQL

Apr 28, 20171 like1,453 views

This document discusses quality assurance processes for PostgreSQL, an open source object-relational database. It covers development practices like code reviews on mailing lists and testing patches through "commitfests". Comprehensive testing is emphasized, including modular unit tests, integration tests, performance benchmarks, and documentation reviews. A variety of tools are also presented to aid testing, debugging, and performance profiling of PostgreSQL code.

Quality Assurance in
PostgreSQL
Anastasia Lubennikova
Aleksander Alekseev

Agenda
● Development
● Testing
● Benchmarking
● Tools
● Other Topics

What is PostgreSQL?
● Open source object-relational database system
● Over 20 years of development
● Hundreds of developers around the world
● Several major vendors

What’s so Special About PostgreSQL?
● Pure C
● Complicated project
● Cross-platform (Windows, Linux, MacOS, Solaris, *BSD, AIX, HP-UX, ...)
● High code quality
● Great documentation
● Major releases every year

Development Process
● Git, C, Autotools, Perl
● Mailing Lists
○ pgsql-hackers@
○ pgsql-committers@
○ pgsql-bugs@
○ etc
● Conferences
○ pgcon.org
○ pgconf.us
○ pgconf.ru
○ … and many more!

Submitting a Patch
● Optional: Proposal and discussion
● Optional: PoC
● Patch is added to commitfest
● While(not ready):
○ Code review
○ Testing
○ Discussion
○ More discussion
○ Rewrite a patch
● Either:
○ Patch is accepted
○ Patch is moved to the next commitfest
○ Patch is returned with a feedback

What to test?
● Modular* tests
○ Indexes
○ Replication
○ Types
○ WAL / Recovery
● Integration/system tests
○ How various features work together?
○ Upgrade, Backup/Restore, etc
● Performance
○ Previous versions
○ Other implementations
● Documentation (SGML)
● Alpha and beta tests

How to test?
● Code review
● Regular SQL input / output tests
○ Can’t test utilities, concurrency, etc..
○ Must be deterministic
● TAP/Perl tests
○ SSL
○ Replication and recovery
● pgTAP framework
● Testgres - python framework

SQL tests
● Can only test SQL-level stuff
● Must be deterministic
● Should be executed to see the exact output

Benchmarks
● Industry standard benchmarks
○ TPC-H
○ TPC-E
○ TPC-B
● Pgbench

Basic Tools
● gcc / clang / ...
● gdb / lldb
● vim + :set spell spelllang=en_us,ru_ru
● Asserts
● wal_consistency_checking = 'all'
● log_level = DEBUG
● amcheck (property based tests)
● sqlsmith (fuzzing)
● valgrind

Links
● https://www.postgresql.org/
● http://postgres-edu.blogspot.ru/search/label/Hacking%20PostgreSQL
● https://github.com/afiskon/pgscripts
● https://habr.ru/p/308442/ + related articles

Thank you for your attention!
● https://postgrespro.com/ (We are hiring! ;)
● a.lubennikova@postgrespro.ru
● a.alekseev@postgrespro.ru

This document discusses data compression techniques for PostgreSQL, including ZSON for compressing JSONB data and pg_protobuf for indexing Protobuf data. It provides overviews of how these extensions work, such as replacing common strings with codes for ZSON and avoiding shared dictionaries for pg_protobuf. The document also shares ideas for future projects, such as adding modification support to pg_protobuf or generating PL/pgSQL procedures for Protobuf fields. Finally, it invites the audience to learn more about these open source projects and announces job opportunities at Postgres Professional.

Protocol buffersManuel Correa

Protocol buffers are a flexible, efficient, and automated mechanism for serializing structured data that is simpler, faster, smaller and generates access classes compared to XML and JSON. It works by defining message types in .proto files that can then be compiled to generate data access classes in various languages like Java, C++, Python and more. Many large companies like Google, Twitter and Oracle use protocol buffers for internal APIs, queue systems and database objects due to its performance advantages over other data formats.

Evolution or stagnation programming languagesDaniele Esposti

Programming languages evolves with the need to the developers, but not all of them evolves at the same speed and sometimes some languages stays almost the same for decades. Is this a sign of stagnation? Is it possible to evolve a language without breaking retrocompatibility? This talk will cover how Python and Javascript approached the problem in a radically different way, with their pro and cons and with the consequences on the community. The main points of this talk are: * comparation of the new features in both languages * strategies used to port or run the code on different versions of the language * impact of the new features and tool on the respective developer’s communities The intent is to start a constructive discussion about the retrocompatility in Python. For this talk you don’t need to have a deep knowledge of Python or Javascript.

Go Programming language, golangBasil N G

The document provides an overview of the Go programming language. It discusses that Go was designed by Google to help solve their large-scale programming problems. It then summarizes Go's history, purpose, features, and syntax elements such as data types, operators, functions, concurrency using goroutines and channels. The document also provides examples of Go code and concludes that Go has emerged as a popular language for cloud infrastructure due to its simplicity, concurrency, and performance.

An introduction to go programming languageTechnology Parser

Go is a new programming language developed by Google as a systems language for building network and cloud services. It was created to address the need for a modern language that supports concurrency and multicore processors. The design goals of Go included being fast, easy to use, and supporting features like garbage collection and parallelism. While the syntax is C-like and it is statically typed, Go also incorporates elements from dynamically typed languages for ease of programming.

A Shallow Survey of Alternative Languages on the JVMCharles Anderson

This document surveys alternative languages that run on the Java Virtual Machine (JVM), including ported languages like JRuby and new languages like Groovy and Scala. It discusses why developers use these languages, noting they provide access to powerful Java platforms and libraries while allowing programming in more modern languages. Issues like performance and compatibility challenges are also outlined. Examples of code in several languages are provided.

Golang (Go Programming Language)ShubhamMishra485

Understanding how concurrency work in osGenchiLu1

This document discusses concurrency in operating systems and different programming languages. It explains how concurrency works at the OS level using schedulers and threads/processes. It then compares implementations of concurrency in Python, Java, and Go. Python uses threads but is constrained by the GIL, Java uses native threads, and Go uses lightweight goroutines scheduled across OS threads. The document cautions that while goroutines make concurrency easy, there are still costs to consider. It concludes by noting languages evolve over time and no approach is inherently unable to handle high concurrency applications.

Getting Started with PHP ExtensionsMichaelBrunoLochemem

Daniele Esposti - Evolution or stagnation programming languages - Codemotion ...Codemotion

HelsinkiJS - Clojurescript for Javascript DevelopersJuho Teperi

This document provides an overview of ClojureScript for JavaScript developers. It introduces ClojureScript and its similarities and differences compared to Clojure and JavaScript. Key points covered include ClojureScript's compilation to JavaScript, its support for immutable data, asynchronous programming using Core.async, and interoperability with JavaScript. Popular ClojureScript libraries and tools for development are also discussed. The document concludes by outlining reasons one might choose to use ClojureScript, such as its ability to target multiple platforms while sharing logic, its ecosystem, and community.

Adopting language server for apache camel feedback from a java/Eclipse plugi...Aurélien Pupier

A First Look at Google's Go Programming LanguageGanesh Samarthyam

Go is a new systems programming language from Google. Go has many interesting features such as 'communication channels' that makes it suitable for use in multi-core machines, and network programming. With Ken Thompson (of Unix fame) as one of its designers, Go has elegant and minimal design that is appealing to most programmers. This talk gives a technical introduction to Go that is of interest to anyone working in system software. [Presentation I have in 2010 - I haven't updated it with recent changes to the Go language]

Go langSuelen Carvalho

Go is a programming language created by Google to help solve problems with large software and hardware systems. It was designed to facilitate development of large codebases by many engineers. Some key problems it aimed to address were slowness, clumsiness and lack of productivity in other languages like C++. Go provides features like garbage collection, concurrency with goroutines and channels, and a standard library, while remaining simple and compiled. It grew from a small project at Google into an open source language adopted by many organizations.

Advantages of Python Learning | Why PythonEvoletTechnologiesCo

This document provides an overview of the Python programming language. It discusses what Python is, how it compares to other languages like Java and C++, common uses of Python, job opportunities in Python, and the future of Python. It also advertises training in Python offered by Evolet Technologies, which aims to teach students to use Python professionally and gain expertise through hands-on learning and practice problems.

An Introduction to GoImesh Gunaratne

Google created Go because existing systems programming languages did not provide efficient compilation, execution, and ease of programming simultaneously. Go combines the ease of dynamically typed languages with the efficiency and safety of compiled, statically typed languages. It has features for concurrency and garbage collection that improve on C, as well as syntax inspired by Python, JavaScript, and Pascal.

Go Programming Language by GoogleUttam Gandhi

Go is a programming language created by Google that aims to be a simple, efficient, and concurrent language. The document provides an overview of the history and features of Go, including its support for garbage collection, concurrency, and ease of programming. Examples are given demonstrating how to write Go code, use interfaces and channels for concurrency, and connect to MongoDB. The document also lists several companies that use Go in production applications.

From NodeJS to RustBastian Gruber

The document discusses choosing Rust over NodeJS for software development. It provides an overview of the speaker's background and experience with Rust. Some key differences between NodeJS and Rust mentioned are that Rust has no need for TypeScript or additional tooling, ships binaries, and allows for more performant code through compiler checks. The document also outlines steps for getting started with Rust, including recommended books and practice projects. Popular Rust libraries for web development are also listed.

My talk on Piter Py 2016Alex Chistyakov

Alex Chistyakov gives a presentation on performance engineering. He discusses using profiling tools like the Poor Man's Profiler and flamegraphs to analyze the performance of a Python program. He profiles a Deluge BitTorrent client running in Docker and finds that it spends most of its time in the libtorrent C++ library with about 30% overhead from Python. The profiling helped identify areas for potential optimization.

ATO 2014 - So You Think You Know 'Go'? The Go Programming LanguageJohn Potocny

JetBrains MPSВладимир Кожаев

The Go programming language - Intro by MyLittleAdventuremylittleadventure

The automation of the process of caring for the quality of the code in PHP an...Krzysztof (Chris) Ozog

This document discusses tools for automating code quality checks in PHP and JavaScript. It begins with an overview of the history of code quality, from unstructured "spaghetti code" to standardized practices. It then covers generally accepted practices like DRY principles. The document outlines PHP standards like PSR-0/4 and tools like PHP CodeSniffer and PHP Mess Detector. Similar JavaScript standards and tools are covered, including JSLint, JsHint, and ESLint. It emphasizes integrating these tools with editors and continuous integration systems to automatically enforce code quality standards.

Go language presentationparamisoft

The document discusses the Go programming language and why it was created. It provides several key points: - Go was created over a decade since a new major systems language emerged, and the computing landscape has changed significantly in areas like software development speed, dependency management, type systems, garbage collection, and parallelism. - Go aims to address these changes with a compiled, garbage-collected language that provides fast compilation, easy dependency analysis, lightweight static types, built-in support for concurrency and communication on multicore systems. - Some of Go's guiding principles in design were to reduce typing, clutter, and complexity while avoiding forward declarations and header files everything is declared once without type hierarchies.

Google Go! languageAndré Mayer

Boosting python web apps with protocol buffers & grpcNaren Arya

This document discusses using Protocol Buffers (Protobuf) and GRPC to boost Python web apps. It begins by explaining why JSON is popular but has limitations. Protobuf provides benefits like binary encoding for smaller size and support for HTTP/2. The key steps to use Protobuf with Python are installing the protoc compiler and generating Python class files from .proto schema files. The document teases additional upcoming slides on working with a real-world example, introducing GRPC, and building GRPC servers and clients in Python using Protobuf to scale microservices.

Dart the better Javascript 2015Jorg Janke

Elephant Roads: a tour of Postgres forksCommand Prompt., Inc

Josh Berkus Most users know that PostgreSQL has a 23-year development history. But did you know that Postgres code is used for over a dozen other database systems? Thanks to our liberal licensing, many companies and open source projects over the years have taken the Postgres or PostgreSQL code, changed it, added things to it, and/or merged it into something else. Illustra, Truviso, Aster, Greenplum, and others have seen the value of Postgres not just as a database but as some darned good code they could use. We'll explore the lineage of these forks, and go into the details of some of the more interesting ones.

More Related Content

What's hot (20)

Understanding how concurrency work in osGenchiLu1

Getting Started with PHP ExtensionsMichaelBrunoLochemem

Daniele Esposti - Evolution or stagnation programming languages - Codemotion ...Codemotion

HelsinkiJS - Clojurescript for Javascript DevelopersJuho Teperi

Adopting language server for apache camel feedback from a java/Eclipse plugi...Aurélien Pupier

A First Look at Google's Go Programming LanguageGanesh Samarthyam

Go langSuelen Carvalho

Advantages of Python Learning | Why PythonEvoletTechnologiesCo

An Introduction to GoImesh Gunaratne

Go Programming Language by GoogleUttam Gandhi

From NodeJS to RustBastian Gruber

My talk on Piter Py 2016Alex Chistyakov

ATO 2014 - So You Think You Know 'Go'? The Go Programming LanguageJohn Potocny

JetBrains MPSВладимир Кожаев

The Go programming language - Intro by MyLittleAdventuremylittleadventure

The automation of the process of caring for the quality of the code in PHP an...Krzysztof (Chris) Ozog

Go language presentationparamisoft

Google Go! languageAndré Mayer

Boosting python web apps with protocol buffers & grpcNaren Arya

Understanding how concurrency work in osGenchiLu1

Getting Started with PHP ExtensionsMichaelBrunoLochemem

Daniele Esposti - Evolution or stagnation programming languages - Codemotion ...Codemotion

HelsinkiJS - Clojurescript for Javascript DevelopersJuho Teperi

Adopting language server for apache camel feedback from a java/Eclipse plugi...Aurélien Pupier

A First Look at Google's Go Programming LanguageGanesh Samarthyam

Go langSuelen Carvalho

Advantages of Python Learning | Why PythonEvoletTechnologiesCo

An Introduction to GoImesh Gunaratne

Go Programming Language by GoogleUttam Gandhi

From NodeJS to RustBastian Gruber

My talk on Piter Py 2016Alex Chistyakov

ATO 2014 - So You Think You Know 'Go'? The Go Programming LanguageJohn Potocny

JetBrains MPSВладимир Кожаев

The Go programming language - Intro by MyLittleAdventuremylittleadventure

The automation of the process of caring for the quality of the code in PHP an...Krzysztof (Chris) Ozog

Go language presentationparamisoft

Google Go! languageAndré Mayer

Boosting python web apps with protocol buffers & grpcNaren Arya

More from Aleksander Alekseev (11)

PostgreSQL Sharding and HA: Theory and Practice (PGConf.ASIA 2017)Aleksander Alekseev

This document summarizes Aleksander Alekseev's talk on PostgreSQL sharding and high availability. The talk introduces replication in PostgreSQL, including physical and logical replication. It discusses solutions for high availability like Stolon and PostgreSQL Multimaster, as well as existing solutions for sharding like manual sharding, Citus, Greenplum, and pg_shardman. It also briefly covers other databases like Amazon Aurora and CockroachDB that are inspired by PostgreSQL but with additional distributed functionality.

Data recovery using pg_filedumpAleksander Alekseev

The document demonstrates how to recover data from PostgreSQL database files using the pg_filedump tool. It shows extracting table data and metadata like the table schema from the heap and system catalog files. Key points extracted include: - pg_filedump can display formatted contents of PostgreSQL files including tables, indexes, and system catalogs - Running pg_filedump on the table file extracted the table data including column types - Further analysis of system catalog files using pg_filedump provided the table and column names and types to fully recover the table schema

Full Text Search in PostgreSQLAleksander Alekseev

This document summarizes full text search capabilities in PostgreSQL. It begins with an introduction and overview of common full text search solutions. It then discusses reasons to use full text search in PostgreSQL, including consistency and no need for additional software. The document covers basics of full text search in PostgreSQL like to_tsvector, to_tsquery, and indexes. It also covers fuzzy full text search using pg_trgm and functions like similarity. Other topics mentioned include ts_headline, ts_rank, and the RUM extension.

pg_filedumpAleksander Alekseev

The document summarizes how to use the pg_filedump tool to recover data from PostgreSQL database files. It provides an example command to dump data from blocks containing integer, boolean, text and timestamp values. The output shows the recovered data includes two items - a text value and Russian text, with metadata on the block offsets, lengths and timestamps. Additional references are provided for more detailed articles on data recovery in Russian.

In-core compression: how to shrink your database size in several timesAleksander Alekseev

The document discusses techniques for compressing database size in Postgres, including: 1. Using in-core block-level compression as a feature of Postgres Pro EE to shrink database size by several times. 2. The ZSON extension provides transparent JSONB compression by replacing common strings with 16-bit codes and compressing the data. 3. Various schema optimizations like proper data types, column ordering, and packing data can reduce size by improving storage layout and enabling TOAST compression.

ZSON, или прозрачное сжатие JSONAleksander Alekseev

Профилирование кода на C/C++ в *nix системахAleksander Alekseev

Новые технологии репликации данных в PostgreSQL - Александр АлексеевAleksander Alekseev

Haskell - это просто - Александр АлексеевAleksander Alekseev

Работа с Akka Cluster - Александр АлексеевAleksander Alekseev

Функциональное программирование - Александр АлексеевAleksander Alekseev

PostgreSQL Sharding and HA: Theory and Practice (PGConf.ASIA 2017)Aleksander Alekseev

Data recovery using pg_filedumpAleksander Alekseev

Full Text Search in PostgreSQLAleksander Alekseev

pg_filedumpAleksander Alekseev

In-core compression: how to shrink your database size in several timesAleksander Alekseev

ZSON, или прозрачное сжатие JSONAleksander Alekseev

Профилирование кода на C/C++ в *nix системахAleksander Alekseev

Новые технологии репликации данных в PostgreSQL - Александр АлексеевAleksander Alekseev

Haskell - это просто - Александр АлексеевAleksander Alekseev

Работа с Akka Cluster - Александр АлексеевAleksander Alekseev

Функциональное программирование - Александр АлексеевAleksander Alekseev

Recently uploaded (19)

OSI TCP IP Protocol Layers description fcbr49917

5-Proses-proses Akuisisi Citra Digital.pptxandani26

IT Services Workflow From Request to Resolutionmzmziiskd

DNS Resolvers and Nameservers (in New Zealand)APNIC

Top Vancouver Green Business Ideas for 2025 Powered by 4GoodHostingsteve198109

Vancouver in 2025 is more than scenic views, yoga studios, and oat milk lattes—it’s a thriving hub for eco-conscious entrepreneurs looking to make a real difference. If you’ve ever dreamed of launching a purpose-driven business, now is the time. Whether it’s urban mushroom farming, upcycled furniture sales, or vegan skincare sold online, your green idea deserves a strong digital foundation. The 2025 Canadian eCommerce landscape is being shaped by trends like sustainability, local innovation, and consumer trust. To stay ahead, eco-startups need reliable hosting that aligns with their values. That’s where 4GoodHosting.com comes in—one of the top-rated Vancouver web hosting providers of 2025. Offering secure, sustainable, and Canadian-based hosting solutions, they help green entrepreneurs build their brand with confidence and conscience. As eCommerce in Canada embraces localism and environmental responsibility, choosing a hosting provider that shares your vision is essential. 4GoodHosting goes beyond just hosting websites—they champion Canadian businesses, sustainable practices, and meaningful growth. So go ahead—start that eco-friendly venture. With Vancouver web hosting from 4GoodHosting, your green business and your values are in perfect sync.

Best web hosting Vancouver 2025 for you businesssteve198109

APNIC Update, presented at NZNOG 2025 by Terry SweetserAPNIC

Perguntas dos animais - Slides ilustrados de múltipla escolhasocaslev

Computers Networks Computers Networks Computers NetworksTito208863

Understanding the Tor Network and Exploring the Deep Webnabilajabin35

While the Tor network, Dark Web, and Deep Web can seem mysterious and daunting, they are simply parts of the internet that prioritize privacy and anonymity. Using tools like Ahmia and onionland search, users can explore these hidden spaces responsibly and securely. It’s essential to understand the technology behind these networks, as well as the risks involved, to navigate them safely. Visit https://torgol.com/

(Hosting PHising Sites) for Cryptography and network securityaluacharya169

project_based_laaaaaaaaaaearning,kelompok 10.pptxredzuriel13

highend-srxseries-services-gateways-customer-presentation.pptxelhadjcheikhdiop

Reliable Vancouver Web Hosting with Local Servers & 24/7 Supportsteve198109

Looking for powerful and affordable web hosting in Vancouver? 4GoodHosting offers premium Canadian web hosting solutions designed specifically for individuals, startups, and businesses across British Columbia. With local data centers in Vancouver and Toronto, we ensure blazing-fast website speeds, superior uptime, and enhanced data privacy—all critical for your business success in today’s competitive digital landscape. Our Vancouver web hosting plans are packed with value—starting as low as $2.95/month—and include secure cPanel management, free domain transfer, one-click WordPress installs, and robust email support with anti-spam protection. Whether you're hosting a personal blog, business website, or eCommerce store, our scalable cloud hosting packages are built to grow with you. Enjoy enterprise-grade features like daily backups, DDoS protection, free SSL certificates, and unlimited bandwidth on select plans. Plus, our expert Canadian support team is available 24/7 to help you every step of the way. At 4GoodHosting, we understand the needs of local Vancouver businesses. That’s why we focus on speed, security, and service—all hosted on Canadian soil. Start your online journey today with a reliable hosting partner trusted by thousands across Canada.

Smart Mobile App Pitch Deck丨AI Travel App Presentation Templateyojeari421237

🚀 Smart Mobile App Pitch Deck – "Trip-A" | AI Travel App Presentation Template This professional, visually engaging pitch deck is designed specifically for developers, startups, and tech students looking to present a smart travel mobile app concept with impact. Whether you're building an AI-powered travel planner or showcasing a class project, Trip-A gives you the edge to impress investors, professors, or clients. Every slide is cleanly structured, fully editable, and tailored to highlight key aspects of a mobile travel app powered by artificial intelligence and real-time data. 💼 What’s Inside: - Cover slide with sleek app UI preview - AI/ML module implementation breakdown - Key travel market trends analysis - Competitor comparison slide - Evaluation challenges & solutions - Real-time data training model (AI/ML) - “Live Demo” call-to-action slide 🎨 Why You'll Love It: - Professional, modern layout with mobile app mockups - Ideal for pitches, hackathons, university presentations, or MVP launches - Easily customizable in PowerPoint or Google Slides - High-resolution visuals and smooth gradients 📦 Format: - PPTX / Google Slides compatible - 16:9 widescreen - Fully editable text, charts, and visuals

White and Red Clean Car Business Pitch Presentation.pptxcanumatown

APNIC -Policy Development Process, presented at Local APIGA Taiwan 2025APNIC

Mobile database for your company telemarketing or sms marketing campaigns. Fr...DataProvider1

Determining Glass is mechanical textileAzizul Hakim