PostgreSQL is, without a doubt, one of the most popular open source databases in the world. Why? Well, there are many reasons, but if I had to pick just one, I’d say it’s extensibility. PostgreSQL isn’t just a database; it’s an ecosystem of extensions that can transform it to tackle any upcoming challenges. By enabling developers to expand database capabilities without hitting a paywall, PostgreSQL has not only stayed relevant but thrived with an ever-growing ecosystem of thousands of extensions. But with great power comes great responsibility (yes, Uncle Ben, I went there), and one area where PostgreSQL has been lagging behind other modern databases is data-at-rest encryption… until now.
The age of compliance
While much recent hype has been around AI, data privacy laws, and the ethical use of machine learning, traditional data security regulations haven’t gone anywhere. In fact, they’re evolving alongside tech advancements. It feels like there is a new regulation, security framework, or another compliance checkbox to tick off every week, and it can be quite overwhelming to stay up-to-date with each change.
For years, many database professionals have debated that Transparent Data Encryption (TDE) is redundant, claiming that encryption-at-rest requirements could be satisfied at the storage layer. While database professionals discussed the necessity of TDE, businesses have been moving in a direction of more security, driven by modern expectations regarding data protection. Regulatory standards are becoming more stringent, and there are several examples where storage encryption alone is explicitly defined as insufficient. A prime example is PCI DSS 4.0, which mandates that Primary Account Numbers (PAN) must be encrypted within the database itself; this means that storage or disk-level encryption alone is no longer enough. PCI DSS is just one of many evolving regulations that explicitly push for stronger database encryption.
Application level encryption
If storage-level encryption isn’t enough, what about application-level encryption? In theory, this is the most secure approach as it enforces strict access controls, aligns with zero-trust principles, and ensures that only authorized applications (and users) can decrypt sensitive data. However, in practice, application-level encryption is often too expensive, too complex, and simply unfeasible for many organizations, especially those relying on legacy systems.
Many large enterprises operate on decades-old applications that weren’t designed with encryption in mind. Retrofitting them to support application-layer encryption requires significant code changes, infrastructure overhauls, and new key management strategies. For many businesses, this level of transformation just isn’t an option due to cost, time, and technical debt.
Tokenization
Another potential approach is tokenization, which replaces sensitive data with non-sensitive placeholders. While this can be a strong compliance-friendly solution, it often requires proprietary third-party software and significant application-level changes. Additionally, it introduces new challenges, such as managing tokenization performance and ensuring seamless integration with existing business processes. If application-level encryption is often a no-go due to the costs it generates, tokenization is often simply too costly and disruptive to implement at scale for many solutions that still need security.
Transparent Data Encryption
This leaves us with Transparent Data Encryption (TDE). TDE offers a pragmatic balance between security and practicality. It encrypts data at the database level, shielding it from unauthorized access while remaining completely transparent to applications. No code changes, no architectural overhauls, just secure data right where it’s stored.
With compliance tightening and legacy constraints preventing many from adopting application-layer encryption or tokenization, TDE isn’t just a “nice-to-have” anymore, it’s becoming a necessity.
In the end, we are looking at the choice between:
- Tokenization – Great, but requires new software (often proprietary) and application-level changes.
- Application level encryption – Super secure but expensive and complex. Not everyone wants to rewrite their entire app.
- Transparent data encryption (TDE) – The easiest approach, where the database handles encryption seamlessly without requiring application changes.
From our vantage point, the best, most cost-effective, and easy-to-implement solution is TDE.
Release Candidate (RC) for Transparent Data Encryption (TDE)
We’re thrilled to announce that with Percona Distribution for PostgreSQL 17.4.1-1, we’ve released the Release Candidate (RC) for TDE!
The Generally Available (GA) release that will be ready for production use is just around the corner, but getting here hasn’t been easy. Our journey to open source TDE was filled with experimentation, iteration, and redesigning parts:
- tde_heap_basic,
- WAL encryption
Reworking the table encryption with tde_heap
With the RC release, we’re officially deprecating tde_heap_basic. The GA version will rely on tde_heap entirely.
Our first attempt, tde_heap_basic worked with the unmodified PostgreSQL Community Server but also had significant drawbacks:
- The performance cost of encrypting data moved in and out of memory rather than storage.
- Indexes weren’t encrypted, making it a partial solution.
We addressed these challenges with tde_heap, a new approach that integrates with PostgreSQL’s Storage Manager (SMGR) and Write-Ahead Log (WAL) based on a couple of small patches to PostgreSQL itself. These small changes make the database more extensible while ensuring efficient and comprehensive encryption.
The SMGR patch is based on previous work from Neon, with improvements for more general use cases (including TDE). We’ve already submitted it for community review, so if you’re a PostgreSQL hacker, please take a look!
Rethinking WAL encryption
The initial WAL encryption method encountered challenges, particularly concerning the replication and the ability to replace encryption keys. Following comprehensive testing and assessment of various options, we implemented modifications that necessitated significant code alterations.
The latest WAL encryption method is incorporated into the RC as a tech preview, and we welcome community input. As WAL encryption is an optional feature, its refinement will not impede the overall GA release of TDE. Nonetheless, we remain dedicated to delivering a production-ready WAL encryption solution promptly, guaranteeing performance and dependability.
pg_tde has a new home
If you’ve stuck with us this far, you’re clearly interested. So here’s another update: pg_tde has moved!
You can now find it under the Percona Server for PostgreSQL repository. Since tde_heap_basic is going away and tde_heap requires a patched PostgreSQL, we didn’t want to confuse people by hosting it separately.
Check it out and give the project a star on GitHub, and check out our TDE solution in the new home
What’s in store short term?
- Deliver a production-ready version of the TDE solution
- Enable integrations with leading security solutions on the market
- Provide troubleshooting materials and documentation to ensure easier adoption
If you use a particular KMS that you would like to see integrated or see use cases that are currently not appropriately addressed by TDE, please let us know!
We need your feedback!
Open source only thrives when the community gets involved. Either you are invested enough to read this far, or you just scrolled to the bottom to see the end notes. Either way, help us out!
- Try TDE out and share your thoughts – Your feedback will shape the final product.
- Join discussions on our Community Forums – More voices = better solutions.
- Give us a GitHub star – It helps more people discover the project.
- Meet us at conferences – We love talking to users face-to-face!
Transparent Data Encryption is a game-changer for PostgreSQL. We’re excited about this milestone, and we hope you are, too. Now, go encrypt some data!
About the Author
Jan WieremjewiczJan is a Senior Product Manager at Percona, leading the products for PostgreSQL. He has vast experience in the development, deployment and maintenance of enterprise systems. Professionally, he is passionate about simple solutions that solve complicated problems and user experience that maximizes the product potential. Privately he is a foodie by day, a tech geek into graphic novels, video and board games by night and parent/spouse in between. Having spent already almost two years in Percona, most of us learned that he has enough energy and topics to fill in any space and time.
