This post was originally published in March 2023 and was updated in March 2025.
When it comes to optimizing MongoDB, compression is a key lever for reducing storage footprint, minimizing I/O bottlenecks, and even speeding up data transfer. The benefits are clear: significant cost savings and the ability to handle more data within the same infrastructure. But with options available, which method should you choose?
This post will compare two prominent compression algorithms used in MongoDB: Snappy vs. Zstd. We’ll examine their application for data block compression and discuss Zstd’s role in network compression, helping you understand the trade-offs and make informed decisions for your specific use case.
Percona Server for MongoDB (PSMDB) supports all types of compression and enterprise-grade features for free. I am using PSMDB 6.0.4 here.
Data compression
MongoDB offers various block compression methods used by the WiredTiger storage engine, like snappy, zlib, and zstd.
When data is written to disk, MongoDB compresses it with a specified block compression method and then writes it to disk. When this data block is read, it decompresses it in memory and presents it to the incoming request.
Block compression is a type of compression that compresses data in blocks rather than compressing the entire data set at once. Block compression can improve performance by allowing data to be read and written in smaller chunks.
By default, MongoDB provides a snappy block compression method for storage and network communication.
Snappy compression is designed to be fast and efficient regarding memory usage, making it a good fit for MongoDB workloads. Snappy is a compression library developed by Google.
Benefits of snappy compression in MongoDB:
- Fast compression and decompression speeds
- Low CPU usage
- A streamable format that allows for quick processing
- Minimal impact on query performance
Zstandard Compression or zstd, another newer block compression method provided by MongoDB starting for v4.2, provides higher compression rates. Zstd is a compression library that Facebook developed.
Zstd typically offers a higher compression ratio than snappy, meaning that it can compress data more effectively and achieve a smaller compressed size for the same input data.
Benefits of zstd compression in MongoDB:
- Higher compression ratios than Snappy
- Highly configurable compression levels
- Fast compression and decompression speeds
- Minimal impact on query performance
To enable zstd block compression, you need to specify the block compressor as “zstd” in the configuration file:
1 2 3 4 5 6 7 8 | storage: engine: wiredTiger wiredTiger: collectionConfig: blockCompressor: zstd blockCompressorQuality: 6 #(available since v5.0) engineConfig: cacheSizeGB: 4 |
In the above example, blockCompressorQuality is set to 6, which is the default.
blockCompressorQuality specifies the level of compression applied when using the zstd compressor. Values can range from 1 to 22.
The higher the specified value for zstdCompressionLevel, the higher the compression which is applied. So, it becomes very important to test for the optimal required use case before implementing it in production.
Here, we are going to test snappy and zstd compression with the following configurations.
Host config: 4vCPU, 14 GB RAM
DB version: PSMDB 6.0.4
OS: CentOS Linux 7
I’ve used mgenerate command to insert a sample document.
1 | mgeneratejs '{"name": "$name", "age": "$age", "emails": {"$array": {"of": "$email", "number": 3}}}' -n 120000000 | mongoimport --uri mongodb://localhost:27017/<db> --collection <coll_name> --mode insert |
Sample record:
1 2 3 4 | _id: ObjectId("64195975e40cea62af1be510"), name: 'Verna Grant', age: 44, |
I’ve created a collection using the below command with a specific block compression method. This does not affect any existing collection or any new collection being created after this.
1 | db.createCollection("user", {storageEngine: {wiredTiger: {configString: "block_compressor=zstd"}}}) |
If any new collection is created in the default manner, it will always be the default snappy or compression method specified in the mongod config file.
At the time of insert ops, no other queries or DML ops were running in the database.
Snappy
Data size: 14.95GB
Data size after compression: 10.75GB
Avg latency: 12.22ms
Avg cpu usage: 34%
Avg insert ops rate: 16K/s
Time taken to import 120000000 document: 7292 seconds
Zstd (with default compression level 6)
Data size: 14.95GB
Data size after compression: 7.69GB
Avg latency: 12.52ms
Avg cpu usage: 31.72%
Avg insert ops rate: 14.8K/s
Time taken to import 120000000 document: 7412 seconds
We can see from the above comparison that we can save almost 3GB of disk space without impacting the CPU or memory.
Network compression
MongoDB also offers network compression.
This can further reduce the amount of data that needs to be transmitted between server and client over the network. This, in turn, requires less bandwidth and network resources, which can improve performance and reduce costs.
It supports the same compression algorithms for network compression, i.e., snappy, zstd, and zlib. All these compression algorithms have various compression ratios and CPU needs.
To enable network compression in mongod and mongos, you can specify the compression algorithm by adding the following line to the configuration file.
1 2 3 | net: compression: compressors: snappy |
We can also use multiple compression algorithms like
1 2 3 | net: compression: compressors: snappy,zstd,zlib |
The client should also use at least one or the same compression method specified in the config to have data over the network compressed, or the data between the client and server would be uncompressed.
In the below example, I am using a python driver to connect to my server with no compression, and zstd compression algorithm
I am doing simple find ops on the sample record shown above.
This is the outbound data traffic without any compression method
Here we can see data transmitted is around 2.33MB/s:
Now, I’ve enabled zstd compression algorithm in both the server and client
1 | client = pymongo.MongoClient("mongodb://user:[email protected]:27017/?replicaSet=rs1&authSource=admin&compressors=zstd") |
Here we can see data avg outbound transmission is around 1MB/s which is almost a 50% reduction.
Note that network compression can have a significant impact on network performance and CPU usage. In my case, there was hardly anything else running, so I did not see any significant CPU usage.
Conclusion
Ultimately, the choice between Snappy and Zstd compression in MongoDB hinges on your specific workload and performance priorities. Snappy often excels in write-heavy scenarios demanding lower CPU overhead, while Zstd typically offers superior compression ratios beneficial for read-intensive workloads or significant storage savings. By understanding these trade-offs for both data block and network-level compression, and ideally testing their impact on your specific application, you can make an informed decision to optimize storage efficiency and database performance in your MongoDB deployment.
I also recommend using Percona Server for MongoDB, which provides MongoDB enterprise-grade features without any license, as it is free. You can learn more about it in the blog MongoDB: Why Pay for Enterprise When Open Source Has You Covered?
Percona also offers some more great products for MongoDB, like Percona Backup for MongoDB, Percona Kubernetes Operator for MongoDB, and Percona Monitoring and Management.
Ready to leave MongoDB, Inc. behind? Percona is the cost-effective, enterprise-grade alternative businesses trust.
About the Author
Santosh VarmaI'm a NoSQL DBA, Close to 9 years of experience. I like exploring best practices around the database and explore new technologies.
