Hybrid Columnar Compression in a non-Exadata System

Hybrid Columnar Compression
for Non-Exadata Databases
Peter Brink, Credit Suisse

Enkitec E4 conference, Dallas
13/14 August 2012

About me

Peter Brink
 15 years experience in Data Warehouse projects within Financials
 13 years working with Oracle databases

Enkitec E4 conference, Dallas Peter Brink
13/14 August 2012 Page 2

Agenda
 Hybrid Columnar Compression without Exadata?
 HCC overview
 HCC proof of concept with ZFS appliance
- Planning the PoC
- Test cases and test results
 Get the most out of HCC
 Discussion


Hybrid Columnar Compression without Exadata?


Hybrid Columnar Compression without Exadata

Oracle Press Release, 30th September 2011
 Oracle Announces Hybrid Columnar Compression Support for ZFS
Storage Appliances and Pillar Axiom Storage Systems


Hybrid Columnar Compression without Exadata
 HCC compression is built into the 11g database,
but only enabled when running on Oracle storage
SQL> select table_name, compression, compress_for
2 from dba_tables
3 where table_name = 'T1_ARCHIVE_HIGH';
TABLE_NAME COMPRESS COMPRESS_FOR
------------------------------ -------- ------------
T1_ARCHIVE_HIGH ENABLED ARCHIVE HIGH

SQL> select c1 from t1_archive_high;
select c1 from t1_archive_high
*
ERROR at line 1:
ORA-64307: hybrid columnar compression is only supported in tablespaces
residing on Exadata storage

see Jonathan Lewis’ trick on how to create (and keep) an HCC
table
 dbms_compression (11.2.0.1)
creates HCC compressed tables during estimation of compression
ratios


Hybrid Columnar Compression overview

Table Compression types
 Basic
9i
only available for bulk operations
 OLTP or Advanced Compression
11g
available for all operations
 Hybrid Columnar Compression
query low
query high } Warehouse compression
X
archive low
archive high } Online archival compression
12c



OLTP Compression
 dictionary compression, self-contained within single blocks
 tables with >255 columns are silently not compressed
 locking works the same as for uncompressed blocks

Initial inserts are PCTFREE triggers Further uncompressed PCTFREE triggers
uncompressd compression inserts compression



 HCC compression only used for bulk operations
conventional inserts and updates fall back to OLTP compression
updates result in rows migrated to new blocks
 Tables are organised in Compression Units (CU)
 Logical structure spanning multiple blocks
 Organised by column during data load
 Columns are compressed individually

Compression Unit, Database Concepts Guide



HCC compression types

Type Description
Query Low • LZO compression algorithm
• Optimised for speed rather than high compression ratio

Query High • ZLIB compression algorithm
Archive Low • ZLIB with higher compression level
• Does not necessarily yield higher compression than
Query High
Archive High • BZIP2 compression
• Yields the highest compression ratio
• Significant cpu overhead



 create table T1 (col1 date, col1 number …)
partition by range col1(
partition P2009 … compress for archive high,
partition P2010 … compress for archive low,
partition P2011 … compress for query high,
partition P2012Q1 … compress for OLTP,
partition P2012Q2 … nocompress
)
different compression types can be used for partitions of a single
table
 alter table T1 compress for query high;
does not result in existing blocks getting compressed
 alter table T1 nocompress;
will not decompress blocks in a table
 The table needs to be rebuilt to apply changed compression settings
to existing data

Introducing the ZFS Storage Appliance

 Oracle Hardware
 Primarily a NAS Storage device
 it runs ZFS inside the device
 Also capable of providing Fibre Channel, iSCSI, etc
 Provides “full service” as NAS storage
 cloning
 Snapshots
 Data deduplication
 Options available from shelves to full racks
3.3 TB to 1.7 PB of raw capacity


HCC proof of concept with ZFS storage appliance

Pre-requirements for HCC compression with ZFS SA
 Database must be running on 11.2.0.3 + patch p13041324
 Database must use NFS datafiles (ideally dNFS)
 Standard requirements for performant NFS implementations
 10 GigE Networking
 Network adjacency (one hop between host and storage)
 Jumbo Frames throughput (MTU 9000)


HCC proof of concept – Application Characteristics

 Risk application chosen for POC
Hybrid between Data Warehouse and OLTP
 10 TB, 2 node RAC cluster used for data loading and reporting
 2.5 TB read-only, archive database

 uses OLTP compression
}
 subset of tables with data older than 6 month not in scope
for POC

 Growth in data volumes driven by:
 business growth
 demand for more detailed analysis
 increased data granularity and risk factors
 more complex risk methodologies


Data loading
 More than 500 GB trade level detail data loaded every week
-> detail data gets purged within a period of one to six weeks
 Aggregated into > 50 GB of final data
-> final data is kept forever
 24 x 5.5 loading activity
 No direct path loads!



Reporting and Analysis
 Only aggregated data is used for reporting purposes
 Reporting activity counts for 80% of database workload
 80% of reports are generated for the last 2 COB dates
data for last 2 COB dates will be in the buffer cache
 Data extracts rely on nested loop joins / index access!


HCC proof of concept – Compression Advisor

DBMS_COMPRESSION.GET_COMPRESSION_RATIO
 Does not require Exadata storage, only needs a 11r2 database
 The advisor gets the ratio by compressing sample data. It will
consume CPU and requires disk space
 Accuracy depends on how representative the sample set is
Sample results:
Compression type Predicted Ratio Ratio during POC
Archive High 17.1 16.7 – 27.5
Archive Low 13.5 12.9 – 20.8
Query High 10.9 10.7 – 17.8
Query Low 5.8 6.6 – 9.5


HCC proof of concept – DML performance

DML Performance Impact
 Updates to compressed data will take longer and expand the space
used
 Sample results (update primary key to a new value for all rows)
Size before Size after (MB) Elapsed time
(MB) (seconds)
Uncompressed 860 860 30
HCC - Query Low 154 694 246

 Updates of a single row of a HCC table locks the Compression Unit
containing the row

Avoid updates of HCC compressed rows


HCC proof of concept – Query Performance

 Positive impact when disk IO can be reduced or avoided
 Negative impact caused by cpu usage for decompression and
additional logical IO when majority of blocks are cached
 Test result from a representative set of report queries:
Elapsed Time CPU time IO Wait time
(s) (s) (s)
Uncached Uncompressed 24356 3542 20652
Query Low 5071 3726 1352
Query High 6681 5064 246
Cached Uncompressed 2171 1711 372
Query Low 2593 2556 33
Query High 3515 3492 30



Disk space consumption:
 HCC will give no benefits on 34% space used for indexes
 Challenges adapting HCC for 44% of table space for trade level
data
less than 30% of total database size in scope


HCC proof of concept – Configuration

Compression Matrix by data type
Compression Size before Size after (GB)
(GB)
Trade level Uncompressed 2037 2037
Aggregated Query High for 1512 139
partitions older than
one week,
latest partitions are
uncompressed
Static Uncompressed 270 270
Audit data Rebuild with Query 745 46
High, then set to
nocompress without
rebuild


HCC proof of concept – Application performance test

 Upgrade to 11.2.0.3 resulted in regressed performance for loading
component. Setting optimizer_features_enable to 11.2.0.2 resolved
this issue
 No further impact on data loads.
Occasional updates to HCC rows resulted in rows migrating, but no
noticeable performance impact and no locking issues
 Reporting application validated previous test results.
Performance improvement for small percentage of reports retrieving
HCC rows


Get the most out of HCC

Design your application to take advantage of HCC
 Get your partitioning right
 Use bulk loads that can use HCC compression
 Avoid updates
 Minimize single row lookups
 In-memory-parallel execution
 Ensure you have sufficient CPU capacity


Get the most out of HCC

Moving data from an Exadata to a non-Exadata platform
 Without ZFS or Pillar storage data in HCC format can not be used
by non-Exadata database
 Using Oracle storage allows accessing HCC blocks when a failover
/ restore of an Exadata database to a non-Exadata platform occurs


Conclusion

 The Oracle ZFS Storage appliance offers genuine storage saving
capabilities through the Hybrid Columnar Compression feature
 Actual storage savings depend on what objects can be compressed.
For the application evaluated only a 20% reduction was achievable
 Natural fit for any Oracle database already using Oracle storage
 Being allowed to use HCC compression without a storage platform
change would be appreciated


Bibliography

 Exadata Hybrid Columnar Compression – Leveraging it fully, Christo
Kutrovsky, UK OUG 2011
 Expert Oracle Exadata, Kerry Osborne, Randy Johnson, Tanel
Poder, Apress, 2011
 Oracle Scratchpad, Jonathan Lewis,
http://jonathanlewis.wordpress.com/category/oracle/exadata/
 Exadata Hybrid Columnar Compression, Oracle
http://www.oracle.com/technetwork/database/features/availability/31
1358-132337.pdf
 oracle.com for documentation on HCC and the ZFS storage
appliance


HCC for Non-Exadata Databases

Discussion


Thanks to Duncan Lawie, Credit Suisse, for his work on this.

Thank you for your attention.

peter.brink@credit-suisse.com


Hybrid Columnar Compression in a non-Exadata System

Recommended

More Related Content

What's hot (20)

Viewers also liked (14)

Similar to Hybrid Columnar Compression in a non-Exadata System (20)

More from Enkitec (20)

Recently uploaded (20)

Hybrid Columnar Compression in a non-Exadata System