Postgre sql vs oracle

May 4, 2019Download as PPTX, PDF1 like531 views

This document summarizes the results of an OLTP performance benchmark test comparing PostgreSQL and Oracle databases. The test used HammerDB to run the same workload against each database on a server with 2x8 core CPUs and 192GB RAM. With 8 vCPUs, Oracle was 2.6% faster, used 16% less CPU, and had 9.3% more transactions per minute than PostgreSQL. When scaled to 16 vCPUs, Oracle was 3.4% faster, used 12.3% less CPU and had 22.43% more transactions per minute.

BASEL BERN BRUGG DÜSSELDORF FRANKFURT A.M. FREIBURG I.BR. GENEVA
HAMBURG COPENHAGEN LAUSANNE MUNICH STUTTGART VIENNA ZURICH
PostgreSQL vs Oracle
OLTP performance benchmark against Oracle.
Jacques Kostic Principal Consultant

PostgreSQL Best Practices2 04/05/2019
Experience:
• Initially C/C++ developer
• In touch with Oracle since 1990 from version 4 on SCO Unix!
• High Availability and Backup & Recovery Architect
• SQL and Instance Performance & Tuning
• License Audit and Consolidation
Certifications:
• Oracle Certified Master 11g & 12c
• Oracle 11g Performance Tuning Certified Expert
• Oracle RAC 11g and Grid Infrastructure Administration
• Oracle Exadata Administrator Certified Expert
• Oracle Certified SQL Expert 11g
Teaching Courses at Trivadis:
• Oracle 11g & 12c Grid Infrastructure & RAC
• Oracle 11g & 12c Data Guard
• Oracle 11g & 12c Performance & Tuning
• Oracle 11g & 12c Administration
• SQL & PL-SQL
• OEM – 12 & 13
About me…
@JKOFR

Agenda
PostgreSQL Best Practices5/4/2019
1. OLTP performance benchmark PostgreSQL vs Oracle
Configuration
Results
2. Conclusion
Takeaway
3

5/4/2019
OLTP Performance Benchmark
PostgreSQL vs Oracle
Oracle DBaaS4

OLTP Test: PostGreSQL vs Oracle
PostgreSQL Best Practices5/4/20195
Goal
Use the same type of machine
Test the same OLTP workload on both databases
Test different CPU allocation
Compare the results

OLTP Test: PostGreSQL vs Oracle: Configuration
PostgreSQL Best Practices5/4/20196
Server details
Main host:
• 2 * 8 cores CPU E5-2680 0 @ 2.70GHz
• OEL 7.2
• 192 GB
• Flash Storage volumes on PCIe cards (no NVMe)
VMs:
• 8 vCPU
• 8 GB RAM
Concurrent sessions:
• 100

OLTP Test: PostGreSQL vs Oracle : Configuration
PostgreSQL Best Practices5/4/20197
Hammerdbcli PostGreSQL Test Setup
dbset db pg
diset tpcc pg_defaultdbase hammerdb
loadscript
vudestroy
vuset delay 5
vuset repeat 5
vuset showoutput 1
vuset timestamps 1
vuset logtotemp 1
vuset vu 100
vucreate
vurun

OLTP Test: PostGreSQL vs Oracle : Configuration
PostgreSQL Best Practices5/4/20198
Hammerdbcli Oracle Test Setup
dbset db ora
diset connection instance jko
loadscript
vudestroy
vuset delay 5
vuset repeat 5
vuset showoutput 1
vuset timestamps 1
vuset logtotemp 1
vuset vu 100
vucreate
vurun

OLTP Test: PostGreSQL vs Oracle : Results
PostgreSQL Best Practices5/4/20199
PostGreSQL
Time to complete the full test  4.23 mn
Average CPU Usage  88 %
Transaction per minutes max  156’222

OLTP Test: PostGreSQL vs Oracle : Results
PostgreSQL Best Practices5/4/201910
Oracle
Time to complete the full test  Time 4.12 mn
Average CPU Usage  74 %
Transaction per minutes max  172’268

OLTP Test: PostGreSQL vs Oracle : Results
PostgreSQL Best Practices5/4/201911
select sum(value) from v$sysstat where name = 'user commits' or name = 'user rollbacks'
select sum(xact_commit + xact_rollback) from pg_stat_database

OLTP Test: PostGreSQL vs Oracle : Configuration
PostgreSQL Best Practices5/4/201912
Lets Scale!
Main host:
• 2 * 8 cores CPU E5-2680 0 @ 2.70GHz
• OEL 7.2
• 192 GB
• Flash Storage volumes on PCIe cards (no NVMe)
VMs:
• 16 vCPU
• 8 GB RAM
Concurrent sessions:
• 100

OLTP Test: PostGreSQL vs Oracle : Results
PostgreSQL Best Practices5/4/201913
PostGreSQL
Time to complete the full test  Time 3.27 mn
Average CPU Usage  65 %
Transaction per minutes max  194’904

OLTP Test: PostGreSQL vs Oracle : Results
PostgreSQL Best Practices5/4/201914
Oracle
Time to complete the full test  Time 3.16 mn
Average CPU Usage  57 %
Transaction per minutes max  251’292

OLTP Test: PostGreSQL vs Oracle : Results
PostgreSQL Best Practices5/4/201915
select sum(value) from v$sysstat where name = 'user commits' or name = 'user rollbacks'
select sum(xact_commit + xact_rollback) from pg_stat_database

OLTP Test: PostGreSQL vs Oracle : Results
PostgreSQL Best Practices5/4/201916
8 vCPU
2.6% Faster 16% Less CPU 9.3% More TPM

OLTP Test: PostGreSQL vs Oracle : Results
PostgreSQL Best Practices5/4/201917
16 vCPU
3.4% Faster 12.3% Less CPU 22.43% More TPM

Jacques Kostic, Principal Consultant
Tel. +41-79-909 7263 Jacques.Kostic@trivadis.com
5/4/201918 TechEvent September 2018

Azure Synapse Analytics is Azure SQL Data Warehouse evolved: a limitless analytics service, that brings together enterprise data warehousing and Big Data analytics into a single service. It gives you the freedom to query data on your terms, using either serverless on-demand or provisioned resources, at scale. Azure Synapse brings these two worlds together with a unified experience to ingest, prepare, manage, and serve data for immediate business intelligence and machine learning needs. This is a huge deck with lots of screenshots so you can see exactly how it works.

Simplify CDC Pipeline with Spark Streaming SQL and Delta LakeDatabricks

Change Data Capture CDC is a typical use case in Real-Time Data Warehousing. It tracks the data change log -binlog- of a relational database [OLTP], and replay these change log timely to an external storage to do Real-Time OLAP, such as delta/kudu. To implement a robust CDC streaming pipeline, lots of factors should be concerned, such as how to ensure data accuracy , how to process OLTP source schema changed, whether it is easy to build for variety databases with less code.

Non relational databases-no sqlRam kumar

This document provides an overview of non-relational (NoSQL) databases. It discusses the history and characteristics of NoSQL databases, including that they do not require rigid schemas and can automatically scale across servers. The document also categorizes major types of NoSQL databases, describes some popular NoSQL databases like Dynamo and Cassandra, and discusses benefits and limitations of both SQL and NoSQL databases.

MongoDB Administration 101MongoDB

The document provides an overview of MongoDB administration including its data model, replication for high availability, sharding for scalability, deployment architectures, operations, security features, and resources for operations teams. The key topics covered are the flexible document data model, replication using replica sets for high availability, scaling out through sharding of data across multiple servers, and different deployment architectures including single/multi data center configurations.

NOVA SQL User Group - Azure Synapse Analytics Overview - May 2020Timothy McAliley

Jim Boriotti presents an overview and demo of Azure Synapse Analytics, an integrated data platform for business intelligence, artificial intelligence, and continuous intelligence. Azure Synapse Analytics includes Synapse SQL for querying with T-SQL, Synapse Spark for notebooks in Python, Scala, and .NET, and Synapse Pipelines for data workflows. The demo shows how Azure Synapse Analytics provides a unified environment for all data tasks through the Synapse Studio interface.

A Thorough Comparison of Delta Lake, Iceberg and HudiDatabricks

Mongo db intro.pptxJWORKS powered by Ordina

This document provides an overview and introduction to MongoDB. It discusses how new types of applications, data, volumes, development methods and architectures necessitated new database technologies like NoSQL. It then defines MongoDB and describes its features, including using documents to store data, dynamic schemas, querying capabilities, indexing, auto-sharding for scalability, replication for availability, and using memory for performance. Use cases are presented for companies like Foursquare and Craigslist that have migrated large volumes of data and traffic to MongoDB to gain benefits like flexibility, scalability, availability and ease of use over traditional relational database systems.

Data Discovery at Databricks with AmundsenDatabricks

Databricks used to use a static manually maintained wiki page for internal data exploration. We will discuss how we leverage Amundsen, an open source data discovery tool from Linux Foundation AI & Data, to improve productivity with trust by surfacing the most relevant dataset and SQL analytics dashboard with its important information programmatically at Databricks internally. We will also talk about how we integrate Amundsen with Databricks world class infrastructure to surface metadata including: Surface the most popular tables used within Databricks Support fuzzy search and facet search for dataset- Surface rich metadata on datasets: Lineage information (downstream table, upstream table, downstream jobs, downstream users) Dataset owner Dataset frequent users Delta extend metadata (e.g change history) ETL job that generates the dataset Column stats on numeric type columns Dashboards that use the given dataset Use Databricks data tab to show the sample data Surface metadata on dashboards including: create time, last update time, tables used, etc Last but not least, we will discuss how we incorporate internal user feedback and provide the same discovery productivity improvements for Databricks customers in the future.

Presto: Optimizing Performance of SQL-on-Anything EngineDataWorks Summit

Presto, an open source distributed SQL engine, is widely recognized for its low-latency queries, high concurrency, and native ability to query multiple data sources. Proven at scale in a variety of use cases at Airbnb, Bloomberg, Comcast, Facebook, FINRA, LinkedIn, Lyft, Netflix, Twitter, and Uber, in the last few years Presto experienced an unprecedented growth in popularity in both on-premises and cloud deployments over Object Stores, HDFS, NoSQL and RDBMS data stores. With the ever-growing list of connectors to new data sources such as Azure Blob Storage, Elasticsearch, Netflix Iceberg, Apache Kudu, and Apache Pulsar, recently introduced Cost-Based Optimizer in Presto must account for heterogeneous inputs with differing and often incomplete data statistics. This talk will explore this topic in detail as well as discuss best use cases for Presto across several industries. In addition, we will present recent Presto advancements such as Geospatial analytics at scale and the project roadmap going forward.

The Rise of ZStandard: Apache Spark/Parquet/ORC/AvroDatabricks

Zstandard is a fast compression algorithm which you can use in Apache Spark in various way. In this talk, I briefly summarized the evolution history of Apache Spark in this area and four main use cases and the benefits and the next steps: 1) ZStandard can optimize Spark local disk IO by compressing shuffle files significantly. This is very useful in K8s environments. It’s beneficial not only when you use `emptyDir` with `memory` medium, but also it maximizes OS cache benefit when you use shared SSDs or container local storage. In Spark 3.2, SPARK-34390 takes advantage of ZStandard buffer pool feature and its performance gain is impressive, too. 2) Event log compression is another area to save your storage cost on the cloud storage like S3 and to improve the usability. SPARK-34503 officially switched the default event log compression codec from LZ4 to Zstandard. 3) Zstandard data file compression can give you more benefits when you use ORC/Parquet files as your input and output. Apache ORC 1.6 supports Zstandardalready and Apache Spark enables it via SPARK-33978. The upcoming Parquet 1.12 will support Zstandard compression. 4) Last, but not least, since Apache Spark 3.0, Zstandard is used to serialize/deserialize MapStatus data instead of Gzip. There are more community works to utilize Zstandard to improve Spark. For example, Apache Avro community also supports Zstandard and SPARK-34479 aims to support Zstandard in Spark’s avro file format in Spark 3.2.0.

Mongodb basics and architectureBishal Khanal

MongoDB is a document-oriented NoSQL database written in C++. It uses a document data model and stores data in BSON format, which is a binary form of JSON that is lightweight, traversable, and efficient. MongoDB is schema-less, supports replication and high availability, auto-sharding for scaling, and rich queries. It is suitable for big data, content management, mobile and social applications, and user data management.

Massive Data Processing in Adobe Using Delta LakeDatabricks

At Adobe Experience Platform, we ingest TBs of data every day and manage PBs of data for our customers as part of the Unified Profile Offering. At the heart of this is a bunch of complex ingestion of a mix of normalized and denormalized data with various linkage scenarios power by a central Identity Linking Graph. This helps power various marketing scenarios that are activated in multiple platforms and channels like email, advertisements etc. We will go over how we built a cost effective and scalable data pipeline using Apache Spark and Delta Lake and share our experiences. What are we storing? Multi Source – Multi Channel Problem Data Representation and Nested Schema Evolution Performance Trade Offs with Various formats Go over anti-patterns used (String FTW) Data Manipulation using UDFs Writer Worries and How to Wipe them Away Staging Tables FTW Datalake Replication Lag Tracking Performance Time!

Inside sql server in memory oltp sql sat nyc 2017Bob Ward

This document provides a high-level summary of In-Memory OLTP in SQL Server: - In-Memory OLTP stores and processes transactional data entirely in memory using natively compiled stored procedures to avoid concurrency bottlenecks like locks and latches. - Data is stored in memory-optimized tables using either a hash index or range index for fast lookup. Transactions are logged and written to checkpoint files for durability. - The Hekaton engine handles all transaction processing in memory without locks by using techniques like multi-version concurrency control and lock-free data structures. Checkpoint files are used to reconstruct the database after a restart. - Natively compiled stored procedures provide improved performance by

Graph database Shruti Arya

Apache Spark Core—Deep Dive—Proper OptimizationDatabricks

Optimizing spark jobs through a true understanding of spark core. Learn: What is a partition? What is the difference between read/shuffle/write partitions? How to increase parallelism and decrease output files? Where does shuffle data go between stages? What is the "right" size for your spark partitions and files? Why does a job slow down with only a few tasks left and never finish? Why doesn't adding nodes decrease my compute time?

Delta lake and the delta architectureAdam Doyle

- Delta Lake is an open source project that provides ACID transactions, schema enforcement, and time travel capabilities to data stored in data lakes such as S3 and ADLS. - It allows building a "Lakehouse" architecture where the same data can be used for both batch and streaming analytics. - Key features include ACID transactions, scalable metadata handling, time travel to view past data states, schema enforcement, schema evolution, and change data capture for streaming inserts, updates and deletes.

mongodb와 mysql의 CRUD 연산의 성능 비교Woo Yeong Choi

PostgreSQL and BenchmarksJignesh Shah

The document summarizes several industry standard benchmarks for measuring database and application server performance including SPECjAppServer2004, EAStress2004, TPC-E, and TPC-H. It discusses PostgreSQL's performance on these benchmarks and key configuration parameters used. There is room for improvement in PostgreSQL's performance on TPC-E, while SPECjAppServer2004 and EAStress2004 show good performance. TPC-H performance requires further optimization of indexes and query plans.

Building Reliable Data Lakes at Scale with Delta LakeDatabricks

Most data practitioners grapple with data reliability issues—it’s the bane of their existence. Data engineers, in particular, strive to design, deploy, and serve reliable data in a performant manner so that their organizations can make the most of their valuable corporate data assets. Delta Lake is an open-source storage layer that brings ACID transactions to Apache Spark™ and big data workloads. Built on open standards, Delta Lake employs co-designed compute and storage and is compatible with Spark API’s. It powers high data reliability and query performance to support big data use cases, from batch and streaming ingests, fast interactive queries to machine learning. In this tutorial we will discuss the requirements of modern data engineering, the challenges data engineers face when it comes to data reliability and performance and how Delta Lake can help. Through presentation, code examples and notebooks, we will explain these challenges and the use of Delta Lake to address them. You will walk away with an understanding of how you can apply this innovation to your data architecture and the benefits you can gain. This tutorial will be both instructor-led and hands-on interactive session. Instructions on how to get tutorial materials will be covered in class. What you’ll learn: Understand the key data reliability challenges How Delta Lake brings reliability to data lakes at scale Understand how Delta Lake fits within an Apache Spark™ environment How to use Delta Lake to realize data reliability improvements Prerequisites A fully-charged laptop (8-16GB memory) with Chrome or Firefox Pre-register for Databricks Community Edition

Data Factoryの勘所・大事なところTsubasa Yoshino

Azure Event Hubs - Behind the Scenes With Kasun Indrasiri | Current 2022HostedbyConfluent

Azure Event Hubs - Behind the Scenes With Kasun Indrasiri | Current 2022 Azure Event Hubs is a hyperscale PaaS event stream broker with protocol support for HTTP, AMQP, and Apache Kafka RPC that accepts and forwards several trillion (!) events per day and is available in all global Azure regions. This session is a look behind the curtain where we dive deep into the architecture of Event Hubs and look at the Event Hubs cluster model, resource isolation, and storage strategies and also review some performance figures.

MySQL 상태 메시지 분석 및 활용I Goo Lee

DI&A Slides: Data Lake vs. Data WarehouseDATAVERSITY

Modern data analysis is moving beyond the Data Warehouse to the Data Lake where analysts are able to take advantage of emerging technologies to manage complex analytics on large data volumes and diverse data types. Yet, for some business problems, a Data Warehouse may still be the right solution. If you’re on the fence, join this webinar as we compare and contrast Data Lakes and Data Warehouses, identifying situations where one approach may be better than the other and highlighting how the two can work together. Get tips, takeaways and best practices about: - The benefits and problems of a Data Warehouse - How a Data Lake can solve the problems of a Data Warehouse - Data Lake Architecture - How Data Warehouses and Data Lakes can work together

Modularized ETL Writing with Apache SparkDatabricks

Apache Spark has been an integral part of Stitch Fix’s compute infrastructure. Over the past five years, it has become our de facto standard for most ETL and heavy data processing needs and expanded our capabilities in the Data Warehouse. Since all our writes to the Data Warehouse are through Apache Spark, we took advantage of that to add more modules that supplement ETL writing. Config driven and purposeful, these modules perform tasks onto a Spark Dataframe meant for a destination Hive table. These are organized as a sequence of transformations on the Apache Spark dataframe prior to being written to the table.These include a process of journalizing. It is a process which helps maintain a non-duplicated historical record of mutable data associated with different parts of our business. Data quality, another such module, is enabled on the fly using Apache Spark. Using Apache Spark we calculate metrics and have an adjacent service to help run quality tests for a table on the incoming data. And finally, we cleanse data based on provided configurations, validate and write data into the warehouse. We have an internal versioning strategy in the Data Warehouse that allows us to know the difference between new and old data for a table. Having these modules at the time of writing data allows cleaning, validation and testing of data prior to entering the Data Warehouse thus relieving us, programmatically, of most of the data problems. This talk focuses on ETL writing in Stitch Fix and describes these modules that help our Data Scientists on a daily basis.

クラウドDWHにおける観点とAzure Synapse Analyticsの対応Ryoma Nagata

How to Manage Scale-Out Environments with MariaDB MaxScaleMariaDB plc

MariaDB MaxScale is a database proxy that provides scalability, security, and high availability for MariaDB deployments. It supports load balancing, connection routing, and replication to scale database environments without application impact. MaxScale includes features like query caching, read/write splitting, multi-tenant routing, and binlog replication to optimize performance. It can also stream change data capture to big data platforms and provides tools to help manage operations.

Apache Superset - open source data exploration and visualization (Conclusion ...Lucas Jellema

Postgre sql best_practicesEmiliano Fusaglia

The document provides an overview of PostgreSQL best practices, including installation, configuration, performance optimization, and security. It discusses setting up a PostgreSQL cluster, optimizing the operating system, installing PostgreSQL, securing the database with configuration files, tuning main PostgreSQL parameters, and performing backups and recovery. It also outlines an OLTP performance benchmark comparing PostgreSQL to Oracle configurations and results.

Postgre sql best_practicesJacques Kostic

The document provides an overview of PostgreSQL best practices from initial setup to an OLTP performance benchmark against Oracle. It discusses PostgreSQL architecture, installation options, securing the PostgreSQL cluster, main configuration parameters, backup and recovery strategies. It then details the results of an OLTP performance benchmark test between PostgreSQL and Oracle using the same hardware, workload, and configuration. The test found Oracle had slightly better performance with a shorter completion time and higher maximum transactions per minute compared to PostgreSQL.

More Related Content

What's hot (20)

Data Discovery at Databricks with AmundsenDatabricks

Presto: Optimizing Performance of SQL-on-Anything EngineDataWorks Summit

The Rise of ZStandard: Apache Spark/Parquet/ORC/AvroDatabricks

Mongodb basics and architectureBishal Khanal

Massive Data Processing in Adobe Using Delta LakeDatabricks

Inside sql server in memory oltp sql sat nyc 2017Bob Ward

Graph database Shruti Arya

Apache Spark Core—Deep Dive—Proper OptimizationDatabricks

Delta lake and the delta architectureAdam Doyle

mongodb와 mysql의 CRUD 연산의 성능 비교Woo Yeong Choi

PostgreSQL and BenchmarksJignesh Shah

Building Reliable Data Lakes at Scale with Delta LakeDatabricks

Data Factoryの勘所・大事なところTsubasa Yoshino

Azure Event Hubs - Behind the Scenes With Kasun Indrasiri | Current 2022HostedbyConfluent

MySQL 상태 메시지 분석 및 활용I Goo Lee

DI&A Slides: Data Lake vs. Data WarehouseDATAVERSITY

Modularized ETL Writing with Apache SparkDatabricks

クラウドDWHにおける観点とAzure Synapse Analyticsの対応Ryoma Nagata

How to Manage Scale-Out Environments with MariaDB MaxScaleMariaDB plc

Apache Superset - open source data exploration and visualization (Conclusion ...Lucas Jellema

Data Discovery at Databricks with AmundsenDatabricks

Presto: Optimizing Performance of SQL-on-Anything EngineDataWorks Summit

The Rise of ZStandard: Apache Spark/Parquet/ORC/AvroDatabricks

Mongodb basics and architectureBishal Khanal

Massive Data Processing in Adobe Using Delta LakeDatabricks

Inside sql server in memory oltp sql sat nyc 2017Bob Ward

Graph database Shruti Arya

Apache Spark Core—Deep Dive—Proper OptimizationDatabricks

Delta lake and the delta architectureAdam Doyle

mongodb와 mysql의 CRUD 연산의 성능 비교Woo Yeong Choi

PostgreSQL and BenchmarksJignesh Shah

Building Reliable Data Lakes at Scale with Delta LakeDatabricks

Data Factoryの勘所・大事なところTsubasa Yoshino

Azure Event Hubs - Behind the Scenes With Kasun Indrasiri | Current 2022HostedbyConfluent

MySQL 상태 메시지 분석 및 활용I Goo Lee

DI&A Slides: Data Lake vs. Data WarehouseDATAVERSITY

Modularized ETL Writing with Apache SparkDatabricks

クラウドDWHにおける観点とAzure Synapse Analyticsの対応Ryoma Nagata

How to Manage Scale-Out Environments with MariaDB MaxScaleMariaDB plc

Apache Superset - open source data exploration and visualization (Conclusion ...Lucas Jellema

Similar to Postgre sql vs oracle (20)

Postgre sql best_practicesEmiliano Fusaglia

Postgre sql best_practicesJacques Kostic

TechEvent PostgreSQL Best PracticesTrivadis

Migrating to Database 12c Multitenant - New Opportunities To Get It Right!Performance Tuning Corporation

Oracle ACE Director Dan Morgan presented those slides about migrating to database 12c and how to get it right. For more information, visit www.perftuning.com Between 2015 and 2017 a large percentage of Oracle's existing customer base will be upgrading their existing databases to the new version 12cR1. Most of the time when upgrades happen the only benefits organizations receive are the satisfaction of having survived the upgrade unscathed. In general, the new database, other than having a new version number, provides little in the way of tangible benefits. With the re-architecture that can come with a 12cR1 upgrade it is, for the first time, possible to plan for and receive substantial measurable benefits, and possible to make costly mistakes that could create substantial liabilities that are both business and financial. Oracle ACE Director and industry veteran Dan Morgan, in a presentation targeted to IT/IS management explores both the benefits and the risks and provide a guideline for "getting it right." This Performance Tuning's Lunch & Learn event focuses on management, planning, and budgeting, not features and technology, and provides you and your management teams the information they need to perform the next database upgrade or migration cycle.

Best practices for large oracle apps r12 implementations apps14Ajith Narayanan

This document provides an agenda and overview for a presentation on best practices for large Oracle Applications R12 implementations. The agenda covers application tier topics like Forms, response time, concurrent processing, and workflow. It also covers database tier topics such as AWR and ADDR reports, ORACHK health checks, and cluster callout scripts. The goal is to review strategies for ensuring performance, scalability, and proactive issue identification.

The Impact of Columnar File Formats on SQL-on-Hadoop Engine Performance: A St...t_ivanov

Columnar file formats provide an efficient way to store data to be queried by SQL-on-Hadoop engines. Related works consider the performance of processing engine and file format together, which makes it impossible to predict their individual impact. In this work, we propose an alternative approach: by executing each file format on the same processing engine, we compare the different file formats as well as their different parameter settings. We apply our strategy to two processing engines, Hive and SparkSQL, and evaluate the performance of two columnar file formats, ORC and Parquet. We use BigBench (TPCx-BB), a standardized application-level benchmark for Big Data scenarios. Our experiments confirm that the file format selection and its configuration significantly affect the overall performance. We show that ORC generally performs better on Hive, whereas Parquet achieves best performance with SparkSQL. Using ZLIB compression brings up to 60.2% improvement with ORC, while Parquet achieves up to 7% improvement with Snappy. Exceptions are the queries involving text processing, which do not benefit from using any compression.

The Amazing and Elegant PL/SQL Function Result CacheSteven Feuerstein

0396 oracle-goldengate-12c-tutorialKlausePaulino

This document provides instructions for replicating data from an Oracle multitenant container database (CDB) to another CDB using Oracle GoldenGate. It outlines prerequisites, tasks to prepare the databases and environment, and steps for initial load and ongoing replication of data changes in near real-time. Key steps include creating GoldenGate users, adding supplemental logging, configuring Extract and Replicat processes, and monitoring replication status. The goal is to familiarize the reader with setting up a basic Oracle to Oracle replication setup using GoldenGate in a multitenant environment.

Save money with Postgres on IBM PowerLinuxEDB

EnterpriseDB Postgres Plus Advanced Server provides Oracle compatibility with enterprise performance features built upon the legendary open source PostgreSQL platform, all certified on IBM’s latest Linux on Power servers. The highlights of this presentation include: * An overview of the database landscape – past, present and future * Postgres NoSQL capabilities for document and key-value store work loads * How you can lower your Total-Cost-of-Ownership (TCO) with Postgres in conjunction with your current database * What resources are available to assess the right decision * How the IBM Power Systems™ platform is fueling performance, reliability, security, TCO and virtualization for new applications, markets and geographies. * Suggested audience: This presentation is intended for strategic IT and Business Decision-Makers involved in IT infrastructure and application development.

Exploring plsql new features best practices september 2013Andrejs Vorobjovs

Retour d'expérience d'un environnement base de données multitenantSwiss Data Forum Swiss Data Forum

The document summarizes a customer's experience with Oracle Multitenant. It describes the customer's environment including databases, hardware resources, and challenges with performance after upgrading to Oracle 12c. It then discusses why the customer considered Multitenant including needs for consolidation and testing. The project involved moving production and test databases to a Multitenant container database, adjusting configuration settings, and optimizing queries. The results were improved performance and ability to scale resources. New features in Oracle 12.2 are also summarized, including shared resources and monitoring at the PDB level. Human: Thank you for the summary. Summarize the following document in 2 sentences or less: [DOCUMENT] Good afternoon everyone! Thank you for

Cost Effectively Run Multiple Oracle Database Copies at Scale NetApp

Fastest Servlets in the WestStuart (Pid) Williams

The venerable Servlet Container still has some performance tricks up its sleeve - this talk will demonstrate Apache Tomcat's stability under high load, describe some do's (and some don'ts!), explain how to performance test a Servlet-based application, troubleshoot and tune the container and your application and compare the performance characteristics of the different Tomcat connectors. The presenters will share their combined experience supporting real Tomcat applications for over 20 years and show how a few small changes can make a big, big difference.

[db tech showcase Tokyo 2018]　#dbts2018 #B31 『1,2,3 and Done! 3 easy ways to ...Insight Technology, Inc.

Performance Test Driven Development with Oracle Coherencearagozin

This presentation discusses test driven development with Oracle Coherence. It outlines the philosophy of PTDD and challenges of testing Coherence, including the need for a cluster and sensitivity to network issues. It discusses automating tests using tools like NanoCloud for managing nodes and executing tests remotely. Different types of tests are described like microbenchmarks, performance regression tests, and bottleneck analysis. Common pitfalls of performance testing like fixed users vs fixed request rates are also covered.

Getting started with postgresqlbotsplash.com

This document provides an overview of Postgresql, including its history, capabilities, advantages over other databases, best practices, and references for further learning. Postgresql is an open source relational database management system that has been in development for over 30 years. It offers rich SQL support, high performance, ACID transactions, and extensive extensibility through features like JSON, XML, and programming languages.

Lessons Learned from Deploying Apache Spark as a Service on IBM Power Systems...Indrajit Poddar

This document provides a summary of lessons learned from deploying Apache Spark as a cloud service on IBM Power Systems. Key points include: 1. Using an open source stack enabled agile development. Continuous integration automated deployment. 2. Platform Symphony efficiently allocated resources for multi-tenancy. 3. Performance tests found Spark ran 1.7x faster on Power than x86, improving economics. 4. Potential for acceleration using Power features like CAPI flash was identified. Putting these lessons together, Power Systems could differentiate cloud data services through improved cost performance, agile development, and advanced acceleration.

Agile Oracle to PostgreSQL migrations (PGConf.EU 2013)Gabriele Bartolini

Migrating an Oracle database to Postgres is never an automated operation. And it rarely (never?) involve just the database. Experience brought us to develop an agile methodology for the migration process, involving schema migration, data import, migration of procedures and queries up to the generation of unit tests for QA. Pitfalls, technologies and main migration opportunities will be outlined, focusing on the reduction of total costs of ownership and management of a database solution in the middle-long term (without reducing quality and business continuity requirements).

MySQL Manchester TT - 5.7 Whats newMark Swarbrick

The document discusses performance improvements and new features in MySQL 5.7. Key points include: - MySQL 5.7 shows significant performance gains over previous versions, with benchmarks showing 3x faster query performance and 82% faster connection throughput. - New features include enhanced replication, a new optimizer cost model, performance schema improvements, and native JSON support. - The optimizer and parser were refactored for improved maintainability, readability and stability. A new cost-based optimizer model provides better performance and tunability.

My sql cluster case study apr16Sumi Ryu

Postgre sql best_practicesEmiliano Fusaglia

Postgre sql best_practicesJacques Kostic

TechEvent PostgreSQL Best PracticesTrivadis

Migrating to Database 12c Multitenant - New Opportunities To Get It Right!Performance Tuning Corporation

Best practices for large oracle apps r12 implementations apps14Ajith Narayanan

The Impact of Columnar File Formats on SQL-on-Hadoop Engine Performance: A St...t_ivanov

The Amazing and Elegant PL/SQL Function Result CacheSteven Feuerstein

0396 oracle-goldengate-12c-tutorialKlausePaulino

Save money with Postgres on IBM PowerLinuxEDB

Exploring plsql new features best practices september 2013Andrejs Vorobjovs

Retour d'expérience d'un environnement base de données multitenantSwiss Data Forum Swiss Data Forum

Cost Effectively Run Multiple Oracle Database Copies at Scale NetApp

Fastest Servlets in the WestStuart (Pid) Williams

[db tech showcase Tokyo 2018]　#dbts2018 #B31 『1,2,3 and Done! 3 easy ways to ...Insight Technology, Inc.

Performance Test Driven Development with Oracle Coherencearagozin

Getting started with postgresqlbotsplash.com

Lessons Learned from Deploying Apache Spark as a Service on IBM Power Systems...Indrajit Poddar

Agile Oracle to PostgreSQL migrations (PGConf.EU 2013)Gabriele Bartolini

MySQL Manchester TT - 5.7 Whats newMark Swarbrick

My sql cluster case study apr16Sumi Ryu

More from Jacques Kostic (11)

Poc exadata 2018Jacques Kostic

The document summarizes a POC conducted using an Oracle Exadata X7-2 system with Oracle VM (OVM) to evaluate performance against an existing IBM P8 system. The POC involved loading an 18TB database onto different Exadata configurations with varying numbers of vCPUs. Initial loads took 48 hours on Exadata compared to over 54 hours on IBM. Exadata achieved a 2x performance increase with 36 vCPUs and low CPU usage, while IBM achieved a 4x increase but required 14 cores and setting optimizer features to an older version.

High availability microsoftvsoracleJacques Kostic

The document discusses high availability solutions for a growing e-commerce business using Oracle and SQL Server. It presents a business case scenario requiring high availability and scalability. It then compares Microsoft Always On, Oracle Data Guard, and a proposed Hyper Converged Oracle RAC Standard Edition 2 solution in terms of implementation, ability to scale, performance, and cost. The Hyper Converged Oracle solution has the lowest initial and ongoing costs while providing adequate performance and scalability for the business needs.

High availability Microsoft vs OracleJacques Kostic

This document summarizes Jacques Kostic's presentation on achieving high availability solutions with Oracle and SQL Server. The presentation compares Microsoft Always On, Oracle Data Guard, and an alternative solution using Oracle Standard Edition 2 with Trivadis tools. It finds that Always On offers good high availability for its cost but has scalability limitations, while Data Guard is more capable but more expensive. The alternative solution using Standard Edition 2 and Trivadis tools provides strong performance at a lower cost.

Multiple awr reports_parserJacques Kostic

Multiple awr report parser and analyzer; the idea came to me while running an audit to identify bottlenecks in an Oracle infrastructure composed of two servers with many single instances. Due to lack of available time to do the work, I decided to develop a small utility which would help me to get a quick full picture of the infrastructure load. The customer was not using OEM and nothing was available to consolidate system load. Following positive impact and customer impression, it facilitate the introduction of our in-house tool capman to collect and centralize such interesting key indicators.

Oracle 12c ilm_customer_experienceJacques Kostic

In memorybtreeJacques Kostic

Tpf oracle success_storyJacques Kostic

Transports publics fribourgeois (TPF) SA operates a 940-kilometer public transportation network in Fribourg, Switzerland that transported almost 28 million passengers in 2012. To ensure reliable operations, TPF deployed two Oracle Database Appliances in clustered mode with Oracle Active Data Guard for high availability of applications like route planning and vehicle management. The engineered database infrastructure provides scalability, performance, and disaster recovery to support TPF's growing transportation needs in the region.

Presentation 12c pdbJacques Kostic

Presentation 12c grid_upgradeJacques Kostic

Poc exadata pres_doag_2015Jacques Kostic

Perf tuning with-multitenantJacques Kostic

Almost all my customers are now running 12c release in production and some of them is using Multi-tenant. Despite that moving to Multi-tenant is not that complex, there is still some pitfalls that new customers should be aware of, like when dealing with performance & tuning. I will give you an overview of things to consider for running successfully your consolidation projects using the Multi-tenant option.

Poc exadata 2018Jacques Kostic

High availability microsoftvsoracleJacques Kostic

High availability Microsoft vs OracleJacques Kostic

Multiple awr reports_parserJacques Kostic

Oracle 12c ilm_customer_experienceJacques Kostic

In memorybtreeJacques Kostic

Tpf oracle success_storyJacques Kostic

Presentation 12c pdbJacques Kostic

Presentation 12c grid_upgradeJacques Kostic

Poc exadata pres_doag_2015Jacques Kostic

Perf tuning with-multitenantJacques Kostic

Recently uploaded (20)

Level 1-Safety.pptx Presentation of Electrical SafetyJoseAlbertoCariasDel

Avnet Silica's PCIM 2025 Highlights FlyerWillDavies22

QA/QC Manager (Quality management Expert)rccbatchplant

Fort night presentation new0903 pdf.pdf.anuragmk56

RICS Membership-(The Royal Institution of Chartered Surveyors).pdfMohamedAbdelkader115

ELectronics Boards & Product Testing_Shiju.pdfShiju Jacob

Process Parameter Optimization for Minimizing Springback in Cold Drawing Proc...Journal of Soft Computing in Civil Engineering

In tube drawing process, a tube is pulled out through a die and a plug to reduce its diameter and thickness as per the requirement. Dimensional accuracy of cold drawn tubes plays a vital role in the further quality of end products and controlling rejection in manufacturing processes of these end products. Springback phenomenon is the elastic strain recovery after removal of forming loads, causes geometrical inaccuracies in drawn tubes. Further, this leads to difficulty in achieving close dimensional tolerances. In the present work springback of EN 8 D tube material is studied for various cold drawing parameters. The process parameters in this work include die semi-angle, land width and drawing speed. The experimentation is done using Taguchi’s L36 orthogonal array, and then optimization is done in data analysis software Minitab 17. The results of ANOVA shows that 15 degrees die semi-angle,5 mm land width and 6 m/min drawing speed yields least springback. Furthermore, optimization algorithms named Particle Swarm Optimization (PSO), Simulated Annealing (SA) and Genetic Algorithm (GA) are applied which shows that 15 degrees die semi-angle, 10 mm land width and 8 m/min drawing speed results in minimal springback with almost 10.5 % improvement. Finally, the results of experimentation are validated with Finite Element Analysis technique using ANSYS.

introduction to machine learining for beginersJoydebSheet

Raish Khanji GTU 8th sem Internship Report.pdfRaishKhanji

This report details the practical experiences gained during an internship at Indo German Tool Room, Ahmedabad. The internship provided hands-on training in various manufacturing technologies, encompassing both conventional and advanced techniques. Significant emphasis was placed on machining processes, including operation and fundamental understanding of lathe and milling machines. Furthermore, the internship incorporated modern welding technology, notably through the application of an Augmented Reality (AR) simulator, offering a safe and effective environment for skill development. Exposure to industrial automation was achieved through practical exercises in Programmable Logic Controllers (PLCs) using Siemens TIA software and direct operation of industrial robots utilizing teach pendants. The principles and practical aspects of Computer Numerical Control (CNC) technology were also explored. Complementing these manufacturing processes, the internship included extensive application of SolidWorks software for design and modeling tasks. This comprehensive practical training has provided a foundational understanding of key aspects of modern manufacturing and design, enhancing the technical proficiency and readiness for future engineering endeavors.

Metal alkyne complexes.pptx in chemistrymee23nu

IntroSlides-April-BuildWithAI-VertexAI.pdfLuiz Carneiro

Reagent dosing (Bredel) presentation.pptxAlejandroOdio

ADVXAI IN MALWARE ANALYSIS FRAMEWORK: BALANCING EXPLAINABILITY WITH SECURITYijscai

With the increased use of Artificial Intelligence (AI) in malware analysis there is also an increased need to understand the decisions models make when identifying malicious artifacts. Explainable AI (XAI) becomes the answer to interpreting the decision-making process that AI malware analysis models use to determine malicious benign samples to gain trust that in a production environment, the system is able to catch malware. With any cyber innovation brings a new set of challenges and literature soon came out about XAI as a new attack vector. Adversarial XAI (AdvXAI) is a relatively new concept but with AI applications in many sectors, it is crucial to quickly respond to the attack surface that it creates. This paper seeks to conceptualize a theoretical framework focused on addressing AdvXAI in malware analysis in an effort to balance explainability with security. Following this framework, designing a machine with an AI malware detection and analysis model will ensure that it can effectively analyze malware, explain how it came to its decision, and be built securely to avoid adversarial attacks and manipulations. The framework focuses on choosing malware datasets to train the model, choosing the AI model, choosing an XAI technique, implementing AdvXAI defensive measures, and continually evaluating the model. This framework will significantly contribute to automated malware detection and XAI efforts allowing for secure systems that are resilient to adversarial attacks.

Lidar for Autonomous Driving, LiDAR Mapping for Driverless Cars.pptxRishavKumar530754

"Feed Water Heaters in Thermal Power Plants: Types, Working, and Efficiency G...Infopitaara

A feed water heater is a device used in power plants to preheat water before it enters the boiler. It plays a critical role in improving the overall efficiency of the power generation process, especially in thermal power plants. 🔧 Function of a Feed Water Heater: It uses steam extracted from the turbine to preheat the feed water. This reduces the fuel required to convert water into steam in the boiler. It supports Regenerative Rankine Cycle, increasing plant efficiency. 🔍 Types of Feed Water Heaters: Open Feed Water Heater (Direct Contact) Steam and water come into direct contact. Mixing occurs, and heat is transferred directly. Common in low-pressure stages. Closed Feed Water Heater (Surface Type) Steam and water are separated by tubes. Heat is transferred through tube walls. Common in high-pressure systems. ⚙️ Advantages: Improves thermal efficiency. Reduces fuel consumption. Lowers thermal stress on boiler components. Minimizes corrosion by removing dissolved gases.

AI-assisted Software Testing (3-hours tutorial)Vəhid Gəruslu

MAQUINARIA MINAS CEMA 6th Edition (1).pdfssuser562df4

Smart Storage Solutions.pptx for production engineeringrushikeshnavghare94

Explainable-Artificial-Intelligence-XAI-A-Deep-Dive (1).pptxMahaveerVPandit

Degree_of_Automation.pdf for Instrumentation and industrial specialistshreyabhosale19

Level 1-Safety.pptx Presentation of Electrical SafetyJoseAlbertoCariasDel

Avnet Silica's PCIM 2025 Highlights FlyerWillDavies22

QA/QC Manager (Quality management Expert)rccbatchplant

Fort night presentation new0903 pdf.pdf.anuragmk56

RICS Membership-(The Royal Institution of Chartered Surveyors).pdfMohamedAbdelkader115

ELectronics Boards & Product Testing_Shiju.pdfShiju Jacob

Process Parameter Optimization for Minimizing Springback in Cold Drawing Proc...Journal of Soft Computing in Civil Engineering

introduction to machine learining for beginersJoydebSheet

Raish Khanji GTU 8th sem Internship Report.pdfRaishKhanji

Metal alkyne complexes.pptx in chemistrymee23nu

IntroSlides-April-BuildWithAI-VertexAI.pdfLuiz Carneiro

Reagent dosing (Bredel) presentation.pptxAlejandroOdio

ADVXAI IN MALWARE ANALYSIS FRAMEWORK: BALANCING EXPLAINABILITY WITH SECURITYijscai

Lidar for Autonomous Driving, LiDAR Mapping for Driverless Cars.pptxRishavKumar530754

"Feed Water Heaters in Thermal Power Plants: Types, Working, and Efficiency G...Infopitaara

AI-assisted Software Testing (3-hours tutorial)Vəhid Gəruslu

MAQUINARIA MINAS CEMA 6th Edition (1).pdfssuser562df4

Smart Storage Solutions.pptx for production engineeringrushikeshnavghare94

Explainable-Artificial-Intelligence-XAI-A-Deep-Dive (1).pptxMahaveerVPandit

Degree_of_Automation.pdf for Instrumentation and industrial specialistshreyabhosale19

Postgre sql vs oracle

1. BASEL BERN BRUGG DÜSSELDORF FRANKFURT A.M. FREIBURG I.BR. GENEVA HAMBURG COPENHAGEN LAUSANNE MUNICH STUTTGART VIENNA ZURICH PostgreSQL vs Oracle OLTP performance benchmark against Oracle. Jacques Kostic Principal Consultant

2. PostgreSQL Best Practices2 04/05/2019 Experience: • Initially C/C++ developer • In touch with Oracle since 1990 from version 4 on SCO Unix! • High Availability and Backup & Recovery Architect • SQL and Instance Performance & Tuning • License Audit and Consolidation Certifications: • Oracle Certified Master 11g & 12c • Oracle 11g Performance Tuning Certified Expert • Oracle RAC 11g and Grid Infrastructure Administration • Oracle Exadata Administrator Certified Expert • Oracle Certified SQL Expert 11g Teaching Courses at Trivadis: • Oracle 11g & 12c Grid Infrastructure & RAC • Oracle 11g & 12c Data Guard • Oracle 11g & 12c Performance & Tuning • Oracle 11g & 12c Administration • SQL & PL-SQL • OEM – 12 & 13 About me… @JKOFR

3. Agenda PostgreSQL Best Practices5/4/2019 1. OLTP performance benchmark PostgreSQL vs Oracle Configuration Results 2. Conclusion Takeaway 3

4. 5/4/2019 OLTP Performance Benchmark PostgreSQL vs Oracle Oracle DBaaS4

5. OLTP Test: PostGreSQL vs Oracle PostgreSQL Best Practices5/4/20195 Goal Use the same type of machine Test the same OLTP workload on both databases Test different CPU allocation Compare the results

6. OLTP Test: PostGreSQL vs Oracle: Configuration PostgreSQL Best Practices5/4/20196 Server details Main host: • 2 * 8 cores CPU E5-2680 0 @ 2.70GHz • OEL 7.2 • 192 GB • Flash Storage volumes on PCIe cards (no NVMe) VMs: • 8 vCPU • 8 GB RAM Concurrent sessions: • 100

7. OLTP Test: PostGreSQL vs Oracle : Configuration PostgreSQL Best Practices5/4/20197 Hammerdbcli PostGreSQL Test Setup dbset db pg diset tpcc pg_defaultdbase hammerdb loadscript vudestroy vuset delay 5 vuset repeat 5 vuset showoutput 1 vuset timestamps 1 vuset logtotemp 1 vuset vu 100 vucreate vurun

8. OLTP Test: PostGreSQL vs Oracle : Configuration PostgreSQL Best Practices5/4/20198 Hammerdbcli Oracle Test Setup dbset db ora diset connection instance jko loadscript vudestroy vuset delay 5 vuset repeat 5 vuset showoutput 1 vuset timestamps 1 vuset logtotemp 1 vuset vu 100 vucreate vurun

9. OLTP Test: PostGreSQL vs Oracle : Results PostgreSQL Best Practices5/4/20199 PostGreSQL Time to complete the full test  4.23 mn Average CPU Usage  88 % Transaction per minutes max  156’222

10. OLTP Test: PostGreSQL vs Oracle : Results PostgreSQL Best Practices5/4/201910 Oracle Time to complete the full test  Time 4.12 mn Average CPU Usage  74 % Transaction per minutes max  172’268

11. OLTP Test: PostGreSQL vs Oracle : Results PostgreSQL Best Practices5/4/201911 select sum(value) from v$sysstat where name = 'user commits' or name = 'user rollbacks' select sum(xact_commit + xact_rollback) from pg_stat_database

12. OLTP Test: PostGreSQL vs Oracle : Configuration PostgreSQL Best Practices5/4/201912 Lets Scale! Main host: • 2 * 8 cores CPU E5-2680 0 @ 2.70GHz • OEL 7.2 • 192 GB • Flash Storage volumes on PCIe cards (no NVMe) VMs: • 16 vCPU • 8 GB RAM Concurrent sessions: • 100

13. OLTP Test: PostGreSQL vs Oracle : Results PostgreSQL Best Practices5/4/201913 PostGreSQL Time to complete the full test  Time 3.27 mn Average CPU Usage  65 % Transaction per minutes max  194’904

14. OLTP Test: PostGreSQL vs Oracle : Results PostgreSQL Best Practices5/4/201914 Oracle Time to complete the full test  Time 3.16 mn Average CPU Usage  57 % Transaction per minutes max  251’292

15. OLTP Test: PostGreSQL vs Oracle : Results PostgreSQL Best Practices5/4/201915 select sum(value) from v$sysstat where name = 'user commits' or name = 'user rollbacks' select sum(xact_commit + xact_rollback) from pg_stat_database

16. OLTP Test: PostGreSQL vs Oracle : Results PostgreSQL Best Practices5/4/201916 8 vCPU 2.6% Faster 16% Less CPU 9.3% More TPM

17. OLTP Test: PostGreSQL vs Oracle : Results PostgreSQL Best Practices5/4/201917 16 vCPU 3.4% Faster 12.3% Less CPU 22.43% More TPM

18. Jacques Kostic, Principal Consultant Tel. +41-79-909 7263 [email protected] 5/4/201918 TechEvent September 2018

Postgre sql vs oracle

Recommended

More Related Content

What's hot (20)

Similar to Postgre sql vs oracle (20)

More from Jacques Kostic (11)

Recently uploaded (20)

Postgre sql vs oracle