PGConf APAC 2018 - Lightening Talk #3: How To Contribute to PostgreSQLPGConf APAC
This document outlines various ways to contribute to the PostgreSQL open source database project. It discusses that PostgreSQL needs support from individuals and companies to continue developing and competing against commercial databases. Contributing provides benefits like being listed as a contributor or sponsor on PostgreSQL's website. The document then lists several contribution methods like making donations, participating in surveys, providing hardware/infrastructure, helping with documentation, answering user questions, reporting bugs, and writing code in the form of tools, extensions, or patches.
PGConf APAC 2018 - A PostgreSQL DBAs Toolbelt for 2018PGConf APAC
There's no need to re-invent the wheel! Dozens of people have already tried...and succeeded. This talk is a categorized and illustrated overview on most popular and/or useful PostgreSQL specific scripts, utilities and whole toolsets that DBAs should be aware of for solving daily tasks. Inlcuding - performance monitoring, logs management/analyzis, identifying/fixing most common adminstration problems around areas of general performance metrics, tuning, locking, indexing, bloat, leaving out high-availability topics. Covered are venerable oldies from wiki.postgresql.org as well as my newer favourites from Github.
While the physical replication in PostgreSQL is quite robust, however, it doesn’t fit well in the picture when:
- You need partial replication only
- You want to replicate between different major versions of PostgreSQL
- You need to replicate multiple databases to the same target
- Transformation of the data is needed
- You want to replicate in order to upgrade without downtime
The answer to these use cases is logical replication
This talk will discuss and cover these use cases followed by a logical replication demo.
PGConf APAC 2018 - PostgreSQL performance comparison in various cloudsPGConf APAC
Speaker: Oskari Saarenmaa
Aiven PostgreSQL is available in five different public cloud providers' infrastructure in more than 60 regions around the world, including 18 in APAC. This has given us a unique opportunity to benchmark and compare performance of similar configurations in different environments.
We'll share our benchmark methods and results, comparing various PostgreSQL configurations and workloads across different clouds.
Challenges in Building a Data PipelineManish Kumar
The document discusses challenges in building a data pipeline including making it highly scalable, available with low latency and zero data loss while supporting multiple data sources. It covers expectations around real-time vs batch processing and streaming vs batch data. Implementation approaches like ETL vs ELT are examined along with replication modes, challenges around schema changes and NoSQL. Effective implementations should address transformations, security, replays, monitoring and more. Reference architectures like Lambda and Kappa are briefly outlined.
This document discusses using ClickHouse for experimentation and metrics at Spotify. It describes how Spotify built an experimentation platform using ClickHouse to provide teams interactive queries on granular metrics data with low latency. Key aspects include ingesting data from Google Cloud Storage to ClickHouse daily, defining metrics through a centralized catalog, and visualizing metrics and running queries using Superset connected to ClickHouse. The platform aims to reduce load on notebooks and BigQuery by serving common queries directly from ClickHouse.
Unifying Frontend and Backend Development with Scala - ScalaCon 2021Taro L. Saito
Scala can be used for developing both frontend (Scala.js) and backend (Scala JVM) applications. A missing piece has been bridging these two worlds using Scala. We built Airframe RPC, a framework that uses Scala traits as a unified RPC interface between servers and clients. With Airframe RPC, you can build HTTP/1 (Finagle) and HTTP/2 (gRPC) services just by defining Scala traits and case classes. It simplifies web application design as you only need to care about Scala interfaces without using existing web standards like REST, ProtocolBuffers, OpenAPI, etc. Scala.js support of Airframe also enables building interactive Web applications that can dynamically render DOM elements while talking with Scala-based RPC servers. With Airframe RPC, the value of Scala developers will be much higher both for frontend and backend areas.
Presto: SQL-on-Anything. Netherlands Hadoop User Group MeetupWojciech Biela
Presto is an open source distributed SQL query engine for running interactive analytic queries against data sources of all sizes ranging from gigabytes to petabytes. Presto was designed and written from the ground up for interactive analytics and approaches the speed of commercial data warehouses while scaling to the size of organizations like Facebook. One key feature in Presto is the ability to query data where it lives via an uniform ANSI SQL interface. Presto’s connector architecture creates an abstraction layer for anything that can be represented in a columnar or row-like format, such as HDFS, Amazon S3, Azure Storage, NoSQL stores, relational databases, Kafka streams and even proprietary data stores. Furthermore, a single Presto query can combine data from multiple sources, allowing for analytics across an entire organization.
Twitter handles billions of events per minute that are logged by clients. They use a scalable architecture with modular client daemons and aggregator daemons to aggregate events into categories on HDFS. To improve scalability, they group categories into category groups and write events to HDFS files together. They also group aggregators to scale independently and isolate resources. Within a single aggregator, they improved memory usage and added microbatching to reduce latency. Going forward, they aim to further reduce latency and improve failure handling.
Efficiently Triaging CI Pipelines with Apache Spark: Mixing 52 Billion Events...Databricks
Continuous integration (CI) pipelines generate massive amounts of messy log data. At Pure Storage engineering, we run over 65,000 tests per day creating a large triage problem. Spark’s flexible computing platform allows us to write a single application for both streaming and batch jobs to understand the state of our CI pipeline. Spark indexes log data for real-time reporting (Streaming), uses Machine Learning for performance modeling and prediction (Batch job), and re-indexes old data for newly encoded patters (Batch job). Previous work on a mixed streaming and batch environment describes the options for persisting data and their trade-offs:
1) short interval buckets which hurts batch performance
2) long interval buckets which increases micro batch time windows
3) additional software on the background to compact the short interval buckets which adds complexity.
This talk will go over how we use the filesystem metadata of our disaggregated compute and storage layers to write over half a million files per day of varied sizes from 52 Billion events and have efficient batch jobs without compaction that allow us to process over 40TB per hour. We will go over the challenges and best practices to achieve efficiency in this mixed environment scenarios.
Observing Intraday Indicators Using Real-Time Tick Data on Apache Superset an...DataWorks Summit
The Central Bank of the Republic of Turkey is primarily responsible for steering the monetary and exchange rate policies in Turkey.
One of the major core functions of the Bank is market operations. In this context, analyzing and interpreting real-time tick data related to money market instruments has become not only a requirement but also a challenge.
For this use case, an API provided by one of the financial data vendors has been used to gather real-time tick data and data routing has been orchestrated by Apache NiFi.
Gathered data is being transferred to Kafka topics and then handed off to Druid for real-time indexing tasks.
Indicators such as effective cost, bid-ask spread, price impact measures, return reversal are calculated using Apache Storm and finally visualized by means of Apache Superset in order to provide decision-makers with a new set of tools.
Introduction to Data Engineer and Data Pipeline at Credit OKKriangkrai Chaonithi
The document discusses the role of data engineers and data pipelines. It begins with an introduction to big data and why data volumes are increasing. It then covers what data engineers do, including building data architectures, working with cloud infrastructure, and programming for data ingestion, transformation, and loading. The document also explains data pipelines, describing extract, transform, load (ETL) processes and batch versus streaming data. It provides an example of Credit OK's data pipeline architecture on Google Cloud Platform that extracts raw data from various sources, cleanses and loads it into BigQuery, then distributes processed data to various applications. It emphasizes the importance of data engineers in processing and managing large, complex data sets.
PGConf.ASIA 2019 Bali - How did PostgreSQL Write Load Balancing of Queries Us...Equnix Business Solutions
Atsushi Mitani from SRA Nishi-Nihon Inc. presented on how to perform write load balancing in PostgreSQL using transactions. He explained that write load distribution is important for systems with high write volumes. PostgreSQL can distribute write load using table partitioning with foreign data wrappers (FDW), which allows partitioning across database instances. Mitani created patches to automate the partitioning setup and load data in parallel to child tables to speed up benchmarking. Benchmark results showed that while increasing child databases improves performance without transactions, increasing parent databases is better with transactions to avoid lock queues. The optimal configuration depends on data size, queries, and hardware.
Presto talk @ Global AI conference 2018 Bostonkbajda
Presented at Global AI Conference in Boston 2018:
http://www.globalbigdataconference.com/boston/global-artificial-intelligence-conference-106/speaker-details/kamil-bajda-pawlikowski-62952.html
Presto, an open source distributed SQL engine, is widely recognized for its low-latency queries, high concurrency, and native ability to query multiple data sources. Proven at scale in a variety of use cases at Facebook, Airbnb, Netflix, Uber, Twitter, LinkedIn, Bloomberg, and FINRA, Presto experienced an unprecedented growth in popularity in both on-premises and cloud deployments in the last few years. Presto is really a SQL-on-Anything engine in a single query can access data from Hadoop, S3-compatible object stores, RDBMS, NoSQL and custom data stores. This talk will cover some of the best use cases for Presto, recent advancements in the project such as Cost-Based Optimizer and Geospatial functions as well as discuss the roadmap going forward.
Postgres Vision 2018: WAL: Everything You Want to KnowEDB
The document is a presentation about PostgreSQL's Write-Ahead Log (WAL) system. It discusses what the WAL is, how it works, and how it is used for tasks like replication, backup and point-in-time recovery. The WAL logs all transactions to prevent data loss during crashes and ensures data integrity. It is critical for high availability and disaster recovery capabilities in PostgreSQL.
Iceberg: a modern table format for big data (Ryan Blue & Parth Brahmbhatt, Netflix)
Presto Summit 2018 (https://www.starburstdata.com/technical-blog/presto-summit-2018-recap/)
Journey of Migrating 1 Million Presto Queries - Presto Webinar 2020Taro L. Saito
Arm Treasure Data utilizes Presto as the query engine processing over 1 million queries per day to support the data business of 500+ companies in three regions; US, EU, and Asia. Arm Treasure Data had been using Presto 0.205 and in 2019 started a big migration project to Presto 317. Although we performed extensive query simulations to check any incompatibilities, we faced many unexpected challenges during the migration at production.
Presto and you are there (Nishant Rayan, Lyft)
Presto Summit 2018 (https://www.starburstdata.com/technical-blog/presto-summit-2018-recap/)
Presentation at SF Kubernetes Meetup (10/30/18), Introducing TiDB/TiKVKevin Xu
This deck was presented at the SF Kubernetes Meetup held at Microsoft's downtown SF office, introducing the architecture of TiDB and TiKV (a CNCF project), key use cases, a user story with Mobike (one of the largest bikesharing platforms in the world), and how TiDB is deployed across different cloud environment using TiDB Operator.
Spark Summit EU talk by Sebastian Schroeder and Ralf SigmundSpark Summit
Otto needed to merge large tracking messages from two Kafka streams with different throughput into a single stream while preserving event timestamps and ensuring at least once delivery. They used Spark Streaming to build a custom event-time merge that used UpdateStateByKey to merge messages, tracked event times to timeout messages, handled uneven throughput, and stored offsets to guarantee at least once delivery. The solution provided excellent performance and scalability while being extensible through custom RDDs.
The document discusses the challenges of handling billions of events per minute at Twitter. It describes Twitter's event log pipeline architecture using microservices, event aggregation with Scribe, and event processing with MapReduce. The pipeline was later modified to use streaming technologies to reduce latency, including Apache Beam and Google Cloud services. Latency for different dataset types ranges from seconds to minutes. Future challenges include scaling further for volume and spikes while maintaining fast failure recovery times.
Pivotal Greenplum provides fast, secure cloud deployments of its data warehouse platform with the same experience across AWS, Azure, and GCP. Deployments are optimized for speed through performance tuning of virtual machines, disks, and networks. Key goals include leveraging cloud features like on-demand provisioning, node replacement, disk snapshots, upgrades, and optional installations through a web interface. Deployments are similar across clouds with comparable parameters, tools, and software versions. Security is ensured through vendor-reviewed templates, password encryption, and network isolation.
in this webinar, we were discussing about an introduction to mikrotik, network management, and the dude.
There are some demos of installing the dude on mikrotik, as well as some the dude features to manage several aspects of network elements.
the recording of this webinar will be uploaded on youtube: https://www.youtube.com/channel/UCI611_IIkQC0rsLWIFIx_yg
in this webinar, we were discussing about a mikrotik feature that is called metarouter, which can allow us to create an independent virtual router instance. we start the presentation from the introduction of mikrotik and GLC, and then the metarouter. we also do demo and QA and the end of presentation.
the recording is available on youtube: https://www.youtube.com/channel/UCI611_IIkQC0rsLWIFIx_yg
Presto: SQL-on-Anything. Netherlands Hadoop User Group MeetupWojciech Biela
Presto is an open source distributed SQL query engine for running interactive analytic queries against data sources of all sizes ranging from gigabytes to petabytes. Presto was designed and written from the ground up for interactive analytics and approaches the speed of commercial data warehouses while scaling to the size of organizations like Facebook. One key feature in Presto is the ability to query data where it lives via an uniform ANSI SQL interface. Presto’s connector architecture creates an abstraction layer for anything that can be represented in a columnar or row-like format, such as HDFS, Amazon S3, Azure Storage, NoSQL stores, relational databases, Kafka streams and even proprietary data stores. Furthermore, a single Presto query can combine data from multiple sources, allowing for analytics across an entire organization.
Twitter handles billions of events per minute that are logged by clients. They use a scalable architecture with modular client daemons and aggregator daemons to aggregate events into categories on HDFS. To improve scalability, they group categories into category groups and write events to HDFS files together. They also group aggregators to scale independently and isolate resources. Within a single aggregator, they improved memory usage and added microbatching to reduce latency. Going forward, they aim to further reduce latency and improve failure handling.
Efficiently Triaging CI Pipelines with Apache Spark: Mixing 52 Billion Events...Databricks
Continuous integration (CI) pipelines generate massive amounts of messy log data. At Pure Storage engineering, we run over 65,000 tests per day creating a large triage problem. Spark’s flexible computing platform allows us to write a single application for both streaming and batch jobs to understand the state of our CI pipeline. Spark indexes log data for real-time reporting (Streaming), uses Machine Learning for performance modeling and prediction (Batch job), and re-indexes old data for newly encoded patters (Batch job). Previous work on a mixed streaming and batch environment describes the options for persisting data and their trade-offs:
1) short interval buckets which hurts batch performance
2) long interval buckets which increases micro batch time windows
3) additional software on the background to compact the short interval buckets which adds complexity.
This talk will go over how we use the filesystem metadata of our disaggregated compute and storage layers to write over half a million files per day of varied sizes from 52 Billion events and have efficient batch jobs without compaction that allow us to process over 40TB per hour. We will go over the challenges and best practices to achieve efficiency in this mixed environment scenarios.
Observing Intraday Indicators Using Real-Time Tick Data on Apache Superset an...DataWorks Summit
The Central Bank of the Republic of Turkey is primarily responsible for steering the monetary and exchange rate policies in Turkey.
One of the major core functions of the Bank is market operations. In this context, analyzing and interpreting real-time tick data related to money market instruments has become not only a requirement but also a challenge.
For this use case, an API provided by one of the financial data vendors has been used to gather real-time tick data and data routing has been orchestrated by Apache NiFi.
Gathered data is being transferred to Kafka topics and then handed off to Druid for real-time indexing tasks.
Indicators such as effective cost, bid-ask spread, price impact measures, return reversal are calculated using Apache Storm and finally visualized by means of Apache Superset in order to provide decision-makers with a new set of tools.
Introduction to Data Engineer and Data Pipeline at Credit OKKriangkrai Chaonithi
The document discusses the role of data engineers and data pipelines. It begins with an introduction to big data and why data volumes are increasing. It then covers what data engineers do, including building data architectures, working with cloud infrastructure, and programming for data ingestion, transformation, and loading. The document also explains data pipelines, describing extract, transform, load (ETL) processes and batch versus streaming data. It provides an example of Credit OK's data pipeline architecture on Google Cloud Platform that extracts raw data from various sources, cleanses and loads it into BigQuery, then distributes processed data to various applications. It emphasizes the importance of data engineers in processing and managing large, complex data sets.
PGConf.ASIA 2019 Bali - How did PostgreSQL Write Load Balancing of Queries Us...Equnix Business Solutions
Atsushi Mitani from SRA Nishi-Nihon Inc. presented on how to perform write load balancing in PostgreSQL using transactions. He explained that write load distribution is important for systems with high write volumes. PostgreSQL can distribute write load using table partitioning with foreign data wrappers (FDW), which allows partitioning across database instances. Mitani created patches to automate the partitioning setup and load data in parallel to child tables to speed up benchmarking. Benchmark results showed that while increasing child databases improves performance without transactions, increasing parent databases is better with transactions to avoid lock queues. The optimal configuration depends on data size, queries, and hardware.
Presto talk @ Global AI conference 2018 Bostonkbajda
Presented at Global AI Conference in Boston 2018:
http://www.globalbigdataconference.com/boston/global-artificial-intelligence-conference-106/speaker-details/kamil-bajda-pawlikowski-62952.html
Presto, an open source distributed SQL engine, is widely recognized for its low-latency queries, high concurrency, and native ability to query multiple data sources. Proven at scale in a variety of use cases at Facebook, Airbnb, Netflix, Uber, Twitter, LinkedIn, Bloomberg, and FINRA, Presto experienced an unprecedented growth in popularity in both on-premises and cloud deployments in the last few years. Presto is really a SQL-on-Anything engine in a single query can access data from Hadoop, S3-compatible object stores, RDBMS, NoSQL and custom data stores. This talk will cover some of the best use cases for Presto, recent advancements in the project such as Cost-Based Optimizer and Geospatial functions as well as discuss the roadmap going forward.
Postgres Vision 2018: WAL: Everything You Want to KnowEDB
The document is a presentation about PostgreSQL's Write-Ahead Log (WAL) system. It discusses what the WAL is, how it works, and how it is used for tasks like replication, backup and point-in-time recovery. The WAL logs all transactions to prevent data loss during crashes and ensures data integrity. It is critical for high availability and disaster recovery capabilities in PostgreSQL.
Iceberg: a modern table format for big data (Ryan Blue & Parth Brahmbhatt, Netflix)
Presto Summit 2018 (https://www.starburstdata.com/technical-blog/presto-summit-2018-recap/)
Journey of Migrating 1 Million Presto Queries - Presto Webinar 2020Taro L. Saito
Arm Treasure Data utilizes Presto as the query engine processing over 1 million queries per day to support the data business of 500+ companies in three regions; US, EU, and Asia. Arm Treasure Data had been using Presto 0.205 and in 2019 started a big migration project to Presto 317. Although we performed extensive query simulations to check any incompatibilities, we faced many unexpected challenges during the migration at production.
Presto and you are there (Nishant Rayan, Lyft)
Presto Summit 2018 (https://www.starburstdata.com/technical-blog/presto-summit-2018-recap/)
Presentation at SF Kubernetes Meetup (10/30/18), Introducing TiDB/TiKVKevin Xu
This deck was presented at the SF Kubernetes Meetup held at Microsoft's downtown SF office, introducing the architecture of TiDB and TiKV (a CNCF project), key use cases, a user story with Mobike (one of the largest bikesharing platforms in the world), and how TiDB is deployed across different cloud environment using TiDB Operator.
Spark Summit EU talk by Sebastian Schroeder and Ralf SigmundSpark Summit
Otto needed to merge large tracking messages from two Kafka streams with different throughput into a single stream while preserving event timestamps and ensuring at least once delivery. They used Spark Streaming to build a custom event-time merge that used UpdateStateByKey to merge messages, tracked event times to timeout messages, handled uneven throughput, and stored offsets to guarantee at least once delivery. The solution provided excellent performance and scalability while being extensible through custom RDDs.
The document discusses the challenges of handling billions of events per minute at Twitter. It describes Twitter's event log pipeline architecture using microservices, event aggregation with Scribe, and event processing with MapReduce. The pipeline was later modified to use streaming technologies to reduce latency, including Apache Beam and Google Cloud services. Latency for different dataset types ranges from seconds to minutes. Future challenges include scaling further for volume and spikes while maintaining fast failure recovery times.
Pivotal Greenplum provides fast, secure cloud deployments of its data warehouse platform with the same experience across AWS, Azure, and GCP. Deployments are optimized for speed through performance tuning of virtual machines, disks, and networks. Key goals include leveraging cloud features like on-demand provisioning, node replacement, disk snapshots, upgrades, and optional installations through a web interface. Deployments are similar across clouds with comparable parameters, tools, and software versions. Security is ensured through vendor-reviewed templates, password encryption, and network isolation.
in this webinar, we were discussing about an introduction to mikrotik, network management, and the dude.
There are some demos of installing the dude on mikrotik, as well as some the dude features to manage several aspects of network elements.
the recording of this webinar will be uploaded on youtube: https://www.youtube.com/channel/UCI611_IIkQC0rsLWIFIx_yg
in this webinar, we were discussing about a mikrotik feature that is called metarouter, which can allow us to create an independent virtual router instance. we start the presentation from the introduction of mikrotik and GLC, and then the metarouter. we also do demo and QA and the end of presentation.
the recording is available on youtube: https://www.youtube.com/channel/UCI611_IIkQC0rsLWIFIx_yg
This document provides an overview and agenda for a webinar on limiting bandwidth using Mikrotik routers. The webinar will be presented by Achmad Mardiansyah from the Indonesian company Garda Lintas Cakrawala (GLC) and will include introductions, an overview of bandwidth and bandwidth management, a demonstration of configuring bandwidth limits on Mikrotik routers, and a question and answer session. Background information is provided on GLC, the trainer, Mikrotik products and RouterOS, Mikrotik router naming conventions, and Mikrotik training and certifications. Key concepts that will be covered include what bandwidth is, considerations for applying bandwidth limits, bandwidth management mechanisms like queueing
In this webinar, we discuss a feature from mikrotik routerOS that is called fastpath. This feature will improve routerboard performance by increasing number of packet per second and lowering CPU usage. Fastpath requires several conditions to be active, and the webinar is talking about them.
the recording is available on youtube (GLC Networks Channel): https://www.youtube.com/channel/UCI611_IIkQC0rsLWIFIx_yg
In this webinar, we were discussing about Dynamic Routing with RIP (Routing Information Protocol). we discussed what is it, why we use it, the benefits of RIP, how it is works, and how its implemented on RouterOS.
the recording is available on youtube (GLC NETWORKS CHANNEL): https://www.youtube.com/channel/UCI611_IIkQC0rsLWIFIx_yg
This document contains an agenda and presentation slides for a webinar hosted by GLC Networks on IPv6 with Mikrotik RouterOS. The webinar covers an introduction to IPv6, how to enable and configure IPv6 on RouterOS, and a demonstration. It provides information on GLC Networks as an Indonesian training and consulting company, the trainer's background, and how to assign IPv6 addresses on RouterOS using SLAAC and DHCPv6 prefix delegation. Slides include comparisons of IPv4 and IPv6, IPv6 addressing rules, and assigning IPv6 addresses to clients.
Policy Based Routing (PBR) on MikrotikGLC Networks
Webinar topic: Policy Based Routing (PBR) on Mikrotik
Presenter: Achmad Mardiansyah
In this webinar series, We are discussing Policy Based Routing (PBR) on Mikrotik
Please share your feedback or webinar ideas here: http://bit.ly/glcfeedback
Check our schedule for future events: https://www.glcnetworks.com/schedule/
Follow our social media for updates: Facebook, Instagram, YouTube Channel, and telegram
Recording is available on Youtube
https://youtu.be/dzbsVZaaaAk
in this webinar, we were discussing about one important mikrotik feature that is firewall, with focus on filter table. we start the presentation from the introduction of mikrotik and firewall, and then the filter table. we also do demo and QA and the end of presentation.
the recording is available on youtube: https://www.youtube.com/channel/UCI611_IIkQC0rsLWIFIx_yg
This document outlines an online training presentation about data analytics using database management systems (DBMS). The agenda includes introductions, an overview of data analytics and DBMS, how to perform data analytics on DBMS using SQL, and a live practice session. The trainer will introduce themselves and their experience before discussing the components of understanding data, moving from data to intelligence, and different levels of data analysis. SQL and how it is used to interact with and retrieve data from DBMS will also be explained.
Evolving ALLSTOCKER: Agile increments with Pharo SmalltalkESUG
ALLSTOCKER is an online marketplace for used construction equipment that uses Pharo Smalltalk. It evolved its development process and architecture over time to handle growth. Originally a single Pharo image, it adopted microservices, multiple Pharo images, and reactive technologies like Elasticsearch and Neo4j to improve search and auctions. This allowed scaling to over 400 machine listings per month while maintaining responsiveness.
#TOA13 - Tech Opoen Air Recommender HackathonTorben Brodt
The document describes the plista Recommender Challenge Hackathon. It provides information on:
- plista's recommendation and advertising network which delivers over 8 billion impressions per month.
- The hackathon challenges participants to develop a recommender that implements plista's API to be evaluated on its success in tracking recommendations. The best recommender that is scalable and works for industry will win.
- Participants can use various programming languages and machine learning libraries. Starting involves registering, implementing examples from the wiki, and getting real-time recommendation data from plista to display on publishers.
- Recommender ideas suggested focusing on implicit feedback, incremental updates, and handling cross-domain recommendations within publisher slices of data.
Webinar topic: Mikrotik Fastpath vs Fasttrack
Presenter: Achmad Mardiansyah
In this webinar series, We are discussing Mikrotik Fastpath vs Fasttrack
Please share your feedback or webinar ideas here: http://bit.ly/glcfeedback
Check our schedule for future events: https://www.glcnetworks.com/en/schedule/
Follow our social media for updates: Facebook, Instagram, YouTube Channel, and telegram also discord
Recording available on Youtube
https://youtu.be/et6MwYY0dvo
Webinar topic: Choosing Mikrotik Platform x86 vs chr
Presenter: Achmad Mardiansyah
In this webinar series, We are discussing Choosing Mikrotik Platform x86 vs chr
Please share your feedback or webinar ideas here: http://bit.ly/glcfeedback
Check our schedule for future events: https://www.glcnetworks.com/en/schedule/
Follow our social media for updates: Facebook, Instagram, YouTube Channel, and telegram also discord
Recording available on Youtube
https://youtu.be/6_PPWS4f2Ek
Creando microservicios con Java y Microprofile - Nicaragua JUGCésar Hernández
En esta sesión los asistentes presenciaron la base teórica y práctica para la creación de micro servicios con Java, JakartaEE, MicroProfile utilizando TomEE como servidor de aplicaciones.
Google Associate Cloud Engineer Certification TipsDaniel Zivkovic
Tips & best practices to prepare for the GCP ACE (Associate Cloud Engineer) Exam by Dan Sullivan - the author of the official Google Cloud Certified study guides!
Event details: https://www.meetup.com/Serverless-Toronto/events/271344917/
Event recording: http://youtube.serverlesstoronto.org/
RSVP for more exciting (online) events at https://www.meetup.com/Serverless-Toronto/events/
GeOrchestra is a free and modular spatial data infrastructure (SDI) software that allows users to easily search, access, share and visualize geospatial data. It provides interoperable OGC web services and REST APIs in a secure manner using modules like GeoNetwork, GeoServer and CAS authentication. GeOrchestra started as a project in 2008 and is now used by various governments and organizations around the world to power their SDIs. It has an active international community and CampToCamp provides commercial support and services for the software.
geOrchestra, a free, modular and secure SDICamptocamp
geOrchestra is a free, modular and interoperable Spatial Data Infrastructure software born in 2009 to meet the requirements of the INSPIRE directive in Europe.
geOrchestra’s core features a security proxy and a single-sign-on authentication system. Next to this core are independant and interoperable modules to build your own Spatial Data Infrastructure:
- a metadata catalog based on GeoNetwork,
- a map and features server: GeoServer,
- a map tile server: GeoWebCache,
- an advanced web map viewer,
- a data extractor,
- and several other admin-oriented modules.
Modules interact using OGC interfaces or REST APIs.
Initially covering Brittany, then France, geOrchestra now spreads worldwide with SDIs in Bolivia, Nicaragua, Switzerland and India.
http://www.georchestra.org/
in this presentation, i will discuss about common issues that might happen when implementing load balancing. i will start the presentation with fundamental concepts that many people missed. and then discuss some implementation of it with demo.
Robert Bates, SVP Sales Engineering of Crunchy Data explains how you can tackle Data Gravity, Kubernetes, and strategies/best practices to run, scale, and leverage stateful containers in production.
PGConf APAC 2018: Sponsored Talk by Fujitsu - The growing mandatory requireme...PGConf APAC
Speaker: Rajni Baliyan
As the volume of data of a personal nature and commodification of information collected and analysed increases; so is the focus on privacy and data security. Many countries are examining international and domestic laws in order to protect consumers and organisations alike.
The Australian Senate has recently passed a bill containing mandatory requirements to notify the privacy commissioner and consumers when data is at risk of causing serious harm in the case of a data breach occurring.
Europe has also announced new laws that allow consumers more control over their data. These laws allow consumers to tell companies to erase any data held about them.
These new laws will have a significant impact on organisations that store personal information.
This talk will examine some of these legislative changes and how specific PostgreSQL features can assist organisations in meeting their obligations and avoid heavy fines associated with breaching them.
The document discusses implementing centralized authorization in PostgreSQL by synchronizing user roles and privileges with an LDAP server. It provides a step-by-step approach to setting up LDAP authentication in PostgreSQL and using scripts to synchronize user roles and privileges between the database and LDAP based on group membership. The synchronization scripts create roles for each LDAP user, grant privileges to roles based on mapping rules, and handle role inheritance.
Speaker: Alexander Kukushkin
Kubernetes is a solid leader among different cloud orchestration engines and its adoption rate is growing on a daily basis. Naturally people want to run both their applications and databases on the same infrastructure.
There are a lot of ways to deploy and run PostgreSQL on Kubernetes, but most of them are not cloud-native. Around one year ago Zalando started to run HA setup of PostgreSQL on Kubernetes managed by Patroni. Those experiments were quite successful and produced a Helm chart for Patroni. That chart was useful, albeit a single problem: Patroni depended on Etcd, ZooKeeper or Consul.
Few people look forward to deploy two applications instead of one and support them later on. In this talk I would like to introduce Kubernetes-native Patroni. I will explain how Patroni uses Kubernetes API to run a leader election and store the cluster state. I’m going to live-demo a deployment of HA PostgreSQL cluster on Minikube and share our own experience of running more than 130 clusters on Kubernetes.
Patroni is a Python open-source project developed by Zalando in cooperation with other contributors on GitHub: https://github.com/zalando/patroni
PGConf APAC 2018 - High performance json postgre-sql vs. mongodbPGConf APAC
Speakers: Dominic Dwyer & Wei Shan Ang
This talk was presented in Percona Live Europe 2017. However, we did not have enough time to test against more scenario. We will be giving an updated talk with a more comprehensive tests and numbers. We hope to run it against citusDB and MongoRocks as well to provide a comprehensive comparison.
https://www.percona.com/live/e17/sessions/high-performance-json-postgresql-vs-mongodb
PGConf APAC 2018 - Monitoring PostgreSQL at ScalePGConf APAC
Speaker: Lukas Fittl
Your PostgreSQL database is one of the most important pieces of your architecture - yet the level of introspection available in Postgres is often hard to work with. Its easy to get very detailed information, but what should you really watch out for, send reports on and alert on?
In this talk we'll discuss how query performance statistics can be made accessible to application developers, critical entries one should monitor in the PostgreSQL log files, how to collect EXPLAIN plans at scale, how to watch over autovacuum and VACUUM operations, and how to flag issues based on schema statistics.
We'll also talk a bit about monitoring multi-server setups, first going into high availability and read standbys, logical replication, and then reviewing how monitoring looks like for sharded databases like Citus.
The talk will primarily describe free/open-source tools and statistics views readily available from within Postgres.
PGConf APAC 2018 - Where's Waldo - Text Search and Pattern in PostgreSQLPGConf APAC
Speaker: Joe Conway
There are many use cases for text search and pattern matching, and there are also a wide variety of techniques available in PostgreSQL to perform text search and pattern matching. Figuring out the best "match" between use case and technique can be confusing. This talk will review the possibilities and provide guidance regarding when to use what method, and especially how to properly deal with the related index methods to ensure speedy searches. This talk covers:
* The primary available search methods
* Examples illustrating when to use each
* Extensive discussion of index use
* Timing comparisons using realistic examples
PGConf APAC 2018 - Managing replication clusters with repmgr, Barman and PgBo...PGConf APAC
Speaker: Ian Barwick
PostgreSQL and reliability go hand-in-hand - but your data is only truly safe with a solid and trusted backup system in place, and no matter how good your application is, it's useless if it can't talk to your database.
In this talk we'll demonstrate how to set up a reliable replication
cluster using open source tools closely associated with the PostgreSQL project. The talk will cover following areas:
- how to set up and manage a replication cluster with `repmgr`
- how to set up and manage reliable backups with `Barman`
- how to manage failover and application connections with `repmgr` and `PgBouncer`
Ian Barwick has worked for 2ndQuadrant since 2014, and as well as making various contributions to PostgreSQL itself, is lead `repmgr` developer. He lives in Tokyo, Japan.
PGConf APAC 2018 - PostgreSQL HA with Pgpool-II and whats been happening in P...PGConf APAC
Speaker: Muhammad Usama
Pgpool-II has been around to complement PostgreSQL over a decade and provides many features like connection pooling, failover, query caching, load balancing, and HA. High Availability (HA) is very critical to most enterprise application, the clients needs the ability to automatically reconnect with a secondary node when the master nodes goes down.
This is where Pgpool-II watchdog feature comes in, the core feature of Pgpool-II provides HA by eliminating the SPOF is the Watchdog. This watchdog feature has been around for a while but it went through major overhauling and enhancements in recent releases. This talk aims to explain the watchdog feature, the recent enhancements went into the watchdog and describe how it can be used to provide PostgreSQL HA and automatic failover.
Their is rising trend of enterprise deployment shifting to cloud based environment, Pgpool II can be used in the cloud without any issues. In this talk we will give some ideas how Pgpool-II is used to provide PostgreSQL HA in cloud environment.
Finally we will summarise the major features that have been added in the recent major release of Pgpool II and whats in the pipeline for the next major release.
This document discusses migrating Oracle databases to EDB Postgres. It outlines the steps to migrate, including assessing the database, preparing the environment, migrating database objects and data, porting applications, testing, integrating, and rolling out the migration. It then provides two case studies of large companies that migrated from Oracle to EDB Postgres to significantly lower costs while still meeting their business and technical requirements.
About a year ago I was caught up in line-of-fire when a production system started behaving abruptly
- A batch process which would finish in 15minutes started taking 1.5 hours
- We started facing OLTP read queries on standby being cancelled
- We faced a sudden slowness on the Primary server and we were forced to do a forceful switch to standby.
We were able to figure out that some peculiarities of the application code and batch process were responsible for this. But we could not fix the application code (as it is packaged application).
In this talk I would like to share more details of how we debugged, what was the problem we were facing and how we applied a work around for it. We also learnt that a query returning in 10minutes may not be as dangerous as a query returning in 10sec but executed 100s of times in an hour.
I will share in detail-
- How to map the process/top stats from OS with pg_stat_activity
- How to get and read explain plan
- How to judge if a query is costly
- What tools helped us
- A peculiar autovacuum/vacuum Vs Replication conflict we ran into
- Various parameters to tune autvacuum and auto-analyze process
- What we have done to work-around the problem
- What we have put in place for better monitoring and information gathering
The document discusses PostgreSQL version 11 and future development. It provides a history of PostgreSQL and its predecessors, describing the development process and community. It summarizes key features committed to version 11, including improvements to partitioning, parallelization, performance and logical replication. It also outlines features proposed for future versions, with a focus on continued enhancements to partitioning and query planning.
This presentation was used by Blair during his talk on Aurora and PostgreSQl compatibility for Aurora at pgDay Asia 2017. The talk was part of dedicated PostgreSQL track at FOSSASIA 2017
This document discusses using drones and PostgreSQL/PostGIS for agricultural applications. It describes how drones can capture imaging data for tasks like measuring crop health through NDVI analysis. PostgreSQL is useful for organizing the large amounts of drone data, like flight plans, sensor readings, and imagery. The document provides an example of importing this data into PostgreSQL and using PostGIS functions to process imagery, extract waypoints of problem areas, and more.
How to teach an elephant to rock'n'rollPGConf APAC
The document discusses techniques for optimizing PostgreSQL queries, including:
1. Using index only scans to efficiently skip large offsets in queries instead of scanning all rows.
2. Pulling the LIMIT clause under joins and aggregates to avoid processing unnecessary rows.
3. Employing indexes creatively to perform DISTINCT operations by scanning the index instead of the entire table.
4. Optimizing DISTINCT ON queries by looping through authors and returning the latest row for each instead of a full sort.
PostgreSQL is one of the most loved databases and that is why AWS could not hold back from offering PostgreSQL as RDS. There are some really nice features in RDS which can be good for DBA and inspiring for Enterprises to build resilient solution with PostgreSQL.
Devrim Gunduz gives a presentation on Write-Ahead Logging (WAL) in PostgreSQL. WAL logs all transactions to files called write-ahead logs (WAL files) before changes are written to data files. This allows for crash recovery by replaying WAL files. WAL files are used for replication, backup, and point-in-time recovery (PITR) by replaying WAL files to restore the database to a previous state. Checkpoints write all dirty shared buffers to disk and update the pg_control file with the checkpoint location.
Lessons PostgreSQL learned from commercial databases, and didn’tPGConf APAC
This is the ppt used by Illay for his presentation at pgDay Asia 2016 - "Lessons PostgreSQL learned from commercial
databases, and didn’t". The talk takes you through some of the really good things that PostgreSQL has done really well and somethings that PostgreSQL can learn from other databases
Query Parallelism in PostgreSQL: What's coming next?PGConf APAC
This presentation was presented by Dilip Kumar (a PostgreSQL contributor) at pgDay Asia 2017. The presentation talks about Prallel query features released in v9.6, the infrastructure for the prallel query feature which was built in previous versions and what is the roadmap for prallel query.
Why we love pgpool-II and why we hate it!PGConf APAC
Pgpool is middleware that works between PostgreSQL clients and servers to provide connection pooling, replication, and load balancing. The presenter's company deployed pgpool in various architectures including master-slave replication and load balancing configurations. They experienced some issues with pgpool like connection errors when using application pooling, lack of guaranteed connection reuse, and bugs. Tips are provided like ensuring synchronized server times and restricting health check users. Pgpool may not be best when automatic node rejoining is needed or during network instability.
Increasing Retail Store Efficiency How can Planograms Save Time and Money.pptxAnoop Ashok
In today's fast-paced retail environment, efficiency is key. Every minute counts, and every penny matters. One tool that can significantly boost your store's efficiency is a well-executed planogram. These visual merchandising blueprints not only enhance store layouts but also save time and money in the process.
Spark is a powerhouse for large datasets, but when it comes to smaller data workloads, its overhead can sometimes slow things down. What if you could achieve high performance and efficiency without the need for Spark?
At S&P Global Commodity Insights, having a complete view of global energy and commodities markets enables customers to make data-driven decisions with confidence and create long-term, sustainable value. 🌍
Explore delta-rs + CDC and how these open-source innovations power lightweight, high-performance data applications beyond Spark! 🚀
HCL Nomad Web – Best Practices und Verwaltung von Multiuser-Umgebungenpanagenda
Webinar Recording: https://www.panagenda.com/webinars/hcl-nomad-web-best-practices-und-verwaltung-von-multiuser-umgebungen/
HCL Nomad Web wird als die nächste Generation des HCL Notes-Clients gefeiert und bietet zahlreiche Vorteile, wie die Beseitigung des Bedarfs an Paketierung, Verteilung und Installation. Nomad Web-Client-Updates werden “automatisch” im Hintergrund installiert, was den administrativen Aufwand im Vergleich zu traditionellen HCL Notes-Clients erheblich reduziert. Allerdings stellt die Fehlerbehebung in Nomad Web im Vergleich zum Notes-Client einzigartige Herausforderungen dar.
Begleiten Sie Christoph und Marc, während sie demonstrieren, wie der Fehlerbehebungsprozess in HCL Nomad Web vereinfacht werden kann, um eine reibungslose und effiziente Benutzererfahrung zu gewährleisten.
In diesem Webinar werden wir effektive Strategien zur Diagnose und Lösung häufiger Probleme in HCL Nomad Web untersuchen, einschließlich
- Zugriff auf die Konsole
- Auffinden und Interpretieren von Protokolldateien
- Zugriff auf den Datenordner im Cache des Browsers (unter Verwendung von OPFS)
- Verständnis der Unterschiede zwischen Einzel- und Mehrbenutzerszenarien
- Nutzung der Client Clocking-Funktion
How Can I use the AI Hype in my Business Context?Daniel Lehner
𝙄𝙨 𝘼𝙄 𝙟𝙪𝙨𝙩 𝙝𝙮𝙥𝙚? 𝙊𝙧 𝙞𝙨 𝙞𝙩 𝙩𝙝𝙚 𝙜𝙖𝙢𝙚 𝙘𝙝𝙖𝙣𝙜𝙚𝙧 𝙮𝙤𝙪𝙧 𝙗𝙪𝙨𝙞𝙣𝙚𝙨𝙨 𝙣𝙚𝙚𝙙𝙨?
Everyone’s talking about AI but is anyone really using it to create real value?
Most companies want to leverage AI. Few know 𝗵𝗼𝘄.
✅ What exactly should you ask to find real AI opportunities?
✅ Which AI techniques actually fit your business?
✅ Is your data even ready for AI?
If you’re not sure, you’re not alone. This is a condensed version of the slides I presented at a Linkedin webinar for Tecnovy on 28.04.2025.
Designing Low-Latency Systems with Rust and ScyllaDB: An Architectural Deep DiveScyllaDB
Want to learn practical tips for designing systems that can scale efficiently without compromising speed?
Join us for a workshop where we’ll address these challenges head-on and explore how to architect low-latency systems using Rust. During this free interactive workshop oriented for developers, engineers, and architects, we’ll cover how Rust’s unique language features and the Tokio async runtime enable high-performance application development.
As you explore key principles of designing low-latency systems with Rust, you will learn how to:
- Create and compile a real-world app with Rust
- Connect the application to ScyllaDB (NoSQL data store)
- Negotiate tradeoffs related to data modeling and querying
- Manage and monitor the database for consistently low latencies
Generative Artificial Intelligence (GenAI) in BusinessDr. Tathagat Varma
My talk for the Indian School of Business (ISB) Emerging Leaders Program Cohort 9. In this talk, I discussed key issues around adoption of GenAI in business - benefits, opportunities and limitations. I also discussed how my research on Theory of Cognitive Chasms helps address some of these issues
TrsLabs - Fintech Product & Business ConsultingTrs Labs
Hybrid Growth Mandate Model with TrsLabs
Strategic Investments, Inorganic Growth, Business Model Pivoting are critical activities that business don't do/change everyday. In cases like this, it may benefit your business to choose a temporary external consultant.
An unbiased plan driven by clearcut deliverables, market dynamics and without the influence of your internal office equations empower business leaders to make right choices.
Getting things done within a budget within a timeframe is key to Growing Business - No matter whether you are a start-up or a big company
Talk to us & Unlock the competitive advantage
Complete Guide to Advanced Logistics Management Software in Riyadh.pdfSoftware Company
Explore the benefits and features of advanced logistics management software for businesses in Riyadh. This guide delves into the latest technologies, from real-time tracking and route optimization to warehouse management and inventory control, helping businesses streamline their logistics operations and reduce costs. Learn how implementing the right software solution can enhance efficiency, improve customer satisfaction, and provide a competitive edge in the growing logistics sector of Riyadh.
Noah Loul Shares 5 Steps to Implement AI Agents for Maximum Business Efficien...Noah Loul
Artificial intelligence is changing how businesses operate. Companies are using AI agents to automate tasks, reduce time spent on repetitive work, and focus more on high-value activities. Noah Loul, an AI strategist and entrepreneur, has helped dozens of companies streamline their operations using smart automation. He believes AI agents aren't just tools—they're workers that take on repeatable tasks so your human team can focus on what matters. If you want to reduce time waste and increase output, AI agents are the next move.
Technology Trends in 2025: AI and Big Data AnalyticsInData Labs
At InData Labs, we have been keeping an ear to the ground, looking out for AI-enabled digital transformation trends coming our way in 2025. Our report will provide a look into the technology landscape of the future, including:
-Artificial Intelligence Market Overview
-Strategies for AI Adoption in 2025
-Anticipated drivers of AI adoption and transformative technologies
-Benefits of AI and Big data for your business
-Tips on how to prepare your business for innovation
-AI and data privacy: Strategies for securing data privacy in AI models, etc.
Download your free copy nowand implement the key findings to improve your business.
TrustArc Webinar: Consumer Expectations vs Corporate Realities on Data Broker...TrustArc
Most consumers believe they’re making informed decisions about their personal data—adjusting privacy settings, blocking trackers, and opting out where they can. However, our new research reveals that while awareness is high, taking meaningful action is still lacking. On the corporate side, many organizations report strong policies for managing third-party data and consumer consent yet fall short when it comes to consistency, accountability and transparency.
This session will explore the research findings from TrustArc’s Privacy Pulse Survey, examining consumer attitudes toward personal data collection and practical suggestions for corporate practices around purchasing third-party data.
Attendees will learn:
- Consumer awareness around data brokers and what consumers are doing to limit data collection
- How businesses assess third-party vendors and their consent management operations
- Where business preparedness needs improvement
- What these trends mean for the future of privacy governance and public trust
This discussion is essential for privacy, risk, and compliance professionals who want to ground their strategies in current data and prepare for what’s next in the privacy landscape.
Special Meetup Edition - TDX Bengaluru Meetup #52.pptxshyamraj55
We’re bringing the TDX energy to our community with 2 power-packed sessions:
🛠️ Workshop: MuleSoft for Agentforce
Explore the new version of our hands-on workshop featuring the latest Topic Center and API Catalog updates.
📄 Talk: Power Up Document Processing
Dive into smart automation with MuleSoft IDP, NLP, and Einstein AI for intelligent document workflows.
Book industry standards are evolving rapidly. In the first part of this session, we’ll share an overview of key developments from 2024 and the early months of 2025. Then, BookNet’s resident standards expert, Tom Richardson, and CEO, Lauren Stewart, have a forward-looking conversation about what’s next.
Link to recording, presentation slides, and accompanying resource: https://bnctechforum.ca/sessions/standardsgoals-for-2025-standards-certification-roundup/
Presented by BookNet Canada on May 6, 2025 with support from the Department of Canadian Heritage.
UiPath Community Berlin: Orchestrator API, Swagger, and Test Manager APIUiPathCommunity
Join this UiPath Community Berlin meetup to explore the Orchestrator API, Swagger interface, and the Test Manager API. Learn how to leverage these tools to streamline automation, enhance testing, and integrate more efficiently with UiPath. Perfect for developers, testers, and automation enthusiasts!
📕 Agenda
Welcome & Introductions
Orchestrator API Overview
Exploring the Swagger Interface
Test Manager API Highlights
Streamlining Automation & Testing with APIs (Demo)
Q&A and Open Discussion
Perfect for developers, testers, and automation enthusiasts!
👉 Join our UiPath Community Berlin chapter: https://community.uipath.com/berlin/
This session streamed live on April 29, 2025, 18:00 CET.
Check out all our upcoming UiPath Community sessions at https://community.uipath.com/events/.
This is the keynote of the Into the Box conference, highlighting the release of the BoxLang JVM language, its key enhancements, and its vision for the future.
Big Data Analytics Quick Research Guide by Arthur MorganArthur Morgan
This is a Quick Research Guide (QRG).
QRGs include the following:
- A brief, high-level overview of the QRG topic.
- A milestone timeline for the QRG topic.
- Links to various free online resource materials to provide a deeper dive into the QRG topic.
- Conclusion and a recommendation for at least two books available in the SJPL system on the QRG topic.
QRGs planned for the series:
- Artificial Intelligence QRG
- Quantum Computing QRG
- Big Data Analytics QRG
- Spacecraft Guidance, Navigation & Control QRG (coming 2026)
- UK Home Computing & The Birth of ARM QRG (coming 2027)
Any questions or comments?
- Please contact Arthur Morgan at [email protected].
100% human made.
2. https://www.2ndQuadrant.com
PGConf APAC
Singapore | March 22-23, 2018
Who am I?
● Head of Marketing & Products @ 2ndQuadrant
● Got ‘pushed’ into PostgreSQL in 2004, ended up falling in
love with it
● 2ndQuadrant is my 4th PostgreSQL company
● Active in the global PostgreSQL community
○ Head of PostgreSQL User Groups
■ Dubai | Islamabad
○ Member organizing team
■ 2Q PGConf | PGConf UK | PGConf APAC
○ Member talk selection committee: PGDay OZ
○ Advocacy at various events across the world
3. https://www.2ndQuadrant.com
PGConf APAC
Singapore | Mar 22-23, 2018
Introducing PostgreSQL
● Database Management System
○ Relational & Object-Relational Features
○ ACID compliant
● Fully Open Source
○ Most liberal open source license: BSD
○ Free to use, free to distribute …
UNLIMITED & FOREVER!
○ Open to enhancements and contributions
● Runs on all major operating systems
○ Linux
○ Unix (AIX, BSD, HP-UX, SGI IRIX, OSX,
Solaris, Tru64)
○ Windows
9. https://www.2ndQuadrant.com
PGConf APAC
Singapore | Mar 22-23, 2018
PostgreSQL - History
● Developed by Stonebraker in 1986-93 @ University of California
at Berkeley, after he left Ingres – hence name Post gres
● Literally a ground-up rethink about how databases should work
○ Multiple CPUs
○ Advanced concurrency design
○ User defined datatypes
○ User defined functions
○ Partial indexes
● All these concepts started with PostgreSQL and then went into
other databases
10. https://www.2ndQuadrant.com
PGConf APAC
Singapore | Mar 22-23, 2018
PostgreSQL - History
● University Postgres 1986-1993
● Postgres95 1994-1995
● PostgreSQL 1996-
○ Phase 1: Avoid crashes 1996-1998
○ Phase 2: SQL Compliance 1998-2001
○ Phase 3: Robustness 2001-2004
○ Phase 4: Enterprise Features 2004-2011
○ Phase 5: World Class 2011-
12. https://www.2ndQuadrant.com
PGConf APAC
Singapore | Mar 22-23, 2018
Major contributor to open source PostgreSQL
● Logical Replication (10.0)
● Parallel Aggregates (9.6)
● Postgres-XL - MPP & horizontally scalable
PostgreSQL (9.5)
● Multi Master Replication - BDR (9.4)
● Hot Standby (9.0)
● Point in time Recovery (8.0)
About 2ndQuadrant
13. https://www.2ndQuadrant.com
PGConf APAC
Singapore | Mar 22-23, 2018
● Biggest collective of PostgreSQL contributors
○ 4 committers (including 1 member of core team,
out of only 5 in the world)
○ 17 major contributors
● Thought leader in PostgreSQL development
○ Major sponsors of the PostgreSQL project
○ The only company to contribute major features
into all of the last 13 releases
About 2ndQuadrant (contd … )
15. https://www.2ndQuadrant.com
PGConf APAC
Singapore | Mar 22-23, 2018
What does that mean for you?
● The same contributors to PostgreSQL are the ones
that make up the 2ndQuadrant Support and Services
teams
● Our extensive contributions mean that no one in the
world knows PostgreSQL better than we do
● We use this knowledge to provide a guaranteed
24-hour bug fix
18. https://www.2ndQuadrant.com
PGConf APAC
Singapore | Mar 22-23, 2018
24/7 Production Support
When it’s critical, you can count on us!
Our Guarantees
● A live local person response to your inquiry within 15 minutes
● A workaround to your problem within 4 hours
● A Bug fix within 24 hours
“The [2ndQuadrant] support is always
committed to the best answer. Not the
first answer.”
- Otávio Sampaio; CIO, Fidelize
19. https://www.2ndQuadrant.com
PGConf APAC
Singapore | Mar 22-23, 2018
Remote DBA
Put your database in Expert hands
● Real-time monitoring and alerts
● 24/7 Production Support Service
● Annual Health Check
● Reduce cost on resources
● Save money on downtime
● Frequent status reports & updates
20. https://www.2ndQuadrant.com
PGConf APAC
Singapore | Mar 22-23, 2018
Reasons to get a Health Check
● Change of usage patterns
● Increase in size & scale
● Hardware Upgrades
● Maximize bang for your buck!
● Validate configuration & deployment
against best practices
PostgreSQL Health Check
Prevention is better than a cure
22. https://www.2ndQuadrant.com
PGConf APAC
Singapore | Mar 22-23, 2018
● A data breach will cost the average
company about $3.8 million
● PostgreSQL Security Audit helps you
✓ Identify breach points in your database
✓ Ensure SLAs are met
✓ Classify Vulnerabilities
✓ Validate Security of Backups and Backup
Channels
✓ Identify Outdated Software and Libraries
✓ Comply with regulations like GDPR, PCI, &
DSS
PostgreSQL Security Audit
Identify threats to harden your database
24. https://www.2ndQuadrant.com
PGConf APAC
Singapore | Mar 22-23, 2018
● Postgres-BDR (Bi-Directional Replication)
○ Multimaster Replication for PostgreSQL
○ Geographically distributed databases
○ Asynchronous logical replication
● pglogical
○ Logical Replication
○ Migrate, upgrade, aggregate, scale out, or integrate
○ Replicate across PostgreSQL major versions
● repmgr
○ High availability and auto-failover management
○ Manages PostgreSQL clusters using Hot Standby capability
○ Simplifies administration and daily management
Popular Tools by 2ndQuadrant
25. https://www.2ndQuadrant.com
PGConf APAC
Singapore | Mar 22-23, 2018
Popular PostgreSQL Tools (contd …)
● Postgres-XL
○ Horizontally scalable PostgreSQL
○ OLTP and OLAP workloads with MPP parallelism
○ Fully ACID compliant including MVCC
● Barman
○ Backup strategy & disaster recovery management
○ Full hot and incremental backups with zero data loss
○ Parallel copy for backup and recovery
● OmniDB
○ Interactive browser-based tool for database management
○ Light weight, responsive interface
○ Easy to manage, SQL editing