Filesystem Performance from a Database Perspective
6 likes3,510 views
How do you choose the right filesystem for your database management system? Administrators have a variety of filesystems to choose from, as well as volume management and hardware or software RAID. This talk will examine how different the performance of filesystems really are, and how do you go about systematically determining which configuration will be the best for your application and hardware. This talk will present data generated by a group of volunteers running performance tests for database tuning. We were curious if the file systems would really behave like we expected them to, especially when used in conjunction with RAID or volume management. There is also more to file systems than how fast we can read to or write to them. Reliability is critical for production environments, and proving that is a key part of evaluating performance. The talk will review and confirm or deny assumptions that many system administrators and developers make about filesystems and databases. Data shared will include baseline throughput determined with exhaustive fio tests.
1 of 57
Downloaded 177 times
Ad
Recommended
Dpdk applications
Dpdk applicationsVipin Varghese Here are some useful GDB commands for debugging:
- break <function> - Set a breakpoint at a function
- break <file:line> - Set a breakpoint at a line in a file
- run - Start program execution
- next/n - Step over to next line, stepping over function calls
- step/s - Step into function calls
- finish - Step out of current function
- print/p <variable> - Print value of a variable
- backtrace/bt - Print the call stack
- info breakpoints/ib - List breakpoints
- delete <breakpoint#> - Delete a breakpoint
- layout src - Switch layout to source code view
- layout asm - Switch layout
Spy hard, challenges of 100G deep packet inspection on x86 platform
Spy hard, challenges of 100G deep packet inspection on x86 platformRedge Technologies This document discusses challenges and approaches for performing deep packet inspection (DPI) at speeds of 100 gigabits per second and beyond on x86 platforms. It begins by explaining why DPI is needed at such high speeds, for tasks like large-scale intrusion detection. It then examines the performance requirements for scanning payloads at 100Gbps rates. The document reviews different software approaches for payload matching, such as regular expressions, and hardware that can assist, such as Intel's Hyperscan technology. It also provides examples of how Hyperscan can be integrated into real-world intrusion detection and prevention systems.
Best Practice of Compression/Decompression Codes in Apache Spark with Sophia...
Best Practice of Compression/Decompression Codes in Apache Spark with Sophia...Databricks Nowadays, people are creating, sharing and storing data at a faster pace than ever before, effective data compression / decompression could significantly reduce the cost of data usage. Apache Spark is a general distributed computing engine for big data analytics, and it has large amount of data storing and shuffling across cluster in runtime, the data compression/decompression codecs can impact the end to end application performance in many ways.
However, there’s a trade-off between the storage size and compression/decompression throughput (CPU computation). Balancing the data compress speed and ratio is a very interesting topic, particularly while both software algorithms and the CPU instruction set keep evolving. Apache Spark provides a very flexible compression codecs interface with default implementations like GZip, Snappy, LZ4, ZSTD etc. and Intel Big Data Technologies team also implemented more codecs based on latest Intel platform like ISA-L(igzip), LZ4-IPP, Zlib-IPP and ZSTD for Apache Spark; in this session, we’d like to compare the characteristics of those algorithms and implementations, by running different micro workloads as well as end to end workloads, based on different generations of Intel x86 platform and disk.
It’s supposedly to be the best practice for big data software engineers to choose the proper compression/decompression codecs for their applications, and we also will present the methodologies of measuring and tuning the performance bottlenecks for typical Apache Spark workloads.
RocksDB Performance and Reliability Practices
RocksDB Performance and Reliability PracticesYoshinori Matsunobu Meta/Facebook's database serving social workloads is running on top of MyRocks (MySQL on RocksDB). This means our performance and reliability depends a lot on RocksDB. Not just MyRocks, but also we have other important systems running on top of RocksDB. We have learned many lessons from operating and debugging RocksDB at scale.
In this session, we will offer an overview of RocksDB, key differences from InnoDB, and share a few interesting lessons learned from production.
Performance Analysis Tools for Linux Kernel
Performance Analysis Tools for Linux Kernellcplcp1 Perf is a collection of Linux kernel tools for performance monitoring and profiling. It provides sampling and profiling of the system to analyze performance bottlenecks. Perf supports hardware events from the CPU performance counters, software events from the kernel, and tracepoint events from the kernel and loaded modules. It offers tools like perf record to sample events and store them, perf report to analyze stored samples, and perf trace to trace system events in real-time.
OSMC 2022 | VictoriaMetrics: scaling to 100 million metrics per second by Ali...
OSMC 2022 | VictoriaMetrics: scaling to 100 million metrics per second by Ali...NETWAYS The growth of observability trends and Kubernetes adoption generates more demanding requirements for monitoring systems. Volumes of time series data increase exponentially, and old solutions just can’t keep up with the pace. The talk will cover how and why we created a new open source time series database from scratch. Which architectural decisions, which trade-offs we had to take in order to match the new expectations and handle 100 million metrics per second with VictoriaMetrics. The talk will be interesting for software engineers and DevOps familiar with observability and modern monitoring systems, or for those who’re interested in building scalable high performant databases for time series.
Untangling cluster management with Helix
Untangling cluster management with HelixKishore Gopalakrishna Helix is a cluster management framework developed by LinkedIn to provide common functionality for distributed data systems (DDSs). It separates coordination and management tasks from the functional components of DDSs. Helix ensures systems satisfy their defined state models while meeting goals for load balancing and throttling state changes. Key features include modeling valid system states and transitions, optimizing resource placement, and responding to failures and elasticity events. The framework has been used to manage several production LinkedIn systems like Espresso, Databus, and a search system.
M|18 How to use MyRocks with MariaDB Server
M|18 How to use MyRocks with MariaDB ServerMariaDB plc MyRocks in MariaDB summarizes MyRocks, a storage engine for MariaDB that is based on RocksDB. It discusses how MyRocks addresses some of the limitations of InnoDB such as high write and space amplification. It provides details on installing and using MyRocks, including data loading techniques, tuning considerations, and replication support. Parallel replication is supported, but the highest isolation level is repeatable-read and row-based replication must be used.
Flink Forward Berlin 2018: Stefan Richter - "Tuning Flink for Robustness and ...
Flink Forward Berlin 2018: Stefan Richter - "Tuning Flink for Robustness and ...Flink Forward Flink's stateful stream processing engine presents a huge variety of optional features and configuration choices to the user. Figuring out the ""optimal"" choices for any production environment and use-case can therefore often be challenging. In this talk, we will explore and discuss the universe of Flink configuration with respect to robustness and performance.
We will start with a closer look under the hood, at core data structures and algorithms, to build the foundation for understanding the impact of tuning parameters and the costs-benefit-tradeoffs that come with certain features and options. In particular, we will focus on state backend choices (Heap vs RocksDB), tuning checkpointing (incremental checkpoints, ...) and recovery (local recovery), file systems, TTL state, and considerations for the network stack. This also includes a discussion about estimating memory requirements and memory partitioning.
Wars of MySQL Cluster ( InnoDB Cluster VS Galera ) 
Wars of MySQL Cluster ( InnoDB Cluster VS Galera ) Mydbops MySQL Clustering over InnoDB engines has grown a lot over the last decade. Galera began working with InnoDB early and then Group Replication came to the environment later, where the features are now rich and robust. This presentation offers a technical comparison of both of them.
MyRocks Deep Dive
MyRocks Deep DiveYoshinori Matsunobu Detailed technical material about MyRocks -- RocksDB storage engine for MySQL -- https://github.com/facebook/mysql-5.6
Diagnosing Problems in Production - Cassandra
Diagnosing Problems in Production - CassandraJon Haddad 1) The document discusses various tools for diagnosing problems in Cassandra production environments, including OpsCenter for monitoring, application metrics collection with Statsd/Graphite, and log aggregation with Splunk or Logstash.
2) Some common issues covered are incorrect server times causing data inconsistencies, tombstone overhead slowing queries, not using the proper snitch, and disk space not being reclaimed on new nodes.
3) Diagnostic tools described are htop, iostat, vmstat, dstat, strace, tcpdump, and nodetool for investigating process activity, disk usage, memory, networking, and Cassandra-specific statistics. GC profiling and query tracing are also recommended.
Performance Wins with eBPF: Getting Started (2021)
Performance Wins with eBPF: Getting Started (2021)Brendan Gregg This document provides an overview of using eBPF (extended Berkeley Packet Filter) to quickly get performance wins as a sysadmin. It recommends installing BCC and bpftrace tools to easily find issues like periodic processes, misconfigurations, unexpected TCP sessions, or slow file system I/O. A case study examines using biosnoop to identify which processes were causing disk latency issues. The document suggests thinking like a sysadmin first by running tools, then like a programmer if a problem requires new tools. It also outlines recommended frontends depending on use cases and provides references to learn more about BPF.
Spark streaming: Best Practices
Spark streaming: Best PracticesPrakash Chockalingam Spark Streaming allows processing of live data streams using Spark. It works by receiving data streams, chopping them into batches, and processing the batches using Spark. This presentation covered Spark Streaming concepts like the lifecycle of a streaming application, best practices for aggregations, operationalization through checkpointing, and achieving high throughput. It also discussed debugging streaming jobs and the benefits of combining streaming with batch, machine learning, and SQL processing.
Kvm performance optimization for ubuntu
Kvm performance optimization for ubuntuSim Janghoon This document discusses various techniques for optimizing KVM performance on Linux systems. It covers CPU and memory optimization through techniques like vCPU pinning, NUMA affinity, transparent huge pages, KSM, and virtio_balloon. For networking, it discusses vhost-net, interrupt handling using MSI/MSI-X, and NAPI. It also covers block device optimization through I/O scheduling, cache mode, and asynchronous I/O. The goal is to provide guidance on configuring these techniques for workloads running in KVM virtual machines.
HDFS on Kubernetes—Lessons Learned with Kimoon Kim
HDFS on Kubernetes—Lessons Learned with Kimoon KimDatabricks There is growing interest in running Apache Spark natively on Kubernetes (see https://github.com/apache-spark-on-k8s/spark). Spark applications often access data in HDFS, and Spark supports HDFS locality by scheduling tasks on nodes that have the task input data on their local disks. When running Spark on Kubernetes, if the HDFS daemons run outside Kubernetes, applications will slow down while accessing the data remotely.
This session will demonstrate how to run HDFS inside Kubernetes to speed up Spark. In particular, it will show how Spark scheduler can still provide HDFS data locality on Kubernetes by discovering the mapping of Kubernetes containers to physical nodes to HDFS datanode daemons. You’ll also learn how you can provide Spark with the high availability of the critical HDFS namenode service when running HDFS in Kubernetes.
How Prometheus Store the Data
How Prometheus Store the DataHao Chen - Understanding Time Series
- What's the Fundamental Problem
- Prometheus Solution (v1.x)
- New Design of Prometheus (v2.x)
- Data Compression Algorithm
Big data skew
Big data skewayan ray The document explains the problem, cause and effect of Data skew. It also explains different techniques to minimize data skew across various big data technologies like mapreduce, hive and pig.
Achieving the ultimate performance with KVM 
Achieving the ultimate performance with KVM ShapeBlue This document summarizes an presentation about achieving ultimate performance with KVM. It discusses optimizing hardware, CPU, memory, networking, and storage for virtual machines. The goal is the lowest cost per delivered resource while meeting performance targets. Specific optimizations mentioned include CPU pinning, huge pages, SR-IOV networking, virtio drivers, and bypassing the host for storage. It cautions that many performance claims use unrealistic benchmarks and hardware configurations unlike real-world usage.
Apache Spark Architecture
Apache Spark ArchitectureAlexey Grishchenko This is the presentation I made on JavaDay Kiev 2015 regarding the architecture of Apache Spark. It covers the memory model, the shuffle implementations, data frames and some other high-level staff and can be used as an introduction to Apache Spark
Approximate nearest neighbor methods and vector models – NYC ML meetup
Approximate nearest neighbor methods and vector models – NYC ML meetupErik Bernhardsson Nearest neighbors refers to something that is conceptually very simple. For a set of points in some space (possibly many dimensions), we want to find the closest k neighbors quickly.
This presentation covers a library called Annoy built my me that that helps you do (approximate) nearest neighbor queries in high dimensional spaces. We're going through vector models, how to measure similarity, and why nearest neighbor queries are useful.
Tuning Apache Kafka Connectors for Flink.pptx
Tuning Apache Kafka Connectors for Flink.pptxFlink Forward Flink Forward San Francisco 2022.
In normal situations, the default Kafka consumer and producer configuration options work well. But we all know life is not all roses and rainbows and in this session we’ll explore a few knobs that can save the day in atypical scenarios. First, we'll take a detailed look at the parameters available when reading from Kafka. We’ll inspect the params helping us to spot quickly an application lock or crash, the ones that can significantly improve the performance and the ones to touch with gloves since they could cause more harm than benefit. Moreover we’ll explore the partitioning options and discuss when diverging from the default strategy is needed. Next, we’ll discuss the Kafka Sink. After browsing the available options we'll then dive deep into understanding how to approach use cases like sinking enormous records, managing spikes, and handling small but frequent updates.. If you want to understand how to make your application survive when the sky is dark, this session is for you!
by
Olena Babenko
Setting up Notifications, Alerts & Webhooks with Flux v2 by Alison Dowdney
Setting up Notifications, Alerts & Webhooks with Flux v2 by Alison DowdneyWeaveworks Watch the recording here: https://youtu.be/cakxixc-yQk
❗️ Notifications & Alerts ⚠️
When operating a cluster, different teams may wish to receive notifications about the status of their GitOps pipelines. For example, the on-call team would receive alerts about reconciliation failures in the cluster, while the dev team may wish to be alerted when a new version of an app was deployed and if the deployment is healthy.
Webhook Receivers
The GitOps toolkit controllers are by design pull-based. In order to notify the controllers about changes in Git or Helm repositories, you can setup webhooks and trigger a cluster reconciliation every time a source changes. Using webhook receivers, you can build push-based GitOps pipelines that react to external events.
Alison Dowdney, Developer Experience Engineer at Weaveworks and CNCF Ambassador, walks through how to define a provider, an alert, git commit status, exposing the webhook receiver and defining a git repository and receiver.
Resources
Flux2 Documentation: https://fluxcd.io/docs/
Flux Guide: Setup Notifications: https://fluxcd.io/docs/guides/notifications/
Flux Guide: Setup Webhook receivers: https://fluxcd.io/docs/guides/webhook-receivers/
Flux Roadmap: https://fluxcd.io/docs/roadmap/
Alison's Demo Repo: https://github.com/alisondy/flux-demos
Intel dpdk Tutorial
Intel dpdk TutorialSaifuddin Kaijar DPDK greatly improves packet processing performance and throughput by allowing applications to directly access hardware and bypass kernel involvement. It can improve performance by up to 10 times, allowing over 80 Mbps throughput on a single CPU or double that with two CPUs. This enables telecom and networking equipment manufacturers to develop products faster and with lower costs. DPDK achieves these gains through techniques like dedicated core affinity, userspace drivers, polling instead of interrupts, and lockless synchronization.
Analyze Virtual Machine Overhead Compared to Bare Metal with Tracing
Analyze Virtual Machine Overhead Compared to Bare Metal with TracingScyllaDB The document compares the performance of running a MySQL database benchmark (Sysbench) on virtual machines versus bare metal machines. On Fedora, the benchmark achieved 6-7% higher transactions per second, queries per second, and lower latency when run on the bare metal host compared to the virtual machine guest. Similarly, on Debian, the benchmark achieved significantly higher transactions per second (over 500 vs under 80) and lower latency when run on bare metal. Tracing tools like trace-cmd can be used to analyze the additional overhead introduced by the virtualization layer.
Deep Dive: Memory Management in Apache Spark
Deep Dive: Memory Management in Apache SparkDatabricks Memory management is at the heart of any data-intensive system. Spark, in particular, must arbitrate memory allocation between two main use cases: buffering intermediate data for processing (execution) and caching user data (storage). This talk will take a deep dive through the memory management designs adopted in Spark since its inception and discuss their performance and usability implications for the end user.
VictoriaMetrics 2023 Roadmap
VictoriaMetrics 2023 RoadmapVictoriaMetrics * Grafana datasource plugin
* OpenTelemetry ingestion protocol support
* Streaming aggregation
* vmalert: UI for rules management
* vmalert hysteresis support
* vmui explore tab
* VictoriaLogs
[Pgday.Seoul 2018] 이기종 DB에서 PostgreSQL로의 Migration을 위한 DB2PG![[Pgday.Seoul 2018] 이기종 DB에서 PostgreSQL로의 Migration을 위한 DB2PG](https://cdn.slidesharecdn.com/ss_thumbnails/04-pgdaydb2pgv1-181112042107-thumbnail.jpg?width=560&fit=bounds)
[Pgday.Seoul 2018] 이기종 DB에서 PostgreSQL로의 Migration을 위한 DB2PGPgDay.Seoul This document discusses DB2PG, a tool for migrating data between different database management systems. It began as an internal project in 2016 and has expanded its supported migration paths over time. It can now migrate schemas, tables, data types and more between Oracle, SQL Server, DB2, MySQL and other databases. The tool uses Java and supports multi-threaded imports for faster migration. Configuration files allow customizing the data type mappings and queries used during migration. The tool is open source and available on GitHub under the GPL v3 license.
PostgreSQL Portland Performance Practice Project - Database Test 2 Filesystem...
PostgreSQL Portland Performance Practice Project - Database Test 2 Filesystem...Mark Wong Fifth presentation in a speaker series sponsored by the Portland State University Computer Science Department. The series covers PostgreSQL performance with an OLTP (on-line transaction processing) workload called Database Test 2 (DBT-2). This presentation goes through results of different hardware RAID configurations to show why it is important to test your own hardware: it might be performing in way you don't expect.
PostgreSQL Portland Performance Practice Project - Database Test 2 Series Ove...
PostgreSQL Portland Performance Practice Project - Database Test 2 Series Ove...Mark Wong Introduction to the Portland State University Computer Science Department sponsored speaker series about Database Test 2 (DBT-2).
Ad
More Related Content
What's hot (20)
Flink Forward Berlin 2018: Stefan Richter - "Tuning Flink for Robustness and ...
Flink Forward Berlin 2018: Stefan Richter - "Tuning Flink for Robustness and ...Flink Forward Flink's stateful stream processing engine presents a huge variety of optional features and configuration choices to the user. Figuring out the ""optimal"" choices for any production environment and use-case can therefore often be challenging. In this talk, we will explore and discuss the universe of Flink configuration with respect to robustness and performance.
We will start with a closer look under the hood, at core data structures and algorithms, to build the foundation for understanding the impact of tuning parameters and the costs-benefit-tradeoffs that come with certain features and options. In particular, we will focus on state backend choices (Heap vs RocksDB), tuning checkpointing (incremental checkpoints, ...) and recovery (local recovery), file systems, TTL state, and considerations for the network stack. This also includes a discussion about estimating memory requirements and memory partitioning.
Wars of MySQL Cluster ( InnoDB Cluster VS Galera )
Wars of MySQL Cluster ( InnoDB Cluster VS Galera ) Mydbops MySQL Clustering over InnoDB engines has grown a lot over the last decade. Galera began working with InnoDB early and then Group Replication came to the environment later, where the features are now rich and robust. This presentation offers a technical comparison of both of them.
MyRocks Deep Dive
MyRocks Deep DiveYoshinori Matsunobu Detailed technical material about MyRocks -- RocksDB storage engine for MySQL -- https://github.com/facebook/mysql-5.6
Diagnosing Problems in Production - Cassandra
Diagnosing Problems in Production - CassandraJon Haddad 1) The document discusses various tools for diagnosing problems in Cassandra production environments, including OpsCenter for monitoring, application metrics collection with Statsd/Graphite, and log aggregation with Splunk or Logstash.
2) Some common issues covered are incorrect server times causing data inconsistencies, tombstone overhead slowing queries, not using the proper snitch, and disk space not being reclaimed on new nodes.
3) Diagnostic tools described are htop, iostat, vmstat, dstat, strace, tcpdump, and nodetool for investigating process activity, disk usage, memory, networking, and Cassandra-specific statistics. GC profiling and query tracing are also recommended.
Performance Wins with eBPF: Getting Started (2021)
Performance Wins with eBPF: Getting Started (2021)Brendan Gregg This document provides an overview of using eBPF (extended Berkeley Packet Filter) to quickly get performance wins as a sysadmin. It recommends installing BCC and bpftrace tools to easily find issues like periodic processes, misconfigurations, unexpected TCP sessions, or slow file system I/O. A case study examines using biosnoop to identify which processes were causing disk latency issues. The document suggests thinking like a sysadmin first by running tools, then like a programmer if a problem requires new tools. It also outlines recommended frontends depending on use cases and provides references to learn more about BPF.
Spark streaming: Best Practices
Spark streaming: Best PracticesPrakash Chockalingam Spark Streaming allows processing of live data streams using Spark. It works by receiving data streams, chopping them into batches, and processing the batches using Spark. This presentation covered Spark Streaming concepts like the lifecycle of a streaming application, best practices for aggregations, operationalization through checkpointing, and achieving high throughput. It also discussed debugging streaming jobs and the benefits of combining streaming with batch, machine learning, and SQL processing.
Kvm performance optimization for ubuntu
Kvm performance optimization for ubuntuSim Janghoon This document discusses various techniques for optimizing KVM performance on Linux systems. It covers CPU and memory optimization through techniques like vCPU pinning, NUMA affinity, transparent huge pages, KSM, and virtio_balloon. For networking, it discusses vhost-net, interrupt handling using MSI/MSI-X, and NAPI. It also covers block device optimization through I/O scheduling, cache mode, and asynchronous I/O. The goal is to provide guidance on configuring these techniques for workloads running in KVM virtual machines.
HDFS on Kubernetes—Lessons Learned with Kimoon Kim
HDFS on Kubernetes—Lessons Learned with Kimoon KimDatabricks There is growing interest in running Apache Spark natively on Kubernetes (see https://github.com/apache-spark-on-k8s/spark). Spark applications often access data in HDFS, and Spark supports HDFS locality by scheduling tasks on nodes that have the task input data on their local disks. When running Spark on Kubernetes, if the HDFS daemons run outside Kubernetes, applications will slow down while accessing the data remotely.
This session will demonstrate how to run HDFS inside Kubernetes to speed up Spark. In particular, it will show how Spark scheduler can still provide HDFS data locality on Kubernetes by discovering the mapping of Kubernetes containers to physical nodes to HDFS datanode daemons. You’ll also learn how you can provide Spark with the high availability of the critical HDFS namenode service when running HDFS in Kubernetes.
How Prometheus Store the Data
How Prometheus Store the DataHao Chen - Understanding Time Series
- What's the Fundamental Problem
- Prometheus Solution (v1.x)
- New Design of Prometheus (v2.x)
- Data Compression Algorithm
Big data skew
Big data skewayan ray The document explains the problem, cause and effect of Data skew. It also explains different techniques to minimize data skew across various big data technologies like mapreduce, hive and pig.
Achieving the ultimate performance with KVM
Achieving the ultimate performance with KVM ShapeBlue This document summarizes an presentation about achieving ultimate performance with KVM. It discusses optimizing hardware, CPU, memory, networking, and storage for virtual machines. The goal is the lowest cost per delivered resource while meeting performance targets. Specific optimizations mentioned include CPU pinning, huge pages, SR-IOV networking, virtio drivers, and bypassing the host for storage. It cautions that many performance claims use unrealistic benchmarks and hardware configurations unlike real-world usage.
Apache Spark Architecture
Apache Spark ArchitectureAlexey Grishchenko This is the presentation I made on JavaDay Kiev 2015 regarding the architecture of Apache Spark. It covers the memory model, the shuffle implementations, data frames and some other high-level staff and can be used as an introduction to Apache Spark
Approximate nearest neighbor methods and vector models – NYC ML meetup
Approximate nearest neighbor methods and vector models – NYC ML meetupErik Bernhardsson Nearest neighbors refers to something that is conceptually very simple. For a set of points in some space (possibly many dimensions), we want to find the closest k neighbors quickly.
This presentation covers a library called Annoy built my me that that helps you do (approximate) nearest neighbor queries in high dimensional spaces. We're going through vector models, how to measure similarity, and why nearest neighbor queries are useful.
Tuning Apache Kafka Connectors for Flink.pptx
Tuning Apache Kafka Connectors for Flink.pptxFlink Forward Flink Forward San Francisco 2022.
In normal situations, the default Kafka consumer and producer configuration options work well. But we all know life is not all roses and rainbows and in this session we’ll explore a few knobs that can save the day in atypical scenarios. First, we'll take a detailed look at the parameters available when reading from Kafka. We’ll inspect the params helping us to spot quickly an application lock or crash, the ones that can significantly improve the performance and the ones to touch with gloves since they could cause more harm than benefit. Moreover we’ll explore the partitioning options and discuss when diverging from the default strategy is needed. Next, we’ll discuss the Kafka Sink. After browsing the available options we'll then dive deep into understanding how to approach use cases like sinking enormous records, managing spikes, and handling small but frequent updates.. If you want to understand how to make your application survive when the sky is dark, this session is for you!
by
Olena Babenko
Setting up Notifications, Alerts & Webhooks with Flux v2 by Alison Dowdney
Setting up Notifications, Alerts & Webhooks with Flux v2 by Alison DowdneyWeaveworks Watch the recording here: https://youtu.be/cakxixc-yQk
❗️ Notifications & Alerts ⚠️
When operating a cluster, different teams may wish to receive notifications about the status of their GitOps pipelines. For example, the on-call team would receive alerts about reconciliation failures in the cluster, while the dev team may wish to be alerted when a new version of an app was deployed and if the deployment is healthy.
Webhook Receivers
The GitOps toolkit controllers are by design pull-based. In order to notify the controllers about changes in Git or Helm repositories, you can setup webhooks and trigger a cluster reconciliation every time a source changes. Using webhook receivers, you can build push-based GitOps pipelines that react to external events.
Alison Dowdney, Developer Experience Engineer at Weaveworks and CNCF Ambassador, walks through how to define a provider, an alert, git commit status, exposing the webhook receiver and defining a git repository and receiver.
Resources
Flux2 Documentation: https://fluxcd.io/docs/
Flux Guide: Setup Notifications: https://fluxcd.io/docs/guides/notifications/
Flux Guide: Setup Webhook receivers: https://fluxcd.io/docs/guides/webhook-receivers/
Flux Roadmap: https://fluxcd.io/docs/roadmap/
Alison's Demo Repo: https://github.com/alisondy/flux-demos
Intel dpdk Tutorial
Intel dpdk TutorialSaifuddin Kaijar DPDK greatly improves packet processing performance and throughput by allowing applications to directly access hardware and bypass kernel involvement. It can improve performance by up to 10 times, allowing over 80 Mbps throughput on a single CPU or double that with two CPUs. This enables telecom and networking equipment manufacturers to develop products faster and with lower costs. DPDK achieves these gains through techniques like dedicated core affinity, userspace drivers, polling instead of interrupts, and lockless synchronization.
Analyze Virtual Machine Overhead Compared to Bare Metal with Tracing
Analyze Virtual Machine Overhead Compared to Bare Metal with TracingScyllaDB The document compares the performance of running a MySQL database benchmark (Sysbench) on virtual machines versus bare metal machines. On Fedora, the benchmark achieved 6-7% higher transactions per second, queries per second, and lower latency when run on the bare metal host compared to the virtual machine guest. Similarly, on Debian, the benchmark achieved significantly higher transactions per second (over 500 vs under 80) and lower latency when run on bare metal. Tracing tools like trace-cmd can be used to analyze the additional overhead introduced by the virtualization layer.
Deep Dive: Memory Management in Apache Spark
Deep Dive: Memory Management in Apache SparkDatabricks Memory management is at the heart of any data-intensive system. Spark, in particular, must arbitrate memory allocation between two main use cases: buffering intermediate data for processing (execution) and caching user data (storage). This talk will take a deep dive through the memory management designs adopted in Spark since its inception and discuss their performance and usability implications for the end user.
VictoriaMetrics 2023 Roadmap
VictoriaMetrics 2023 RoadmapVictoriaMetrics * Grafana datasource plugin
* OpenTelemetry ingestion protocol support
* Streaming aggregation
* vmalert: UI for rules management
* vmalert hysteresis support
* vmui explore tab
* VictoriaLogs
[Pgday.Seoul 2018] 이기종 DB에서 PostgreSQL로의 Migration을 위한 DB2PG
[Pgday.Seoul 2018] 이기종 DB에서 PostgreSQL로의 Migration을 위한 DB2PGPgDay.Seoul This document discusses DB2PG, a tool for migrating data between different database management systems. It began as an internal project in 2016 and has expanded its supported migration paths over time. It can now migrate schemas, tables, data types and more between Oracle, SQL Server, DB2, MySQL and other databases. The tool uses Java and supports multi-threaded imports for faster migration. Configuration files allow customizing the data type mappings and queries used during migration. The tool is open source and available on GitHub under the GPL v3 license.
Similar to Filesystem Performance from a Database Perspective (20)
PostgreSQL Portland Performance Practice Project - Database Test 2 Filesystem...
PostgreSQL Portland Performance Practice Project - Database Test 2 Filesystem...Mark Wong Fifth presentation in a speaker series sponsored by the Portland State University Computer Science Department. The series covers PostgreSQL performance with an OLTP (on-line transaction processing) workload called Database Test 2 (DBT-2). This presentation goes through results of different hardware RAID configurations to show why it is important to test your own hardware: it might be performing in way you don't expect.
PostgreSQL Portland Performance Practice Project - Database Test 2 Series Ove...
PostgreSQL Portland Performance Practice Project - Database Test 2 Series Ove...Mark Wong Introduction to the Portland State University Computer Science Department sponsored speaker series about Database Test 2 (DBT-2).
Performance Whack-a-Mole Tutorial (pgCon 2009) 
Performance Whack-a-Mole Tutorial (pgCon 2009) PostgreSQL Experts, Inc. The document discusses tools for troubleshooting database performance issues. It describes operating system tools like ps, vmstat, iostat that can help identify hardware and resource bottlenecks. It also covers PostgreSQL-specific tools like the pg_stat views and logs that provide insight into query performance and activity. Benchmarks like pgbench, bonnie++, and the more complex DBT2 are presented as options for reproducing and analyzing problems in a controlled way. The overall approach presented is to start with less invasive tools and progress to more targeted benchmarks if needed to pinpoint severe issues.
Inspection and maintenance tools (Linux / OpenStack)
Inspection and maintenance tools (Linux / OpenStack)Gerard Braad This handout is part of the training at UnitedStack and will introduce you to several inspection and maintenance tools.
It is generated from the slides at: http://gbraad.gitlab.io/tools-training/
Source: https://gitlab.com/gbraad/tools-training
PostgreSQL Portland Performance Practice Project - Database Test 2 Howto
PostgreSQL Portland Performance Practice Project - Database Test 2 HowtoMark Wong Fourth presentation in a speaker series sponsored by the Portland State University Computer Science Department. The series covers PostgreSQL performance with an OLTP (on-line transaction processing) workload called Database Test 2 (DBT-2). This presentation is a set of examples to go along with the live presentation given on March 12, 2009.
Performance Whack A Mole
Performance Whack A Moleoscon2007 The document discusses establishing a performance baseline for a PostgreSQL database. It recommends gathering hardware, operating system, database, and application configuration details. The baseline involves configuring these layers with generally recommended settings, including updating hardware/OS, using appropriate filesystem and PostgreSQL configuration settings, and setting up regular maintenance tasks. Establishing a baseline configuration helps identify potential performance issues and allows comparison to other systems.
Introduction to clarity
Introduction to clarityDharshana Kasun Warusavitharana Clarity is a test automation framework that makes testing easy, organized, relevant and optimized. It allows automating scenarios across multiple WSO2 products using different browsers, databases, operating systems and application servers. Clarity uses TestNG annotations for test execution structure and listeners for operations like server startup/shutdown, report generation and artifact deployment/clean up. Test cases in Clarity utilize utility classes and optimized APIs while following an execution template.
Ceph Day Melbourne - Troubleshooting Ceph 
Ceph Day Melbourne - Troubleshooting Ceph Ceph Community This document provides troubleshooting guidance for issues with Ceph. It begins by suggesting identifying the problem domain as either performance, hang, crash, or unexpected behavior. For each problem, it recommends tools and techniques for further investigation such as debugging logs, profiling tools, and source code analysis. Debugging steps include establishing baselines, identifying implicated hosts or subsystems, increasing log verbosity, and tracing transactions through logs. The document emphasizes starting at the user end and working back towards Ceph to isolate issues.
Systems Automation with Puppet
Systems Automation with Puppetelliando dias The document discusses systems automation using Puppet. It provides an overview of Puppet including its components like types, dependencies, facts, classes, nodes, templates and modules. It describes how Puppet manages the configuration of systems through declarative specifications and enforces consistency across infrastructure. The document also gives examples of using Puppet to manage files, packages and services to maintain configurations over time.
Database Hardware Benchmarking
Database Hardware BenchmarkingCommand Prompt., Inc Greg Smith
Cover how to use simple low-level tools such as memtest86, dd, bonnie++, and sysbench to benchmark the hardware of a server intended for database use. A heavy dose of vendor management suggestions will be included as well, for the inevitable time when your shiny new server fails to deliver the performance it should.
Performance Whackamole (short version)
Performance Whackamole (short version)PostgreSQL Experts, Inc. The document discusses performance troubleshooting for databases. It provides an overview of common issues ("moles") that can impact database performance and tools/techniques for identifying and resolving them. Some key points:
- Most database performance issues are not actually problems with the database itself but other areas like hardware, OS, middleware, or application code.
- A small number (less than 10%) of issues usually account for the vast majority (90%) of performance degradation.
- The first steps in troubleshooting are establishing a baseline configuration and gathering performance metrics from across the full software stack using tools like OS monitoring utilities, database admin views, and benchmarks.
- Common types of performance issues ("moles") include
Evergreen Sysadmin Survival Skills
Evergreen Sysadmin Survival SkillsEvergreen ILS Slides that accompanied a three-hour crash training course on sysadmin survival skills useful for sysadmins of Evergreen open source library software. Session led by Don McMorris, Equinox Software.
Open Source Data Deduplication
Open Source Data DeduplicationRedWireServices Data deduplication is a hot topic in storage and saves significant disk space for many environments, with some trade offs. We’ll discuss what deduplication is and where the Open Source solutions are versus commercial offerings. Presentation will lean towards the practical – where attendees can use it in their real world projects (what works, what doesn’t, should you use in production, etcetera).
MySQL Tuning
MySQL TuningFord AntiTrust This document provides guidance on tuning MySQL for optimal performance. It discusses adjusting various configuration settings related to I/O, memory allocation, query caching, and InnoDB settings. Tuning aspects like I/O, queries, maintenance and configurations are recommended to maximize speed within the constraints of other services. Transaction logs and temporary file storage especially impact performance as heavy consumers of I/O.
Hands-on Lab: How to Unleash Your Storage Performance by Using NVM Express™ B...
Hands-on Lab: How to Unleash Your Storage Performance by Using NVM Express™ B...Odinot Stanislas (FR)
Voici un excellent document qui explique étape après étape comment installer, monitorer et surtout correctement benchmarker ses SSD PCIe/NVMe (pas si simple que ça). Autre élément clé : comment analyser la charge I/O de véritables applications? Combien d'IOPS, en read, en write, quelle bande passante et surtout quel impact sur la durée de vie des SSD? Bref à mettre en toute les mains, et un merci à mon collègue Andrey Kudryavtsev.
(EN)
An excellent content which describe step by step how to install, monitor and benchmark PCIe/NVMe SSD (many trick not so simple). Another key learning: how to measure real I/O activities on a real workload? How many R/W IOPS, block size, throughtput, and finally what's the impact on SSD endurance and (real)life? A must read, and a huge thanks to my colleague Andrey Kudryavtsev.
Auteurs/Authors:
Andrey Kudryavtsev, SSD Solution Architect, Intel Corporation
Zhdan Bybin, Application Engineer, Intel Corporation
Nagios Conference 2012 - Dan Wittenberg - Case Study: Scaling Nagios Core at ...
Nagios Conference 2012 - Dan Wittenberg - Case Study: Scaling Nagios Core at ...Nagios Dan Wittenberg's presentation on using Nagios at a Fortune 50 Company
The presentation was given during the Nagios World Conference North America held Sept 25-28th, 2012 in Saint Paul, MN. For more information on the conference (including photos and videos), visit: http://go.nagios.com/nwcna
Resume_CQ_Edward
Resume_CQ_Edwardcaiqi wang This resume summarizes the qualifications of Edward Wang. He has over 10 years of experience in performance engineering, testing, and analysis. Some of his responsibilities include performance testing, analysis and tuning of Veritas/Symantec products, virtualization solutions from VMware, and storage arrays. He is proficient with various performance benchmarking and debugging tools. He has delivered performance improvements and resolved issues for numerous projects.
pg_proctab: Accessing System Stats in PostgreSQL
pg_proctab: Accessing System Stats in PostgreSQLMark Wong pc_proctab is a collection of PostgreSQL stored functions that allow you to access the operating system process table using SQL. See examples on how to use these stored functions to collect processor and I/O statistics on SQL statements run against the database.
Measuring Firebird Disk I/O
Measuring Firebird Disk I/OMind The Firebird This document discusses two methods for measuring Firebird disk I/O: 1) Using the MON$IO_STATS tables within Firebird to track page reads, writes, fetches, and marks, and 2) Using the host operating system's performance monitoring tools like Windows Performance Monitor. It notes some limitations with MON$IO_STATS and provides examples of specific disk and process counters to log. The document also covers estimating required IOPS based on potential disk throughput and accounting for factors like RAID write penalties.
SiteGround Tech TeamBuilding
SiteGround Tech TeamBuildingMarian Marinov The document describes Linux containerization and virtualization technologies including containers, control groups (cgroups), namespaces, and backups. It discusses:
1) How cgroups isolate and limit system resources for containers through mechanisms like cpuset, cpuacct, cpu, memory, blkio, and freezer.
2) How namespaces isolate processes by ID, mounting, networking, IPC, and other resources to separate environments for containers.
3) The new backup system which uses thin provisioning and snapshotting to efficiently backup container environments to backup servers and restore individual accounts or full servers as needed.
Ad
More from Mark Wong (17)
OHAI, my name is Chelnik! PGCon 2014 Mockumentary
OHAI, my name is Chelnik! PGCon 2014 MockumentaryMark Wong A hipster crochet elephant named Chelnik from Portland, Oregon attends the PGCon 2014 conference, registers for the event, signs up for a lightning talk, discusses ideas for hacking PostgreSQL with Tom, and asks to be picked in the charity auction at the end of the conference.
OHAI, my name is Chelnik! Postgres Open 2013 Report
OHAI, my name is Chelnik! Postgres Open 2013 ReportMark Wong Chelnik, a hipster crochet elephant from Portland, Oregon, attended the Postgres Open 2013 conference where they met many people, learned about database topics like locks, normalization, and replication, and participated in presentations on Puppet, JSON, and statistics. Chelnik is now looking to be adopted from the charity auction on the last day of the conference.
collectd & PostgreSQL
collectd & PostgreSQLMark Wong This presentation is primarily focused on how to use collectd (http://collectd.org/) to gather data from the Postgres statistics tables. Examples of how to use collectd with Postgres will be shown. There is some hackery involved to make collectd do a little more and collect more meaningful data from Postgres. These small patches will be explored. A small portion of the discussion will be about how to visualize the data.
Android & PostgreSQL
Android & PostgreSQLMark Wong The document summarizes Mark Wong's presentation on using PostgreSQL with Android applications. It provides an overview and code samples for connecting to a PostgreSQL database from an Android application using the PostgreSQL JDBC driver. It also covers topics like executing queries, listening for notifications, and using prepared statements. The slides are available online and questions from the audience are welcomed.
PGTop for Android: Things I learned making this app
PGTop for Android: Things I learned making this appMark Wong This is about a weekend he spent slapping together an Android app that talks directly to Postgres using the Postgres JDBC interface. He'll focus more on the specifics of the development environment, how to use JDBC to connect to Postgres, and gotchas
encountered along the way and not so much on general Android application programming.
Introduction to PostgreSQL
Introduction to PostgreSQLMark Wong The document outlines an introduction to databases presentation using PostgreSQL. It includes an introduction to databases concepts, an overview of PostgreSQL, demonstrations of SQL commands like CREATE TABLE, INSERT, SELECT and JOIN in psql, and discussions of database administration and GUI tools. Exercises are provided for attendees to practice the concepts covered.
Developing PGTop for Android
Developing PGTop for AndroidMark Wong Mark Wong presented on developing PGTop for Android, a PostgreSQL monitoring application for Android devices. He discussed the development environment including Java, the Android SDK, and optionally Eclipse. He also demonstrated connecting to a PostgreSQL database using the JDBC driver, executing queries, and provided examples of other PostgreSQL and JDBC functions that could be used.
Pg in-the-brazilian-armed-forces-presentation
Pg in-the-brazilian-armed-forces-presentationMark Wong The document discusses the use of PostgreSQL and other open source software by the Brazilian Armed Forces. It describes how the Brazilian Army uses PostgreSQL in its Opus Project, a system for managing over 1,800 military properties. It also discusses how the Brazilian Air Force uses PostgreSQL for various logistics projects like Sisub, a food supply management system used across 54 database servers in 19 states. The document highlights the cost savings, reliability, and other benefits the Armed Forces have experienced from adopting PostgreSQL and open source software.
pg_proctab: Accessing System Stats in PostgreSQL
pg_proctab: Accessing System Stats in PostgreSQLMark Wong pg_proctab is a collection of PostgreSQL stored functions that provide access to the operating system process table using SQL. We'll show you which functions are available and where they collect the data, and give examples of their use to collect processor and I/O statistics on SQL queries.
pg_proctab: Accessing System Stats in PostgreSQL
pg_proctab: Accessing System Stats in PostgreSQLMark Wong pg_proctab is a collection of PostgreSQL stored functions that provide access to the operating system process table using SQL. We'll show you which functions are available and where they collect the data, and give examples of their use to collect processor and I/O statistics on SQL queries. These stored functions currently only work on Linux-based systems.
PostgreSQL Portland Performance Practice Project - Database Test 2 Tuning
PostgreSQL Portland Performance Practice Project - Database Test 2 TuningMark Wong This document summarizes the results of tuning a PostgreSQL database using the tool pgtune. It shows the performance improvements achieved by configuring various PostgreSQL configuration options like shared_buffers and work_mem as recommended by pgtune. It also discusses the effects of changing the effective_cache_size parameter and physical disk layout optimizations on an older system with multiple disk arrays. Overall, pgtune helped improve performance by 7% and separating data files from logs saw further gains.
PostgreSQL Portland Performance Practice Project - Database Test 2 Workload D...
PostgreSQL Portland Performance Practice Project - Database Test 2 Workload D...Mark Wong Third presentation in a speaker series sponsored by the Portland State University Computer Science Department. The series covers PostgreSQL performance with an OLTP (on-line transaction processing) workload called Database Test 2 (DBT-2). This presentation go into detail about what the workload does.
PostgreSQL Portland Performance Practice Project - Database Test 2 Background
PostgreSQL Portland Performance Practice Project - Database Test 2 BackgroundMark Wong Portland State University Computer Science Department sponsored speaker series. Background information for Database Test 2 (DBT-2).
pg_top is 'top' for PostgreSQL: pg_top + pg_proctab
pg_top is 'top' for PostgreSQL: pg_top + pg_proctabMark Wong Introductory presentation including pg_top and pg_proctab for PostgreSQL.
Linux Filesystems, RAID, and more
Linux Filesystems, RAID, and moreMark Wong Simple filesystem test results on linux using software and hardware RAID and LVM. Some operating systems parameters also examined.
pg_top is 'top' for PostgreSQL
pg_top is 'top' for PostgreSQLMark Wong pg_top allows you to monitor PostgreSQL processes to view the currently running SQL statement of a process, the query plan of a currently running SELECT statement, locks held by a process, user table statistics, and user index statistics.
What Is Going On?
What Is Going On?Mark Wong A introduction to using system tools to identify what the system is doing. This will go over using top and iostat to determine what the system is physically doing, and then using tools like ps and querying the PostgreSQL system catalog tables to determine what queries are running, what locks have been acquired, and where the tables and indexes are physically located on the system in order to correlate the physical activity to what the database is doing.
Ad
Recently uploaded (20)
Noah Loul Shares 5 Steps to Implement AI Agents for Maximum Business Efficien...
Noah Loul Shares 5 Steps to Implement AI Agents for Maximum Business Efficien...Noah Loul Artificial intelligence is changing how businesses operate. Companies are using AI agents to automate tasks, reduce time spent on repetitive work, and focus more on high-value activities. Noah Loul, an AI strategist and entrepreneur, has helped dozens of companies streamline their operations using smart automation. He believes AI agents aren't just tools—they're workers that take on repeatable tasks so your human team can focus on what matters. If you want to reduce time waste and increase output, AI agents are the next move.
How analogue intelligence complements AI
How analogue intelligence complements AIPaul Rowe
Artificial Intelligence is providing benefits in many areas of work within the heritage sector, from image analysis, to ideas generation, and new research tools. However, it is more critical than ever for people, with analogue intelligence, to ensure the integrity and ethical use of AI. Including real people can improve the use of AI by identifying potential biases, cross-checking results, refining workflows, and providing contextual relevance to AI-driven results.
News about the impact of AI often paints a rosy picture. In practice, there are many potential pitfalls. This presentation discusses these issues and looks at the role of analogue intelligence and analogue interfaces in providing the best results to our audiences. How do we deal with factually incorrect results? How do we get content generated that better reflects the diversity of our communities? What roles are there for physical, in-person experiences in the digital world?
Into The Box Conference Keynote Day 1 (ITB2025)
Into The Box Conference Keynote Day 1 (ITB2025)Ortus Solutions, Corp This is the keynote of the Into the Box conference, highlighting the release of the BoxLang JVM language, its key enhancements, and its vision for the future.
Social Media App Development Company-EmizenTech
Social Media App Development Company-EmizenTechSteve Jonas EmizenTech is a trusted Social Media App Development Company with 11+ years of experience in building engaging and feature-rich social platforms. Our team of skilled developers delivers custom social media apps tailored to your business goals and user expectations. We integrate real-time chat, video sharing, content feeds, notifications, and robust security features to ensure seamless user experiences. Whether you're creating a new platform or enhancing an existing one, we offer scalable solutions that support high performance and future growth. EmizenTech empowers businesses to connect users globally, boost engagement, and stay competitive in the digital social landscape.
Increasing Retail Store Efficiency How can Planograms Save Time and Money.pptx
Increasing Retail Store Efficiency How can Planograms Save Time and Money.pptxAnoop Ashok In today's fast-paced retail environment, efficiency is key. Every minute counts, and every penny matters. One tool that can significantly boost your store's efficiency is a well-executed planogram. These visual merchandising blueprints not only enhance store layouts but also save time and money in the process.
Cyber Awareness overview for 2025 month of security
Cyber Awareness overview for 2025 month of securityriccardosl1 Cyber awareness training educates employees on risk associated with internet and malicious emails
TrustArc Webinar: Consumer Expectations vs Corporate Realities on Data Broker...
TrustArc Webinar: Consumer Expectations vs Corporate Realities on Data Broker...TrustArc Most consumers believe they’re making informed decisions about their personal data—adjusting privacy settings, blocking trackers, and opting out where they can. However, our new research reveals that while awareness is high, taking meaningful action is still lacking. On the corporate side, many organizations report strong policies for managing third-party data and consumer consent yet fall short when it comes to consistency, accountability and transparency.
This session will explore the research findings from TrustArc’s Privacy Pulse Survey, examining consumer attitudes toward personal data collection and practical suggestions for corporate practices around purchasing third-party data.
Attendees will learn:
- Consumer awareness around data brokers and what consumers are doing to limit data collection
- How businesses assess third-party vendors and their consent management operations
- Where business preparedness needs improvement
- What these trends mean for the future of privacy governance and public trust
This discussion is essential for privacy, risk, and compliance professionals who want to ground their strategies in current data and prepare for what’s next in the privacy landscape.
Top 10 IT Help Desk Outsourcing Services
Top 10 IT Help Desk Outsourcing ServicesInfrassist Technologies Pvt. Ltd. IT help desk outsourcing Services can assist with that by offering availability for customers and address their IT issue promptly without breaking the bank.
Rusty Waters: Elevating Lakehouses Beyond Spark
Rusty Waters: Elevating Lakehouses Beyond Sparkcarlyakerly1 Spark is a powerhouse for large datasets, but when it comes to smaller data workloads, its overhead can sometimes slow things down. What if you could achieve high performance and efficiency without the need for Spark?
At S&P Global Commodity Insights, having a complete view of global energy and commodities markets enables customers to make data-driven decisions with confidence and create long-term, sustainable value. 🌍
Explore delta-rs + CDC and how these open-source innovations power lightweight, high-performance data applications beyond Spark! 🚀
Cybersecurity Identity and Access Solutions using Azure AD
Cybersecurity Identity and Access Solutions using Azure ADVICTOR MAESTRE RAMIREZ Cybersecurity Identity and Access Solutions using Azure AD
AI Changes Everything – Talk at Cardiff Metropolitan University, 29th April 2...
AI Changes Everything – Talk at Cardiff Metropolitan University, 29th April 2...Alan Dix Talk at the final event of Data Fusion Dynamics: A Collaborative UK-Saudi Initiative in Cybersecurity and Artificial Intelligence funded by the British Council UK-Saudi Challenge Fund 2024, Cardiff Metropolitan University, 29th April 2025
https://alandix.com/academic/talks/CMet2025-AI-Changes-Everything/
Is AI just another technology, or does it fundamentally change the way we live and think?
Every technology has a direct impact with micro-ethical consequences, some good, some bad. However more profound are the ways in which some technologies reshape the very fabric of society with macro-ethical impacts. The invention of the stirrup revolutionised mounted combat, but as a side effect gave rise to the feudal system, which still shapes politics today. The internal combustion engine offers personal freedom and creates pollution, but has also transformed the nature of urban planning and international trade. When we look at AI the micro-ethical issues, such as bias, are most obvious, but the macro-ethical challenges may be greater.
At a micro-ethical level AI has the potential to deepen social, ethnic and gender bias, issues I have warned about since the early 1990s! It is also being used increasingly on the battlefield. However, it also offers amazing opportunities in health and educations, as the recent Nobel prizes for the developers of AlphaFold illustrate. More radically, the need to encode ethics acts as a mirror to surface essential ethical problems and conflicts.
At the macro-ethical level, by the early 2000s digital technology had already begun to undermine sovereignty (e.g. gambling), market economics (through network effects and emergent monopolies), and the very meaning of money. Modern AI is the child of big data, big computation and ultimately big business, intensifying the inherent tendency of digital technology to concentrate power. AI is already unravelling the fundamentals of the social, political and economic world around us, but this is a world that needs radical reimagining to overcome the global environmental and human challenges that confront us. Our challenge is whether to let the threads fall as they may, or to use them to weave a better future.
Web and Graphics Designing Training in Rajpura
Web and Graphics Designing Training in RajpuraErginous Technology Web & Graphics Designing Training at Erginous Technologies in Rajpura offers practical, hands-on learning for students, graduates, and professionals aiming for a creative career. The 6-week and 6-month industrial training programs blend creativity with technical skills to prepare you for real-world opportunities in design.
The course covers Graphic Designing tools like Photoshop, Illustrator, and CorelDRAW, along with logo, banner, and branding design. In Web Designing, you’ll learn HTML5, CSS3, JavaScript basics, responsive design, Bootstrap, Figma, and Adobe XD.
Erginous emphasizes 100% practical training, live projects, portfolio building, expert guidance, certification, and placement support. Graduates can explore roles like Web Designer, Graphic Designer, UI/UX Designer, or Freelancer.
For more info, visit erginous.co.in , message us on Instagram at erginoustechnologies, or call directly at +91-89684-38190 . Start your journey toward a creative and successful design career today!
Role of Data Annotation Services in AI-Powered Manufacturing
Role of Data Annotation Services in AI-Powered ManufacturingAndrew Leo From predictive maintenance to robotic automation, AI is driving the future of manufacturing. But without high-quality annotated data, even the smartest models fall short.
Discover how data annotation services are powering accuracy, safety, and efficiency in AI-driven manufacturing systems.
Precision in data labeling = Precision on the production floor.
Quantum Computing Quick Research Guide by Arthur Morgan
Quantum Computing Quick Research Guide by Arthur MorganArthur Morgan This is a Quick Research Guide (QRG).
QRGs include the following:
- A brief, high-level overview of the QRG topic.
- A milestone timeline for the QRG topic.
- Links to various free online resource materials to provide a deeper dive into the QRG topic.
- Conclusion and a recommendation for at least two books available in the SJPL system on the QRG topic.
QRGs planned for the series:
- Artificial Intelligence QRG
- Quantum Computing QRG
- Big Data Analytics QRG
- Spacecraft Guidance, Navigation & Control QRG (coming 2026)
- UK Home Computing & The Birth of ARM QRG (coming 2027)
Any questions or comments?
- Please contact Arthur Morgan at [email protected].
100% human made.
Unlocking the Power of IVR: A Comprehensive Guide
Unlocking the Power of IVR: A Comprehensive Guidevikasascentbpo Streamline customer service and reduce costs with an IVR solution. Learn how interactive voice response systems automate call handling, improve efficiency, and enhance customer experience.
TrsLabs Consultants - DeFi, WEb3, Token Listing
TrsLabs Consultants - DeFi, WEb3, Token ListingTrs Labs Enter the world of web3 with experienced strategic partner.
Open your business to new opportunities.
Mastering Advance Window Functions in SQL.pdf
Mastering Advance Window Functions in SQL.pdfSpiral Mantra How well do you really know SQL?📊
.
.
If PARTITION BY and ROW_NUMBER() sound familiar but still confuse you, it’s time to upgrade your knowledge
And you can schedule a 1:1 call with our industry experts: https://spiralmantra.com/contact-us/ or drop us a mail at [email protected]
Generative Artificial Intelligence (GenAI) in Business
Generative Artificial Intelligence (GenAI) in BusinessDr. Tathagat Varma My talk for the Indian School of Business (ISB) Emerging Leaders Program Cohort 9. In this talk, I discussed key issues around adoption of GenAI in business - benefits, opportunities and limitations. I also discussed how my research on Theory of Cognitive Chasms helps address some of these issues
Build Your Own Copilot & Agents For Devs
Build Your Own Copilot & Agents For DevsBrian McKeiver May 2nd, 2025 talk at StirTrek 2025 Conference.