FastNetMon and Metrics

Feb 25, 20220 likes319 views

Pavel Odintsov introduces FastNetMon DDoS prevention and how they migrated analytics to ClickHouse to handle large data volumes. Pavel is CTO and co-founder of FastNetMon LTD. FastNetMon: https://fastnetmon.com/ Meetup: https://www.meetup.com/San-Francisco-Bay-Area-ClickHouse-Meetup/events/282872933/

Hello
2
I’m Pavel Odintsov, the author of FastNetMon: https://fastnetmon.com
Ways to contact me:
● linkedin.com/in/podintsov
● github.com/pavel-odintsov
● twitter.com/odintsov_pavel
● IRC, Libera Chat, pavel_odintsov
● pavel@fastnetmon.com

What is FastNetMon
3
● DDoS detection Tool
● Amazon AWS VPC Flow Analytics
● Google Compute VPC Flow Analytics
● BGP Blackhole Automation Tool
● BGP Automation Tool
● Traﬃc Visibility Tool
● Network Automation Tool
● Network Traﬃc Engineering Tool

FastNetMon: Detection Logic
6
Detection type:
• Threshold based (based on host’s average traﬃc)
THRESHOLD TYPES:
• USING TOTAL TRAFFIC
• USING TOTAL PPS RATE
• PER PROTOCOL

FastNetMon: More Tra ic Reports in Grafana
8

Our Challenges with Metrics
10
● Cardinality: medium sized network has ~1m IPv4 hosts with 32
metrics each
● Value range from 0 to UINT64_MAX: traﬃc in mbit / s or packet /s
● 1s precision
● Insert in very large batches
● Customers love top-k queries

Graphite As Storage for Metrics, 2015
11
● “.” as delimiter. IPs and preﬁxes look ugly: 10_1_2_3 , 10_1_2_3_24
● Limited by performance of single CPU core
● Disk space hungry datastore format
● Whisper is simple and easy to implement
● Graphite is not well maintained and broken in recent Debian /
Ubuntu

InﬂuxDB As Storage for Metrics, 2016
15
● Allows “.” in metric names
● Pretty compact datastore format
● Automated retention
● Native Grafana support
● Can use multiple CPU cores
● Easy installation, just single binary
● May need tens of minutes for loading with large database
● Uses lots of memory
● Top-k query is extremely slow
● Does not scale after 2m metrics per second
● Queries over few days of data are very slow

ClickHouse As Storage for Metrics, 2022
16
● Allows “.” in metric names
● Pretty compact datastore format
● Automated retention
● Plugin for Grafana
● Can use multiple CPU cores
● Requires SSE 4.2 :(
● Top-k query is pretty fast
● Supports unlimited cardinality
● Queries over few days of data can be ﬁnished in reasonable time
● Ability to store ﬂows!

Clickhouse vs InﬂuxDB
17
InfluxDB ClickHouse
Cardinality < 1m of unique series Battle tested with 16m+ unique series
Metrics / s < 1m per second 10m+ per second
Top-k performance Extremely slow Good
Data format Inefficient, text Very Efficient, binary
Query syntax Counterintuitive Well known SQL
Multi CPU support Limited Brilliant, scales linearly
Grafana Native Plugin based

FastNetMon: our community
23
● Site: https://fastnetmon.com
● GitHub: https://github.com/pavel-odintsov/fastnetmon
● IRC: #fastnetmon at Libra Chat
● Telegram: https://t.me/fastnetmon
● Slack: http://bit.ly/2o5Idx8
● LinkedIN: https://www.linkedin.com/company/fastnetmon/
● Facebook: https://www.facebook.com/fastnetmon/
● WhatsApp:
https://chat.whatsapp.com/JjwF855pwZvIIasTUsZ7EO

THANKS!
ANY QUESTIONS?
You can ﬁnd me at:
⬥ @odintsov_pavel
⬥ pavel@fastnetmon.com
⬥ linkedin.com/in/podintsov
24

This document summarizes a presentation about designing systems to handle high loads when Chuck Norris is your customer. It discusses scaling architectures vertically and horizontally, RESTful principles, using NoSQL databases like MongoDB, caching with Memcached, search engines like Sphinx, video/image storage, and bandwidth management. It emphasizes that the right technology depends on business needs, and high-load systems require robust architectures, qualified developers, and avoiding single points of failure.

Zero Downtime JEE ArchitecturesAlexander Penev

This document discusses zero downtime architectures. It defines zero downtime as services being available to end users at all times. It identifies sources of planned and unplanned downtime. It proposes concepts like independent application groups, redundant infrastructure within and between datacenters, and replicating data between datacenters to reduce downtime. It provides examples of implementing high availability for networks, applications, and databases. It also discusses development guidelines and monitoring to support zero downtime operations.

Netflix Open Source Meetup Season 4 Episode 2aspyker

In this episode, we will take a close look at 2 different approaches to high-throughput/low-latency data stores, developed by Netflix. The first, EVCache, is a battle-tested distributed memcached-backed data store, optimized for the cloud. You will also hear about the road ahead for EVCache it evolves into an L1/L2 cache over RAM and SSDs. The second, Dynomite, is a framework to make any non-distributed data-store, distributed. Netflix's first implementation of Dynomite is based on Redis. Come learn about the products' features and hear from Thomson and Reuters, Diego Pacheco from Ilegra and other third party speakers, internal and external to Netflix, on how these products fit in their stack and roadmap.

Web performance mercadolibre - ECI 2013Santiago Aimetta

The document discusses techniques for improving web performance, including reducing time to first byte, using content delivery networks and HTTP compression, caching resources, keeping connections alive and reducing request sizes. It also covers optimizing images, loading JavaScript asynchronously to avoid blocking, and prefetching content. The overall goal is to reduce page load times and improve user experience.

Dynomite @ RedisConf 2017Ioannis Papapanagiotou

- Dynomite is a framework that makes non-distributed databases distributed by adding a proxy layer, auto-sharding, replication across datacenters, and more. - Netflix uses Dynomite to provide high availability and scalability for several internal microservices and tools like Conductor, an orchestration engine that uses Redis. - Conductor allows defining workflows as code and executing them in a distributed and scalable way using Dynomite and Dyno Queues.

What is Nginx and Why You Should to Use it with Wordpress HostingWPSFO Meetup Group

SPDY and What to Consider for HTTP/2.0Mike Belshe

NetflixOSS Meetup season 3 episode 1Ruslan Meshenberg

This summary provides an overview of the key points from the document in 3 sentences: The document outlines the agenda for Season 3 Episode 1 of the Netflix OSS podcast, which includes lightning talks on 8 new projects including Atlas, Prana, Raigad, Genie 2, Inviso, Dynomite, Nicobar, and MSL. Representatives from Netflix, IBM Watson, Nike Digital, and Pivotal then each provide a 3-5 minute presentation on their featured project. The presentations describe the motivation, features and benefits of each project for observability, integration with the Netflix ecosystem, automation of Elasticsearch deployments, job scheduling, dynamic scripting for Java, message security, and developing microservices

USENIX LISA15: How TubeMogul Handles over One Trillion HTTP Requests a MonthNicolas Brousse

TubeMogul grew from few servers to over two thousands servers and handling over one trillion http requests a month, processed in less than 50ms each. To keep up with the fast growth, the SRE team had to implement an efficient Continuous Delivery infrastructure that allowed to do over 10,000 puppet deployment and 8,500 application deployment in 2014. In this presentation, we will cover the nuts and bolts of the TubeMogul operations engineering team and how they overcome challenges.

Web performance optimization - MercadoLibrePablo Moretti

The document provides techniques and tools for improving web performance. It discusses how reducing response times can directly impact revenues and user experience. It then covers various ways to optimize the frontend, including reducing time to first byte through DNS optimization and caching, using content delivery networks, HTTP compression, keeping connections alive, parallel downloads, and prefetching. It also discusses optimizing images, JavaScript loading, and introducing new formats like WebP. The overall document aims to educate on measuring and enhancing web performance.

Netflix Data Pipeline With KafkaAllen (Xiaozhong) Wang

This document summarizes Netflix's use of Kafka in their data pipeline. It discusses how Netflix evolved from using S3 and EMR to introducing Kafka and Kafka producers and consumers to handle 400 billion events per day. It covers challenges of scaling Kafka clusters and tuning Kafka clients and brokers. Finally, it outlines Netflix's roadmap which includes contributing to open source projects like Kafka and testing failure resilience.

Netflix Data Pipeline With KafkaSteven Wu

This document summarizes Netflix's use of Kafka in their data pipeline. It discusses the evolution of Netflix's data pipeline to incorporate Kafka to handle 400 billion events per day. It describes how Netflix uses Kafka clusters with different priorities and configurations. It also outlines some of the challenges of using Kafka at Netflix's scale, such as Zookeeper client issues and cluster scaling, and the solutions Netflix developed to address these challenges.

Big data at scrapinghubDana Brophy

Design patterns for scaling web applicationsIvan Dimitrov

This document discusses various patterns for scaling web applications. It begins by describing single machine, two-tier, and multi-tier web application architectures. It then covers message bus and service-oriented architectures. The document discusses scaling up approaches like identifying bottlenecks, utilizing bottlenecks, adjusting non-bottlenecks, elevating bottlenecks, and reviewing processes. Specific scaling techniques covered include caching, threading and queues, and using content delivery networks.

Netty trainingJackson dos Santos Olveira

Netty trainingMarcelo Serpa

This document provides an overview of Netty, an asynchronous event-driven network application framework for Java. It discusses key Netty concepts like the event loop, channels, pipelines and handlers. It also provides examples of building an echo server and client with Netty. Additionally, it mentions several large companies that use Netty in production and open-source projects built with Netty. Finally, it discusses some alternative solutions to Netty like ServiceTalk, Armeria and Reactor Netty that provide additional features on top of the Netty core.

Node.js scaling in highloadTimur Shemsedinov

This document outlines different steps in scaling Node.js applications from 2012 to 2019. It begins with using Node.js in cluster mode with Nginx as a reverse proxy. It progresses to using CDNs for static files, in-memory databases, and eventually custom protocols for real-time data synchronization across servers and clients. Key aspects discussed include data synchronization, offline capabilities, interactivity, scalability, and high connectivity. Alternative approaches and bad practices are also addressed.

Logs @ OVHcloudOVHcloud

Logging at OVHcloud : Logs Data platform est la plateforme de collecte, d'analyse et de gestion centralisée de logs d'OVHcloud. Cette plateforme a pour but de répondre aux challenges que constitue l'indexation de plus de 4000 milliards de logs par une entreprise comme OVHcloud. Cette présentation vous décrira l'architecture générale de Logs Data Platform autour de ses composants centraux Elasticsearch et Graylog et vous décrira les différentes problématiques de scalabilité, disponibilité, performance et d'évolutivité qui sont le quotidien de l'équipe Observability à OVHcloud.

RedisConf17 - Dynomite - Making Non-distributed Databases DistributedRedis Labs

Dynomite is a framework that makes non-distributed databases distributed by adding a proxy layer, auto-sharding, replication across datacenters, and more. It is used at Netflix to power various services by sitting on top of Redis and providing high availability, scalability, and tunable consistency. Conductor is a workflow orchestration engine used by Netflix that stores workflow definitions and state in Dynomite to allow reusable and controllable workflow processes.

Protecting the Web at a scale using consul and Elk / Valentin Chernozemski (S...Ontico

HighLoad++ 2017 Зал «Мумбай», 7 ноября, 16:00 Тезисы: http://www.highload.ru/2017/abstracts/3065.html Brute-force attacks against web based applications are on the raise. You will be presented with an architecture built on top of ELK (https://www.elastic.co/products) and consul (https://www.consul.io/) that is capable of reliably detecting, analysing and mitigating large scale brute-force attacks against Wordpress, Drupal, Magento and Joomla based web sites in near real time. ...

Archmage, Pinterest’s Real-time Analytics Platform on DruidImply

In this talk, we will talk about: 1) the motivation of switching from Hbase backed analytics system to Druid 2) the architecture design of Druid as a platform in Pinterest (Archmage, Hadoop, Kafka) including a query interface, Archmage, a thrift service in front of Druid which exposes a thrift api to company-wise clients, handles Druid broker hosts discovery, serves as a relay to broker hosts to abstract the async HTTP connection and provides query optimizations transparent to clients including directly translating fixed pattern SQL to Druid native JSON queries to save planning time. In addition, we’ll cover the production Hadoop batch and Kafka real time ingestion pipeline setup and the reason we picked a pull-based solution instead of a push-based solution for real time ingestion. 3) We will also talk about the use cases currently running in production on this platform including their data volume, QPS, Druid cluster setup, the unique challenges we met while onboarding and how we addressed them with extensive tunings to meet SLA and lessons learned for use cases including: partner insights, which provides partners with stats on organic pins; realtime spam detection, which detects user login related anomaly events and pin related spamming events like pin creation and repin; and migrating the backend from Presto to Druid for Ads related experiments data analysis.

Cloud arch patternsCorey Huinker

This document discusses PostgreSQL database architecture patterns for running PostgreSQL at scale when a relational database as a service like Amazon RDS won't meet needs. It describes challenges faced with MySQL, Redshift and Vertica and how PostgreSQL was better suited through techniques like partitioning by date, TOAST compression, foreign data wrappers, and poor man's parallel processing. Key takeaways are that PostgreSQL supported scaling to petabytes of data, sub-second queries across large date ranges, and custom extensions needed while avoiding limitations and expenses of other database options.

Screaming Fast Wpmudjcp

This document discusses optimizing a WordPress installation for performance. It recommends using Nginx as a reverse proxy cache in front of Apache and PHP to cache both logged-in and logged-out content. It achieved near elimination of problems, doubled network throughput, halved memory usage, and allowed disabling of plugins by caching everything. Results included atom feeds increasing in speed from 6 requests/second to over 7000 requests/second.

Instruments to play microserviceChandresh Pancholi

This document discusses microservices architecture and the various tools needed to build microservices. It begins by explaining the pros and cons of monolith applications versus microservices. It then covers requirements for building microservices like security, asynchronous communication, monitoring, logging, tracing, service discovery, configuration management, and circuit breakers. Specific open source tools are mentioned for each requirement area, including RabbitMQ, Kafka, Prometheus, Zipkin, Elasticsearch, Eureka, Spring Cloud Config, and Kubernetes. The document concludes by asking if the audience has any additional questions.

Introduction to Apache KafkaRicardo Bravo

This document provides an overview of Apache Kafka including its main components, architecture, and ecosystem. It describes how LinkedIn used Kafka to solve their data pipeline problem by decoupling systems and allowing for horizontal scaling. The key elements of Kafka are producers that publish data to topics, the Kafka cluster that stores streams of records in a distributed, replicated commit log, and consumers that subscribe to topics. Kafka Connect and the Schema Registry are also introduced as part of the Kafka ecosystem.

AWS big-data-demystified #1.1 | Big Data Architecture Lessons Learned | EnglishOmid Vahdaty

Big Data in 200 km/h | AWS Big Data Demystified #1.3 Omid Vahdaty

What we're about A while ago I entered the challenging world of Big Data. As an engineer, at first, I was not so impressed with this field. As time went by, I realised more and more, The technological challenges in this area are too great to master by one person. Just look at the picture in this articles, it only covers a small fraction of the technologies in the Big Data industry… Consequently, I created a meetup detailing all the challenges of Big Data, especially in the world of cloud. I am using AWS infrastructure to answer the basic questions of anyone starting their way in the big data world. how to transform data (TXT, CSV, TSV, JSON) into Parquet, ORCwhich technology should we use to model the data ? EMR? Athena? Redshift? Spectrum? Glue? Spark? SparkSQL?how to handle streaming?how to manage costs?Performance tips?Security tip?Cloud best practices tips? Some of our online materials: Website: https://big-data-demystified.ninja/ Youtube channels: https://www.youtube.com/channel/UCzeGqhZIWU-hIDczWa8GtgQ?view_as=subscriber https://www.youtube.com/channel/UCMSdNB0fGmX5dXI7S7Y_LFA?view_as=subscriber Meetup: https://www.meetup.com/AWS-Big-Data-Demystified/ https://www.meetup.com/Big-Data-Demystified Facebook Group : https://www.facebook.com/groups/amazon.aws.big.data.demystified/ Facebook page (https://www.facebook.com/Amazon-AWS-Big-Data-Demystified-1832900280345700/) Audience: Data Engineers Data Science DevOps Engineers Big Data Architects Solution Architects CTO VP R&D

AWS Big Data Demystified #1.2 | Big Data architecture lessons learned Omid Vahdaty

A while ago I entered the challenging world of Big Data. As an engineer, at first, I was not so impressed with this field. As time went by, I realised more and more, The technological challenges in this area are too great to master by one person. Just look at the picture in this articles, it only covers a small fraction of the technologies in the Big Data industry… Consequently, I created a meetup detailing all the challenges of Big Data, especially in the world of cloud. I am using AWS & GCP and Data Center infrastructure to answer the basic questions of anyone starting their way in the big data world. how to transform data (TXT, CSV, TSV, JSON) into Parquet, ORC,AVRO which technology should we use to model the data ? EMR? Athena? Redshift? Spectrum? Glue? Spark? SparkSQL? GCS? Big Query? Data flow? Data Lab? tensor flow? how to handle streaming? how to manage costs? Performance tips? Security tip? Cloud best practices tips? In this meetup we shall present lecturers working on several cloud vendors, various big data platforms such hadoop, Data warehourses , startups working on big data products. basically - if it is related to big data - this is THE meetup. Some of our online materials (mixed content from several cloud vendor): Website: https://big-data-demystified.ninja (under construction) Meetups: https://www.meetup.com/Big-Data-Demystified https://www.meetup.com/AWS-Big-Data-Demystified/ You tube channels: https://www.youtube.com/channel/UCMSdNB0fGmX5dXI7S7Y_LFA?view_as=subscriber https://www.youtube.com/channel/UCzeGqhZIWU-hIDczWa8GtgQ?view_as=subscriber Audience: Data Engineers Data Science DevOps Engineers Big Data Architects Solution Architects CTO VP R&D

Building an Analytic Extension to MySQL with ClickHouse and Open Source.pptxAltinity Ltd

Building an Analytic Extension to MySQL with ClickHouse and Open Source In this webinar Percona and Altinity offer suggestions and tips on how to recognize when MySQL is overburdened with analytics and can benefit from ClickHouse’s unique capabilities. Also, they will walk you through important patterns for integrating MySQL and ClickHouse which will enable the building of powerful and cost-efficient applications that leverage the strengths of both databases.

Cloud Native ClickHouse at Scale--Using the Altinity Kubernetes Operator-2022...Altinity Ltd

Over the last few years Kubernetes has transitioned from an object of curiosity and fear to a robust platform for big data. Watch this webinar and you will learn how the Altinity Kubernetes Operator for ClickHouse enables users to run high performance analytics on ClickHouse. You will see a simple installation and teach you how to scale it into a cluster that can analyze 100s of terabytes of data. Along the way we’ll share our lessons for ClickHouse on Kubernetes in Altinity.Cloud. We built it on Kubernetes using the Altinity Operator and now run hundreds of clusters in the cloud. You can too!

More Related Content

Similar to FastNetMon and Metrics (20)

USENIX LISA15: How TubeMogul Handles over One Trillion HTTP Requests a MonthNicolas Brousse

Web performance optimization - MercadoLibrePablo Moretti

Netflix Data Pipeline With KafkaAllen (Xiaozhong) Wang

Netflix Data Pipeline With KafkaSteven Wu

Big data at scrapinghubDana Brophy

Design patterns for scaling web applicationsIvan Dimitrov

Netty trainingJackson dos Santos Olveira

Netty trainingMarcelo Serpa

Node.js scaling in highloadTimur Shemsedinov

Logs @ OVHcloudOVHcloud

RedisConf17 - Dynomite - Making Non-distributed Databases DistributedRedis Labs

Protecting the Web at a scale using consul and Elk / Valentin Chernozemski (S...Ontico

Archmage, Pinterest’s Real-time Analytics Platform on DruidImply

Cloud arch patternsCorey Huinker

Screaming Fast Wpmudjcp

Instruments to play microserviceChandresh Pancholi

Introduction to Apache KafkaRicardo Bravo

AWS big-data-demystified #1.1 | Big Data Architecture Lessons Learned | EnglishOmid Vahdaty

Big Data in 200 km/h | AWS Big Data Demystified #1.3 Omid Vahdaty

AWS Big Data Demystified #1.2 | Big Data architecture lessons learned Omid Vahdaty

USENIX LISA15: How TubeMogul Handles over One Trillion HTTP Requests a MonthNicolas Brousse

Web performance optimization - MercadoLibrePablo Moretti

Netflix Data Pipeline With KafkaAllen (Xiaozhong) Wang

Netflix Data Pipeline With KafkaSteven Wu

Big data at scrapinghubDana Brophy

Design patterns for scaling web applicationsIvan Dimitrov

Netty trainingJackson dos Santos Olveira

Netty trainingMarcelo Serpa

Node.js scaling in highloadTimur Shemsedinov

Logs @ OVHcloudOVHcloud

RedisConf17 - Dynomite - Making Non-distributed Databases DistributedRedis Labs

Protecting the Web at a scale using consul and Elk / Valentin Chernozemski (S...Ontico

Archmage, Pinterest’s Real-time Analytics Platform on DruidImply

Cloud arch patternsCorey Huinker

Screaming Fast Wpmudjcp

Instruments to play microserviceChandresh Pancholi

Introduction to Apache KafkaRicardo Bravo

AWS big-data-demystified #1.1 | Big Data Architecture Lessons Learned | EnglishOmid Vahdaty

Big Data in 200 km/h | AWS Big Data Demystified #1.3 Omid Vahdaty

AWS Big Data Demystified #1.2 | Big Data architecture lessons learned Omid Vahdaty

More from Altinity Ltd (20)

Building an Analytic Extension to MySQL with ClickHouse and Open Source.pptxAltinity Ltd

Cloud Native ClickHouse at Scale--Using the Altinity Kubernetes Operator-2022...Altinity Ltd

Building an Analytic Extension to MySQL with ClickHouse and Open SourceAltinity Ltd

This is a joint webinar Percona - Altinity. In this webinar we will discuss suggestions and tips on how to recognize when MySQL is overburdened with analytics and can benefit from ClickHouse’s unique capabilities. We will then walk through important patterns for integrating MySQL and ClickHouse which will enable the building of powerful and cost-efficient applications that leverage the strengths of both databases.

Fun with ClickHouse Window Functions-2021-08-19.pdfAltinity Ltd

Fun with ClickHouse Window Functions | Altinity Webinar Window functions have arrived in ClickHouse! Our webinar will start with an introduction to standard window function syntax and show how it is implemented in ClickHouse. We’ll next show you problems that you can now solve easily using window functions. Finally, we’ll compare window functions to arrays, another powerful ClickHouse feature. There will be time for questions with our SQL experts. Join us for a complete overview of this long-awaited feature! Speakers: Robert Hodges, CEO @Altinity Vitaliy Zakaznikov, QA Manager and Architect @Altinity

Cloud Native Data Warehouses - Intro to ClickHouse on Kubernetes-2021-07.pdfAltinity Ltd

Robert Hodges from Altinity, an enterprise provider of ClickHouse and developer of the ClickHouse Kubernetes operator, provides an introduction to running ClickHouse on Kubernetes. The presentation demonstrates how to deploy ClickHouse and Zookeeper on Kubernetes using the ClickHouse Kubernetes operator. It shows how to define ClickHouse installations using custom resources, access ClickHouse, and update the cluster configuration, such as changing the number of shards and replicas or ClickHouse version. The operator automatically applies configuration changes to pods.

Building High Performance Apps with Altinity Stable Builds for ClickHouse | A...Altinity Ltd

Altinity Stable Builds offer a ClickHouse distribution that is ready for production use and with 3 years of maintenance. Our webinar introduces the special features of Stable Builds and describes how we build them from ClickHouse Long-Term Support (LTS) releases. We’ll show you how to find them and install them yourself, then guide you through the important topic of upgrading. We’ll also walk through how to use Altinity Stable Builds in Altinity.Cloud, our managed ClickHouse platform for high-performance analytics.

Application Monitoring using Open Source - VictoriaMetrics & Altinity ClickHo...Altinity Ltd

Application Monitoring using Open Source - VictoriaMetrics & Altinity ClickHouse Webinar Slides Monitoring is the key to the successful operation of any software service, but commercial solutions are complex, expensive, and slow. Let us show you how to build monitoring that is simple, cost-effective, and fast using open-source stacks easily accessible to any developer. We’ll start with the elements of monitoring systems: data ingest, query engine, visualization, and alerting. We’ll then explain and contrast two implementation approaches. The first uses VictoriaMetrics, a fast-growing, high-performance time series database that uses PromQL for queries. The second is based on ClickHouse, a popular real-time analytics database that speaks SQL. Fast, affordable monitoring is within reach. This webinar provides designs and working code to get you there. Presented by: Roman Khavronenko, Co-Founder at VictoriaMetrics Robert Hodges, CEO at Altinity

Own your ClickHouse data with Altinity.Cloud Anywhere-2023-01-17.pdfAltinity Ltd

The document discusses Altinity.Cloud Anywhere, a service that allows users to run ClickHouse databases on their own Kubernetes clusters. It provides automation of ClickHouse operations and management through the Altinity Connector. Users can prepare their own Kubernetes environment, connect it to Altinity.Cloud, and then launch and manage ClickHouse clusters on their infrastructure. Advanced topics covered include how the service works internally and how to get support from Altinity.

ClickHouse ReplacingMergeTree in Telecom AppsAltinity Ltd

Adventures with the ClickHouse ReplacingMergeTree EngineAltinity Ltd

Building a Real-Time Analytics Application with Apache Pulsar and Apache PinotAltinity Ltd

This document provides an overview of building a real-time analytics application with Apache Pulsar and Apache Pinot. It introduces Mary Grygleski and Mark Needham, describes what real-time analytics is, and discusses the properties of real-time analytics systems. It then demonstrates how to ingest data from the Wikimedia recent changes feed into Pulsar and Pinot for real-time analytics and builds a dashboard with the data using Streamlit.

Altinity Webinar: Introduction to Altinity.Cloud-Platform for Real-Time Data.pdfAltinity Ltd

Altinity Webinar: Introduction to Altinity.Cloud-Platform for Real-Time Data - Presentation Slides Altinity.Cloud is a fully automated cloud service for ClickHouse that is optimized for real-time analytics. In this webinar, we’ll explain how Altinity.Cloud works, then show how to set up your first ClickHouse cluster. We’ll then tour important features like scale-up, scale-out, uptime schedules, and DBA tools to analyze your tables. You’ll learn everything necessary to start working on real-time analytics today. Bring your questions! Presenters: Robert Hodges & Alexander Zaitsev Note: This webinar will be recorded and later posted on our Webinar page (https://altinity.com/webinarspage/) or Altinity official Youtube channel (https://www.youtube.com/@Altinity).

OSA Con 2022 - What Data Engineering Can Learn from Frontend Engineering - Pe...Altinity Ltd

OSA Con 2022 - Welcome to OSA CON Version 2022 - Robert Hodges - Altinity.pdfAltinity Ltd

OSA Con 2022 - Using ClickHouse Database to Power Analytics and Customer Enga...Altinity Ltd

OSA Con 2022: Using ClickHouse Database to Power Analytics and Customer Engagement Platform Prafulla Gupta - Times Internet This talk covers how we empowered Product Managers and Editors at Times Internet by developing an in-house product, GrowthRx, using Clickhouse Open Source Database to track and analyze user behavior to increase user retention and customer engagement. Times Internet is India's largest digital news publisher, which manages leading brands like Times of India, Economic Times, Navbharat Times, etc, where we are tracking more than 10 billion events per month in the ClickHouse Database.

OSA Con 2022 - Tips and Tricks to Keep Your Queries under 100ms with ClickHou...Altinity Ltd

- ClickHouse can query 170 billion rows at 500 queries per second with a 99th percentile latency of 110ms through careful data modeling, query optimization, and use of materialized views. - To achieve low latency at high query rates, it is important to reduce the amount of data scanned by queries through techniques like sorting keys, data compression, and reducing data cardinality. - Materialized views can reduce data sizes by 1000-10,000x and are critical for maintaining low query latencies on large datasets. Dividing data into read and write replicas also improves query performance.

OSA Con 2022 - The Open Source Analytic Universe, Version 2022 - Robert Hodge...Altinity Ltd

OSA Con 2022 - Switching Jaeger Distributed Tracing to ClickHouse to Enable A...Altinity Ltd

The document discusses how OpsVerse migrated their Jaeger distributed tracing storage from Cassandra to ClickHouse for improved performance monitoring. Jaeger is an open source distributed tracing system that was originally designed to use Elasticsearch or Cassandra for storage. While Cassandra worked well for basic functionality, it lacked capabilities for advanced analytics. ClickHouse supports richer query functions and better handles large datasets. The document outlines the steps OpsVerse took to implement the ClickHouse storage plugin for Jaeger and deploy ClickHouse on Kubernetes using the ClickHouse Operator. This migration enabled more insightful performance monitoring and analytics.

OSA Con 2022 - Streaming Data Made Easy - Tim Spann & David Kjerrumgaard - St...Altinity Ltd

OSA Con 2022 - State of Open Source Databases - Peter Zaitsev - Percona.pdfAltinity Ltd

OSA Con 2022 - State of Open Source Databases Peter Zaitsev - Percona It has been an exciting year in the open-source database industry, with more choices, more cloud, and key changes in the industry. We will dive into the key developments over 2022, including the most important open-source database software releases in general, the significance of cloud-native solutions in a multi-vendor multi-cloud world, the new criticality of security challenges, and the evolution of the open-source software industry.