Observability; a gentle introduction

Jun 23, 20220 likes272 views

As engineers we spend much of our time getting stuff to production and making sure our infrastructure doesn’t burn down out right. We however spend very little time learning to understand and respond to outages. Does our platform degrade in a graceful way or what does a high cpu load really mean? What can we learn from level 1 outages to be able to run our platforms more reliably. Plenty of people are jumping on the new hype, Observability, lots of them are replacing their “legacy” monitoring stack. Not all of them achieve the goals they set. But observability is not a tool — it is a property of a system. Moving from many small black boxes to a more holistic view of your system. In this talk we ll talk about how to prepare teams to tweak their testing and monitoring setup and work instructions to quickly observe, react to and resolve problems. We look at improving your monitoring by adapting your culture and then maybe your tooling. Where we as engineers not only write, maintain and operate our software platforms but actively pursue ways to learn and predict its (non-functional) behavior. Furthermore we ll discuss the need for and the options of not only monitoring our platforms and it's envitable outages, but also their (potential) length and impact. We ll look at tools like at using Service Level Objects for ways to prepare teams to tweak their testing and monitoring setup and runbooks to quickly observe, react to and resolve problems.

Observability (o11y):
a gentle introdcution
Bram Vogelaar
@attachmentgenie

$ whoami
• Used to be a Molecular Biologist
• Then became a Dev
• Now an Ops
• Currently Cloud Engineer @ The Factory

SNMP pulls hundreds of checks, logs live in databases
Push thousands of time-series to Graphite + statsd , Logstash for
logs
Prometheus scrapes millions of metrics, Logs are everywhere,
Traces are a thing now
How it started -> How is it going

The monitoring everything age
The act of observing changes the system being observed
Base CPU load significantly visible
aka HDDs and CPUs killed by monitoring love

Monitoring solutions were not catching up
While Applications were deployed 10x a day, we were doing
manual configuration of Single Source of Truth
Eventual consistency with huge latency
Causing Chaos and Distrust in the Observer

“Observability is defined as a measure of how
well internal states of a system can be inferred
from knowledge of its external outputs.”
Engineering Definition

$R.E.D & U.S.E U = Utilization, as canonically defined S = Concurrency E = Error Rate, as a throughput metric R = Request Throughput, in requests per second E = Request Error Rate, as either a throughput metric or a fraction of overall throughput D = Latency, Residence Time, or Response Time; all three are widely used$

Large complex systems will always be in a
(somewhat) degraded state

Get engineers to be comfortable facing failure

monitoring – luck == Observability
Cultural Definition

SL{O,I,A}
Service-Level Agreement (SLA) => Contract
Service-Level Objective (SLO) => Promise
Service-Level Indicator (SLI) => Observe

Only alert on SLO violations
And definitely on HDD space useage
Don’t alert on symptoms
Do alert on the rate you are using the error budget

monitoring – luck == Observability
monitoring + devops == Observability
Bram’s Definition

Contact
bram@attachmentgenie.com
@attachmentgenie
https://www.slideshare.net/attachmentgenie

This document discusses operations, monitoring, and observability. It provides an overview of each topic. For operations, it describes different models from manual to proactive. For monitoring, it explains that the goal is to understand what is broken and why by looking at symptoms and causes. It also discusses monitoring methodologies like using key metrics and thresholds. For observability, it defines it as understanding a system more fully by capturing metrics, events, and traces. It explains the three pillars of observability - metrics, logging, and tracing - and how they provide visibility into reliability, bottlenecks, and request flows.

ObservabilityDiego Pacheco

This document discusses concepts related to observability including Prometheus, ELK stack, OpenTracing, and Victoria Metrics. It provides examples of setting up Prometheus and Grafana to monitor metrics from applications instrumented with exporters. It also demonstrates setting up Filebeat, Logstash and Elasticsearch (ELK stack) to monitor logs and send them to Elasticsearch. Additionally, it shows how to implement OpenTracing in a Java application and visualize traces using Jaeger. Finally, it outlines an exercise to build a microservices ecommerce application incorporating logging, metrics and tracing using the discussed tools.

Observability Enes Altınok

This document discusses observability and its three pillars: logs, metrics, and traces. It introduces common observability tools like Elastic Stack, Prometheus, and Jaeger. Logs should be aggregated and indexed, metrics can use recording rules and alerting, and traces enable root cause analysis. Best practices include monitoring components, testing configurations, and retaining sufficient log data. Observability provides insight into systems from external outputs and context about internal states.

Observability at Scale Knoldus Inc.

Observability has emerged as one of the hottest topics on the DevOps landscape. Organizations seek to improve visibility into their cloud infrastructure and applications and identify production issues that may negatively impact #customerexperience. ➡️ But what are some of the best practices for scaling observability for modernapplications? ➡️ What challenges are #cloudplatforms facing? Explore how to overcome the challenges and unlock speed, observability, and automation across your DevOps lifecycle.

Observability vs APM vs Monitoring Comparisonjeetendra mandal

APM is a tool that monitors application performance and user experience by tracking metrics like load and KPIs. It allows seeing how applications are used by real users and identifying problems that impact sales or brand experience. Observability aggregates data from logs, metrics, and traces to assess overall system health, while APM directly focuses on gauging user experience. Both ensure good user experience but in different ways - APM actively collects data related to response time, while observability passively examines various data sources. Monitoring tracks predefined metrics over time to understand system status, but observability analyzes related data to determine the root cause of issues.

Observability for modern applications MoovingON

The document discusses observability for modern applications. It describes observability as a measure of how well internal states of a system can be inferred from external outputs. It outlines three pillars of observability: event logs, metrics, and tracing. It provides examples of how AWS services like CloudWatch, CloudWatch Logs, and AWS X-Ray can be used to gain observability. It also discusses key concepts like segments, subsegments, and service maps for tracing and provides code examples for instrumenting applications to generate metrics and traces.

ObservabilityMaganathin Veeraragaloo

More Than Monitoring: How Observability Takes You From Firefighting to Fire P...DevOps.com

For some, observability is just a hollow rebranding of monitoring, for others it’s monitoring on steroids. But what if we told you observability is the new way to find out why—not just if—your distributed system or application isn’t working as expected? Today, we see that traditional monitoring approaches can fall short if a system or application doesn’t adequately externalize its state. This is truer as workloads move into the cloud and leverage ephemeral technologies, such as microservices and containers. To reach observability, IT and DevOps teams need to correlate different sources from logs, metrics, traces, events and more. This becomes even more challenging when defining the online revenue impact of a failed container—after all, this is what really matters to the business. This webinar will cover: The differences between observability and monitoring Why it is a bigger challenge in a multicloud and containerized world How observability results in less firefighting and more fire prevention How new platforms can help gain observability (on premises and in the cloud) for containers, microservices and even SAP or mainframes

Observability – the good, the bad, and the uglyTimetrix

This document discusses observability and incident management. It notes that incidents are expensive and reduce credibility. Common causes of outages include changes, network failures, bugs, human errors, hardware failures, and unspecified issues. The timeline of an outage includes detection, investigation, escalation, and fixing. Many companies have a "zoo" of monitoring solutions that are difficult to manage. Common anti-patterns include an exponential growth of metrics that nobody understands. The document advocates focusing on key performance indicator metrics and using time-series databases, distributed tracing, and machine learning to more quickly detect anomalies and reduce incident timelines. It describes an open source project called Timetrix that combines metrics, events and traces for improved observability.

Intro to open source observability with grafana, prometheus, loki, and tempo(...LibbySchulze

This document provides an introduction to open source observability tools including Grafana, Prometheus, Loki, and Tempo. It summarizes each tool and how they work together. Prometheus is introduced as a time series database that collects metrics. Loki is described as a log aggregation system that handles logs at scale without high costs. Tempo is explained as a tracing system that allows tracing from logs, metrics, and between services. The document emphasizes that these tools can be run together to gain observability across an entire system from logs to metrics to traces.

Cloud-Native ObservabilityTyler Treat

What is observability and how is it different from traditional monitoring? How do we effectively monitor and debug complex, elastic microservice architectures? In this interactive discussion, we’ll answer these questions. We’ll also introduce the idea of an “observability pipeline” as a way to empower teams following DevOps practices. Lastly, we’ll demo cloud-native observability tools that fit this “observability pipeline” model, including Fluentd, OpenTracing, and Jaeger.

Combining logs, metrics, and traces for unified observabilityElasticsearch

Monitoring and observabilityTheo Schlossnagle

In this session we’ll leave the need for performance a foregone conclusion and take a whirlwind tour through the complexity of modern Internet architectures. The complexities lead to evil optimization problems and significant challenges troubleshooting production issues to a speedy and successful end. Starting with the simple facts that you can’t fix what you can’t see and you can’t improve what you can’t measure, we’ll discuss what needs monitoring and why. We’ll talk about unlikely allies in the fight for time and budget to instrument systems, applications and processes for observability. You’ll leave the session with a better understanding of what it looks like to troubleshoot the storm of a malfunctioning large architecture and some tools and techniques you can use to not be swallowed by the Kraken.

OpenTelemetry For ArchitectsKevin Brockhoff

Monitoring and observabilityDanylenko Max

Monitoring involves collecting logs, metrics and alerts to detect issues, while observability provides insight into internal system states. The presenter faced problems determining causes of performance drops. They will discuss starting with monitoring basics like logging, tracing and metrics. They will then explain how to transition to domain-oriented observability through techniques like aspect-oriented programming to better understand the system. Observability aims to answer any questions about internal states using monitoring tools.

Observability in the world of microservicesChandresh Pancholi

The document discusses observability in microservices and provides an overview of key concepts. It introduces One Concern, which monitors buildings and natural disasters, and describes the differences between monoliths and microservices. It then covers the three pillars of observability - monitoring, logging, and tracing - and provides examples of tools for each. The rest of the document focuses on Jaeger, describing its architecture, benefits, features, terminology, and includes a demo. It concludes by mentioning One Concern is hiring.

THE STATE OF OPENTELEMETRY, DOTAN HOROVITS, Logz.ioDevOpsDays Tel Aviv

Everyone wants observability into their system, but find themselves with too many vendors and tools, each with its own API, SDK, agent and collectors. In this talk I will present OpenTelemetry, an ambitious open source project with the promise of a unified framework for collecting observability data. With OpenTelemetry you could instrument your application in a vendor-agnostic way, and then analyze the telemetry data in your backend tool of choice, whether Prometheus, Jaeger, Zipkin, or others. I will cover the current state of the various projects of OpenTelemetry (across programming languages, exporters, receivers, protocols), some of which not even GA yet, and provide useful guidance on how to get started with it.

How to Move from Monitoring to Observability, On-Premises and in a Multi-Clou...Splunk

With the acceleration of customer and business demands, site reliability engineers and IT Ops analysts now require operational visibility into their entire architecture, something that traditional APM tools, dev logging tools, and SRE tools aren’t equipped to provide. Observability enables you to inspect and understand your IT stack on premises and in the cloud(s); It’s no longer about whether your system works (monitoring), but being able to task why it is not working? (Observability). This presentation will outline key steps to take to move from monitoring to observability.

Observability For You and Me with OpenTelemetryEric D. Schabell

Are you interested in dipping your toes in the cloud native observability waters, but as an engineer you are not sure where to get started with tracing problems through your microservices and application landscapes on Kubernetes? Then this is the session for you, where we take you on your first steps in an active open-source project that offers a buffet of languages, challenges, and opportunities for getting started with telemetry data. The project is called openTelemetry, but before diving into the specifics, we’ll start with de-mystifying key concepts and terms such as observability, telemetry, instrumentation, cardinality, percentile to lay a foundation. After understanding the nuts and bolts of observability and distributed traces, we’ll explore the openTelemetry community; its Special Interest Groups (SIGs), repositories, and how to become not only an end-user, but possibly a contributor.We will wrap up with an overview of the components in this project, such as the Collector, the OpenTelemetry protocol (OTLP), its APIs, and its SDKs. Attendees will leave with an understanding of key observability concepts, become grounded in distributed tracing terminology, be aware of the components of openTelemetry, and know how to take their first steps to an open-source contribution! Key Takeaways: Open source, vendor neutral instrumentation is an exciting new reality as the industry standardizes on openTelemetry for observability. OpenTelemetry is on a mission to enable effective observability by making high-quality, portable telemetry ubiquitous. The world of observability and monitoring today has a steep learning curve and in order to achieve ubiquity, the project would benefit from growing our contributor community.

Elastic Observability FaithWestdorp

George Kobar, a community advocate for Capgemini, shared information on observability at a Meetup for New Application Development. The document defines observability as the combination of monitoring, metrics, and logging. It presents a typical observability stack that collects data from various sources to provide visibility for development, operations, and business teams through tools that analyze application performance, uptime, logs, metrics, and business KPIs. The stack advocates an elastic approach to storing all operational data together in Elasticsearch for unified access and analysis.

Service Mesh - ObservabilityAraf Karsh Hamid

ObservabilityMartin Gross

Observability refers to the ability to infer the internal state of a system from its external outputs. It is a property of the system, not an action like monitoring. For a system to be observable, it must externalize its state through logs, metrics, and events. Improving observability involves monitoring all components of an application from the front-end to backend services to infrastructure. Common metrics include requests processed, errors encountered, and response times for applications as well as CPU usage, disk I/O, and network traffic for infrastructure. Observability extends monitoring by helping understand why a system is not working in addition to whether it is working.

Getting started with Site Reliability Engineering (SRE)Abeer R

"Getting started with Site Reliability Engineering (SRE): A guide to improving systems reliability at production" This is an intro guide to share some of the common concepts of SRE to a non-technical audience. We will look at both technical and organizational changes that should be adopted to increase operational efficiency, ultimately benefiting for global optimizations - such as minimize downtime, improve systems architecture & infrastructure: - improving incident response - Defining error budgets - Better monitoring of systems - Getting the best out of systems alerting - Eliminating manual, repetitive actions (toils) by automation - Designing better on-call shifts/rotations How to design the role of the Site Reliability Engineer (who effectively works between application development teams and operations support teams)

Observability, what, why and howNeeraj Bagga

OSMC 2022 | OpenTelemetry 101 by Dotan Horovit s.pdfNETWAYS

This document provides an overview of OpenTelemetry, including: - OpenTelemetry is an observability framework that assists in generating and capturing telemetry data from cloud-native software across traces, metrics, and logs. - It includes vendor-agnostic APIs, SDKs, and tools for generating, collecting, and exporting telemetry data to analysis tools. - OpenTelemetry has reached general availability for tracing and is in release candidate for metrics, with client libraries available for many popular programming languages.

Logging and observabilityAnton Drukh

This document discusses observability and logging practices at Snyk. It defines observability and distinguishes it from monitoring. It describes Snyk's current logging implementation including structured logging to stdout and sending logs to logz.io. It recommends logging best practices like separating log generation from collection and using request IDs and durations. It provides an example demo and discusses benefits like improved troubleshooting and developer productivity.

CD using ArgoCD(KnolX).pdfKnoldus Inc.

Observability & DatadogJamesAnderson599331

Prometheus (Prometheus London, 2016)Brian Brazil

Brian Brazil is an engineer passionate about reliable software operations. He worked at Google SRE for 7 years and is the founder of Prometheus, an open source time series database designed for monitoring system and service metrics. Prometheus supports metric labeling, unified alerting and graphing, and is efficient, decentralized, reliable, and opinionated in how it encourages good monitoring practices.

An Introduction to Prometheus (GrafanaCon 2016)Brian Brazil

Often what you monitor and get alerted on is defined by your tools, rather than what makes the most sense to you and your organisation. Alerts on metrics such as CPU usage which are noisy and rarely spot real problems, while outages go undetected. Monitoring systems can also be challenging to maintain, and overall provide a poor return on investment. In the past few years several new monitoring systems have appeared with more powerful semantics and which are easier to run, which offer a way to vastly improve how your organisation operates and prepare you for a Cloud Native environment. Prometheus is one such system. This talk will look at the monitoring ideal and how whitebox monitoring with a time series database, multi-dimensional labels and a powerful querying/alerting language can free you from midnight pages.

More Related Content

What's hot (20)

Observability – the good, the bad, and the uglyTimetrix

Intro to open source observability with grafana, prometheus, loki, and tempo(...LibbySchulze

Cloud-Native ObservabilityTyler Treat

Combining logs, metrics, and traces for unified observabilityElasticsearch

Monitoring and observabilityTheo Schlossnagle

OpenTelemetry For ArchitectsKevin Brockhoff

Monitoring and observabilityDanylenko Max

Observability in the world of microservicesChandresh Pancholi

THE STATE OF OPENTELEMETRY, DOTAN HOROVITS, Logz.ioDevOpsDays Tel Aviv

How to Move from Monitoring to Observability, On-Premises and in a Multi-Clou...Splunk

Observability For You and Me with OpenTelemetryEric D. Schabell

Elastic Observability FaithWestdorp

Service Mesh - ObservabilityAraf Karsh Hamid

ObservabilityMartin Gross

Getting started with Site Reliability Engineering (SRE)Abeer R

Observability, what, why and howNeeraj Bagga

OSMC 2022 | OpenTelemetry 101 by Dotan Horovit s.pdfNETWAYS

Logging and observabilityAnton Drukh

CD using ArgoCD(KnolX).pdfKnoldus Inc.

Observability & DatadogJamesAnderson599331

Observability – the good, the bad, and the uglyTimetrix

Intro to open source observability with grafana, prometheus, loki, and tempo(...LibbySchulze

Cloud-Native ObservabilityTyler Treat

Combining logs, metrics, and traces for unified observabilityElasticsearch

Monitoring and observabilityTheo Schlossnagle

OpenTelemetry For ArchitectsKevin Brockhoff

Monitoring and observabilityDanylenko Max

Observability in the world of microservicesChandresh Pancholi

THE STATE OF OPENTELEMETRY, DOTAN HOROVITS, Logz.ioDevOpsDays Tel Aviv

How to Move from Monitoring to Observability, On-Premises and in a Multi-Clou...Splunk

Observability For You and Me with OpenTelemetryEric D. Schabell

Elastic Observability FaithWestdorp

Service Mesh - ObservabilityAraf Karsh Hamid

ObservabilityMartin Gross

Getting started with Site Reliability Engineering (SRE)Abeer R

Observability, what, why and howNeeraj Bagga

OSMC 2022 | OpenTelemetry 101 by Dotan Horovit s.pdfNETWAYS

Logging and observabilityAnton Drukh

CD using ArgoCD(KnolX).pdfKnoldus Inc.

Observability & DatadogJamesAnderson599331

Similar to Observability; a gentle introduction (20)

Prometheus (Prometheus London, 2016)Brian Brazil

An Introduction to Prometheus (GrafanaCon 2016)Brian Brazil

Prometheus - Open Source Forum JapanBrian Brazil

Prometheus is a next-generation monitoring system. It lets you see you not just what your systems look like from the outside, but also gives visibility into the internals and business aspects of your systems. This allows everyone to benefit, including both operations and developers. This talk will look at the concepts behind monitoring with Prometheus, how it's designed, why it's suitable for Cloud Native environments and how you can get involved.

Overview of Site Reliability Engineering (SRE) & best practicesAshutosh Agarwal

Monitoring What Matters: The Prometheus Approach to Whitebox Monitoring (Berl...Brian Brazil

Prometheus is an open-source monitoring system that allows for whitebox monitoring through metrics collected from inside systems and applications. It provides the ability to alert on high-level symptoms, debug issues through customizable dashboards, and perform complex queries across metrics. Prometheus empowers building monitoring that matters through alerting on important business metrics, gaining insight via dashboards, and integrating with other systems via open interfaces.

Performance monitoring for Docker - Lucerne meetupStijn Polfliet

Performance monitoring for Docker Challenges - Anomaly detection - CoScale demo For more info about how to use CoScale Docker monitoring, some reading material here: http://www.coscale.com/blog/how-to-monitor-docker-containers-with-coscale and http://www.coscale.com/blog/how-to-monitor-your-kubernetes-cluster A summary of CoScale Docker performance monitoring can be found here: http://www.coscale.com/docker-monitoring

Application Logging Good Bad Ugly ... Beautiful?Anton Chuvakin

When Platform Engineers meet SREs - The Birth of O11y-as-a-Service SuperpowersEric D. Schabell

Monitoring the behavior of a system is essential to ensuring its long-term effectiveness. However, managing an end-to-end observability stack can feel like stepping into quicksand, without a clear plan you’re risking sinking deeper into system complexities. In this talk, we’ll explore how combining two worlds—developer platforms and observability—can help tackle the feeling of being off the beaten cloud native path. We’ll discuss how to build paved paths, ensuring that adopting new developer tooling feels as seamless as possible. Further, we’ll show how to avoid getting lost in the sea of telemetry data generated by our systems. Implementing the right strategies and centralizing data on a platform ensures both developers and SREs stay on top of things. Practical examples are used to map out creating your very own Internal Developer Platform (IDP) with observability integrated from day 1.

Complex Event ProcessingMatthew Versaggi

This document provides an overview of complex event processing (CEP). It defines key CEP terms and describes an example use case of monitoring login and data access events in a military system. The document outlines the architecture for representing events as Java objects, describes rules for detecting complex patterns involving multiple related events over time, and shows examples of implemented rules. It also discusses Allen's temporal operators for reasoning about time in rules and provides diagrams to illustrate several of the operators. The document concludes by comparing forward chaining and backward chaining inference approaches in CEP.

How to not fail at security data analytics (by CxOSidekick)Dinis Cruz

1. The document discusses the challenges of obtaining security-related data from different sources and transporting it to a central platform for analysis. It addresses questions about data volume, collection methods, filtering and formatting. 2. Setting up a security data pipeline involves determining what data to collect from various host systems, networks, and applications. Data must then be forwarded from collectors to a central platform while managing bandwidth, latency, and failures. 3. Collecting the right security-related data is vital for detecting threats and being able to investigate incidents. The document argues for collecting most available data by default and filtering out exceptions, rather than only collecting predefined types of data.

Prometheus for Monitoring Metrics (Fermilab 2018)Brian Brazil

2011-05-02 - VU Amsterdam - Testing safety critical systemsJaap van Ekris

What does "monitoring" mean? (FOSDEM 2017)Brian Brazil

Monitoring can mean very different things to different people, and this often leads to confusion and misunderstandings. There are many offerings both free software and commercials, and it's not always clear where each fits in the bigger picture. This talk will look a bit at the history of monitoring, and then into the general categories of Metrics, Logs, Profiling and Distributed tracing and how each of these is important in Cloud-based environment. Video: https://www.youtube.com/watch?v=hCBGyLRJ1qo

Christopher Judd: Monitor All Your Stuff with OpenTelemetry and AWSAWS Chicago

Monitoring Distributed SystemsAleksandr Tavgen

2010-03-31 - VU Amsterdam - Experiences testing safety critical systemsJaap van Ekris

1) Testing safety critical systems is challenging because software often contains errors and failures can have catastrophic consequences, so systems must be designed and tested to extremely high standards of reliability. 2) The document discusses standards like IEC 61508 that provide requirements for safety integrity levels and risk management in developing safety critical systems. 3) Rigorous verification techniques are needed including reviews, static analysis, unit testing with high code coverage, integration testing of components, system testing of full environments, and acceptance testing of real systems.

Six Mistakes of Log Management 2008Anton Chuvakin

The document discusses six common mistakes made in security log management: 1) not logging at all, 2) not looking at the logs, 3) storing logs for too short a time, 4) prioritizing log records before collection, 5) ignoring logs from applications, and 6) treating logs from different systems in silos. It emphasizes the importance of centralized log management to enable security investigations, incident response, auditing and regulatory compliance.

Evolution of Monitoring and Prometheus (Dublin 2018)Brian Brazil

Observability foundations in dynamically evolving architecturesBoyan Dimitrov

Holistic application health monitoring, request tracing across distributed systems, instrumentation, business process SLAs - all of them are integral parts of today’s technical stacks. Nevertheless many teams decide to integrate observability last which makes it an almost impossible challenge - especially if you have to deal with hundreds and thousands of services. Therefore starting early is essential and in this talk we are going to see how we can solve those challenges early and explore the foundations of building and evolving complex microservices platforms in respect to observability. We are going to share some of the best practices and quick wins that allow us to correlate different telemetry systems and gradually build up towards more sophisticated use-cases. We are also going to look at some of the standard AWS services such as X-Ray and Cloudwatch that help us get going "for free" and then discuss more complex tooling and integrations building up towards a fully integrated ecosystem. As part of this talk we are also going to share some of the learnings we have made at Sixt on this topic and we are going to introduce some of the solutions that help us operate our microservices stack

Dynamic Population Discovery for Lateral Movement (Using Machine Learning)Rod Soto

Prometheus (Prometheus London, 2016)Brian Brazil

An Introduction to Prometheus (GrafanaCon 2016)Brian Brazil

Prometheus - Open Source Forum JapanBrian Brazil

Overview of Site Reliability Engineering (SRE) & best practicesAshutosh Agarwal

Monitoring What Matters: The Prometheus Approach to Whitebox Monitoring (Berl...Brian Brazil

Performance monitoring for Docker - Lucerne meetupStijn Polfliet

Application Logging Good Bad Ugly ... Beautiful?Anton Chuvakin

When Platform Engineers meet SREs - The Birth of O11y-as-a-Service SuperpowersEric D. Schabell

Complex Event ProcessingMatthew Versaggi

How to not fail at security data analytics (by CxOSidekick)Dinis Cruz

Prometheus for Monitoring Metrics (Fermilab 2018)Brian Brazil

2011-05-02 - VU Amsterdam - Testing safety critical systemsJaap van Ekris

What does "monitoring" mean? (FOSDEM 2017)Brian Brazil

Christopher Judd: Monitor All Your Stuff with OpenTelemetry and AWSAWS Chicago

Monitoring Distributed SystemsAleksandr Tavgen

2010-03-31 - VU Amsterdam - Experiences testing safety critical systemsJaap van Ekris

Six Mistakes of Log Management 2008Anton Chuvakin

Evolution of Monitoring and Prometheus (Dublin 2018)Brian Brazil

Observability foundations in dynamically evolving architecturesBoyan Dimitrov

Dynamic Population Discovery for Lateral Movement (Using Machine Learning)Rod Soto

More from Bram Vogelaar (20)

Terraforming your Platform Engineering organisation.pptxBram Vogelaar

Have we just reached the limits of shifting left? Are we now limited by "Devs doing Ops?” After the emergence of SRE, we’re now in the age of platform engineering. Platform engineering massively improves the developer experience. It enables teams to do exactly what needed to be done to get features in front of users, instead of being bogged down by things that have no unique selling point for their platform or product. Providing, creating, and maintaining the right permissions and team structures can be quite tedious and when done manually will result in (at best) broken and (at worst) dangerous team structures. In the last couple of years we have seen a general "shift-left" approach in the DevOps space. This gave engineers the power to be in control, but also the burden and steep learning curve of having to know all the layers and best security practices involved. By using HashiCorp Waypoint as a starting point, we can have well-defined and proven building blocks while letting the engineering teams be in control. This talk discusses how to create a team structure in GitHub that can be used to create and maintain teams and permissions in HashiCorp Vault and Boundary. After which Vault can be used to sync secrets from Vault to appropriate GitHub projects.

Secure second days operations with Boundary and Vault.pdfBram Vogelaar

Cost reconciliation in a post CMDB worldBram Vogelaar

Back in the day in a IT company long ago, where the BOFHs roamed and the ITIL was strong. We used to keep long lists of CIs that used to enviably and hopelessly out of date. Because we either didnt care, know or bother keeping up to date. That was totally fine in a relatively static environment the IT company of long ago. We would have our yearly inventory day and forget about it again. Of course we all use some form of infrastructure as code right now. Some of us might go as far that "if it isn't in code it doesnt exist", but can we truly say that whatever is in the OpenTofu state really is the only thing running? What about that recurring 1$ in that dormant AWS account, where is that coming from? How about the playground projects the CEO likes to play around with in his sparetime? or that one time the opentofu destroy didnt exit cleanly and some resources weren't cleaned during that timeout, did we really manually cleanup all resources?

Self scaling Multi cloud nomad workloadsBram Vogelaar

During this talk we will discuss the problems encountered and the solutions implemented while dealing with building out a multi-cloud strategy. We will start by taking our first steps building a multi-region, multi-cloud Nomad cluster and discussing some pitfalls we encountered, since not all cloud providers are built the same. We’ll finish our talk by diving into ingress patterns and Consul config to be able to survive pretty much any outage or price change. Maintaining these config can be quite cumbersome but they’re are also a prime target to automate using Consul watches.

Scraping metrics for fun and profitBram Vogelaar

We all know the wall of illegible wall of small graphs, that we like to present to (senior) management and auditors as proof that we do observability. It doesnt matter that the person ( and the associated knowledge) has long since left the company, nor that the dashboard doesnt autorefresh and still show the same data from when we turned on the monitoring PC. In an ever changing IT landscape we deserve better than that. We shouldn't have to rely on the gut instinct of the senior engineer on deck about the general shape of the dashboard to know where to start fixing whatever it is that needs fixing. We should aim to collect and present such information both from the it and business side that let a relatively inexperienced oncall engineer differentiate between a P1 incident and a major client/customer/continet going to bed. We should be telling beautiful stories with data and dashboards in such a way that (even) management can pull up global dashboard and can determine business relevant information like "is our advertisement campaign having any impact". We should also not be afraid to have multiple dashboards that show different relevant information to stack holders (e.g sales figures for management and links to runbooks for engineers)

10 things i learned building nomad-packsBram Vogelaar

This document discusses Nomad packs, which are a templating and packaging tool for easily deploying popular applications to Nomad. Key points include: - Nomad packs use HashiCorp Configuration Language (HCL) templates and Go template syntax to define jobs. - Pack structure includes templates, variables, metadata, and output files. Common patterns can be abstracted and reused across packs. - Packs can be run via the nomad-pack CLI and support variables, registries for finding packs, and CI/CD workflows. - The author discusses ideas for the future including pre-commit hooks, locals support, meta packages, and dependency health checks.

10 things I learned building Nomad packsBram Vogelaar

Easy Cloud Native Transformation with NomadBram Vogelaar

HashiCorp Nomad is a flexible and straightforward scheduler and orchestrator to deploy and manage containers and non-containerized applications across on-prem and clouds at scale. Nomad can be seen as: - an alternative to Kubernetes to deploy and scale containers without complexity - a supplement to Kubernetes to implement a multi-orchestrator platform On the other hand, this session will present how to ease the Cloud Native Transformation using Nomad.

Uncomplicated NomadBram Vogelaar

The nomad team has been working hard to make sure that running Nomad is as easy as possible. For that they have introduced native service discovery and secret management and the ecosystem is picking up speed and tools like Traefik and Prometheus now support it out of the box. With the introduction of nomad-pack and the community registry it is now easier than ever to start nomad jobs. During this talk we ll discuss how to plumb all these together and why running Nomad has never been easier.

Running Trusted Payload with Nomad and WaypointBram Vogelaar

Things like Infrastructure as Code, Service Discovery and Config Management can and have helped us to quickly build and rebuild infrastructure but we haven't nearly spend enough time to train our self to review, monitor and respond to outages. With the the introduction of CI/CD best practices into our day to day workflows we protect ourselves for introducing "bad" code into production and exposing flaws to our (end-)users. But what about influences from bad actors in- and out-side our projects. This talk will focus on the additional steps we can add to our Waypoint build pipelines to also protect ourselves to so called supply chain attacks while running our jobs in Nomad. We ll discuss scanning for vulnerabilities in incoming code, packages and images and signing the content artefacts we trust before exposing them to our users.

Easy Cloud Native Transformation using HashiCorp NomadBram Vogelaar

Nomad is an open source tool for dynamic workload scheduling that can be used to easily transform applications to be cloud native. The document discusses how Nomad was used to deploy a personal blog to AWS, including writing a Nomad job file and using features like spreading tasks across hosts and auto-scaling. It also covers integrating Nomad with other HashiCorp tools like Consul for service discovery, Vault for access control, and the Nomad autoscaler for auto-scaling. Examples are provided of monitoring metrics with Prometheus and visualizing with Grafana.

Securing Prometheus exporters using HashiCorp VaultBram Vogelaar

Things like Infrastructure as Code, Service Discovery and Config Management can and have helped us to quickly build and rebuild infrastructure but we haven't nearly spend enough time to train our self to review, monitor and respond to outages. Does our platform degrade in a graceful way or what does a high cpu load really mean? What can we learn from level 1 outages to be able to run our platforms more reliably. This talk will focus on on creating a secure prometheus exporter ecosystem using HashiCorp Vault where we can we be sure that we are not leaking any business metrics from our observability stack. After which we ll investigate how to automatically rotate the certificates we created to do so.

CICD using jenkins and NomadBram Vogelaar

Things like Infrastructure as Code, Service Discovery and Config Management can and have helped us to quickly build and rebuild infrastructure but we haven't nearly spend enough time to train our self to review, monitor and respond to outages. Does our platform degrade in a graceful way or what does a high cpu load really mean? What can we learn from level 1 outages to be able to run our platforms more reliably. This talk will focus on on setting up a CICD pipeline using Jenkins. We start by configuring Jenkins to use our Nomad platform to autoscale job runners. After which we ll look at using the newly released nomad-pack tool to convert, deploy and test and existing nomad job.

Bootstrapping multidc observability stackBram Vogelaar

Running trusted payloads with Nomad and WaypointBram Vogelaar

Things like Infrastructure as Code, Service Discovery and Config Management can and have helped us to quickly build and rebuild infrastructure but we haven't nearly spend enough time to train our self to review, monitor and respond to outages. With the the introduction of CI/CD best practices into our day to day workflows we protect ourselves for introducing "bad" code into production and exposing flaws to our (end-)users. But what about influences from bad actors in- and out-side our projects. This talk will focuss on the addtional steps we can add to our Waypoint build pipelines to also protect ourselves to so called supply chain attacks while running our jobs in Nomad. We ll discuss scanning for vulnerabilities in incoming code, packages and images and signing the content artifacts we trust before exposing them to our users.

Gamification of Chaos TestingBram Vogelaar

Athletes, Firemen and Doctors train everyday to be the best at their chosen profession. As engineers, we spend much of our time getting stuff to production and making sure our infrastructure doesn’t burn down out right. In this talk, we'll discuss the need for and the options of creating a game day culture. Where we as engineers not only write, maintain and operate our software platforms but actively pursue ways to learn and predict its (non-functional) behavior. We'll look at tools like toxiproxy and the simian army for ways to prepare teams to tweak their testing and monitoring setup and work instructions to quickly observe, react to and resolve problems.

Puppet and the HashiStackBram Vogelaar

Puppet is an open source configuration management tool that can be used to automate the configuration and management of infrastructure and applications. It uses a client-server architecture and declarative language to define and enforce the desired state of systems. Other HashiCorp tools like Packer, Terraform, Vault and Nomad can integrate with Puppet for tasks like infrastructure provisioning, secrets management and workload orchestration. Bolt is a task orchestration tool from Puppet that can be used to automate operational tasks across infrastructure defined by tools like Terraform. Consul provides service discovery and configuration for the Puppet infrastructure.

Bootstrapping multidc observability stackBram Vogelaar

Creating Reusable Puppet ProfilesBram Vogelaar

"Roles and Profiles" is now the ubiquitous design pattern to create your puppet code tree. In this talk we will discuss writing reusable and maintainable profiles. We ll start by introducing creating module structures and will move on to type hinting and setting appropriate defaults. Finally we ll discuss the importance and the enforcing of code style conventions that allows multiple teams or projects to inner-source profiless

Gamification of Chaos TestingBram Vogelaar

1. Practice and experimentation are important for building expertise, whether for athletes, musicians, pilots, doctors, or scientists. Pre- and post-analysis helps identify areas for improvement. 2. Simulations that mimic real-world stressors are critical for pilots, doctors, and firefighters to practice their skills and prevent mistakes. Checklists can help ensure important steps are not missed. 3. When experimenting scientifically, experimental design and accounting for biases are important. Replicates, controls, and representative samples help establish confidence in results.

Terraforming your Platform Engineering organisation.pptxBram Vogelaar

Secure second days operations with Boundary and Vault.pdfBram Vogelaar

Cost reconciliation in a post CMDB worldBram Vogelaar

Self scaling Multi cloud nomad workloadsBram Vogelaar

Scraping metrics for fun and profitBram Vogelaar

10 things i learned building nomad-packsBram Vogelaar

10 things I learned building Nomad packsBram Vogelaar

Easy Cloud Native Transformation with NomadBram Vogelaar

Uncomplicated NomadBram Vogelaar

Running Trusted Payload with Nomad and WaypointBram Vogelaar

Easy Cloud Native Transformation using HashiCorp NomadBram Vogelaar

Securing Prometheus exporters using HashiCorp VaultBram Vogelaar

CICD using jenkins and NomadBram Vogelaar

Bootstrapping multidc observability stackBram Vogelaar

Running trusted payloads with Nomad and WaypointBram Vogelaar

Gamification of Chaos TestingBram Vogelaar

Puppet and the HashiStackBram Vogelaar

Bootstrapping multidc observability stackBram Vogelaar

Creating Reusable Puppet ProfilesBram Vogelaar

Gamification of Chaos TestingBram Vogelaar

Recently uploaded (20)

Special Meetup Edition - TDX Bengaluru Meetup #52.pptxshyamraj55

TrustArc Webinar: Consumer Expectations vs Corporate Realities on Data Broker...TrustArc

Most consumers believe they’re making informed decisions about their personal data—adjusting privacy settings, blocking trackers, and opting out where they can. However, our new research reveals that while awareness is high, taking meaningful action is still lacking. On the corporate side, many organizations report strong policies for managing third-party data and consumer consent yet fall short when it comes to consistency, accountability and transparency. This session will explore the research findings from TrustArc’s Privacy Pulse Survey, examining consumer attitudes toward personal data collection and practical suggestions for corporate practices around purchasing third-party data. Attendees will learn: - Consumer awareness around data brokers and what consumers are doing to limit data collection - How businesses assess third-party vendors and their consent management operations - Where business preparedness needs improvement - What these trends mean for the future of privacy governance and public trust This discussion is essential for privacy, risk, and compliance professionals who want to ground their strategies in current data and prepare for what’s next in the privacy landscape.

Build Your Own Copilot & Agents For DevsBrian McKeiver

2025-05-Q4-2024-Investor-Presentation.pptxSamuele Fogagnolo

Drupalcamp Finland – Measuring Front-end Energy ConsumptionExove

Into The Box Conference Keynote Day 1 (ITB2025)Ortus Solutions, Corp

Mobile App Development Company in Saudi ArabiaSteve Jonas

EmizenTech is a globally recognized software development company, proudly serving businesses since 2013. With over 11+ years of industry experience and a team of 200+ skilled professionals, we have successfully delivered 1200+ projects across various sectors. As a leading Mobile App Development Company In Saudi Arabia we offer end-to-end solutions for iOS, Android, and cross-platform applications. Our apps are known for their user-friendly interfaces, scalability, high performance, and strong security features. We tailor each mobile application to meet the unique needs of different industries, ensuring a seamless user experience. EmizenTech is committed to turning your vision into a powerful digital product that drives growth, innovation, and long-term success in the competitive mobile landscape of Saudi Arabia.

tecnologias de las primeras civilizaciones.pdffjgm517

AI Changes Everything – Talk at Cardiff Metropolitan University, 29th April 2...Alan Dix

Talk at the final event of Data Fusion Dynamics: A Collaborative UK-Saudi Initiative in Cybersecurity and Artificial Intelligence funded by the British Council UK-Saudi Challenge Fund 2024, Cardiff Metropolitan University, 29th April 2025 https://alandix.com/academic/talks/CMet2025-AI-Changes-Everything/ Is AI just another technology, or does it fundamentally change the way we live and think? Every technology has a direct impact with micro-ethical consequences, some good, some bad. However more profound are the ways in which some technologies reshape the very fabric of society with macro-ethical impacts. The invention of the stirrup revolutionised mounted combat, but as a side effect gave rise to the feudal system, which still shapes politics today. The internal combustion engine offers personal freedom and creates pollution, but has also transformed the nature of urban planning and international trade. When we look at AI the micro-ethical issues, such as bias, are most obvious, but the macro-ethical challenges may be greater. At a micro-ethical level AI has the potential to deepen social, ethnic and gender bias, issues I have warned about since the early 1990s! It is also being used increasingly on the battlefield. However, it also offers amazing opportunities in health and educations, as the recent Nobel prizes for the developers of AlphaFold illustrate. More radically, the need to encode ethics acts as a mirror to surface essential ethical problems and conflicts. At the macro-ethical level, by the early 2000s digital technology had already begun to undermine sovereignty (e.g. gambling), market economics (through network effects and emergent monopolies), and the very meaning of money. Modern AI is the child of big data, big computation and ultimately big business, intensifying the inherent tendency of digital technology to concentrate power. AI is already unravelling the fundamentals of the social, political and economic world around us, but this is a world that needs radical reimagining to overcome the global environmental and human challenges that confront us. Our challenge is whether to let the threads fall as they may, or to use them to weave a better future.

Manifest Pre-Seed Update | A Humanoid OEM Deeptech In Francechb3

SAP Modernization: Maximizing the Value of Your SAP S/4HANA Migration.pdfPrecisely

Generative Artificial Intelligence (GenAI) in BusinessDr. Tathagat Varma

Role of Data Annotation Services in AI-Powered ManufacturingAndrew Leo

AI EngineHost Review: Revolutionary USA Datacenter-Based Hosting with NVIDIA ...SOFTTECHHUB

I started my online journey with several hosting services before stumbling upon Ai EngineHost. At first, the idea of paying one fee and getting lifetime access seemed too good to pass up. The platform is built on reliable US-based servers, ensuring your projects run at high speeds and remain safe. Let me take you step by step through its benefits and features as I explain why this hosting solution is a perfect fit for digital entrepreneurs.

Cyber Awareness overview for 2025 month of securityriccardosl1

IEDM 2024 Tutorial2_Advances in CMOS Technologies and Future Directions for C...organizerofv

Dev Dives: Automate and orchestrate your processes with UiPath MaestroUiPathCommunity

This session is designed to equip developers with the skills needed to build mission-critical, end-to-end processes that seamlessly orchestrate agents, people, and robots. 📕 Here's what you can expect: - Modeling: Build end-to-end processes using BPMN. - Implementing: Integrate agentic tasks, RPA, APIs, and advanced decisioning into processes. - Operating: Control process instances with rewind, replay, pause, and stop functions. - Monitoring: Use dashboards and embedded analytics for real-time insights into process instances. This webinar is a must-attend for developers looking to enhance their agentic automation skills and orchestrate robust, mission-critical processes. 👨‍🏫 Speaker: Andrei Vintila, Principal Product Manager @UiPath This session streamed live on April 29, 2025, 16:00 CET. Check out all our upcoming Dev Dives sessions at https://community.uipath.com/dev-dives-automation-developer-2025/.

Massive Power Outage Hits Spain, Portugal, and France: Causes, Impact, and On...Aqusag Technologies

Designing Low-Latency Systems with Rust and ScyllaDB: An Architectural Deep DiveScyllaDB

Want to learn practical tips for designing systems that can scale efficiently without compromising speed? Join us for a workshop where we’ll address these challenges head-on and explore how to architect low-latency systems using Rust. During this free interactive workshop oriented for developers, engineers, and architects, we’ll cover how Rust’s unique language features and the Tokio async runtime enable high-performance application development. As you explore key principles of designing low-latency systems with Rust, you will learn how to: - Create and compile a real-world app with Rust - Connect the application to ScyllaDB (NoSQL data store) - Negotiate tradeoffs related to data modeling and querying - Manage and monitor the database for consistently low latencies

ThousandEyes Partner Innovation Updates for May 2025ThousandEyes