Why Open Source Works for DevOps Monitoring
2 likes234 views
The document discusses the integration of InfluxDB and Grafana, emphasizing their utility in collecting, storing, and visualizing time series data for applications such as DevOps monitoring and real-time analytics. It outlines the basic setup, common configurations, and performance enhancement strategies for using these tools together. The document also provides links to resources for further learning and notes upcoming events related to Grafana and InfluxDB.
![© 2017 InfluxData. All rights reserved.7
Pain points solved
“Out of all of the
monitors there [in our
office], we only have a
couple that show some
custom dashboards.
The rest is entirely
Grafana running on
InfluxDB.”
Jack Tench, Senior Software Engineer
NewVoiceMedia
“We are getting good
mileage in a short time
period from our
investment in TICK and
Grafana.”
Hans Gustavson, Senior Director SRE
Coupa
“We also store
annotation of events,
curl calls from, say,
Jenkins, for deployment,
pushing those points
into InfluxDB and then
we’re able to use those
for annotations with
Grafana.”
Robert Allen, Director of Engineering
Houghton Mifflin Harcourt](https://image.slidesharecdn.com/influxdbandgrafana-nov2018-181114153051/85/Why-Open-Source-Works-for-DevOps-Monitoring-7-320.jpg)
Why Open Source Works for DevOps Monitoring
- 1. Chris Churilo | InfluxData & David Kaltschmidt | Grafana November 2018 How to use InfluxDB & Grafana
- 2. © 2018 InfluxData. All rights reserved.2 Agenda • What is InfluxDB & Grafana • Why use them together • Basic Setup • Common Configurations • Feature Overview • How to Make Your Solution Efficient
- 3. © 2018 InfluxData. All rights reserved.3 © 2018 InfluxData. All rights reserved.3 What is InfluxDB & Grafana
- 4. © 2017 InfluxData. All rights reserved.4 The Market-Leading Time Series Platform Built specifically for metrics, events, and other time-based data The open platform for beautiful analytics & monitoring The leading open source software for time series analytics
- 5. © 2018 InfluxData. All rights reserved.5 © 2018 InfluxData. All rights reserved.5 Time series data is a sequence of data points, typically consisting of successive measurements made from the same source over a time interval. Plot the points on a graph and one of your axes would always be time.
- 6. © 2018 InfluxData. All rights reserved.6 Time series databases are Optimized for collecting, storing, retrieving & processing of time series data Compare this to ✓ Document databases Optimized for storing JSON documents ✓ Search databases Optimized for full-text searches ✓ Traditional relational Databases optimized for the tabular storage of related data in rows & columns
- 7. © 2017 InfluxData. All rights reserved.7 Pain points solved “Out of all of the monitors there [in our office], we only have a couple that show some custom dashboards. The rest is entirely Grafana running on InfluxDB.” Jack Tench, Senior Software Engineer NewVoiceMedia “We are getting good mileage in a short time period from our investment in TICK and Grafana.” Hans Gustavson, Senior Director SRE Coupa “We also store annotation of events, curl calls from, say, Jenkins, for deployment, pushing those points into InfluxDB and then we’re able to use those for annotations with Grafana.” Robert Allen, Director of Engineering Houghton Mifflin Harcourt
- 8. © 2017 InfluxData. All rights reserved.8 © 2018 InfluxData. All rights reserved.8 InfluxData 4 Open Source projects Telegraf: 200+ collection agents InfluxDB: Time Series database Chronograf: Admin UI for the DB Kapacitor: Native data processing engine Grafana Open Source Visualization and Dashboarding tool The tool for beautiful monitoring and metric analytics & dashboards InfluxDB Announced Nov ‘13 14,780 2,153 Telegraf 5,829 2,185 Grafana 24,798 4,526
- 9. © 2017 InfluxData. All rights reserved.9 © 2018 InfluxData. All rights reserved.9 ✓ Need to collect and act on time series data for managing DevOps, IoT or Real Time Analytics projects ✓ InfluxData & Grafana are perfect partners to collect, store, and visualize/dashboard metrics and events ✓ Open Source: try it for free and make them better with your contributions Why would you use these two projects together?
- 10. © 2018 InfluxData. All rights reserved.10 © 2018 InfluxData. All rights reserved.10 Basic Setup
- 11. © 2018 InfluxData. All rights reserved.11 © 2018 InfluxData. All rights reserved.11 Architecture
- 12. © 2018 InfluxData. All rights reserved.12 © 2018 InfluxData. All rights reserved.12 Download https://portal.influxdata.com/downloads Open Source Trials for InfluxCloud & InfluxEnterprise https://grafana.com/get Open Source Trials for Hosted Grafana
- 13. © 2017 InfluxData. All rights reserved.13 © 2018 InfluxData. All rights reserved.13 Creating a database CREATE DATABASE mydb Last login: Mon Oct 19 10:50:43 on ttys006 ~$ influx Connected to http://localhost:8086 version 0.9 InfluxDB shell 0.9 > create database mydb >
- 14. © 2017 InfluxData. All rights reserved.14 © 2018 InfluxData. All rights reserved.14 Verifying that it was created SHOW DATABASES > show databases name: databases --------------- name _internal mydb
- 15. © 2017 InfluxData. All rights reserved.15 © 2018 InfluxData. All rights reserved.15 Using the database we just created USE mydb > use mydb Using database mydb >
- 16. © 2017 InfluxData. All rights reserved.16 © 2018 InfluxData. All rights reserved.16 Inserting data into the database insert cpu,host=server1,location=us-west value=10 insert cpu,host=server1,location=london value=11 insert cpu,host=server2,location=us-west value=12
- 17. © 2017 InfluxData. All rights reserved.17 © 2018 InfluxData. All rights reserved.17 Verifying that the data was written SELECT * FROM cpu PS. Be careful! This query can be very expensive. SHOW SERIES SHOW MEASUREMENTS
- 18. © 2017 InfluxData. All rights reserved.18 © 2018 InfluxData. All rights reserved.18 Demo Grafana local installation: http://localhost:3000
- 19. © 2017 InfluxData. All rights reserved.19 © 2018 InfluxData. All rights reserved.19 Demo - Added datasource and dashboard panels - Table panel to explore data (unset all query options) - Graph panel to look at data - Changed query options and checked in Query Inspector - Added telegraf datasource, stacked CPU graphs - Template variable based on cpu - Annotations based on logs INSERT logs,webinar=devops.com value=10
- 20. © 2017 InfluxData. All rights reserved.20 © 2018 InfluxData. All rights reserved.20 Common Configurations to enhance performance ● More Grafana graphs = slower in-browser experience ● Use different graph types ○ Single-stat ○ Templating ○ Annotations ● Collapse rows ○ Avoids rendering of unneeded graphs ○ Saves space; looks cleaner ● Large number of series = slow query builder dropdowns
- 21. © 2017 InfluxData. All rights reserved.21 © 2018 InfluxData. All rights reserved.21 Common Configurations to enhance performance ● InfluxDB does not cache; every query recalculates results ○ When InfluxDB queries give an error… ○ ...don’t hit repeatedly click the refresh button ● Downsample high fidelity data for better performance ○ Continuous queries ■ Periodically runs a user-specified query ■ Saves results back into new retention policy ○ Retention policies ■ Allows data to be expired after a set duration ○ Kapacitor
- 22. © 2017 InfluxData. All rights reserved.22 © 2018 InfluxData. All rights reserved.22 Summary ● Grafana & InfluxDB are great for collecting, storing, and visualizing metrics and events for DevOps monitoring, IoT & Real-Time analytics applications ● Popular Open Source - download it ● Learn more ○ https://docs.influxdata.com/influxdb/latest/ ○ http://docs.grafana.org/ ● Events: ○ Grafanacon in LA February 25-26, 2019 ○ InfluxDays in NYC March 13-14, 2019
- 23. Questions?