Introducing workload analysis
1 like1,088 views
The document discusses workload analysis, emphasizing its importance for gaining insights into database usage, optimizing resource allocation, and maintaining quality of service. It outlines modern methods, including gathering metrics and applying machine learning for real-time monitoring and predictive alerts. The SKySQL app is highlighted as a tool for visualizing database workloads and historical trends, aiding in proactive database management.
Introducing workload analysis
- 1. Introducing workload analysis Shane K Johnson Senior Director of Product Marketing MariaDB Corporation 1
- 2. Workload analysis – why 2 ● Gain deeper insights into database usage ● Optimize resource allocation ○ Reduce costs, improve performance ○ Rinse and repeat (e.g., day vs. night, weekday vs. weekend) ● Take proactive measures (vs. reactive) ● Maintain quality of service (QoS) ● Build a foundation for autonomous services
- 3. Workload analysis – definition ● Categories ○ Transactional vs. analytical ○ Read vs. write vs. mixed ○ Too simplistic ● Discrete queries ○ Which ones to optimize for? ○ Which ones will be hurt? ○ Too many different queries Conventional 3 ● Resource based ● Database state ● Identifiable ● Time bound ○ Cycles and patterns ○ Evolution ● Statistical ○ Distributions ○ Properties Modern
- 4. Workload analysis – insight ● Is the workload changing? ● Are workload changes getting smaller or bigger? ● Do workload changes justify further resource optimization? ● How are workload changes impacting the business? 4
- 5. Workload analysis – application ● Most important metrics ● Define the workload ● Strong correlation ● Change with/to workload ● Learned by WLA Critical metrics 5 ● Time intervals ● Temporal changes ● Trends and spikes Historical context
- 6. Workload analysis – coming next ● Dynamic vs. static ● Per workload vs. global ● Based on change ○ Similarity index ○ Rate, distribution, spread ● No more ○ Manual analysis ○ Needle in a haystack ● Personalized health checks Proactive monitoring ● Maintaining consistency ● Learned QoS metrics ● Predictive alerts ○ Or, autonomous changes Quality of Service (QoS) 6
- 8. Workload analysis 8 ● SkySQL app that lets users explore database workloads that were automatically detected by our Machine Learning platform. ● It gives easy access to interactive visualizations to help users understand how database workloads change over time.
- 9. Machine learning pipeline 9 1. Collect database metrics at 5-sec intervals 2. Extract data from Monitor repository, on an hourly basis 3. Preprocess data to reduce ”noise” and strongly correlated metrics 4. Apply Deep Learning to create working tensor a. 2000+ sample data points, 600+ model steps b. Approximately 100+ critical features 5. Cluster the matrix into workloads that exhibit similar behavior 6. Visualize via D3
- 10. Daily max over time 10 Visualize changes in the daily maximum values of 100+ metrics, making it easy to identify historical trends and recurring patterns
- 11. Correlated metrics 11 Visualize the collective impact of correlated metrics, identified by deep-learning, on all database workloads (i.e., metrics that change together).
- 12. Distribution impact 12 Visualize the spread and distribution of metrics so DBAs can anticipate and optimize resources usage like memory for performance
- 13. Metric relationships 13 Ability to pair any of the 100+ critical features to see how they change relative to each other.
- 14. DEMO 14