Propelling IoT Innovation with Predictive Analytics
0 likes318 views
The document discusses the shift towards an on-demand economy, emphasizing the need for enterprises to transition from batch processing to real-time operations to meet consumer expectations. It highlights the benefits of in-memory databases, which store data in memory rather than on disk, enabling faster data processing and analytics. Additionally, it introduces Powerstream, a predictive analytics tool for monitoring wind turbine health in real-time.
Propelling IoT Innovation with Predictive Analytics
- 1. (c) Nikita Shamgunov and MemSQL
- 2. Topics • The On-Demand Economy • From In-Memory Compu8ng to In-Memory Databases • Renewable Energy and PowerStream • Demo and Q&A (c) Nikita Shamgunov and MemSQL
- 3. The On-Demand Economy (c) Nikita Shamgunov and MemSQL
- 4. We live in an on-demand economy (c) Nikita Shamgunov and MemSQL
- 5. Consumers are condi.oned to instant services, like Uber, Stripe, and Airbnb (c) Nikita Shamgunov and MemSQL
- 6. Where does that leave enterprises? (c) Nikita Shamgunov and MemSQL
- 7. Racing to meet internal and external expecta1ons for speed and personaliza,on (c) Nikita Shamgunov and MemSQL
- 8. Batch processing is the enterprise enemy (c) Nikita Shamgunov and MemSQL
- 9. Enterprises must move from overnight to Real-&me, intra-day opera&ons (c) Nikita Shamgunov and MemSQL
- 10. Harnessing data in real -me (c) Nikita Shamgunov and MemSQL
- 11. Achieving sub 100 ms latency • Real-'me monitoring and analy'cs on streaming video • Proac'vely diagnose issues in real-'me • Deliver be9er viewer experience (c) Nikita Shamgunov and MemSQL
- 12. Massive Ingest AND Analy1cs • Instant accuracy to the latest repin • Build real-5me analy5c applica5ons • 1 GB/sec totaling 72 TB/day (c) Nikita Shamgunov and MemSQL
- 13. What is an in-memory database? (c) Nikita Shamgunov and MemSQL
- 14. In-Memory Databases... • Use memory instead of disk (c) Nikita Shamgunov and MemSQL
- 15. In-Memory Databases... • Use memory instead of disk (c) Nikita Shamgunov and MemSQL
- 16. In-Memory Databases... • Use memory instead of disk • Do not (need to) save data on disk (c) Nikita Shamgunov and MemSQL
- 17. In-Memory Databases... • Use memory instead of disk • Do not (need to) save data on disk (c) Nikita Shamgunov and MemSQL
- 18. In-Memory Databases... • Use memory instead of disk • Do not (need to) save data on disk • Put the whole dataset in memory (c) Nikita Shamgunov and MemSQL
- 19. In-Memory Databases... • Use memory instead of disk • Do not (need to) save data on disk • Put the whole dataset in memory (c) Nikita Shamgunov and MemSQL
- 20. In-Memory Databases... • Use memory instead of disk • Do not (need to) save data on disk • Put the whole dataset in memory Well, some)mes... (c) Nikita Shamgunov and MemSQL
- 21. Wikipedia says... In-memory databases primarily rely on main-memory for storage. (c) Nikita Shamgunov and MemSQL
- 22. In-Memory Databases • Are durable to disk (and respect ACID) (c) Nikita Shamgunov and MemSQL
- 23. In-Memory Databases • Are durable to disk (and respect ACID) • Can spill on disk or pin data in-memory (and take advantage of it) (c) Nikita Shamgunov and MemSQL
- 24. In-Memory Databases • Are durable to disk (and respect ACID) • Can spill on disk or pin data in-memory (and take advantage of it) • Tradeoffs are suited to systems with lots of memory (c) Nikita Shamgunov and MemSQL
- 25. In-Memory Databases • Are durable to disk (and respect ACID) • Can spill on disk or pin data in-memory (and take advantage of it) • Tradeoffs are suited to systems with lots of memory • Tend to be distributed systems (c) Nikita Shamgunov and MemSQL
- 26. In-Memory Databases • Are durable to disk (and respect ACID) • Can spill on disk or pin data in-memory (and take advantage of it) • Tradeoffs are suited to systems with lots of memory • Tend to be distributed systems • Have a different set of boClenecks (c) Nikita Shamgunov and MemSQL
- 27. Bold Claim (c) Nikita Shamgunov and MemSQL
- 28. All database workloads will be running on in-memory databases (c) Nikita Shamgunov and MemSQL
- 29. Why? • Memory is ge,ng cheaper (about 40% every year) (c) Nikita Shamgunov and MemSQL
- 30. Why? • Memory is ge,ng cheaper (about 40% every year) • Cache is the new RAM (RAM is the new disk, disk is the new tape, etc) (c) Nikita Shamgunov and MemSQL
- 31. Why? • Memory is ge,ng cheaper (about 40% every year) • Cache is the new RAM (RAM is the new disk, disk is the new tape, etc) • In-memory databases leverage SSD (no random writes) (c) Nikita Shamgunov and MemSQL
- 32. Why? • Memory is ge,ng cheaper (about 40% every year) • Cache is the new RAM (RAM is the new disk, disk is the new tape, etc) • In-memory databases leverage SSD (no random writes) • NVRAM is coming (and could be cheaper than SSD) (c) Nikita Shamgunov and MemSQL
- 33. Why? • Memory is ge,ng cheaper (about 40% every year) • Cache is the new RAM (RAM is the new disk, disk is the new tape, etc) • In-memory databases leverage SSD (no random writes) • NVRAM is coming (and could be cheaper than SSD) In-memory databases are tuned to modern hardware and modern workloads (c) Nikita Shamgunov and MemSQL
- 34. Renewable Energy in the News (c) Nikita Shamgunov and MemSQL
- 35. (c) Nikita Shamgunov and MemSQL
- 36. (c) Nikita Shamgunov and MemSQL
- 37. (c) Nikita Shamgunov and MemSQL
- 38. Introducing PowerStream (c) Nikita Shamgunov and MemSQL
- 39. MemSQL PowerStream Predic'ng the global health of wind turbines (c) Nikita Shamgunov and MemSQL
- 40. (c) Nikita Shamgunov and MemSQL
- 41. (c) Nikita Shamgunov and MemSQL
- 42. (c) Nikita Shamgunov and MemSQL
- 43. Streamliner Architecture for PowerStream (c) Nikita Shamgunov and MemSQL
- 44. (c) Nikita Shamgunov and MemSQL
- 45. (c) Nikita Shamgunov and MemSQL
- 46. Live Demo(c) Nikita Shamgunov and MemSQL
- 47. Demo Sequence • Powerstream user interface • Showcase largest windfarms • Real-8me simula8ons • Witness live opera8ons • Ease of new pipeline setup • Ka>a subscrip8on (c) Nikita Shamgunov and MemSQL
- 48. Enabling predic.ve analy.cs • Use exis(ng models from SAS • Create models in Spark MLlib • Predic(ve scoring as part of the pipeline (c) Nikita Shamgunov and MemSQL
- 49. From a Real-Time Dashboard to Predic5ve Applica5ons (c) Nikita Shamgunov and MemSQL
- 50. (c) Nikita Shamgunov and MemSQL
- 51. (c) Nikita Shamgunov and MemSQL
- 52. (c) Nikita Shamgunov and MemSQL