How to Optimize Sales Analytics Using 10x the Data at 1/10th the Cost
Download as PPTX, PDF•0 likes•193 views
The document discusses optimizing sales analytics by leveraging vast data resources cost-effectively. It covers techniques for quick data analysis, real-time KPI tracking, and creating a scalable data infrastructure for decision-making at Wayfair. Key concepts include data velocity, self-service tools for data access, and the importance of data literacy among employees.
How to Optimize Sales Analytics Using 10x the Data at 1/10th the Cost
- 1. HOW TO OPTIMIZE SALES ANALYTICS USING 10X THE DATA AT 1/10TH THE COST
- 2. 2 Today, we’ll learn how to ● Perform sales analysis on billions of rows of data ● Add new dimensions and hierarchies for drill-down analysis in seconds ● Build “what-if” analyses using 20x faster using OLAP ● Analyze near real-time data for more accurate KPIs ● Reported consistent KPIs across BI tools
- 3. 3 Today’s Speakers AD of Product, Wayfair @wayfairtech Matt is the Associate Director of Product Management on the Data Infrastructure team at Wayfair where he has worked for over 5 years. Prior to Wayfair, he worked at Verifone and Curb in product management roles. He’s a graduate of Dickinson and holds a Data Science specialization from Johns Hopkins. Matt Hartwig Chief Strategy Officer, AtScale @dmariani Dave is one of the co-founders of AtScale and is currently the Chief Strategy Officer. Prior to AtScale, Dave was VP of Engineering at Klout & at Yahoo! where he built the world's largest multi-dimensional cube for BI on Hadoop. Dave is a Big Data visionary & serial entrepreneur. Dave Mariani
- 4. What is Wayfair? 4 Wayfair is a Clear Leader in Home Goods ~$600B+ total addressable market rapidly moving from brick and mortar to online Utilizing in-house software development capabilities to build and leverage proprietary technology Highly recognized brand in North America and Europe with increasing engagement from repeat customers Partnering with fragmented and largely unbranded supplier base of over 12,000 suppliers Investing in specialized logistics network, international markets, and existing teams to continue outsized share-taking Co-founders are largest shareholders, with focus on sustainable long-term growth, operational discipline, and customer- first orientation
- 5. 5 Optimizing Sales Analytics ~900M Queries/Dashboard loads yearly It’s a... Data Driven Business ~250M By People Analysts BI Developers Data Scientists By hundreds of
- 6. 6 Optimizing Sales Analytics What do I do? Data Infrastructure Team applications/users Data Platform Access Enrich Store Product Merch Ad Tech Storefront Operations BI/ DS core data platform We provide application datastores, data movement, and analytics & data science tools to enable developers and analysts across Wayfair to store, secure, enrich, and present data.
- 7. 7 Optimizing Sales Analytics What is Velocity: Velocity is how we talk about speed and scale in all things at Wayfair - design, development, decision making, etc. It’s not enough to grow today; it’s about building our growth in a sustainable way that enables continued momentum. What about for Data: The speed with which Wayfair can go from data collection to driving business decisions, outcomes and insights.
- 8. 8 Optimizing Sales Analytics Storefront Analysis Decision A customer clicks on a product page but doesn’t proceed to purchase An analyst identifies that we’re seeing lower conversion rate after a recent deploy We roll back that recent deploy and see conversion rate recover to previous baseline Now do this better and faster on repeat with ever increasing system complexity, size, and organizational sprawl. That is Data Velocity.
- 9. 9 Optimizing Sales Analytics Big data.. Typical Problems 1 Data Everywhere Existing data warehouse and data lake systems store hundreds of thousands of data sets, many of which were copies of one another and not intended for others to use. Hard to find what you need. Long lead times Scaling on-premise infrastructure had long lead times, challenges with physical hardware, power/network constraints. 2 3 Fragmented Tool Space Mix of legacy BI tools, relational databases, and open-source big data tooling. Fragmented IAM Patchwork access control, no central identity provider. Employees often stuck in ticket hell. Rapid Data Volume Growth Over 100% YoY growth in both data volume produced and data accessed. 4 5
- 10. 10 A lot goes into solving this at size and scale Data Curation / Transformation: That data is further enriched, transformed, and curated downstream. Often to power decision support and business intelligence systems but also other software apps. Application Data Exchange: Data needs to flow from production applications into many downstream processes across software, analytics, and data science. Self Service Tooling: Once data is curated and enriched, it need to be accessible through self-service BI Tools that enable uniform and equal access to data at Wayfair. Data Literacy: Every employee at Wayfair needs to be empowered to make data informed decisions through training and support. Employees need opportunities to develop their data instincts. Scalable Infrastructure: At the base layer is infrastructure that can power the exchange, enrichment, and access of our data at increased and accelerating scale. The Pillars of Data Velocity at Wayfair
- 11. 11 A lot goes into solving this at size and scale Data Curation / Transformation: That data is further enriched, transformed, and curated downstream. Often to power decision support and business intelligence systems but also other software apps. Application Data Exchange: Data needs to flow from production applications into many downstream processes across software, analytics, and data science. Self Service Tooling: Once data is curated and enriched, it need to be accessible through self-service BI Tools that enable uniform and equal access to data at Wayfair. Data Literacy: Every employee at Wayfair needs to be empowered to make data informed decisions through training and support. Employees need opportunities to develop their data instincts. Scalable Infrastructure: At the base layer is infrastructure that can power the exchange, enrichment, and access of our data at increased and accelerating scale. The Pillars of Data Velocity at Wayfair
- 12. 12 Our transformation is underway
- 13. 13 How To Optimize Sales Analytics Using 10X the Data at 1/10th the Cost Dave Mariani, Founder and Chief Strategy Officer, AtScale
- 14. The Cloud Analytics Stack 14 COMPONENT CONSUMPTION VISUALIZATION, ANALYSIS, REPORTING SEMANTIC LAYER QUERY ACCESS, FILTERING, MASKING, AUDITING PREPARED DATA DATA PROCESSING, MODELING RAW DATA DATA STORAGE, ENCRYPTION DATA TRANSFORMATION ETL,MERGING, AGGREGATION LAYER (FUNCTION) BI Tools AI/ML Tools Applications Multi-dimensional Engine Data Governance Engine Virtualization Engine Data Warehouse File Access Engine ETL Engine File System (Data Lake) Data Catalog
- 15. Today’s Use Case 15 Using Excel, create a model that will forecast inventory quantities for the 2020-Q4 using SafeGraph’s foot traffic data
- 16. 16 Step 1 Load Foot Traffic Data & Sales History
- 17. Challenge #1: Data Integration is Slow & Cumbersome 17 DEMOED SOLUTION Leverage data virtualization to access data quickly & easily ALTERNATIVES 1. Build a data pipeline using tools like Hive, Databricks, etc. 2. Use ETL/ELT tools like Informatica, Talend, Matillion, etc.
- 18. 18 Step 2 Create an Excel Model to Forecast Sales for 2020-Q4
- 19. Challenge #2: Complex Calculations are Hard to Share 19 DEMOED SOLUTION Leverage OLAP & MDX to compute calculations server-side ALTERNATIVES 1. Use Excel spreadsheets to compute cell-based calculations 2. Use advanced SQL functions to calculate metrics
- 20. 20 Step 3 Refresh Forecast for 2021-Q1
- 21. Challenge #3: Getting Up to Date Data is Slow & Manual 21 DEMOED SOLUTION Leverage Time Relative functions & direct connections to data ALTERNATIVES 1. Update data manually by repeating data preparation 2. Build logic & data prep into a custom application
- 22. Summary 22 ▵ Leverage virtualization to deliver faster time to insight ▵ Leverage OLAP to share “single source of truth” calculations ▵ Leverage “live” (direct) data connections to reduce data latency ▵ Build upon a cloud-based, scalable data platform
- 23. www.atscale.com
Editor's Notes
- #15: AtScale is built to leverage the efficiencies and performance of the cloud for the data consumer whether you’re on premise or in the cloud (or both). We connect people to data. We do that without moving data and without complexity—leveraging existing investments in big data platforms, applications and tools. We also do that consistently, securely and with one set of semantics—and without interrupting existing data usage so that data workers no longer have to understand how or where it is stored. Performance Optimizing performance is difficult and that’s where we focus our energies. AtScale’s data warehouse virtualization can reduce queries performance from 5 weeks to 5 seconds—automatically optimizing each time a user queries the database. Security Because we haven’t copied the data and applied new code or embedded rules, we’ve reduced the amount of complexity and maintain consistent data lineage throughout the data lifecycle. AtScale not only leverages existing data security and governance but applies an additional layer so that data can be ported to new data tools, applications and platforms. Agility What’s more powerful is we create simple interface to querying data and building models for data science and analytics data workers with deep integrations with BI and AI/ML tools. For the first time, users (and IT) have visibilities into how data is being queried and used throughout the organization (no more data silos).
- #16: Today we'll show you how to increase your data velocity to report on sales. This will include reporting on billions of rows of data in a popular BI tool that can be used across the business and performs at conversational speeds