Alluxio Monthly Webinar | Simplify Data Access for AI in Multi-Cloud
0 likes146 views
Alluxio Monthly Webinar May. 14, 2024 For more Alluxio Events: https://www.alluxio.io/events/ Speaker: - ChanChan Mao (Developer Advocate, Alluxio) - Bin Fan (VP of Technology, Alluxio) Running AI/ML workloads in different clouds present unique challenges. The key to a manageable multi-cloud architecture is the ability to seamlessly access data across environments with high performance and low cost. This webinar is designed for data platform engineers, data infra engineers, data engineers, and ML engineers who work with multiple data sources in hybrid or multi-cloud environments. Chanchan and Bin will guide the audience through using Alluxio to greatly simplify data access and make model training and serving more efficient in these environments. You will learn: - How to access data in multi-region, hybrid, and multi-cloud like accessing a local file system - How to run PyTorch to read datasets and write checkpoints to remote storage with Alluxio as the distributed data access layer - Real-world examples and insights from tech giants like Uber, AliPay and more
Ad
More Related Content
Similar to Alluxio Monthly Webinar | Simplify Data Access for AI in Multi-Cloud (20)
More from Alluxio, Inc. (20)
Ad
Recently uploaded (20)
![Get & Download Wondershare Filmora Crack Latest [2025]](https://cdn.slidesharecdn.com/ss_thumbnails/revolutionizingresidentialwi-fi-250422112639-60fb726f-250429170801-59e1b240-thumbnail.jpg?width=560&fit=bounds)
Ad
Alluxio Monthly Webinar | Simplify Data Access for AI in Multi-Cloud
- 1. Simplify Data Access for AI in Multi-Cloud Bin Fan, VP of Technology @ Alluxio - [email protected] ChanChan Mao, Developer Advocate @ Alluxio - [email protected] 1 Optimizing PyTorch Training and Serving in Practice
- 2. VP of Technology @ Alluxio Bin Fan Developer Advocate @ Alluxio ChanChan Mao
- 3. 3 Letʼs recap the Multi-Cloud series so far… Why a Multi-Cloud Strategy Matters for Your AI Platform ● A multi-cloud architecture allows organizations to leverage different cloud services to meet diverse workload demands while maximizing efficiency, reducing costs, and avoiding vendor lock-in. Architecting Data Platform Across Regions & Clouds ● Efficient data access is key to a successful heterogeneous data platform, whether your data is distributed across multiple datacenters and/or clouds. Cloud-Native Model Training on Distributed Data ● The evolution of the modern data stack created challenges for data locality in the multi-region/cloud ML pipeline - resulting in high latency, expensive cloud costs, and lack of reliability. 01 02 03
- 4. ALLUXIO 4 Data Access Patterns of AI/ML 4
- 7. Cloud Data Access Patterns: Multi-Cloud/Multi-Region Data Access 7
- 8. Hybrid/Multi-Cloud ML Platforms Online ML platform Serving cluster Models Training Data Models 1 2 3 Offline training platform Training cluster DC/Cloud A DC/Cloud B 8 Separation of compute and storage
- 9. Challenges of Data Access 1. Performance ● Pulling data from cloud storage is hurting training/serving. 2. Cost ● Repeatedly requesting data from cloud storage is costly. 3. Reliability ● Availability is the key for every service in cloud. 4. Usability ● Manual data management is unfavorable. 9
- 10. Data access: 1. Read data directly from cloud storage 2. Copy data from cloud to local before training 3. Local cache layer for data reuse 4. Distributed cache system Model access: 1. Pull models directly from cloud storage Existing Solutions 10
- 11. Option 1: Read From Cloud Storage ● Easy to set up ● Performance are not ideal ■ Model access: Models are repeatedly pulled from cloud storage ■ Data access: Reading data can take more time than actual training 82% of the time spent by DataLoader 11
- 12. Option 2: Copy Data To Local Before Training ● Data is now local ■ Faster access + less cost ● Management is hard ■ Must manually delete training data after use ● Local storage space is limited ■ Dataset is huge - limited benefits 12
- 13. Option 3: Local Cache for Data Reuse Examples: S3FS built-in local cache, Alluxio Fuse SDK ● Reused data is local ■ Faster access + less cost ● Cache layer provider helps data management ■ No manual deletion/supervision ● Cache space is limited ■ Dataset is huge - limited benefits 13
- 14. Option 4: Distributed Cache System Clients Worker Worker Worker … ● Training data and trained models can be kept in cache - distributed. ● Typically with data management functionalities. 14
- 15. ALLUXIO 15 Alluxio as an example 15
- 16. Clients Worker Worker … Masters Worker ● Use consistent hashing to cache both data and metadata on workers. ● Worker nodes have plenty space for cache. Training data and models only need to be pulled once from cloud storage. Cost -- ● No more single point of failure. Reliability ++ ● No more performance bottleneck on masters. Performance ++ ● Data management system. Consistent Hashing for caching 16
- 17. By the numbers ● High Scalability ■ One worker supports 30 - 50 million files ■ Scale linearly - easy to support 10 billions of files ● High Availability ■ 99.99% uptime ■ No single point of failure ● High Performance ■ Faster data loading ● Cloud-native K8s Operator and CSI-FUSE for data access management 17
- 18. Alluxio FUSE ● Expose the Alluxio file system as a local file system. ● Can access the cloud storage just as accessing local storage. ○ cat, ls ○ f = open(“a.txt”, “r”) ● Very low impact for end users 18
- 19. Alluxio CSI x Alluxio FUSE for Data Access ● FUSE: Turn remote dataset in cloud into local folder for training ● CSI: Launch Alluxio FUSE pod only when dataset is needed Alluxio Fuse pod Fuse Container Host Machine Application pod Application Container Persistent volume + claim mount mount 19
- 20. ALLUXIO 20 Data Access Management for PyTorch 20
- 21. Under Storage Integration with PyTorch Training (Alluxio) Training Node Get Task Info Alluxio Client PyTorch Get Cluster Info Send Result Cache Cluster Service Registry Cache Worker Cache Worker Execute Task Cache Worker Cache Client Find Worker(s) Affinity Block Location Policy Client-side load balance 1 2 3 4 5 Cache miss - Under storage task 21
- 22. Data Loading Performance ImageNet (subset) 22 Yelp review
- 23. 23 Training Directly from Storage (S3-FUSE) - > 80% of total time is spent in DataLoader - Result in Low GPU Utilization Rate (<20%) GPU Utilization Improvement
- 24. Training with Alluxio-FUSE - Reduced DataLoader Rate from 82% to 1% (82X) - Increase GPU Utilization Rate from 17% to 93% (5X) GPU Utilization Improvement
- 25. ALLUXIO 25 How to enable Python Applications 25
- 26. Use Alluxio - Ray Integration as an example 26 Ray Dataloader fsspec - Alluxio impl Alluxio Python client Ray etcd Alluxio Worker REST API server Alluxio Worker REST API server PyArrow Dataset loading Registration Get worker addresses
- 27. Alluxio+Ray Benchmark – Small Files ● Dataset ○ 130GB imagenet dataset ● Process Settings ○ 4 train workers ○ 9 process reading ● Active Object Store Memory ○ 400-500 MiB 27
- 28. Alluxio+Ray Benchmark – Large Parquet files ● Dataset ○ 200MiB files, adds up to 60GiB ● Process Settings ○ 28 train workers ○ 28 process reading ● Active Object Store Memory ○ 20-30 GiB 28
- 29. Cost Saving – Egress/Data Transfer Fees 29
- 30. Cost Saving – API Calls/S3 Operations (List, Get) List/Get API calls only access Alluxio 30
- 32. Alluxio Benefits 30-50% 90% + Reduce 30%+ time compare consuming from Cloud object storage Manage the on-going training dataset from cold storage Alluxio server data to GPU with advanced caching capability Avoid 50%+ data copy Stable GPU utilization no matter where you start GPU cluster Virtual layer to different storage Use case Autonomous driving
- 33. 33
- 34. Any Questions? Scan the QR code for a Linktree including great learning resources, exciting meetups & a community of data & AI infra experts! 34
- 35. Thank you! 35 Up Next: Awesome AI Dev Tools - Mon May 20 @ Github, SF https://lu.ma/s2ghbk5i Speak at an Alluxio event: https://forms.gle/iJX9GTMaAVQdzKc28