AI/ML Infra Meetup | RAYvolution - The Last Mile: Mastering AI Deployment with Ray

Jan 24, 20250 likes215 views

AI/ML Infra Meetup Jan. 23, 2025 Organized by Alluxio For more Alluxio Events: https://www.alluxio.io/events/ Speaker: - Robert Nishihara (Co-Founder @ Anyscale) You won't want to miss this talk presented by Robert Nishihara, Co-Founder of Anyscale, which is packed with insights on using Ray to conquer the last-mile challenges in AI deployment.

Three Developments in AI Infra
Robert Nishihara
Co-creator of Ray, Co-founder of Anyscale

My Background
2013: Began PhD in AI
2016: Started Ray along with Philipp Moritz and Ion Stoica
2019: Started Anyscale to commercialize Ray

AI/ML Infra Meetup | RAYvolution - The Last Mile: Mastering AI Deployment with Ray

Three Industry Trends
How the world has changed since we started Ray

Three Industry Trends
Scaling Laws
● Larger models, more compute, more data → Better results
● Top priority for many AI infra teams: “enable training on 100x more data”
● ML platform / infra teams are on the critical path
● Scaling data -> scaling compute

This document provides an overview and introduction to big data. It discusses the technical challenges of big data including issues of volume, variety, velocity and veracity. It also discusses solutions like Hadoop, MapReduce, and big data databases. Additionally, it covers big data analytics including different levels of analytics maturity and techniques like data mining, machine learning, and predictive analytics. Finally, it provides resources for learning more about big data including online courses, sandbox environments, open source tools, and public datasets.

Hadoop and SAP BI Praveen Kumar (Tyagi)

This document provides an overview of big data and Hadoop. It discusses that Hadoop is a software framework for distributed processing of large datasets across clusters of computers. It can handle terabytes or petabytes of data stored on hundreds or thousands of nodes. The document also summarizes key Hadoop concepts like HDFS for storage, MapReduce for processing, and YARN for resource management. It highlights top reasons for using big data and Hadoop like scalability, flexibility to handle all data formats, and ability to use inexpensive commodity hardware. Finally, it presents a typical big data architecture showing data sources, storage, processing, visualization, and machine learning capabilities.

data analytics lecture2.pptxNamrataBhatt8

This document discusses big data analytics. It covers the four V's of big data (volume, variety, velocity, and veracity), drivers of big data, an introduction to big data analytics including what it is and key technologies used, benefits including improved customer service and decision making, and applications in various industries like healthcare, manufacturing, and banking. The learning objectives are to understand the four V's of big data, drivers of big data, what big data analytics is, and its benefits and applications.

Information Security AnalyticsAmrit Chhetri

This document summarizes Amrit Chhetri's presentation on information security analytics. The presentation covers topics such as big data introduction and history, advantages of big data solutions, trends in big data analytics, big data adoption trends, big data software stacks, platforms for big data analytics, programming platforms for big data, and applications of big data analytics in telecommunications and information security. It also includes a demonstration of running a word count example using MapReduce on a big data platform and configuring a Cloudera big data platform.

Big data Introduction by MohanVenkata Reddy Konasani

This document provides an introduction to big data and Hadoop. It discusses the three V's of big data: volume, variety, and velocity. Examples are given of the large amounts of data generated daily from various sources. The growth and market opportunity for big data technologies is also discussed. Common use cases for big data in different industries are outlined. The document then covers Hadoop components and how Hadoop HDFS and MapReduce work. Other Hadoop technologies like Hive, Pig, and Zookeeper are introduced. Benefits of Hadoop and commercial Hadoop distributions are summarized. Finally, technologies alternative to Hadoop like HPCC and SAP HANA are briefly described.

Problem Definition muAoPS | Analytics Problem Solving | Mu Sigman40077943

Driving Customer Loyalty with Azure Machine LearningCCG

Learn how you can leverage the elastic, on-demand processing power of Microsoft Azure to create faster, more applicable analytics by viewing this informative webinar. Data Scientist and Author, Ahmed Sherif, demonstrates key analytic use cases that can be spun up quickly with minimal effort and maximum return on investment. To watch the full recording of this webinar, visit http://ccgbi.com/resources/webinars/driving-customer-loyalty-with-AML

SurveyAnalytics MaxDiff WebinarQuestionPro

This document provides an overview and agenda for a webinar on using MaxDiff analysis to design products people want to buy. It introduces the speakers, Chris Robson from Parametric Marketing and Esther LaVielle from SurveyAnalytics. The agenda includes an introduction to MaxDiff analysis, a demonstration of building a MaxDiff survey in SurveyAnalytics, reviewing MaxDiff reporting tools, and a Q&A session.

SurveyAnalytics MaxDiff Webinar SlidesQuestionPro

Big Data Developer Career Path: Job & Interview PreparationIntellipaat

Big Data - A Real Life RevolutionCapgemini

Cloud as a Data PlatformAndrei Savu

Big Data LDN 2018: THE PATH TO ENTERPRISE AI: TALES FROM THE FIELDMatt Stubbs

Date: 14th November 2018 Location: AI Lab Theatre Time: 11:50 - 12:20 Speaker: Romain Fouache Organisation: Dataiku About: Enterprise AI is a target state where every business process is AI-augmented and every employee is an AI beneficiary. But is that really attainable? And, if so, what is the path to get there? In this talk, Kurt Muehmel, VP Sales Engineering at Dataiku, will share learnings from the field, describing how companies of different sizes and across different sectors have begun this journey. Some are farther along than others, and by making the right decisions now and avoiding stumbling blocks, you can to supercharge your quest to this AI-fuelled future.

R for SAS Users Complement or Replace Two StrategiesRevolution Analytics

Big Data ChallengesArtem Rodichev

Advanced Analytics: Going From Big Data to Big Answersajayc47

Why Everything You Know About bigdata Is A LieSunil Ranka

Gse uk-cedrinemadera-2018-sharedcedrinemadera

Big data analytics presented at meetup big data for decision makersRuhollah Farchtchi

What Managers Need to Know about Data ScienceAnnie Flippo

A6 big data_in_the_cloudDr. Wilfred Lin (Ph.D.)

This document discusses how big data and analytics are moving from on-premises data warehouses to hybrid cloud environments that leverage technologies like Hadoop, Spark, and machine learning. It provides examples of how Oracle is helping customers with this transition by offering big data cloud services that give them flexibility to run workloads both on-premises and in the cloud while simplifying data management and enabling new types of advanced analytics.

Agile data scienceJoel Horwitz

Agile Data Science is a lean methodology that is adopted from Agile Software Development. At the core it centers around people, interactions, and building minimally viable products to ship fast and often to solicit customer feedback. In this presentation, I describe how this work was done in the past with examples. Get started today with our help by visiting http://www.alpinenow.com

Data Science as a Commodity: Use MADlib, R, & other OSS Tools for Data Scienc...Sarah Aerni

Slides from the Pivotal Open Source Hub Meetup "Data Science as a Commodity: Use MADlib, R, & other OSS Tools for Data Science!" As the need for data science as a key differentiator grows in all industries, from large corporations to startups, the need to get to results quickly is enabled by sharing ideas and methods in the community. The data science team at Pivotal leverages and contributes to this community of publicly available and open source technologies as part of their practice. We will share the resources we use by highlighting specific toolkits for building models (e.g. MADlib, R) and visualization (e.g. Gephi and Circos) along with their benefits and limitations by sharing examples from Pivotal's data science engagements. At the end of this session we hope to have answered the questions: Where can I get started with Data Science? Which toolkit is most appropriate for building a model with my dataset? How can I visualize my results to have the greatest impact? Bio: Sarah Aerni is a member of the Pivotal Data Science team with a focus on healthcare and life science. She has a background in the field of Bioinformatics, developing tools to help biomedical researchers understand their data. She holds a B.S. In Biology with a specialization in Bioinformatics and minor in French Literature from UCSD, and an M.S. and Ph.D in Biomedical Informatics from Stanford University. During her time as a researcher she focused on the interface between machine learning and biology, building computational models enabling research for a broad range of fields in biomedicine. She also co-founded a start-up providing informatics services to researchers and small companies. At Pivotal she works with customers in life science and healthcare building models to derive insight and business value from their data.

Unit-I_Big data life cycle.pptx, sources of Big DataRajendraKankrale1

Maxdiff webinar_10_19_10QuestionPro

02 a holistic approach to big dataRaul Chong

Digital Transformation: How to Run Best-in-Class IT Operations in a World of ...Precisely

IT leaders looking to move beyond reactive and ad hoc troubleshooting need to find the intersection of maintaining existing systems while still driving innovation - solving for the present while preparing for the future. Identifying ways to bring existing infrastructure and legacy systems into the modern world can create the business advantage you need. View the conversation with Splunk’s Chief Technology Advocate, Andi Mann and Syncsort’s Chief Product Officer, David Hodgson where we discuss the digital transformation taking place in IT and how machine learning and AI are helping IT leaders create a more business-centric view of their world including: • The importance of data sharing and collaboration between mainframe and distributed IT • The value of integrating legacy data sources and existing infrastructure into the modern world • Achieving an end to end view of IT operations and application performance with machine learning

Big Data Matching - How to Find Two Similar Needles in a Really Big HaystackPrecisely

When consolidating multiple sources of information from across your organization, how do you find the records that relate to the same customer, the same company or the same product? This is the challenge faced by many businesses today when putting a data lake to work. The problem is made far worse when different systems may not have the same contact entered the same way. Is Bob Smith the same as Robert Smith? How about Dr. Robert L. Smith - is he the same person? What about Syncsort, Inc and Sinksort Corp.? Are those the same company? One must compare each individual record to every other record in the dataset with some very sophisticated matching algorithms to determine who is who, and you may have to compare the data multiple times in multiple ways to resolve each entity. Just to add to the difficulty, let’s say your organization has very large volumes of records in your data lake - you don’t have to compare a thousand records to a thousand other records multiple times - you must compare a million to a million, or 100 million to 100 million. This kind of compute intensive comparison can bring even a powerful cluster to its knees. This is a problem Syncsort customers must solve, and we have developed some very powerful and intelligent software to tackle it. View this presentation as we discuss the challenges of entity resolution at scale, how Syncsort’s Trillium data quality software line has tackled them successfully in production clusters and see a demonstration of this software in action.

How Coupang Leverages Distributed Cache to Accelerate ML Model TrainingAlluxio, Inc.

Alluxio Tech Talk Webinar Apr. 22, 2025 Organized by Alluxio For more Alluxio Events: https://www.alluxio.io/events/ Speaker: - Hyun Jung Baek (Staff Backend Engineer @ Coupang) Description Coupang is a leading e-commerce company in South Korea, with over 50,000 employees and $20+ billion in annual revenue. Coupang's AI platform team builds and manages a large-scale AI platform in AWS for machine learning engineers to train models that enhance and customize product search results and product recommendations for its 100+ million customers. As the search and recommendation models evolve, optimizing the underlying infrastructure for AI/ML workloads is essential for the e-commerce business. Coupang's platform team actively sought to improve their model training pipeline to boost machine learning engineers' productivity, publish models to production faster, and reduce operational costs. Coupang focused on addressing several key areas: - Shortening data preparation and model training time - Improving GPU utilization in training clusters in different regions - Reducing S3 API and egress costs incurred from copying large training datasets across regions - Simplifying the operational complexity of storage system management In this tech talk, Hyun Jung Baek, Staff Backend Engineer at Coupang, will share best practices for leveraging Alluxio to power search and recommendation model training infrastructure. Hyun will discuss: - How Coupang builds a world-class large-scale AI platform for machine learning engineers to deliver better search and recommendation models - How adding distributed caching to their multi-region AI infrastructure improves GPU utilization, accelerates end-to-end training time, and significantly reduces cross-region data transfer costs. - How to simplify platform operations and to easily deploy the same architecture to new GPU clusters.

Alluxio Webinar | Inside Deepseek 3FS: A Deep Dive into AI-Optimized Distribu...Alluxio, Inc.

Alluxio Webinar Apr 1, 2025 For more Alluxio Events: https://www.alluxio.io/events/ Speaker: Stephen Pu (Staff Software Engineer @ Alluxio) Deepseek’s recent announcement of the Fire-flyer File System (3FS) has sparked excitement across the AI infra community, promising a breakthrough in how machine learning models access and process data. In this webinar, an expert in distributed systems and AI infrastructure will take you inside Deepseek 3FS, the purpose-built file system for handling large files and high-bandwidth workloads. We’ll break down how 3FS optimizes data access and speeds up AI workloads as well as the design tradeoffs made to maximize throughput for AI workloads. This webinar you’ll learn about how 3FS works under the hood, including: ✅ The system architecture ✅ Core software components ✅ Read/write flows ✅ Data distribution/placement algorithms ✅ Cluster/node management and disaster recovery Whether you’re an AI researcher, ML engineer, or infrastructure architect, this deep dive will give you the technical insights you need to determine if 3FS is the right solution for you.

More Related Content

Similar to AI/ML Infra Meetup | RAYvolution - The Last Mile: Mastering AI Deployment with Ray (20)

SurveyAnalytics MaxDiff Webinar SlidesQuestionPro

Big Data Developer Career Path: Job & Interview PreparationIntellipaat

Big Data - A Real Life RevolutionCapgemini

Cloud as a Data PlatformAndrei Savu

Big Data LDN 2018: THE PATH TO ENTERPRISE AI: TALES FROM THE FIELDMatt Stubbs

R for SAS Users Complement or Replace Two StrategiesRevolution Analytics

Big Data ChallengesArtem Rodichev

Advanced Analytics: Going From Big Data to Big Answersajayc47

Why Everything You Know About bigdata Is A LieSunil Ranka

Gse uk-cedrinemadera-2018-sharedcedrinemadera

Big data analytics presented at meetup big data for decision makersRuhollah Farchtchi

What Managers Need to Know about Data ScienceAnnie Flippo

A6 big data_in_the_cloudDr. Wilfred Lin (Ph.D.)

Agile data scienceJoel Horwitz

Data Science as a Commodity: Use MADlib, R, & other OSS Tools for Data Scienc...Sarah Aerni

Unit-I_Big data life cycle.pptx, sources of Big DataRajendraKankrale1

Maxdiff webinar_10_19_10QuestionPro

02 a holistic approach to big dataRaul Chong

Digital Transformation: How to Run Best-in-Class IT Operations in a World of ...Precisely

Big Data Matching - How to Find Two Similar Needles in a Really Big HaystackPrecisely

SurveyAnalytics MaxDiff Webinar SlidesQuestionPro

Big Data Developer Career Path: Job & Interview PreparationIntellipaat

Big Data - A Real Life RevolutionCapgemini

Cloud as a Data PlatformAndrei Savu

Big Data LDN 2018: THE PATH TO ENTERPRISE AI: TALES FROM THE FIELDMatt Stubbs

R for SAS Users Complement or Replace Two StrategiesRevolution Analytics

Big Data ChallengesArtem Rodichev

Advanced Analytics: Going From Big Data to Big Answersajayc47

Why Everything You Know About bigdata Is A LieSunil Ranka

Gse uk-cedrinemadera-2018-sharedcedrinemadera

Big data analytics presented at meetup big data for decision makersRuhollah Farchtchi

What Managers Need to Know about Data ScienceAnnie Flippo

A6 big data_in_the_cloudDr. Wilfred Lin (Ph.D.)

Agile data scienceJoel Horwitz

Data Science as a Commodity: Use MADlib, R, & other OSS Tools for Data Scienc...Sarah Aerni

Unit-I_Big data life cycle.pptx, sources of Big DataRajendraKankrale1

Maxdiff webinar_10_19_10QuestionPro

02 a holistic approach to big dataRaul Chong

Digital Transformation: How to Run Best-in-Class IT Operations in a World of ...Precisely

Big Data Matching - How to Find Two Similar Needles in a Really Big HaystackPrecisely

More from Alluxio, Inc. (20)

How Coupang Leverages Distributed Cache to Accelerate ML Model TrainingAlluxio, Inc.

Alluxio Webinar | Inside Deepseek 3FS: A Deep Dive into AI-Optimized Distribu...Alluxio, Inc.

AI/ML Infra Meetup | Building Production Platform for Large-Scale Recommendat...Alluxio, Inc.

AI/ML Infra Meetup | How Uber Optimizes LLM Training and FinetuneAlluxio, Inc.

AI/ML Infra Meetup | Optimizing ML Data Access with Alluxio: Preprocessing, ...Alluxio, Inc.

AI/ML Infra Meetup Mar. 06, 2025 Organized by Alluxio For more Alluxio Events: https://www.alluxio.io/events/ Speaker: - Bin Fan (VP of Technology @ Alluxio) In this talk, Bin Fan shares his insights on data access challenges in ML applications, with particular emphasis on how Alluxio's distributed caching helps bridge the gap between storage and compute in preprocessing, pretraining and inference.

AI/ML Infra Meetup | Deployment, Discovery and Serving of LLMs at Uber ScaleAlluxio, Inc.

Alluxio Webinar | What’s New in Alluxio AI: 3X Faster Checkpoint File Creatio...Alluxio, Inc.

Alluxio Webinar Feb. 25, 2025 For more Alluxio Events: https://www.alluxio.io/events/ Speaker: Bill Hodak (VP of Marketing and Product Marketing, Alluxio) Tom Luckenbach (Solutions Engineering Manager, Alluxio) Join us to learn about the latest release of Alluxio Enterprise AI. In this webinar, we’ll provide an overviewof the new features and capabilities of Alluxio Enterprise AI, built to accelerate AI workloads and maximize GPU utilization. Key highlights include: - New caching mode accelerates AI checkpoints - Advanced cache eviction policies provide fine-grained control - Python SDK integrations enhance AI framework compatibility - A demo of Alluxio accelerating AI training workloads in AWS

AI/ML Infra Meetup | A Faster and More Cost Efficient LLM Inference StackAlluxio, Inc.

AI/ML Infra Meetup Jan. 23, 2025 Organized by Alluxio For more Alluxio Events: https://www.alluxio.io/events/ Speaker: - Junchen Jiang (Assistant Professor @ University of Chicago) LLM inference can be huge, particularly, with long contexts. In this on-demand video, Junchen Jiang, Assistant Professor at University of Chicago, presents a 10x solution for long contexts inference: an easy-to-deploy stack over multiple vLLM engines with tailored KV-cache backend.

AI/ML Infra Meetup | Balancing Cost, Performance, and Scale - Running GPU/CPU...Alluxio, Inc.

Alluxio Webinar | Accelerate AI: Alluxio 101Alluxio, Inc.

Alluxio Webinar Dec. 3, 2024 For more Alluxio Events: https://www.alluxio.io/events/ Speaker: Bill Hodak (VP of Marketing and Product Marketing, Alluxio) In the rapidly evolving landscape of AI and machine learning, Platform and Data Infrastructure Teams face critical challenges in building and managing large-scale AI platforms. Performance bottlenecks, scalability of the platform, and scarcity of GPUs pose significant challenges in supporting large-scale model training and serving. In this talk, we will introduce how Alluxio helps Platform and Data Infrastructure teams deliver faster, more scalable platforms to ML Engineering teams developing and training AI models. Alluxio’s highly-distributed cache accelerates AI workloads by eliminating data loading bottlenecks and maximizing GPU utilization. Customers report up to 4x faster training performance with high-speed access to petabytes of data spread across billions of files regardless of persistent storage type or proximity to GPU clusters. Alluxio’s architecture lowers data infrastructure costs, increases GPU utilization, and enables workload portability for navigating GPU scarcity challenges.

AI/ML Infra Meetup | The power of Ray in the era of LLM and multi-modality AIAlluxio, Inc.

AI/ML Infra Meetup Nov. 7, 2024 Organized by Alluxio For more Alluxio Events: https://www.alluxio.io/events/ Speaker: - Zhe Zhang (Distinguished Engineer @ NVIDIA) In this talk, Zhe Zhang (NVIDIA, ex-Anyscale) introduced Ray and its applications in the LLM and multi-modal AI era. He shared his perspective on ML infrastructure, noting that it presents more unstructured challenges, and recommended using Ray and Alluxio as solutions for increasingly data-intensive multi-modal AI workloads.

AI/ML Infra Meetup | Exploring Distributed Caching for Faster GPU Training wi...Alluxio, Inc.

AI/ML Infra Meetup Nov. 7, 2024 Organized by Alluxio For more Alluxio Events: https://www.alluxio.io/events/ Speaker: - Bin Fan (Founding Engineer, VP of Technology @ Alluxio) As large-scale machine learning becomes increasingly GPU-centric, modern high-performance hardware like NVMe storage and RDMA networks (InfiniBand or specialized NICs) are becoming more widespread. To fully leverage these resources, it’s crucial to build a balanced architecture that avoids GPU underutilization. In this talk, we will explore various strategies to address this challenge by effectively utilizing these advanced hardware components. Specifically, we will present experimental results from building a Kubernetes-native distributed caching layer, utilizing NVMe storage and high-speed RDMA networks to optimize data access for PyTorch training.

AI/ML Infra Meetup | Big Data and AI, Zoom DevelopersAlluxio, Inc.

AI/ML Infra Meetup Nov. 7, 2024 Organized by Alluxio For more Alluxio Events: https://www.alluxio.io/events/ Speaker: - Sandeep Manchem (ML Platform Engineering Manager @ Zoom) In this talk, Sandeep Manchem (Zoom) discussed big data and AI, covering typical platform architecture and data challenges. We had engaging discussions about ensuring data safety and compliance in Big Data and AI applications.

AI/ML Infra Meetup | TorchTitan, One-stop PyTorch native solution for product...Alluxio, Inc.

AI/ML Infra Meetup Nov. 7, 2024 Organized by Alluxio For more Alluxio Events: https://www.alluxio.io/events/ Speaker: - Tianyu Liu (Research Scientist @ Meta) TorchTitan is a proof-of-concept for Large-scale LLM training using native PyTorch. It is a repo that showcases PyTorch's latest distributed training features in a clean, minimal codebase. In this talk, Tianyu will share TorchTitan’s design and optimizations for the Llama 3.1 family of LLMs, spanning 8 billion to 405 billion parameters, and showcase its performance, composability, and scalability.

Alluxio Webinar | Model Training Across Regions and Clouds – Challenges, Solu...Alluxio, Inc.

Alluxio Webinar October.15, 2024 For more Alluxio Events: https://www.alluxio.io/events/ Speaker: - Tom Luckenbach (Solutions Engineering Manager, Alluxio) AI training workloads running on compute engines like PyTorch, TensorFlow, and Ray require consistent, high-throughput access to training data to maintain high GPU utilization. However, with the decoupling of compute and storage and with today’s hybrid and multi-cloud landscape, AI Platform and Data Infrastructure teams are struggling to cost-effectively deliver the high-performance data access needed for AI workloads at scale. Join Tom Luckenbach, Alluxio Solutions Engineering Manager, to learn how Alluxio enables high-speed, cost-effective data access for AI training workloads in hybrid and multi-cloud architectures, while eliminating the need to manage data copies across regions and clouds. What Tom will share: - AI data access challenges in cross-region, cross-cloud architectures. - The architecture and integration of Alluxio with frameworks like PyTorch, TensorFlow, and Ray using POSIX, REST, or Python APIs across AWS, GCP and Azure. - A live demo of an AI training workload accessing cross-cloud datasets leveraging Alluxio's distributed cache, unified namespace, and policy-driven data management. - MLPerf and FIO benchmark results and cost-savings analysis.

AI/ML Infra Meetup | Scaling Experimentation Platform in Digital Marketplaces...Alluxio, Inc.

AI/ML Infra Meetup Aug. 29, 2024 Organized by Alluxio For more Alluxio Events: https://www.alluxio.io/events/ Speaker: - Koundinya Pidaparthi (VP of Analytics @ Poshmark) Scaling experimentation in digital marketplaces is crucial for driving growth and enhancing user experiences. However, varied methodologies and a lack of experiment governance can hinder the impact of experimentation leading to inconsistent decision-making, inefficiencies, and missed opportunities for innovation. At Poshmark, we developed a homegrown experimentation platform, Lightspeed, that allowed us to make reliable and confident reads on product changes, which led to a 10x growth in experiment velocity and positive business outcomes along the way. This session will provide a deep dive into the best practices and lessons learned from successful implementations of large-scale experiments. We will explore the importance of experimentation, overcome scalability challenges, and gain insights into the frameworks and technologies that enable effective testing.

AI/ML Infra Meetup | Scaling Vector Databases for E-Commerce Visual Search: A...Alluxio, Inc.

AI/ML Infra Meetup Aug. 29, 2024 Organized by Alluxio For more Alluxio Events: https://www.alluxio.io/events/ Speaker: - Mahesh Pasupuleti (VP of DS, ML & Data Infra @ Poshmark) In the rapidly evolving world of e-commerce, visual search has become a game-changing technology. Poshmark, a leading fashion resale marketplace, has developed Posh Lens – an advanced visual search engine that revolutionizes how shoppers discover and purchase items. Under the hood of Posh Lens lies Milvus, a vector database enabling efficient product search and recommendation across our vast catalog of over 150 million items. However, with such an extensive and growing dataset, maintaining high-performance search capabilities while scaling AI infrastructure presents significant challenges. In this talk, Mahesh Pasupuleti shares: - The architecture and strategies to scale Milvus effectively within the Posh Lens infrastructure - Key considerations include optimizing vector indexing, managing data partitioning, and ensuring query efficiency amidst large-scale data growth - Distributed computing principles and advanced indexing techniques to handle the complexity of Poshmark's diverse product catalog

Alluxio Webinar | Optimize, Don't Overspend: Data Caching Strategy for AI Wor...Alluxio, Inc.

Alluxio Webinar Sept. 10, 2024 For more Alluxio Events: https://www.alluxio.io/events/ Speaker: - Jingwen Ouyang (Senior Program Manager, Alluxio) As machine learning and deep learning models grow in complexity, AI platform engineers and ML engineers face significant challenges with slow data loading and GPU utilization, often leading to costly investments in high-performance computing (HPC) storage. However, this approach can result in overspending without addressing the core issues of data bottlenecks and infrastructure complexity. A better approach is adding a data caching layer between compute and storage, like Alluxio, which offers a cost-effective alternative through its innovative data caching strategy. In this webinar, Jingwen will explore how Alluxio's caching solutions optimize AI workloads for performance, user experience and cost-effectiveness. What you will learn: - The I/O bottlenecks that slow down data loading in model training - How Alluxio's data caching strategy optimizes I/O performance for training and GPU utilization, and significantly reduces cloud API costs - The architecture and key capabilities of Alluxio - Using Rapid Alluxio Deployer to install Alluxio and run benchmarks in AWS in just 30 minutes

AI/ML Infra Meetup | Maximizing GPU Efficiency : Optimizing Model Training wi...Alluxio, Inc.

AI/ML Infra Meetup Aug. 29, 2024 Organized by Alluxio For more Alluxio Events: https://www.alluxio.io/events/ Speaker: - Bin Fan (VP of Technology, Founding Engineer @OpenAI) In the rapidly evolving landscape of AI and machine learning, infra teams face critical challenges in managing large-scale data for AI. Performance bottlenecks, cost inefficiencies, and management complexities pose significant challenges for AI platform teams supporting large-scale model training and serving. In this talk, Bin Fan will discuss the challenges of I/O stalls that lead to suboptimal GPU utilization during model training. He will present a reference architecture for running PyTorch jobs with Alluxio in cloud environments, demonstrating how this approach can significantly enhance GPU efficiency. What you will learn: - How to identify GPU utilization and I/O-related performance bottlenecks in model training - Leverage GPU anywhere to maximize resource utilization - Best practices for monitoring and optimizing GPU usage across training and serving pipelines - Strategies for reducing cloud costs and simplifying management of AI infrastructure at scale

AI/ML Infra Meetup | Preference Tuning and Fine Tuning LLMsAlluxio, Inc.