Ai platform at scale

Oct 4, 20192 likes992 views

The document discusses designing scalable platforms for artificial intelligence (AI) and machine learning (ML). It outlines several challenges in developing AI applications, including technical debts, unpredictability, different data and compute needs compared to traditional software. It then reviews existing commercial AI platforms and common components of AI platforms, including data access, ML workflows, computing infrastructure, model management, and APIs. The rest of the document focuses on eBay's Krylov project as an example AI platform, outlining its architecture, challenges of deploying platforms at scale, and needed skill sets on the platform team.

AI Platform at Scale
Designing scalable platform for AI
Henry Saputra

Motivation for an AI Platform
● AI == ML for context of this presentation
● Developing AI Applications can easily incur technical debts
● Traditional software development assumes predictability during the lifetime
● Bring your own software and hardware
● Explainability and correctness are hard to quantify
● Data access and management is different from traditional software
● Compute and scale of workloads is different from traditional software

$AI and ML code only small fraction ... Reference: https://papers.nips.cc/paper/5656-hidden-technical-debt-in-machine-learning-systems.pdf$

AI Platform in the wild
● eBay Krylov
● Facebook FBLearner Flow
● Uber Michelangelo
● Google TFX
● Salesforce Einstein Platform
● Amazon Sagemaker

Problems to Solve?
● Reduce plumbing work by data scientists
● Dependency on data pipeline, compute infrastructure, and networking
● Large variance of quality, metrics, and measurement of success
● Research vs Applied
● Online vs Oﬄine
● Collaboration requires different approach - Eg: Machine Learning models not
directly re-usable
● Undeclared consumers

Goals of an AI Platform
● Provides a system where data scientists could build reliable, secured, easy, reproducible, and
automated AI model training, and scoring/ inference at scale.
● Address the problem of platform approach to uniﬁed infrastructure to run AI and ML jobs - no
longer running inside data scientists computer
● Standardizing on tools and pipeline to simplify AI and ML jobs from training to deploy models
● AI and ML algorithms should be implemented once and shareable
● Enable parallelism and distributed jobs to accelerate and scale
● Support exploration of metrics about past experiments
● Secure and Easy to use

Common Architecture and Components
● Access to Data - Data analysis, Feature store, Data Lake, Data Format
● ML Workﬂow or Pipeline - DAG, Orchestration vs Choreography
● Domain Speciﬁc Language (DSL)
● Computing Platform and Infrastructure - Cloud vs In-house
○ “Tall” instances, GPU accelerated
○ Distributed computing framework
○ Fast network for data ingest
○ Data locality to compute resources
○ Containers and Microservices
● Models and Experiments lifecycle and management
● Models deployment and serving ﬂow - Batch and Realtime
● Metrics and monitoring - dashboards, reports, logs
● APIs - UI, CLI, Program bindings/ SDK, RESTful, RPC
● Supported ML libraries

Challenges of Deploying AI Platform at Scale
● Deﬁning the “right” architecture
● Open source - build vs buy? Early stage for AI Platform
● Extendible and Scale - horizontal vs vertical
● Secure environment for data access and compute
● Standards and common tooling for ML development - reduce complexity
● Sharing and re-use of algorithms and models
● Reduce tech debts - fast moving
● Tech refresh of hardware - Cloud vs In-house

Future Looking ...
● AutoML
● Online training/ learning and edge devices update
● Distributed Deep Learning for training - model vs data parallelism
● Graph as machine learning
● Improve of computing infrastructure hardware - GPU, TPU
● Faster network
● Next generation of storage for ML use cases
● Better support for AI applications - update and retrain models from devices
● Support for newer AI computing paradigm at scale - generative models,
reinforcement learning

Who do we need in AI Platform Team?
● Engineers and scientists
● Product Management
● Runtime support and infrastructure

This document discusses MLOps and Kubeflow. It begins with an introduction to the speaker and defines MLOps as addressing the challenges of independently autoscaling machine learning pipeline stages, choosing different tools for each stage, and seamlessly deploying models across environments. It then introduces Kubeflow as an open source project that uses Kubernetes to minimize MLOps efforts by enabling composability, scalability, and portability of machine learning workloads. The document outlines key MLOps capabilities in Kubeflow like Jupyter notebooks, hyperparameter tuning with Katib, and model serving with KFServing and Seldon Core. It describes the typical machine learning process and how Kubeflow supports experimental and production phases.

Productionizing Machine Learning with a Microservices ArchitectureDatabricks

SQL Analytics Powering Telemetry Analysis at ComcastDatabricks

Comcast is one of the leading providers of communications, entertainment, and cable products and services. At the heart of it is Comcast RDK providing the backbone of telemetry to the industry. RDK (Reference Design Kit) is pre-bundled opensource firmware for a complete home platform covering video, broadband and IoT devices. RDK team at Comcast analyzes petabytes of data, collected every 15 minutes from 70 million devices (video and broadband and IoT devices) installed in customer homes. They run ETL and aggregation pipelines and publish analytical dashboards on a daily basis to reduce customer calls and firmware rollout. The analysis is also used to calculate WIFI happiness index which is a critical KPI for Comcast customer experience. In addition to this, RDK team also does release tracking by analyzing the RDK firmware quality. SQL Analytics allows customers to operate a lakehouse architecture that provides data warehousing performance at data lake economics for up to 4x better price/performance for SQL workloads than traditional cloud data warehouses. We present the results of the “Test and Learn” with SQL Analytics and the delta engine that we worked in partnership with the Databricks team. We present a quick demo introducing the SQL native interface, the challenges we faced with migration, The results of the execution and our journey of productionizing this at scale.

Microsoft AI Platform OverviewDavid Chou

Microsoft provides an AI platform and tools for developers to build, train, and deploy intelligent applications and services. Key elements of Microsoft's AI offerings include: - A unified AI platform spanning infrastructure, tools, and services to make AI accessible and useful for every developer. - Powerful tools for AI development including deep learning frameworks, coding and management tools, and AI services for tasks like computer vision, natural language processing, and more. - Capabilities for training models at scale using GPU accelerated compute on Azure and deploying trained models as web APIs, mobile apps, or other applications. - A focus on trusted, responsible, and inclusive AI that puts users in control and augments rather than replaces human

Introduction to Data EngineeringHadi Fadlallah

MLOps and Data Quality: Deploying Reliable ML Models in ProductionProvectus

Looking to build a robust machine learning infrastructure to streamline MLOps? Learn from Provectus experts how to ensure the success of your MLOps initiative by implementing Data QA components in your ML infrastructure. For most organizations, the development of multiple machine learning models, their deployment and maintenance in production are relatively new tasks. Join Provectus as we explain how to build an end-to-end infrastructure for machine learning, with a focus on data quality and metadata management, to standardize and streamline machine learning life cycle management (MLOps). Agenda - Data Quality and why it matters - Challenges and solutions of Data Testing - Challenges and solutions of Model Testing - MLOps pipelines and why they matter - How to expand validation pipelines for Data Quality

Introduction to Data EngineeringDurga Gadiraju

As part of this session, I will be giving an introduction to Data Engineering and Big Data. It covers up to date trends. * Introduction to Data Engineering * Role of Big Data in Data Engineering * Key Skills related to Data Engineering * Role of Big Data in Data Engineering * Overview of Data Engineering Certifications * Free Content and ITVersity Paid Resources Don't worry if you miss the video - you can click on the below link to go through the video after the schedule. https://youtu.be/dj565kgP1Ss * Upcoming Live Session - Overview of Big Data Certifications (Spark Based) - https://www.meetup.com/itversityin/events/271739702/ Relevant Playlists: * Apache Spark using Python for Certifications - https://www.youtube.com/playlist?list=PLf0swTFhTI8rMmW7GZv1-z4iu_-TAv3bi * Free Data Engineering Bootcamp - https://www.youtube.com/playlist?list=PLf0swTFhTI8pBe2Vr2neQV7shh9Rus8rl * Join our Meetup group - https://www.meetup.com/itversityin/ * Enroll for our labs - https://labs.itversity.com/plans * Subscribe to our YouTube Channel for Videos - http://youtube.com/itversityin/?sub_confirmation=1 * Access Content via our GitHub - https://github.com/dgadiraju/itversity-books * Lab and Content Support using Slack

What is data engineering?yongdam kim

MLOps journey at Swisscom: AI Use Cases, Architecture and Future VisionBATbern

What powers the AI/ML services of Switzerland's leading telecommunication company? In this talk, we will provide an overview of the different AI/ML projects at Swisscom, from Conversational AI and Recommender Systems to Anomaly Detection. Moreover, we will show how we automate, scale, and operationalise these ML pipelines in production, highlighting the MLOps techniques and open source tools that are used. Finally, we will present Swisscom's roadmap towards the cloud with AWS and discuss how we envision a common MLOps solution for the organisation.

Emerging Trends in Data EngineeringAnanth PackkilDurai

This document discusses 7 emerging trends in data engineering: 1) Data discovery and metadata management using open source tools like Amundsen and Marquez. 2) Data mesh and domain ownership. 3) Data observability using tools like DBT, Great Expectations, and Dagster. 4) Data lakehouse using Apache Iceberg and Delta Lake. 5) Modern data stacks using tools for extraction, transformation, data warehouses, governance, and BI. 6) Industrialized machine learning using frameworks like TensorFlow and PyTorch. 7) Prioritizing diversity, privacy, and AI ethics through techniques like explainable AI and privacy-preserving modeling.

Mohamed Sabri: Operationalize machine learning with KubeflowEdunomica

Data Product ArchitecturesBenjamin Bengfort

This document discusses data product architectures and provides examples of different architectures for data products, including the lambda architecture, analyst architecture, recommender architecture, and partisan discourse architecture. It also discusses common design principles for data product architectures, such as using microservices with stateful backend services and database-backed APIs. Key aspects of data product architectures include handling training data and models, making predictions via APIs, updating models and annotations, and designing flexible systems that can incorporate new models and data.

Pinecone Vector Database.pdfAniruddha Chakrabarti

Getting Started with Databricks SQL AnalyticsDatabricks

It has long been said that business intelligence needs a relational warehouse, but that view is changing. With the Lakehouse architecture being shouted from the rooftops, Databricks have released SQL Analytics, an alternative workspace for SQL-savvy users to interact with an analytics-tuned cluster. But how does it work? Where do you start? What does a typical Data Analyst’s user journey look like with the tool? This session will introduce the new workspace and walk through the various key features – how you set up a SQL Endpoint, the query workspace, creating rich dashboards and connecting up BI tools such as Microsoft Power BI. If you’re truly trying to create a Lakehouse experience that satisfies your SQL-loving Data Analysts, this is a tool you’ll need to be familiar with and include in your design patterns, and this session will set you on the right path.

MLOps - The Assembly Line of MLJordan Birdsell

The catalyst for the success of automobiles came not through the invention of the car but rather through the establishment of an innovative assembly line. History shows us that the ability to mass produce and distribute a product is the key to driving adoption of any innovation, and machine learning is no different. MLOps is the assembly line of Machine Learning and in this presentation we will discuss the core capabilities your organization should be focused on to implement a successful MLOps system.

Importance of Data - Where to find it, how to store, manipulate, and characterize it Artificial Intelligence (AI)- Introduction to AI & ML Technologies/ Applications Machine Learning (ML), Basic Machine Learning algorithms. Applications of AI & ML in Marketing, Sales, Finance, Operations, Supply Chain & Human Resources Data Governance Legal and Ethical Issues Robotic Process Automation (RPA) Internet of Things (IoT) Cloud Computing

AI Modernization at AT&T and the Application to Fraud with DatabricksDatabricks

AT&T has been involved in AI from the beginning, with many firsts; “first to coin the term AI”, “inventors of R”, “foundational work on Conv. Neural Nets”, etc. and we have applied AI to hundreds of solutions. Today we are modernizing these AI solutions in the cloud with the help of Databricks and a variety of in-house developments. This talk will highlight our AI modernization effort along with its application to Fraud which is one of our biggest benefitting applications.

Elastic-EngineeringAraf Karsh Hamid

Analytics in Power Platform: What are my options?Juan Carlos Gonzalez

Ml ops past_present_futureNisha Talagala

Architecting a datalakeLaurent Leturgez

This document discusses architecting a data lake. It begins by introducing the speaker and topic. It then defines a data lake as a repository that stores enterprise data in its raw format including structured, semi-structured, and unstructured data. The document outlines some key aspects to consider when architecting a data lake such as design, security, data movement, processing, and discovery. It provides an example design and discusses solutions from vendors like AWS, Azure, and GCP. Finally, it includes an example implementation using Azure services for an IoT project that predicts parts failures in trucks.

When NOT to use Apache Kafka?Kai Wähner

Apache Kafka is the de facto standard for data streaming to process data in motion. With its significant adoption growth across all industries, I get a very valid question every week: When NOT to use Apache Kafka? What limitations does the event streaming platform have? When does Kafka simply not provide the needed capabilities? How to qualify Kafka out as it is not the right tool for the job? This session explores the DOs and DONTs. Separate sections explain when to use Kafka, when NOT to use Kafka, and when to MAYBE use Kafka. No matter if you think about open source Apache Kafka, a cloud service like Confluent Cloud, or another technology using the Kafka protocol like Redpanda or Pulsar, check out this slide deck. A detailed article about this topic: https://www.kai-waehner.de/blog/2022/01/04/when-not-to-use-apache-kafka/

Sawtooth Windows for Feature AggregationsDatabricks

In this talk about zipline, we will introduce a new type of windowing construct called a sawtooth window. We will describe various properties about sawtooth windows that we utilize to achieve online-offline consistency, while still maintaining high-throughput, low-read latency and tunable write latency for serving machine learning features.We will also talk about a simple deployment strategy for correcting feature drift – due operations that are not “abelian groups”, that operate over change data.

The DevOps JourneyMicro Focus

Micro Focus Software Delivery and Testing Jan De Coster Presentation on the Journey to DevOps in the recent Micro Focus #DevDay Copenhagen. Micro Focus enables enterprise software organizations to build innovative software and accelerate application delivery to meet the needs of the business. Whatever the challenges and infrastructures, our core principle—of reusing what already works to minimize business risk while supporting modern software practices—has positioned our customers to be better prepared to support the digital transformation of the business. Build, test and deliver innovative software faster with less risk. April 2017.

Pourquoi Leroy Merlin a besoin d'un Knowledge Graph ?Neo4j

Building the Artificially Intelligent EnterpriseDatabricks

Mike Ferguson is Managing Director of Intelligent Business Strategies Limited and specializes in business intelligence/analytics and data management. He discusses building the artificially intelligent enterprise and transitioning to a self-learning enterprise. Some key challenges discussed include the siloed and fractured nature of current data and analytics efforts, with many tools and scripts in use without integration. He advocates sorting out the data foundation, implementing DataOps and MLOps, creating a data and analytics marketplace, and integrating analytics into business processes to drive value from AI.

Big data architectures and the data lakeJames Serra

The document provides an overview of big data architectures and the data lake concept. It discusses why organizations are adopting data lakes to handle increasing data volumes and varieties. The key aspects covered include: - Defining top-down and bottom-up approaches to data management - Explaining what a data lake is and how Hadoop can function as the data lake - Describing how a modern data warehouse combines features of a traditional data warehouse and data lake - Discussing how federated querying allows data to be accessed across multiple sources - Highlighting benefits of implementing big data solutions in the cloud - Comparing shared-nothing, massively parallel processing (MPP) architectures to symmetric multi-processing (

Building a Scalable and reliable open source ML Platform with MLFlowGoDataDriven

This document discusses building a scalable and open source machine learning platform. It introduces MLOps and describes ING's ML batch platform use case. The machine learning lifecycle is presented, noting that operationalizing machine learning models is difficult due to infrastructure deployment challenges, lack of collaboration and standardization. An ideal MLOps approach is described with flexible, scalable, automated and standardized processes. Benefits of ING's MLOps approach include increased efficiency, speed, quality, security and auditability. Open source tools that could be leveraged are also presented.

Digital Reinvention by NRBWilliam Poos

How to reinvent your organization in an iterative and pragmatic way? This is the result of using our digital toolbox. It allows you to transform your business model, expand your ecosystem by setting up your digital platform. This reinvention is also supported by the adaptation of your governance allowing you to innovate while guaranteeing the performance of your organization. For any information / suggestion / collaboration - [email protected] Comment réinventer votre organisation de manière itérative et pragmatique ? C'est le résultat de l'utilisation de notre boîte à outils digitale. Elle vous permet de transformer votre modèle métier, d'étendre votre écosystème en mettant en place votre plateforme digitale. Cette réinvention est également supportée par l'adaptation de votre gouvernance vous permettant d'innover tout en garantissant la performance de votre organisation. Pour toute information / suggestion / collaboration - [email protected]

More Related Content

What's hot (20)

What is data engineering?yongdam kim

MLOps journey at Swisscom: AI Use Cases, Architecture and Future VisionBATbern

Emerging Trends in Data EngineeringAnanth PackkilDurai

Mohamed Sabri: Operationalize machine learning with KubeflowEdunomica

Data Product ArchitecturesBenjamin Bengfort

Pinecone Vector Database.pdfAniruddha Chakrabarti

Getting Started with Databricks SQL AnalyticsDatabricks

MLOps - The Assembly Line of MLJordan Birdsell

AI Modernization at AT&T and the Application to Fraud with DatabricksDatabricks

Elastic-EngineeringAraf Karsh Hamid

Analytics in Power Platform: What are my options?Juan Carlos Gonzalez

Ml ops past_present_futureNisha Talagala

Architecting a datalakeLaurent Leturgez

When NOT to use Apache Kafka?Kai Wähner

Sawtooth Windows for Feature AggregationsDatabricks

The DevOps JourneyMicro Focus

Pourquoi Leroy Merlin a besoin d'un Knowledge Graph ?Neo4j

Building the Artificially Intelligent EnterpriseDatabricks

Big data architectures and the data lakeJames Serra

What is data engineering?yongdam kim

MLOps journey at Swisscom: AI Use Cases, Architecture and Future VisionBATbern

Emerging Trends in Data EngineeringAnanth PackkilDurai

Mohamed Sabri: Operationalize machine learning with KubeflowEdunomica

Data Product ArchitecturesBenjamin Bengfort

Pinecone Vector Database.pdfAniruddha Chakrabarti

Getting Started with Databricks SQL AnalyticsDatabricks

MLOps - The Assembly Line of MLJordan Birdsell

AI Modernization at AT&T and the Application to Fraud with DatabricksDatabricks

Elastic-EngineeringAraf Karsh Hamid

Analytics in Power Platform: What are my options?Juan Carlos Gonzalez

Ml ops past_present_futureNisha Talagala

Architecting a datalakeLaurent Leturgez

When NOT to use Apache Kafka?Kai Wähner

Sawtooth Windows for Feature AggregationsDatabricks

The DevOps JourneyMicro Focus

Pourquoi Leroy Merlin a besoin d'un Knowledge Graph ?Neo4j

Building the Artificially Intelligent EnterpriseDatabricks

Big data architectures and the data lakeJames Serra

Similar to Ai platform at scale (20)

Building a Scalable and reliable open source ML Platform with MLFlowGoDataDriven

Digital Reinvention by NRBWilliam Poos

ICP for Data- Enterprise platform for AI, ML and Data ScienceKaran Sachdeva

DDDP 2019 - Brown to GreenJohn Archer

Deploying ML models in the enterprisedoppenhe

The talk was given at OReilly Strata Data Conference September 2018 in NYC All the conferences and thought leaders have been painting a vision of the businesses of the future being powered by data, but if we’re honest with ourselves, the vast majority of our massive data science investments are being deployed to PowerPoint or maybe a business dashboard. Productionizing your machine learning (ML) portfolio is the next big step on the path to ROI from AI. You probably started out years ago on a “big data” initiative: You collected and cleaned your data and built data warehouses, and when those filled up you upgraded to data lakes. You hired data engineers and data scientists, and around the organization, everyone brushed up their SQL querying skills and got some licenses to Tableau and PowerBI. Then you saw what Google, Uber, Facebook, and Amazon were doing with machine learning to automate business processes and customer interactions. To not get broadsided, you hired more data scientists and machine learning engineers. They were put on your teams and started using your big data investments to train models. But what you probably found is that your tech stack and DevOps processes don’t fit ML models. Unlike most of your systems, ML models require short spikes of massive compute; they are often written in different languages than your core code; they need different hardware to perform well; one model probably has applications across many teams; and the people making the models often don’t have the engineering experience to write production code but need to iterate faster than traditional engineers. Expecting your engineering and DevOps teams to deploy ML models well is like showing up to Seaworld with a giraffe since they are already handling large mammals. There is a path forward. Almost five years ago Algorithmia launched a marketplace for models, functions, and algorithms. Today 65,000 developers are on the platform deploying 4,500 models—the result has been a layer of tools and best practices to make deploying ML models frictionless, scalable, and low maintenance. The company refers to it as the “AI layer.” Drawing on this experience, Diego Oppenheimer covers the strategic and technical hurdles each company must overcome and the best practices developed while deploying over 4,000 ML models for 70,000 engineers. Topics include: Best practices for your organization Continuous model deployment Varying languages (Your code base probably isn’t in Python or R, but your ML models probably are.) Managing your portfolio of ML models Standardize versioning Enabling models across your organization Analytics on how and where models are being used Maintaining auditability

Building a Real-Time Security Application Using Log Data and Machine Learning...Sri Ambati

World Artificial Intelligence Conference Shanghai 2018Adam Gibson

It Consulting & Services - Black Basil TechnologiesBlack Basil Technologies

Bitkom Cray presentation - on HPC affecting big data analytics in FSPhilip Filleul

ODSC East 2020 Accelerate ML Lifecycle with Kubernetes and Containerized Da...Abhinav Joshi

This deck provide an overview of containers and Kubernetes, and how these technologies can help solve the challenges faced by data scientists, ML engineers, and application developers. Next, it showcases the key capabilities required in a containers and kubernetes platform to help data scientists easily use technologies like Jupyter Notebooks, ML frameworks, programming languages to innovate faster. Finally it discusses the available platform options (e.g. KubeFlow, Open Data Hub, etc.), and some examples of how data scientists are accelerating their ML initiatives with containers and kubernetes platform.

Making machine learning model deployment boring - Big Data Expo 2019webwinkelvakdag

The free lunch for machine learning is over. Organizations are quickly ramping up their abilities to automate and professionalize their machine learning processes and infrastructure. As a consequence organizational goals, processes and requirements put an increasing burden on teams to put machine learning models in production. We believe much of this burden relates to engineering issues, which with proper abstractions can be greatly reduced for product teams. In this presentation we will talk about the organizational context of ING and the design our Machine Learning Platform. In the first part we will sketch some organizational context and the requirements it brings. Next, we will picture the kind of use cases and user journey we have in mind. Finally, we will present how these considerations led the platform design we are currently deploying.

Data Engineer's Lunch #60: Series - Developing Enterprise ConsciousnessAnant Corporation

Machine Learning at Scale with MLflow and Apache SparkDatabricks

This document summarizes the challenges faced by SocGen, a large French bank, in implementing machine learning at scale using Spark and MLflow. Some key challenges included: 1) Keeping data and models local for regulatory reasons while performing training and prediction, 2) Ensuring reliability when moving models between prototyping and production phases, 3) Managing different Python package dependencies, 4) Tracking and managing many models, and 5) Ensuring high availability of the tracking server. The presentation provided a concrete example of using Spark, MLflow, and Kafka to periodically retrain a model for scoring news articles and handling user feedback in a scalable and reliable way.

Navigating the ML Pipeline Jungle with MLflow: Notes from the Field with Thun...Databricks

Plumbing has been a key focus of modern software engineering, with our API/services/containers/devops driven landscape so it may come as a surprise that plumbing is where AI projects tend to fail. But it is precisely because our modern software development focuses on decoupled plumbing that we have struggled to handle the rise of AI. Specifically, companies are able to use AI effectively when they are able to create end-to-end AI model factories that explicitly account for coupling between data, models, and code. In this talk, I will be walking through what a model factory is and how MLFlow’s design supports the creation of end-to-end model factories as well as sharing best practices I’ve observed helping customers from startups to Fortune 50s create, productionize, and scale end-to-end ML pipelines, and watching those pipelines produce serious, game changing business impact.

ML Platform Q1 Meetup: Airbnb's End-to-End Machine Learning InfrastructureFei Chen

Cloud computingjun28Dennis Ebenezer

This document discusses cloud computing concepts, technologies, and business implications. It provides an introduction to cloud models like IaaS, PaaS, and SaaS and demonstrates cloud capabilities through examples of Amazon AWS, Google App Engine, and Windows Azure. The document also discusses enabling technologies for cloud computing like virtualization and programming models for big data like MapReduce and Hadoop.

Cloud computingjun28Aravindharamanan S

Emergence of cloud computing and internet of things an overviewSelvaraj Kesavan

This document provides an overview of cloud computing and the Internet of Things (IoT). It defines cloud computing as the on-demand delivery of computing resources over the Internet. It describes the common cloud service models of SaaS, PaaS, and IaaS. It also discusses characteristics of cloud computing like resource pooling, elasticity, and measured service. The document then introduces IoT and describes components of an IoT system including sensors, gateways, communication protocols, cloud platforms, and applications. It provides examples of services from major cloud platforms and discusses tools for visualizing IoT data.

Dell AI Telecom WebinarBill Wong

This document discusses how telecom companies can leverage artificial intelligence and analytics to drive digital transformation. It identifies key opportunities for AI including improving the customer experience, fraud mitigation, and predictive maintenance. It then outlines the components of a telecom data lake that can support these advanced analytics initiatives. Examples of AI use cases for different telecom business functions like marketing, network operations, and security are also provided. The document argues that a data lake platform optimized for analytics can help telecom companies achieve business and innovation goals through improved operations, new revenue streams, and lower costs.

Cloud Computing concepts and technologiesssuser4c9444

Building a Scalable and reliable open source ML Platform with MLFlowGoDataDriven

Digital Reinvention by NRBWilliam Poos

ICP for Data- Enterprise platform for AI, ML and Data ScienceKaran Sachdeva

DDDP 2019 - Brown to GreenJohn Archer

Deploying ML models in the enterprisedoppenhe

Building a Real-Time Security Application Using Log Data and Machine Learning...Sri Ambati

World Artificial Intelligence Conference Shanghai 2018Adam Gibson

It Consulting & Services - Black Basil TechnologiesBlack Basil Technologies

Bitkom Cray presentation - on HPC affecting big data analytics in FSPhilip Filleul

ODSC East 2020 Accelerate ML Lifecycle with Kubernetes and Containerized Da...Abhinav Joshi

Making machine learning model deployment boring - Big Data Expo 2019webwinkelvakdag

Data Engineer's Lunch #60: Series - Developing Enterprise ConsciousnessAnant Corporation

Machine Learning at Scale with MLflow and Apache SparkDatabricks

Navigating the ML Pipeline Jungle with MLflow: Notes from the Field with Thun...Databricks

ML Platform Q1 Meetup: Airbnb's End-to-End Machine Learning InfrastructureFei Chen

Cloud computingjun28Dennis Ebenezer

Cloud computingjun28Aravindharamanan S

Emergence of cloud computing and internet of things an overviewSelvaraj Kesavan

Dell AI Telecom WebinarBill Wong

Cloud Computing concepts and technologiesssuser4c9444

Recently uploaded (20)

Artificial Intelligence (AI) basics.pptxaditichinar

Avnet Silica's PCIM 2025 Highlights FlyerWillDavies22

DT REPORT by Tech titan GROUP to introduce the subject design ThinkingDhruvChotaliya2

"Boiler Feed Pump (BFP): Working, Applications, Advantages, and Limitations E...Infopitaara

A Boiler Feed Pump (BFP) is a critical component in thermal power plants. It supplies high-pressure water (feedwater) to the boiler, ensuring continuous steam generation. ⚙️ How a Boiler Feed Pump Works Water Collection: Feedwater is collected from the deaerator or feedwater tank. Pressurization: The pump increases water pressure using multiple impellers/stages in centrifugal types. Discharge to Boiler: Pressurized water is then supplied to the boiler drum or economizer section, depending on design. 🌀 Types of Boiler Feed Pumps Centrifugal Pumps (most common): Multistage for higher pressure. Used in large thermal power stations. Positive Displacement Pumps (less common): For smaller or specific applications. Precise flow control but less efficient for large volumes. 🛠️ Key Operations and Controls Recirculation Line: Protects the pump from overheating at low flow. Throttle Valve: Regulates flow based on boiler demand. Control System: Often automated via DCS/PLC for variable load conditions. Sealing & Cooling Systems: Prevent leakage and maintain pump health. ⚠️ Common BFP Issues Cavitation due to low NPSH (Net Positive Suction Head). Seal or bearing failure. Overheating from improper flow or recirculation.

Metal alkyne complexes.pptx in chemistrymee23nu

Reagent dosing (Bredel) presentation.pptxAlejandroOdio

some basics electrical and electronics knowledgenguyentrungdo88

π0.5: a Vision-Language-Action Model with Open-World GeneralizationNABLAS株式会社

今回の資料「Transfusion / π0 / π0.5」は、画像・言語・アクションを統合するロボット基盤モデルについて紹介しています。拡散×自己回帰を融合したTransformerをベースに、π0.5ではオープンワールドでの推論・計画も可能に。 This presentation introduces robot foundation models that integrate vision, language, and action. Built on a Transformer combining diffusion and autoregression, π0.5 enables reasoning and planning in open-world settings.

Smart_Storage_Systems_Production_Engineering.pptxrushikeshnavghare94

Raish Khanji GTU 8th sem Internship Report.pdfRaishKhanji

This report details the practical experiences gained during an internship at Indo German Tool Room, Ahmedabad. The internship provided hands-on training in various manufacturing technologies, encompassing both conventional and advanced techniques. Significant emphasis was placed on machining processes, including operation and fundamental understanding of lathe and milling machines. Furthermore, the internship incorporated modern welding technology, notably through the application of an Augmented Reality (AR) simulator, offering a safe and effective environment for skill development. Exposure to industrial automation was achieved through practical exercises in Programmable Logic Controllers (PLCs) using Siemens TIA software and direct operation of industrial robots utilizing teach pendants. The principles and practical aspects of Computer Numerical Control (CNC) technology were also explored. Complementing these manufacturing processes, the internship included extensive application of SolidWorks software for design and modeling tasks. This comprehensive practical training has provided a foundational understanding of key aspects of modern manufacturing and design, enhancing the technical proficiency and readiness for future engineering endeavors.

IntroSlides-April-BuildWithAI-VertexAI.pdfLuiz Carneiro

Level 1-Safety.pptx Presentation of Electrical SafetyJoseAlbertoCariasDel

The Gaussian Process Modeling Module in UQLabJournal of Soft Computing in Civil Engineering

We introduce the Gaussian process (GP) modeling module developed within the UQLab software framework. The novel design of the GP-module aims at providing seamless integration of GP modeling into any uncertainty quantification workflow, as well as a standalone surrogate modeling tool. We first briefly present the key mathematical tools on the basis of GP modeling (a.k.a. Kriging), as well as the associated theoretical and computational framework. We then provide an extensive overview of the available features of the software and demonstrate its flexibility and user-friendliness. Finally, we showcase the usage and the performance of the software on several applications borrowed from different fields of engineering. These include a basic surrogate of a well-known analytical benchmark function; a hierarchical Kriging example applied to wind turbine aero-servo-elastic simulations and a more complex geotechnical example that requires a non-stationary, user-defined correlation function. The GP-module, like the rest of the scientific code that is shipped with UQLab, is open source (BSD license).

Data Structures_Searching and Sorting.pptxRushaliDeshmukh2

Degree_of_Automation.pdf for Instrumentation and industrial specialistshreyabhosale19

Explainable-Artificial-Intelligence-XAI-A-Deep-Dive (1).pptxMahaveerVPandit

MAQUINARIA MINAS CEMA 6th Edition (1).pdfssuser562df4

Introduction to FLUID MECHANICS & KINEMATICSnarayanaswamygdas

Fluid mechanics is the branch of physics concerned with the mechanics of fluids (liquids, gases, and plasmas) and the forces on them. Originally applied to water (hydromechanics), it found applications in a wide range of disciplines, including mechanical, aerospace, civil, chemical, and biomedical engineering, as well as geophysics, oceanography, meteorology, astrophysics, and biology. It can be divided into fluid statics, the study of various fluids at rest, and fluid dynamics. Fluid statics, also known as hydrostatics, is the study of fluids at rest, specifically when there's no relative motion between fluid particles. It focuses on the conditions under which fluids are in stable equilibrium and doesn't involve fluid motion. Fluid kinematics is the branch of fluid mechanics that focuses on describing and analyzing the motion of fluids, such as liquids and gases, without considering the forces that cause the motion. It deals with the geometrical and temporal aspects of fluid flow, including velocity and acceleration. Fluid dynamics, on the other hand, considers the forces acting on the fluid. Fluid dynamics is the study of the effect of forces on fluid motion. It is a branch of continuum mechanics, a subject which models matter without using the information that it is made out of atoms; that is, it models matter from a macroscopic viewpoint rather than from microscopic. Fluid mechanics, especially fluid dynamics, is an active field of research, typically mathematically complex. Many problems are partly or wholly unsolved and are best addressed by numerical methods, typically using computers. A modern discipline, called computational fluid dynamics (CFD), is devoted to this approach. Particle image velocimetry, an experimental method for visualizing and analyzing fluid flow, also takes advantage of the highly visual nature of fluid flow. Fundamentally, every fluid mechanical system is assumed to obey the basic laws : Conservation of mass Conservation of energy Conservation of momentum The continuum assumption For example, the assumption that mass is conserved means that for any fixed control volume (for example, a spherical volume)—enclosed by a control surface—the rate of change of the mass contained in that volume is equal to the rate at which mass is passing through the surface from outside to inside, minus the rate at which mass is passing from inside to outside. This can be expressed as an equation in integral form over the control volume. The continuum assumption is an idealization of continuum mechanics under which fluids can be treated as continuous, even though, on a microscopic scale, they are composed of molecules. Under the continuum assumption, macroscopic (observed/measurable) properties such as density, pressure, temperature, and bulk velocity are taken to be well-defined at "infinitesimal" volume elements—small in comparison to the characteristic length scale of the system, but large in comparison to molecular length scale

Lidar for Autonomous Driving, LiDAR Mapping for Driverless Cars.pptxRishavKumar530754

Introduction to Zoomlion Earthmoving.pptxAS1920

Artificial Intelligence (AI) basics.pptxaditichinar

Avnet Silica's PCIM 2025 Highlights FlyerWillDavies22

DT REPORT by Tech titan GROUP to introduce the subject design ThinkingDhruvChotaliya2

"Boiler Feed Pump (BFP): Working, Applications, Advantages, and Limitations E...Infopitaara

Metal alkyne complexes.pptx in chemistrymee23nu

Reagent dosing (Bredel) presentation.pptxAlejandroOdio

some basics electrical and electronics knowledgenguyentrungdo88

π0.5: a Vision-Language-Action Model with Open-World GeneralizationNABLAS株式会社

Smart_Storage_Systems_Production_Engineering.pptxrushikeshnavghare94

Raish Khanji GTU 8th sem Internship Report.pdfRaishKhanji

IntroSlides-April-BuildWithAI-VertexAI.pdfLuiz Carneiro

Level 1-Safety.pptx Presentation of Electrical SafetyJoseAlbertoCariasDel

The Gaussian Process Modeling Module in UQLabJournal of Soft Computing in Civil Engineering

Data Structures_Searching and Sorting.pptxRushaliDeshmukh2

Degree_of_Automation.pdf for Instrumentation and industrial specialistshreyabhosale19

Explainable-Artificial-Intelligence-XAI-A-Deep-Dive (1).pptxMahaveerVPandit

MAQUINARIA MINAS CEMA 6th Edition (1).pdfssuser562df4

Introduction to FLUID MECHANICS & KINEMATICSnarayanaswamygdas

Lidar for Autonomous Driving, LiDAR Mapping for Driverless Cars.pptxRishavKumar530754

Introduction to Zoomlion Earthmoving.pptxAS1920

Ai platform at scale

1. AI Platform at Scale Designing scalable platform for AI Henry Saputra

2. Motivation for an AI Platform ● AI == ML for context of this presentation ● Developing AI Applications can easily incur technical debts ● Traditional software development assumes predictability during the lifetime ● Bring your own software and hardware ● Explainability and correctness are hard to quantify ● Data access and management is different from traditional software ● Compute and scale of workloads is different from traditional software

3. AI and ML code only small fraction ... Reference: https://papers.nips.cc/paper/5656-hidden-technical-debt-in-machine-learning-systems.pdf

4. AI Platform in the wild ● eBay Krylov ● Facebook FBLearner Flow ● Uber Michelangelo ● Google TFX ● Salesforce Einstein Platform ● Amazon Sagemaker

5. Problems to Solve? ● Reduce plumbing work by data scientists ● Dependency on data pipeline, compute infrastructure, and networking ● Large variance of quality, metrics, and measurement of success ● Research vs Applied ● Online vs Oﬄine ● Collaboration requires different approach - Eg: Machine Learning models not directly re-usable ● Undeclared consumers

6. Goals of an AI Platform ● Provides a system where data scientists could build reliable, secured, easy, reproducible, and automated AI model training, and scoring/ inference at scale. ● Address the problem of platform approach to uniﬁed infrastructure to run AI and ML jobs - no longer running inside data scientists computer ● Standardizing on tools and pipeline to simplify AI and ML jobs from training to deploy models ● AI and ML algorithms should be implemented once and shareable ● Enable parallelism and distributed jobs to accelerate and scale ● Support exploration of metrics about past experiments ● Secure and Easy to use

7. Common Architecture and Components ● Access to Data - Data analysis, Feature store, Data Lake, Data Format ● ML Workflow or Pipeline - DAG, Orchestration vs Choreography ● Domain Specific Language (DSL) ● Computing Platform and Infrastructure - Cloud vs In-house ○ “Tall” instances, GPU accelerated ○ Distributed computing framework ○ Fast network for data ingest ○ Data locality to compute resources ○ Containers and Microservices ● Models and Experiments lifecycle and management ● Models deployment and serving flow - Batch and Realtime ● Metrics and monitoring - dashboards, reports, logs ● APIs - UI, CLI, Program bindings/ SDK, RESTful, RPC ● Supported ML libraries

8. AI Platform at eBay - Krylov Project

9. Challenges of Deploying AI Platform at Scale ● Deﬁning the “right” architecture ● Open source - build vs buy? Early stage for AI Platform ● Extendible and Scale - horizontal vs vertical ● Secure environment for data access and compute ● Standards and common tooling for ML development - reduce complexity ● Sharing and re-use of algorithms and models ● Reduce tech debts - fast moving ● Tech refresh of hardware - Cloud vs In-house

10. Future Looking ... ● AutoML ● Online training/ learning and edge devices update ● Distributed Deep Learning for training - model vs data parallelism ● Graph as machine learning ● Improve of computing infrastructure hardware - GPU, TPU ● Faster network ● Next generation of storage for ML use cases ● Better support for AI applications - update and retrain models from devices ● Support for newer AI computing paradigm at scale - generative models, reinforcement learning

11. Who do we need in AI Platform Team? ● Engineers and scientists ● Product Management ● Runtime support and infrastructure

12. eBay Krylov High Level Architecture

13. eBay Krylov Cluster Deployment

14. eBay Krylov Cluster Infrastructure

15. eBay Krylov Dashboard

16. Thank You

Ai platform at scale

Recommended

More Related Content

What's hot (20)

Similar to Ai platform at scale (20)

Recently uploaded (20)

Ai platform at scale