Containerized architectures for deep learning
1 like199 views
This document discusses containerized architectures for deep learning using Kubernetes and Kubeflow. It motivates the use of machine learning pipelines for improving data wrangling, system unification, composability, complexity management, portability, and model quality. It then summarizes popular machine learning pipeline tools like Apache Airflow, Kubeflow, TensorFlow Extended, and MLflow. It introduces Kubernetes and containers, and argues that Kubernetes and Kubeflow can provide composability, portability, and scalability for machine learning workloads. It demonstrates a deep learning model deployed on Kubernetes with Kubeflow through a doppelganger image similarity search app.
Ad
More Related Content
What's hot (19)
Similar to Containerized architectures for deep learning (20)
![Hybrid Cloud, Kubeflow and Tensorflow Extended [TFX]](https://cdn.slidesharecdn.com/ss_thumbnails/tfx-kfp-191031073013-thumbnail.jpg?width=560&fit=bounds)
![[DSC Europe 23] Milos Grubjesic Empowering Business with Pepsico s Advanced M...](https://cdn.slidesharecdn.com/ss_thumbnails/dsceurope23milosgrubjesicempoweringbusinesswithpepsicosadvancedmlplatform3-231128225740-80a4b074-thumbnail.jpg?width=560&fit=bounds)
Ad
Recently uploaded (20)
Ad
Containerized architectures for deep learning
- 1. Containerized architectures for deep learning Antje Barth @anbarth
- 2. Me Data Enthusiast Technical Evangelist AI / ML / Deep Learning Container / Kubernetes Big Data #CodeLikeAGirl
- 3. Agenda โข Motivation โข ML pipeline tools and platforms โข Machine Learning on Kubernetes โข Deep Learning Demo โข Conclusion
- 4. Agenda โข Motivation โข ML pipeline tools and platforms โข Machine Learning on Kubernetes โข Deep Learning Demo โข Conclusion
- 6. ML โ Helicopter view How good are your predictions?โข Accuracy โข Optimization
- 8. ML โ The (enterprise) reality โข Wrangle large datasets โข Unify disparate systems โข Composability โข Manage pipeline complexity โข Improve training/serving consistency โข Improve portability โข Improve model quality โข Manage versions Building a model Data ingestion Data analysis Data transform Data validation Data splitting Ad-hoc Training Model validation Logging Roll-out Serving Monitoring Distributed Training Training at scale Data Versioning HP Tuning Experiment Tracking Feature Store SYSTEM 1 SYSTEM 2 SYSTEM 3 SYSTEM 4 SYSTEM 5 SYSTEM 6 SYSTEM 3.5 SYSTEM 1.5
- 10. The rise of ML pipeline tools & platforms
- 11. Agenda โข Motivation โข ML pipeline tools and platforms โข Machine Learning on Kubernetes โข Deep Learning Demo โข Conclusion
- 12. Quick comparison Apache Airflow is a platform to programmatically author, schedule and monitor workflows. The Kubeflow project is dedicated to making deployments of machine learning (ML) workflows on Kubernetes simple, portable and scalable. TensorFlow Extended (TFX) is an end-to-end platform for deploying production ML pipelines. MLflow is an open source platform to manage the ML lifecycle, including experimentation, reproducibility and deployment. https://airflow.apache.org/ https://www.kubeflow.org/ https://www.tensorflow.org/ tfx https://mlflow.org/
- 13. How to scale to production? Composability Portability Scalability
- 14. Wait a minuteโฆ
- 16. Virtual Machines are Computers in a Box Containers are Applications in a Box Containers?
- 18. Kubernetes is an API and agents The Kubernetes API provides containers with a scheduling, configuration, network, and storage The Kubernetes runtime manages the containers
- 19. Agenda โข Motivation โข ML pipeline tools and platforms โข Machine Learning on Kubernetes โข Deep Learning Demo โข Conclusion
- 20. Machine Learning on Kubernetes โข Kubernetes-native โข Run wherever k8s runs โข Move between local โ dev โ test โ prod โ cloud โข Use k8s to manage ML tasks โข CRDs (UDTs) for distributed training โข Adopt k8s patterns โข Microservices โข Manage infrastructure declaratively โข Support for multiple ML frameworks โข Tensorflow, Pytorch, Scikit, Xgboost, etc.
- 22. Introducing Kubeflow Make it easy for everyone to develop, deploy, and manage portable, scalable ML everywhere.
- 24. Composability โข Build and deploy re-usable, portable, scalable, machine learning workflows based on Docker containers. โข Use the libraries/ frameworks of your choice Example: KubeFlow "deployer" component lets you deploy as a plain TF Serving model server: https://github.com/kubeflow/pipelines/tree/ master/components/kubeflow/deployer
- 25. METADATA SERVING Back to our ML enterprise workflow! Building a model Data ingestion Data analysis Data transform Data validation Data splitting Ad-hoc Training Model validation Logging Roll-out Serving Monitoring Distributed Training Training at scale Data Versioning HP Tuning Experiment Tracking Feature Store
- 26. Portability Containers for Deep Learning Container runtime Infrastructure NVIDIA drivers Host OS Packages: TensorFlow mkl cudnn cublas Nccl CUDA toolkit CPU: GPU: TensorFlow Container Image Keras horovod numpy scipy othersโฆ scikit- learn pandas openmpi Python ML environments that are:
- 27. TensorFlow mkl cudnn cublas Nccl CUDA toolkit CPU: GPU: TensorFlow Container Image Keras horovod numpy scipy othersโฆ scikit- learn pandas openmpi Python Container runtime Development System NVIDIA drivers Host OS Container registry push pull TensorFlow mkl cudnn cublas Nccl CUDA toolkit CPU: GPU: TensorFlow Container Image Keras horovod numpy scipy othersโฆ scikit- learn pandas openmpi Python Container runtime Training Cluster NVIDIA drivers Host OS
- 28. Scalability โข Kubernetes - Autoscaling Jobs โข Describe the job, let Kubernetes take care of the rest โข CPU, RAM, Accelerators โข TF Jobs delete themselves when finished, node pool will auto scale back down Model works great! But I need six nodes. Data Scientist IT Ops Credit: @aronchick
- 29. Scalability โข Kubernetes - Autoscaling Jobs โข Describe the job, let Kubernetes take care of the rest โข CPU, RAM, Accelerators โข TF Jobs delete themselves when finished, node pool will auto scale back down Data Scientist IT Ops apiVersion: "kubeflow.org/v1alpha1" kind: "TFJob" spec: replicaSpecs: replicas: 6 CPU: 1 GPU: 1 containers: gcr.io/myco/myjob:1.0 Credit: @aronchick
- 30. Scalability โข Kubernetes - Autoscaling Jobs โข Describe the job, let Kubernetes take care of the rest โข CPU, RAM, Accelerators โข TF Jobs delete themselves when finished, node pool will auto scale back down Data Scientist IT Ops GPU GPU GPU GPU GPU GPU Credit: @aronchick
- 31. Scalability โข Kubernetes - Autoscaling Jobs โข Describe the job, let Kubernetes take care of the rest โข CPU, RAM, Accelerators โข TF Jobs delete themselves when finished, node pool will auto scale back down Jobโs done! Data Scientist IT Ops Credit: @aronchick
- 32. Agenda โข Motivation โข ML pipeline tools and platforms โข Container > Kubernetes > Kubeflow โข Deep Learning Demo โข Conclusion
- 34. Implementing Image Similarity search
- 35. Recap: The โKubeโflow โข Deploy Kubernetes & Kubeflow โข Experiment in Jupyter โข Build Docker Image โข Train at Scale โข Build Model Server โข Deploy Model โข Integrate Model into App โข Operate Model Training Model Serving Pod Pod Pod Kubernetes Worker Nodes #1 #2 #3 Jupyter Notebook Seldon Core Engine Seldon Core Engine Doppelganger Model Doppelganger Model Istio Gateway (Traffic Routing) {REST API} curlโฆ Dockerfile Training Job Dockerfile Inference Service Data Scientist Pod Train Model Pod Train Model
- 36. Agenda โข Motivation โข ML pipeline tools and platforms โข Machine Learning on Kubernetes โข Deep Learning Demo โข Conclusion
- 37. Conclusion & Take-aways โข Platform matters โข Composability โ Portability โ Scalability โข Containerized architectures โข Kubernetes + Machine Learning = Kubeflow โข Start building! https://github.com/antje/doppelganger
- 38. More information โข Kubeflow https://www.kubeflow.org/ https://github.com/kubeflow/kubeflow โข Tensorflow Extended (TFX) https://www.tensorflow.org/tfx โข The Definitive Guide to Machine Learning Platforms https://twimlai.com/mlplatforms-ebook/ โข Amazon Elastic Kubernetes Service (Amazon EKS) https://eksworkshop.com https://github.com/aws-samples/machine-learning-using-k8s
- 39. Session page on conference website OโReilly Events App Rate todayโs session