132523
132523
com
https://ebookmeta.com/product/implementing-mlops-in-the-
enterprise-a-production-first-approach-1st-edition-yaron-
haviv/
OR CLICK BUTTON
DOWNLOAD NOW
https://ebookmeta.com/product/implementing-mlops-early-release-noah-
gift/
ebookmeta.com
https://ebookmeta.com/product/photos-from-beijing-1989-1st-edition-
wesley-gibbs/
ebookmeta.com
The Master and Margarita Mikhail Bulgakov
https://ebookmeta.com/product/the-master-and-margarita-mikhail-
bulgakov/
ebookmeta.com
https://ebookmeta.com/product/mcgraw-hill-education-sat-subject-test-
math-level-1-4th-ed-4th-edition-john-j-diehl/
ebookmeta.com
https://ebookmeta.com/product/react-up-running-building-web-
applications-second-edition-stoyan-stefanov/
ebookmeta.com
https://ebookmeta.com/product/the-highest-good-in-aristotle-and-kant-
first-edition-edition-joachim-aufderheide-editor/
ebookmeta.com
C C Course for Beginners 5e Callie Bond
https://ebookmeta.com/product/c-c-course-for-beginners-5e-callie-bond/
ebookmeta.com
Implementing
MLOps in the
Enterprise
A Production-First
Approach
Yaron Haviv
& Noah Gift
Implementing MLOps in the Enterprise
This practical guide will help your organization bring data
science to life for different real-world MLOps scenarios. Senior “The authors excel in
data scientists, MLOps engineers, and ML engineers will learn
presenting complex
how to tackle challenges that prevent many businesses from
concepts in a clear
moving ML models to production and scaling their AI initiatives.
and relatable manner.
Authors Yaron Haviv and Noah Gift take a production-first Their emphasis on the
approach. Rather than beginning with the ML model, you’ll learn importance of ROI, risk
how to design a continuous operational pipeline—while making
management, and
sure that various components and practices can map into it.
strategic technology
By automating as many components as possible, and making
the process fast and repeatable, your pipeline can scale to match
adoption provides
your organization’s needs. practical guidance for
organizations looking to
This book will show you how to generate rapid business value
leverage ML effectively.”
while answering dynamic MLOps requirements. You’ll learn the
—Dhanasekar Sundararaman
foundations of the MLOps process, including its technological
Researcher, Microsoft
and business value, and discover how to:
• Build and structure effective MLOps pipelines
Yaron Haviv is a serial entrepreneur
• Efficiently scale MLOps across your organization with deep technological experience in
• Explore common MLOps use cases data, cloud, AI, and networking. Yaron
is the cofounder and CTO of Iguazio,
• Build MLOps pipelines for hybrid deployments, real-time which was acquired by McKinsey and
predictions, and composite AI Company in 2023. He is an author,
• Prepare for and adapt to the future of MLOps keynote speaker, and contributor to
various AI associations, publications,
• Use pretrained models like Hugging Face and OpenAI to and communities.
complement your MLOps strategy
Noah Gift is the founder of Pragmatic
AI Labs. He lectures in the data science
programs at universities including
Northwestern, Duke, UC Berkeley,
UNC Charlotte, and the University
of Tennessee.
The O’Reilly logo is a registered trademark of O’Reilly Media, Inc. Implementing MLOps in the Enterprise,
the cover image, and related trade dress are trademarks of O’Reilly Media, Inc.
The views expressed in this work are those of the authors and do not represent the publisher’s views.
While the publisher and the authors have used good faith efforts to ensure that the information and
instructions contained in this work are accurate, the publisher and the authors disclaim all responsibility
for errors or omissions, including without limitation responsibility for damages resulting from the use
of or reliance on this work. Use of the information and instructions contained in this work is at your
own risk. If any code samples or other technology this work contains or describes is subject to open
source licenses or the intellectual property rights of others, it is your responsibility to ensure that your use
thereof complies with such licenses and/or rights.
978-1-098-13658-1
[LSI]
Table of Contents
Preface. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . ix
iii
Data Collection and Preparation 29
Data Storage and Ingestion 30
Data Exploration and Preparation 33
Data Labeling 35
Feature Stores 36
Model Development and Training 38
Writing and Maintaining Production ML Code 39
Tracking and Comparing Experiment Results 42
Distributed Training and Hyperparameter Optimization 44
Building and Testing Models for Production 45
Deployment (and Online ML Services) 48
From Model Endpoints to Application Pipelines 49
Online Data Preparation 51
Continuous Model and Data Monitoring 52
Monitoring Data and Concept Drift 54
Monitoring Model Performance and Accuracy 57
The Strategy of Pretrained Models 58
Building an End-to-End Hugging Face Application 59
Flow Automation (CI/CD for ML) 61
Conclusion 64
Critical Thinking Discussion Questions 65
Exercises 65
iv | Table of Contents
4. Working with Data and Feature Stores. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 91
Data Versioning and Lineage 92
How It Works 93
Common ML Data Versioning Tools 95
Data Preparation and Analysis at Scale 105
Structured and Unstructured Data Transformations 106
Distributed Data Processing Architectures 107
Interactive Data Processing 108
Batch Data Processing 110
Stream Processing 114
Stream Processing Frameworks 115
Feature Stores 117
Feature Store Architecture and Usage 118
Ingestion and Transformation Service 119
Feature Storage 120
Feature Retrieval (for Training and Serving) 121
Feature Stores Solutions and Usage Example 122
Using Feast Feature Store 123
Using MLRun Feature Store 126
Conclusion 130
Critical Thinking Discussion Questions 131
Exercises 131
Table of Contents | v
6. Deployment of Models and AI Applications. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 161
Model Registry and Management 161
Solution Examples 163
SageMaker Example 163
MLflow Example 165
MLRun Example 166
Model Serving 168
Amazon SageMaker 170
Seldon Core 171
MLRun Serving 173
Advanced Serving and Application Pipelines 176
Implementing Scalable Application Pipelines 177
Model Routing and Ensembles 187
Model Optimization and ONNX 189
Data and Model Monitoring 190
Integrated Model Monitoring Solutions 192
Standalone Model Monitoring Solutions 197
Model Retraining 200
When to Retrain Your Models 201
Strategies for Data Retraining 202
Model Retraining in the MLOps Pipeline 203
Deployment Strategies 203
Measuring the Business Impact 206
Conclusion 206
Critical Thinking Discussion Questions 207
Exercises 207
vi | Table of Contents
Building and Running an Automated Training and Validation Pipeline 234
Real-Time Application Pipeline 238
Defining a Custom Model Serving Class 238
Building an Application Pipeline with Enrichment and Ensemble 238
Testing the Application Pipeline Locally 240
Deploying and Testing the Real-Time Application Pipeline 241
Model Monitoring 242
CI/CD and Continuous Operations 243
Conclusion 246
Critical Thinking Discussion Questions 246
Exercises 246
8. Building Scalable Deep Learning and Large Language Model Projects. . . . . . . . . . . . . 247
Distributed Deep Learning 248
Horovod 249
Ray 250
Data Gathering, Labeling, and Monitoring in DL 251
Data Labeling Pitfalls to Avoid 252
Data Labeling Best Practices 253
Data Labeling Solutions 254
Using Foundation Models as Labelers 256
Monitoring DL Models with Unstructured Data 257
Build Versus Buy Deep Learning Models 258
Foundation Models, Generative AI, LLMs 259
Risks and Challenges with Generative AI 262
MLOps Pipelines for Efficiently Using and Customizing LLMs 267
Application Example: Fine-Tuning an LLM Model 269
Conclusion 281
Critical Thinking Discussion Questions 281
Exercises 282
Index. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 353
As MLOps veterans, we have often seen the following scenario play out across
enterprises building their data science practices.
Traditionally, when enterprises built their data science practice, they would start by
building a model in the lab, with a small team, often working on their laptops and
with a small, manually extracted dataset. They developed the model in operational
isolation, and the results were incorporated manually into applications. Then, once
the model was complete and predicting with accuracy, the true struggle of trying to
bring it to production, to generate real business value, began.
At this point, the enterprise faced challenges such as ingestion of production data,
large scale training, serving in real-time, and monitoring/management of the models
in production. These hurdles would often take months to overcome, presenting a
huge cost in resources and lost time.
The AI pipeline is siloed, with teams working in isolation and with many different
tools and frameworks that don’t necessarily play well with each other. This results
in a huge waste of resources and businesses not being able to capitalize on their
investment in data science. According to Gartner, as many as 85% of data science
projects fall short of expectations.
In this book, we propose a mindset shift, one that addresses these existing challenges
that prevent bringing models to production. We recommend a production-first
approach: starting out not with the model but rather by designing a continuous
operational pipeline, and then making sure the various components and practices
map into it. By automating as many components as possible and making the process
fast and repeatable, the pipeline can scale along with the organization’s needs and
provide rapid business value while answering dynamic and enterprise MLOps needs.
Today, more businesses understand the vast potential of AI models to positively
impact the business across many new use cases. And with generative AI opening
up new opportunities for business innovation across industries, it seems that AI
ix
adoption and usage are set to skyrocket in the coming years. This book explores how
to bring data science to life for these real-world MLOps scenarios.
• Chapters 1–3 show how organizations should approach MLOps, how data sci‐
ence teams can get started, and what to prepare for your first MLOps project.
• Chapters 4–7 explain the components of a resilient and scalable MLOps pipeline
and how to build a machine learning pipeline that scales across the organization.
• Chapter 8 covers deep learning pipelines and also dives into GenAI and LLMs.
• Chapters 9 and 10 show how to adapt pipelines for specific verticals and use
cases, like hybrid deployments, real-time predictions, composite AI, and so on.
Throughout the book, you will find real code examples to interactively try out for
yourself.
x | Preface
After reading this book, you will be a few steps closer to being able to:
Preface | xi
Using Code Examples
Supplemental material (code examples, exercises, and so on) is available for down‐
load at https://github.com/mlrun/demo-fraud and https://github.com/mlrun/demo-llm-
tuning.
If you have a technical question or a problem using the code examples, please send
email to [email protected].
This book is here to help you get your job done. In general, if example code is
offered with this book, you may use it in your programs and documentation. You
do not need to contact us for permission unless you’re reproducing a significant
portion of the code. For example, writing a program that uses several chunks of code
from this book does not require permission. Selling or distributing examples from
O’Reilly books does require permission. Answering a question by citing this book
and quoting example code does not require permission. Incorporating a significant
amount of example code from this book into your product’s documentation does
require permission.
We appreciate, but generally do not require, attribution. An attribution usually
includes the title, author, publisher, and ISBN. For example: “Implementing MLOps in
the Enterprise by Yaron Haviv and Noah Gift (O’Reilly). Copyright 2024 Yaron Haviv
and Noah Gift, 978-1-098-13658-1.”
If you feel your use of code examples falls outside fair use or the permission given
above, feel free to contact us at [email protected].
Our unique network of experts and innovators share their knowledge and expertise
through books, articles, and our online learning platform. O’Reilly’s online learning
platform gives you on-demand access to live training courses, in-depth learning
paths, interactive coding environments, and a vast collection of text and video from
O’Reilly and 200+ other publishers. For more information, visit https://oreilly.com.
xii | Preface
How to Contact Us
Please address comments and questions concerning this book to the publisher:
We have a web page for this book, where we list errata, examples, and any additional
information. You can access this page at https://oreil.ly/mlops-in-the-enterprise.
Email [email protected] to comment or ask technical questions about this
book.
For news and information about our books and courses, visit https://oreilly.com.
Find us on LinkedIn: https://linkedin.com/company/oreilly-media.
Follow us on Twitter: https://twitter.com/oreillymedia.
Watch us on YouTube: https://youtube.com/oreillymedia.
Acknowledgments
We’d like to thank the people behind the scenes who assisted, guided, and supported
us throughout this book’s journey. Without them, this book wouldn’t have been
brought to life.
Thank you to the dedicated team at O’Reilly, who provided feedback and guidance,
drove the writing process of this book, and helped polish the content. We’d especially
like to thank Corbin Collins for being our partner throughout the process, paying
close attention to all the details and helping us meet deadlines, and to Nicole Butter‐
field, for her unwavering support and valuable input.
We’re deeply appreciative of our tech reviewers, Dhanasekar Sundararaman, Tigran
Harutyunyan, Nivas Durairaj, and Noga Cohen for their expertise and wisdom.
Preface | xiii
Another Random Scribd Document
with Unrelated Content
AGED 38.'"
"I don't like that one bit," said Stephen; "it has got too
many hard words in it."
"'IN MEMORY
OF
JOHN POWELL.
DIED IN 1781.
ALSO MARY, RELICT OF
THE ABOVE, WHO DIED
JANUARY 20, 1827,
AGED 87.
ALSO TWO GRANDCHILDREN,
WHO DIED YOUNG.'"
"No, it's a bit of grey hair; she cut it off her mother's
head when she was dead, and she says it's a relict. I don't
know what she means, but she keeps it locked up ever so
safe."
"I hope John Powell didn't lock Mary up," said Stephen.
"She must have got out if he did," said Audrey, "for she
lived a long, long, long time after him. He died in 1781, and
she didn't die not until 1827; let me count up, it's quite a
long sum. Why, it's forty-six years, Stephen!"
CHAPTER V
The Collection
STEPHEN had now quite settled upon the grave which
he was to make his especial care, but he promised not to
begin his work until Audrey had chosen hers. She was very
undecided for a long time, but at length she chose one,
sacred to the memory of another John.
"'BENEATH IS DEPOSITED
ALL THAT WAS MORTAL OF
JOHN HUTTON,
WHO DIED THE 12TH OF APRIL, 1793,
AGED 47.'"
"If Stephen's father will give him a basin, I will give you
one, Audrey," said Granny Robin.
"And I'll get you both an old sponge," said Mr. Robin,
who was smoking his pipe in the window.
At the end of it, they were far from satisfied with their
work.
"I expect she did," said Audrey; "I wonder what has
become of her. Do you think she will ever come to see how
nice we have made her John's grave, Granny Robin?"
"We know very little about it, Stephen," said the old
woman, "but we can't help thinking about it, and dreaming
about it; and I always think of it as a beautiful garden,
where the King walks with His friends. I may be wrong,
Stephie, but that's what I always see in my mind when I
think of it."
"The two grandchildren who died young will like being in
the garden," said Stephen. "Do you think they're glad they
died young, Granny Robin?"
"I think they are, Stephie," she said; "they did not have
to tread far on life's rough ways; their little feet reached the
garden long, long years ago."
"I think you will, Stephie; I feel almost sure you will,"
she said.
"If I see any very dear little children playing under the
trees of the garden," said little Stephen, "I might ask them,
'Are you the two grandchildren who died young?' And then
they could tell me, couldn't they?"
"God bless you, my dear little lad!" was all the answer
Granny Robin gave him.
CHAPTER VI
Angels' Visits
It was only when Audrey told her that if she did so, it
would spoil everything—for the old woman would be sure to
come with her, and would perhaps be angry with them for
doing it without her leave—it was only then that Aunt
Cordelia consented to try the undignified descent.
"Do you think they know what we've been doing?" said
little Stephen.
"Perhaps the angels will tell them when they go back to-
night," said Stephen. "They are sure to notice it when they
come to look at the graves, and I think the little children
will be glad when they hear."
His own grave, as he loved to call it, was lying full in the
pure, silvery light. He could see the flowers he had planted
distinctly, and he could even distinguish some of the words
on the old tombstone. He loved to fancy to himself that the
angels were glad to see it looking so beautiful, that they
were pleased with what he had done, and that they were
lingering round it with bright and happy faces. Some of the
other graves were lying in shadow, but the angels, so he
thought, had gathered round the one upon which he had
bestowed so much care, and were unwilling to leave it
behind.
It was not until clouds came drifting across the sky, and
one of them was driven over the face of the moon, and the
whole churchyard was left in darkness again; it was not
until every ray of moonlight had disappeared, that little
Stephen crept back to bed. The angels were gone, he said,
as he laid his head on the pillow; they had flown away to
the King's Garden, and perhaps, even then, they were
telling the two grandchildren who died young that the
flowers were blooming on their grave, and that it was no
longer forsaken and desolate.
CHAPTER VII
The Mysterious Light
"GRANNY ROBIN—" said Stephen, when he and Audrey
were leaning on her window-seat on the bright Sunday
afternoon which followed that busy Saturday, "Granny
Robin, do you think I shall die young?"
"I can't tell, my dear child," said the old woman, as she
stroked Stephen's little thin hand; "only the dear Lord
above knows that."
"If you have come to Jesus you will, Stephen," she said.
"Yes," said the child; "we would take hold of hands and
go together."
"You have not far to go," said Granny Robin, as she laid
down her knitting and put her arms round the two children.
"Yes, but the barges are ugly," said Audrey. "But the
boats are lovely, and sometimes they have a sail up, and
then I like them best of all. One day me and Stephen went
down on the new walk by the river, and we sat on one of
the seats and watched them."
"No, let's stay outside," said the little girl. "Your father's
at chapel, and Aunt Cordelia's at church, and it's much
darker in than out."
But day after day of that week went by, and they never
saw it again.
"We must wait and see," said Audrey. "I do hope it will
come, and then we will fetch them to see it, and they will all
believe it."
"I never came away," said the old woman; "I'm there
now!"
"I think Granny Robin means her soul can see," said
Audrey.
But Mr. Robin put on his cap, and came with Stephen as
quickly as he could.
"I can't see anything," said Mr. Robin; "it's all dark
inside."
CHAPTER IX
Under the Yew Tree
When she lay awake at night, and felt lonely in the little
top room where she slept, she would creep out of bed and
look into the churchyard, and above the old tower to the
stars shining overhead, and would say to herself:
And then she never felt afraid; for were not the angels
watching over all the Children of Light, their little brothers
and sisters in the Kingdom?