5/7/23, 1:33 PM Using FastAPI to deploy Machine Learning models | by Carl W.

Handlin | Rappi Tech

Published in Rappi Tech

Carl W. Handlin Following

May 25, 2022 · 6 min read · Listen

Save

Using FastAPI to deploy Machine Learning

models
Use FastAPI and Docker to deploy a classification Machine
Learning model built in Scikit-Learn

This tutorial follows on the previous tutorial: Serve your first model with
Scikit-Learn + Flask + Docker: 👇

https://engineering.rappi.com/using-fastapi-to-deploy-machine-learning-models-cd5ed7219ea 1/14
5/7/23, 1:33 PM Using FastAPI to deploy Machine Learning models | by Carl W. Handlin | Rappi Tech

Serve your first model with Scikit-Learn + Flask + Docker

Learn about implementing a microservice for serving a
prediction machine learning model built-in Scikit-Learn
engineering.rappi.com

In this tutorial, we will implement the same microservice to serve a

machine learning model for classification built on Scikit-Learn but using
FastAPI instead of Flask. Why? Well, FastAPI is a modern, fast (high-
performance) and relevant framework for building web APIs with Python, a
good alternative to Flask, and has gained popularity in recent years. Created
by Sebastián Ramírez back in 2018 is usually praised by its superb
documentation and great design.

You can check the full FastAPI documentation here: 👇

GitHub - tiangolo/fastapi: FastAPI framework, high

performance, easy to learn, fast to code, ready…
FastAPI framework, high performance, easy to learn, fast to
code, ready for production Documentation…
github.com

FastAPI compares favorably to Flask on several aspects. Flask does not

provide data validation, automatic documentation or asynchronous
capabilities (without any extension). FastAPI also tends to be consistently
faster than Flask as presented by Christopher GS in “FastAPI vs Flask — The
Complete Guide” [1], by using ASGI (Asynchronous Server Gateway
Interface) instead of WSGI:

https://engineering.rappi.com/using-fastapi-to-deploy-machine-learning-models-cd5ed7219ea 2/14
5/7/23, 1:33 PM Using FastAPI to deploy Machine Learning models | by Carl W. Handlin | Rappi Tech

Web Frameworks Benchmark

In this tutorial we will create a Docker image for deploying a model using
FastAPI as part of the stack, you can find the full code and documentation
here: 👇

GitHub - cwallaceh/sklearn_fastapi_docker: Deployment

of ML models using Python's Scikit-Learn +…
You can't perform that action at this time. You signed in with
another tab or window. You signed out in another tab or…
github.com

Setting up the Environment

First we now need to set up our working environment, for this, it is useful to
start by creating a virtual environment for the project. This is an important
step because, by default, every Python project on your system will use the
same directories to store and retrieve site-packages. By creating a virtual
environment we help to create an isolated environment for Python projects,
to keep track of every package. This way each project can have its own
dependencies, regardless of what dependencies every other project has.

Install the package needed for creating a virtual environment*:

https://engineering.rappi.com/using-fastapi-to-deploy-machine-learning-models-cd5ed7219ea 3/14
5/7/23, 1:33 PM Using FastAPI to deploy Machine Learning models | by Carl W. Handlin | Rappi Tech

$ pip install virtualenv

*If you are using Python 3, then you should already have the virtualenv
module from the standard library installed.

Next up, create a new virtual environment with the name venv:

$ virtualenv venv

It’s also helpful to specify the Python version used to create the environment
by using the option -p and the Python path (This is helpful later when
building the Docker image to match the same Python version).

$ virtualenv -p path_to_python venv

Finally, we can activate the virtual environment:

$ source venv/bin/activate

Commands are a bit different if you are working with Windows, however, this is a
really great guide to get going with virtual environments in Windows:

https://medium.com/co-learning-lounge/create-virtual-environment-python-
windows-2021-d947c3a3ca78

Python packages
Once we have the virtual environment activated we need to install the
software dependencies (site-packages), a list of the Python packages and
their versions are listed in the file requirements.txt:

https://engineering.rappi.com/using-fastapi-to-deploy-machine-learning-models-cd5ed7219ea 4/14
5/7/23, 1:33 PM Using FastAPI to deploy Machine Learning models | by Carl W. Handlin | Rappi Tech

1 anyio==3.5.0
2 asgiref==3.5.0
3 click==8.1.2
4 cycler==0.11.0
5 fastapi==0.75.2
6 fonttools==4.32.0
7 h11==0.13.0
8 idna==3.3
9 joblib==1.1.0
10 kiwisolver==1.4.2
11 matplotlib==3.5.1
12 numpy==1.22.3
13 packaging==21.3
14 pandas==1.4.2
15 Pillow==9.1.0
16 pydantic==1.9.0
17 pyparsing==3.0.8
18 python-dateutil==2.8.2
19 pytz==2022.1
20 scikit-learn==1.0.2
21 scipy==1.8.0
22 six==1.16.0
23 sklearn==0.0
24 sniffio==1.2.0
25 starlette==0.17.1
26 threadpoolctl==3.1.0
27 typing extensions==4.2.0

We can install the dependencies by running the command:

$ pip install -r requirements.txt

After the installation is done, we can check we have correctly installed all
the packages in our virtual environment:

$ pip freeze

https://engineering.rappi.com/using-fastapi-to-deploy-machine-learning-models-cd5ed7219ea 5/14
5/7/23, 1:33 PM Using FastAPI to deploy Machine Learning models | by Carl W. Handlin | Rappi Tech

We should see the same list in the file requirements.txt, this means that now
we are ready to train our model.

After setting up the virtual environment, installing the required Python

packages, we can move forward to training the model and building the
application. But now let’s look at the project structure:

(root) sklearn_fastapi_docker/
├── code/
│ └── train.py
├── data/
│ └── breast_cancer.csv
├── model/
│ └── model_binary.dat.gz
├── ms/
│ ├── __init__.py
│ └── functions.py
├── tests/
│ └── example_calls.txt
├── .gitignore
├── Dockerfile
├── LICENSE
├── README.md
├── main.py
└── requirements.txt

In the folder code/, we will find the main script for training our model
train.py,
Open in apphere we basically follow the standard data science process for access
Get unlimited
training a model: loading the data, preprocessing, training and finally
saving the model inside the folder model/.

$ python code/train.py

Web application
Now we are ready to create our FastAPI application!

https://engineering.rappi.com/using-fastapi-to-deploy-machine-learning-models-cd5ed7219ea 6/14
5/7/23, 1:33 PM Using FastAPI to deploy Machine Learning models | by Carl W. Handlin | Rappi Tech

We will start by creating the web application using FastAPI following the
script in main.py:

197 3

https://engineering.rappi.com/using-fastapi-to-deploy-machine-learning-models-cd5ed7219ea 7/14
5/7/23, 1:33 PM Using FastAPI to deploy Machine Learning models | by Carl W. Handlin | Rappi Tech

1 # Local imports
2 import datetime
3
4 # Third party imports
5 from pydantic import BaseModel, Field
6
7 from ms import app
8 from ms.functions import get_model_response
9
10
11 model_name = "Breast Cancer Wisconsin (Diagnostic)"
12 version = "v1.0.0"
13
14
15 # Input for data validation
16 class Input(BaseModel):
17 concavity_mean: float = Field(..., gt=0)
18 concave_points_mean: float = Field(..., gt=0)
19 perimeter_se: float = Field(..., gt=0)
20 area_se: float = Field(..., gt=0)
21 texture_worst: float = Field(..., gt=0)
22 area_worst: float = Field(..., gt=0)
23
24 class Config:
25 schema_extra = {
26 "concavity_mean": 0.3001,
27 "concave_points_mean": 0.1471,
28 "perimeter_se": 8.589,
29 "area_se": 153.4,
30 "texture_worst": 17.33,
31 "area_worst": 2019.0,
32 }
33
34
35 # Ouput for data validation
36 class Output(BaseModel):
37 label: str
38 prediction: int
39
40
41 @app.get('/info')
42 async def model_info():
43 """Return model information, version, how to call"""
44 return {
https://engineering.rappi.com/using-fastapi-to-deploy-machine-learning-models-cd5ed7219ea 8/14
5/7/23, 1:33 PM Using FastAPI to deploy Machine Learning models | by Carl W. Handlin | Rappi Tech
44 return {
Here
45
we create the routes for the endpoints of our application, an /info route
"name": model_name,
with
46 the GET method version
"version": to retrieve the information from the model and a
47
/predict }
route with the POST method to receive input data, run the data
48
through the pipeline and produce a prediction for the client. The file
49
ms/functions.py
50 contains the auxiliary functions in helping run the model:
@app.get('/health')
51 async def service_health():
52 """Return service health"""
1 import pandas as pd
53 return {
2 from ms import model
54 "ok"
3
55 }
4
56
5 def predict(X, model):
57
6 prediction = model.predict(X)[0]
7 return prediction
8
9
10 def get_model_response(input):
11 X = pd.json_normalize(input.__dict__)
12 prediction = predict(X, model)
13 if prediction == 1:
14 label = "M"
15 else:
16 label = "B"
17 return {
18 'label': label,
19 'prediction': int(prediction)
20 }

At the beginning of the post we mentioned that one of the advantages of

using FastAPI is data validation. FastAPI combines Python type hints and
the Pydantic package to add this behavior. In the ‘Input’ and ‘Output’ classes
declared in main.py we provided the expected variables, types and
additional validations and constraints for each input and output of the
model, which is useful in the case of ML ensuring correct operation of the
machine learning model!

We can now go ahead and test our application by running:

https://engineering.rappi.com/using-fastapi-to-deploy-machine-learning-models-cd5ed7219ea 9/14
5/7/23, 1:33 PM Using FastAPI to deploy Machine Learning models | by Carl W. Handlin | Rappi Tech

$ uvicorn main:app

This command will run the development server locally and listen to port
8000 where we can test our application! The folder /tests contain some
example calls to test that our application is up and running:

$ curl -X GET http://localhost:8000/info

The service should respond:

{
"name": "Breast Cancer Wisconsin (Diagnostic)",
"version": "v1.0.0"
}

FastAPI provide some additional amazing features such a automatic

documentation, with the application running we can visit the endpoint /docs
for the swagger documentation of our recently created API, and /redoc for
redocly documentation:

https://engineering.rappi.com/using-fastapi-to-deploy-machine-learning-models-cd5ed7219ea 10/14
5/7/23, 1:33 PM Using FastAPI to deploy Machine Learning models | by Carl W. Handlin | Rappi Tech

Swagger documentation with endpoints and validations

Redoc documentation with endpoints and validations

Container
Finally, we can create our container, following the Dockerfile we give the
instructions to create an image with everything we need to run the
application we just created.

https://engineering.rappi.com/using-fastapi-to-deploy-machine-learning-models-cd5ed7219ea 11/14
5/7/23, 1:33 PM Using FastAPI to deploy Machine Learning models | by Carl W. Handlin | Rappi Tech

1 # Base image
2 FROM python:3.10
3
4 # Set working directory
5 WORKDIR /app
6
7 # Copy files
8 COPY main.py /app
9 COPY requirements.txt /app
10 COPY model /app/model
11 COPY ms /app/ms
12
13 # Install dependencies
14 RUN pip install -r requirements.txt
15
16 # Run the application
17 EXPOSE 8000
18 CMD ["uvicorn", "main:app", "--host", "0.0.0.0", "--port", "8000"]

An image is essentially built from the instructions for a complete and

executable version of an application, which relies on the host OS kernel. We
start from the official base image of python:3.10, copy our application files
(including the app, the model, and the requirements), install the
requirements, and run the application by using uvicorn exposed through the
port 8000.

We are ready to build the image by running:

$ docker build . -t sklearn_fastapi_docker

The process should take a couple of minutes, but in the end, we are going to
have a Docker image with the name sklearn_fastapi_docker which will serve
as the base for our application.

Serving and Testing

We are now ready to serve our containerized application as a service! We can
test it locally by running:

$ docker run -p 8000:8000 sklearn_fastapi_docker

https://engineering.rappi.com/using-fastapi-to-deploy-machine-learning-models-cd5ed7219ea 12/14
5/7/23, 1:33 PM Using FastAPI to deploy Machine Learning models | by Carl W. Handlin | Rappi Tech

This will launch the application at localhost where we will be able to test it
on the port 8000, we can use another example call to test it with a single
observation:

curl -H "Content-Type: application/json" -d '{

"concavity_mean": 0.3001,
"concave_points_mean": 0.1471,
"perimeter_se": 8.589,
"area_se": 153.4,
"texture_worst": 17.33,
"area_worst": 2019.0
}' -XPOST http://0.0.0.0:8000/predict

The application should respond:

{
"label": "M",
"prediction": 1
}

And we are done! Our application is ready to be served as a prediction

service!

I must add that working with FastAPI feels very easy for creating this type of
applications for machine learning models. Data validation and
documentation provide two benefits of working with the tool without the
need for additional components. Give it a try!

References
[1] Christopher Samiullah, 2021 “FastAPI vs Flask — The Complete Guide”
https://christophergs.com/python/2021/06/16/python-flask-fastapi/

Fastapi Machine Learning Artificial Intelligence Docker Scikit Learn

https://engineering.rappi.com/using-fastapi-to-deploy-machine-learning-models-cd5ed7219ea 13/14
5/7/23, 1:33 PM Using FastAPI to deploy Machine Learning models | by Carl W. Handlin | Rappi Tech

https://engineering.rappi.com/using-fastapi-to-deploy-machine-learning-models-cd5ed7219ea 14/14