Auth0 Com Blog Sqlalchemy Orm Tutorial for Python Developers
Auth0 Com Blog Sqlalchemy Orm Tutorial for Python Developers
SQLAlchemy is a library that facilitates the communication between Python programs and databases. Most of the
times, this library is used as an Object Relational Mapper (ORM) tool that translates Python classes to tables on
relational databases and automatically converts function calls to SQL statements. SQLAlchemy provides a standard
interface that allows developers to create database-agnostic code to communicate with a wide variety of database
engines.
Python DBAPI
The Python DBAPI (an acronym for DataBase API) was created to specify how Python modules that integrate with
databases should expose their interfaces. Although we won't interact with this API directly—we will use
SQLAlchemy as a facade to it—it's good to know that it defines how common functions like connect , close ,
commit , and rollback must behave.
In this article, we are going to install and use the most popular PostgreSQL DBAPI implementation available: psycopg .
SQLAlchemy Engines
Whenever we want to use SQLAlchemy to interact with a database, we need to create an Engine. Engines, on
SQLAlchemy, are used to manage two crucial factors: Pools and Dialects. The following two sections will explain what
these two concepts are, but for now it suffices to say that SQLAlchemy uses them to interact with DBAPI functions.
To create an engine and start interacting with databases, we have to import the create_engine function from the
sqlalchemy library and issue a call to it:
engine =
create_engine('postgresql://usr:pass@localhost:5432/sqlalchemy')
This example creates a PostgreSQL engine to communicate with an instance running locally on port 5432
(the default one). It also defines that it will use usr and pass as the credentials to interact with the
TABLE OF CONTENTSsqlalchemy database. Note that, creating an engine does not connect to the database instantly. This
process is postponed to when it's needed (like when we submit a query, or when create/update a row in a table).
Since SQLAlchemy relies on the DBAPI specification to interact with databases, the most common database
management systems available are supported. PostgreSQL, MySQL, Oracle, Microsoft SQL Server, and SQLite are all
examples of engines that we can use alongside with SQLAlchemy.
There are various implementations of the connection pool pattern available on SQLAlchemy . For example, creating an
Engine through the create_engine() function usually generates a QueuePool. This kind of pool comes configured
with some reasonable defaults, like a maximum pool size of 5 connections.
As usual production-ready programs need to override these defaults (to fine-tune pools to their needs), most of the
different implementations of connection pools provide a similar set of configuration options. The following list shows
the most common options with their descriptions:
pool_size: Sets the number of connections that the pool will handle.
pool_recycle: Configures the maximum age (in seconds) of connections in the pool.
pool_timeout: Identifies how many seconds the program will wait before giving up
on getting a connection from the pool.
SQLAlchemy Dialects
As SQLAlchemy is a facade that enables Python developers to create applications that communicate to different
database engines through the same API, we need to make use of Dialects. Most of the popular relational databases
available out there adhere to the SQL (Structured Query Language) standard, but they also introduce proprietary
variations. These variations are the solely responsible for the existence of dialects.
For example, let's say that we want to fetch the first ten rows of a table called people . If our data was being held by
a Microsoft SQL Server database engine, SQLAlchemy would need to issue the following query:
Therefore, to know precisely what query to issue, SQLAlchemy needs to be aware of the type of the database that it is
dealing with. This is exactly what Dialects do. They make SQLAlchemy aware of the dialect it needs to talk.
Firebird
Microsoft SQL Server
MySQL
Oracle
PostgreSQL
SQLite
Sybase
SQLAlchemy ORM
ORM, which stands for Object Relational Mapper, is the specialization of the Data Mapper design pattern that
addresses relational databases like MySQL, Oracle, and PostgreSQL. Mappers are responsible for moving data
between objects and a database while keeping them independent of each other. As object-oriented programming
languages and relational databases structure data on different ways, we need specific code to translate from one
schema to the other.
To understand how we use SQLAlchemy data types to map properties of Python classes into columns on a relation
database table, let's analyze the following example:
class Product(Base):
__tablename__ = 'products'
id=Column(Integer, primary_key=True)
title=Column('title', String(32))
in_stock=Column('in_stock', Boolean)
quantity=Column('quantity', Integer)
price=Column('price', Numeric)
In the code snippet above, we are defining a class called Product that has six properties. Let's take a look at what
these properties do:
The __tablename__ property tells SQLAlchemy that rows of the products table must be mapped to this class.
The id property identifies that this is the primary_key in the table and that its type is Integer.
The title property indicates that a column in the table has the same name of the property and that its type
is String.
The in_stock property indicates that a column in the table has the same name of the property and that its type
is Boolean.
The quantity property indicates that a column in the table has the same name of the property and that its type
is Integer.
The price property indicates that a column in the table has the same name of the property and that its type
is Numeric.
Seasoned developers will notice that (usually) relational databases do not have data types with these exact names.
SQLAlchemy uses these types as generic representations to what databases support and use the dialect configured to
understand what types they translate to. For example, on a PostgreSQL database, the title would be mapped to a
varchar column.
Note that this section will be an overview of all these types, but in the
class Article(Base):
__tablename__= 'articles'
id = Column(Integer, primary_key
=True)
comments= relationship
("Comment")
class Comment(Base):
__tablename__= 'comments'
SQLAlchemy ORM in Practice action we will do a hands-on to practice mapping classes into
tables and to learn how to insert, extract, and remove data from these tables.
The first type, One To Many, is used to mark that an instance of a class can be associated with many instances of
another class. For example, on a blog engine, an instance of the Article class could be associated with many
instances of the Comment class. In this case, we would map the mentioned classes and its relation as follows:
The second type, Many To One, refers to the same relationship described above but from the other perspective. To
give a different example, let's say that we want to map the relationship between instances of
Tire to an instance of a Car . As many tires belong to one car and this car contains many tires, we would map this
relation as follows:
class Tire(Base):
__tablename__= 'tires'
id = Column(Integer, primary_key
=True)
car_id = Column(Integer, ForeignKey('cars.id'))
car = relationship
("Car")
class Car(Base):
__tablename__= 'cars'
id = Column(Integer, primary_key
=True)
The third type, One To One, refers to relationships where an instance of a particular class may only be associated with
one instance of another class, and vice versa. As an example, consider the relationship between a Person and a
MobilePhone . Usually, one person possesses one mobile phone and this mobile phone belongs to this person only.
To map this relationship on SQLAlchemy, we would create the following code:
class Person(Base):
__tablename__= 'people'
id = Column(Integer, primary_key
=True)
mobile_phone= relationship
("MobilePhone"
, uselist=False, back_populates
="person")
class MobilePhone(Base):
__tablename__= 'mobile_phones'
id = Column(Integer, primary_key
=True)
person_id= Column(Integer, ForeignKey('people.id'))
person = relationship
("Person", back_populates
="mobile_phone"
)
In this example, we pass two extra parameters to the relationship function. The first one, uselist=False ,
makes SQLAlchemy understand that mobile_phone will hold only a single instance and
not an array (multiple) of instances. The second one, back_populates , instructs SQLAlchemy to populate the other
side of the mapping.
The last type supported by SQLAlchemy, Many To Many, is used when instances of a particular class can have zero or
more associations to instances of another class. For example, let's say that we are mapping the relationship of
instances of Student and instances of Class in a system that manages a school. As many students can participate in
many classes, we would map the relationship as follows:
students_classes_association
= Table('students_classes'
, Base.metadata,
Column('student_id'
, Integer, ForeignKey('students.id'
)),
Column('class_id', Integer, ForeignKey('classes.id'
))
class Student(Base):
__tablename__= 'students'
id = Column(Integer, primary_key
=True)
classes = relationship
("Class", secondary=students_classes_association
)
class Class(Base):
__tablename__= 'classes'
id = Column(Integer, primary_key
=True)
In this case, we had to create a helper table to persist the association between instances of Student and
instances of Class , as this wouldn't be possible without an extra table. Note that, to make SQLAlchemy
aware of the helper table, we passed it in the secondary parameter of the relationship function.
reason, we need to delete rows from shopping_carts we will need to delete the related rows from
shopping_carts_products as well. Otherwise we will end up with a lot of garbage and unfulfilled references
in our database.
To make this kind of operation easy to maintain, SQLAlchemy ORM enables developers to map cascade behavior when
using relationship() constructs. Like that, when operations are performed on parent objects, child objects get
updated/deleted as well. The following list provides a brief explanation of the most used cascade strategies on
SQLAlchemy ORM:
If more information about this feature is needed, the SQLAlchemy documentation provides an excellent chapter about
Cascades.
SQLAlchemy Sessions
Sessions, on SQLAlchemy ORM, are the implementation of the Unit of Work design pattern. As explained by Martin
Fowler, a Unit of Work is used to maintain a list of objects affected by a business transaction and to coordinate the
writing out of these changes. This means that all modifications tracked by Sessions (Units of Works) will be applied to
the underlying database together, or none of them will. In other words, Sessions are used to guarantee the database
consistency.
The official SQLAlchemy ORM documentation about Sessions gives a great explanation how changes are tracked, how
to get sessions, and how to create ad-hoc sessions. However, in this article, we will use the most basic form of session
creation:
As we can see from the code snippet above, we only need one step to get sessions. We need to create a session
factory that is bound to the SQLAlchemy engine. After that, we can just issue calls to this session factory to get our
sessions.
# create an engine
engine = create_engine
('postgresql://usr:pass@localhost:5432/sqlalchemy'
)
# create a Session
session = Session()
SQLAlchemy in Practice
Now that we got a better understanding of the most important pieces of SQLAlchemy, it's time to start practicing it. In
the following sections, we will create a small project based on pipenv —a Python dependency manager—and add
some classes to it. Then we will map these classes to tables persisted to a
Running PostgreSQL
To be able to practice our new skills and to learn how to query data on SQLAlchemy, we will need a database to
support our examples. As already mentioned, SQLAlchemy provides support for many different databases engines, but
the instructions that follow will focus on PostgreSQL. There are many ways to get an instance of PostgreSQL. One of
them is to use some cloud provider like Heroku or ElephantSQL (both of them have free tiers). Another possibility is to
install PostgreSQL locally on our current environment. A third option is to run a PostgreSQL instance inside a Docker
container.
The third option is probably the best choice because it has the performance of an instance running locally, it's free
forever, and because it's easy to create and destroy Docker instances. The only (small) disadvantage is that we need to
install Docker locally.
After having Docker installed, we can create and destroy dockerized PostgreSQL instances with the following
commands:
-e POSTGRES_USER
=usr \
-e POSTGRES_DB=sqlalchemy \
-p 5432:5432 \
-d postgres
# stop instance
docker stop sqlalchemy-orm-psql
# destroy instance
docker rm sqlalchemy-orm-psql
The first command, the one that creates the PostgreSQL instance, contains a few parameters that are worth
inspecting:
-e POSTGRES_DB : Defines the main (and only) database available in the PostgreSQL instance.
-p 5432:5432 : Defines that the local 5432 port will tunnel connections to the same port in the Docker
instance.
-d postgres : Defines that this Docker instance will be created based on the official PostgreSQL repository.
Installing SQLAlchemy Dependencies
In this tutorial, we will need to install only two packages: sqlalchemy and psycopg2 . The first dependency refers
to SQLAlchemy itself and the second one, psycopg2 , is the PostgreSQL driver that SQLAlchemy will use to
communicate with the database. To install these dependencies, we will use pipenv as shown:
This command will download both libraries and make them available in our Python virtual environment. Note that to
run the scripts that we are going to create, we first need to spawn the virtual environment shell. That is, before
executing python somescript.py , we need to execute pipenv shell . Otherwise, Python won't be able to
find the installed dependencies, as they are just available in our new virtual environment.
# coding=utf-8
engine = create_engine
('postgresql://usr:pass@localhost:5432/sqlalchemy'
)
Session = sessionmaker
(bind=engine)
Base = declarative_base
()
a SQLAlchemy Engine that will interact with our dockerized PostgreSQL database, a
SQLAlchemy ORM session factory bound to this engine, and a base class for our
classes definitions.
Now let's create and map the Movie class. To do this, let's create a new file called movie.py and add the following
code to it:
The definition of this class and its mapping characteristics is quite simple. We start by making this class
extend the Base class defined in the base.py module and then we add four properties to it:
# coding=utf-8
class Movie(Base):
__tablename__= 'movies'
id = Column(Integer, primary_key
=True)
title = Column(String)
release_date= Column(Date)
def __init__(self, title, release_date
):
self
.title = title
self
.release_date= release_date
1. A __tablename__ to indicate what is the name of the table that will support this class.
2. An id to represent the primary key in the table.
3. A title of type String .
The next class that we will create and map is the Actor class. Let's create a file called actor.py and add the
following code to it:
# coding=utf-8
from sqlalchemy
import Column
, String
, Integer
, Date
class Actor(Base):
__tablename__
= 'actors'
id = Column
(Integer
, primary_key
=True)
name= Column
(String
)
birthday
= Column
(Date)
def __init__
(self, name, birthday
):
self
.name = name
self
.birthday= birthday
The definition of this class is pretty similar to the previous one. The differences are that
Actorthehas a
name instead of a title , a birthday instead of a release_date , and that it points to a table called actors
instead of movies .
As many movies can have many actors and vice-versa, we will need to create a Many To Many relationship
between these two classes. Let's create this relationship by updating the movie.py file as follows:
# coding=utf-8
movies_actors_association
= Table(
'movies_actors'
, Base.metadata,
Column('movie_id', Integer, ForeignKey('movies.id')),
Column('actor_id', Integer, ForeignKey('actors.id'))
)
class Movie(Base):
__tablename__= 'movies'
id = Column(Integer, primary_key
=True)
title = Column(String)
release_date= Column(Date)
actors = relationship
("Actor", secondary=movies_actors_association
)
self
.release_date= release_date
The difference between this version and the previous one is that:
added the actors property to Movie and configured the movies_actors_association as the
intermediary table.
The next class that we will create is Stuntman . In our tutorial, a particular Actor will have only one
Stuntman and this Stuntman will work only with this Actor . This means that we need to create the
The fourth and final class that we will map in our tutorial is ContactDetails . Instances of this class will hold a
phone_number and an address of a particular Actor , and one Actor will be able to have many
Stuntmanclass andO
aneT
oOnerelationship between these classes. To accomplish that, let's create a file
called stuntman
.py and add the following code to it:
# coding=utf-8
from sqlalchemy
importColumn
, String
, Integer
, Boolean
, ForeignKey
from sqlalchemy
.orm importrelationship
, backref
from baseimportBase
classStuntman
(Base
):
__tablename__
= 'stuntmen'
id = Column
(Integer
, primary_key
=True
)
name
= Column
(String
)
active
= Column
(Boolean
)
actor_id
= Column
(Integer
, ForeignKey
('actors.id'
))
actor
= relationship
("Actor"
, backref
=backref
("stuntman"
, uselist
=False
))
def __init__
(self
, name
, active
, actor
):
self
.name= name
self
.active= active
self
.actor= actor
ContactDetails associated. Therefore, we will need to use the Many To One relationship pattern to map this
association. To create this class and this association, let's create a file called contact_details.py and add the
following source code to it:
# coding=utf-8
from sqlalchemy import Column, String, Integer, ForeignKey
from sqlalchemy.orm import relationship
class ContactDetails
(Base):
__tablename__= 'contact_details'
id = Column(Integer, primary_key
=True)
phone_number= Column(String)
address = Column(String)
actor_id= Column(Integer, ForeignKey('actors.id'))
actor = relationship
("Actor", backref="contact_details"
)
As we can see, creating a Many To One association is kinda similar to creating a One To One association. The
difference is that in the latter we instructed SQLAlchemy not to use lists. This instruction ends up restricting the
association to a single instance instead of a list of instances.
# coding=utf-8
# 1 - imports
from datetime import date
session.add(dwayne_contact_2
)
session.add(mark_contact
)
session.add(matt_stuntman
)
session.add(dwayne_stuntman
)
session.add(mark_stuntman
)
1. The first section imports the classes that we created, the SQLAlchemy engine, the Base class, the session
factory, and date from the datetime module.
2. The second section instructs SQLAlchemy to generate the database schema. This generation occurs based on
the declarations that we made while creating the four main classes that compose our tutorial.
3. The third section extracts a new session from the session factory.
4. The fourth section creates three instances of the Movie class.
5. The fifth section creates three instances of the Actor class.
6. The sixth section adds actors to movies. Note that the Pain & Gain movie references two actors: Dwayne
Johnson and Mark Wahlberg.
7. The seventh section creates instances of the ContactDetails class and defines what actors these instances
are associated to.
8. The eighth section defines three stuntmen and also defines what actors these stuntmen are associated to.
9. The ninth section uses the current session to save the movies, actors, contact details, and stuntmen created.
Note that we haven't explicitly saved actors. This is not needed because SQLAlchemy, by default, uses the
save-update cascade strategy.
10. The tenth section commits the current session to the database and closes it.
To run this Python script, we can simply issue the python inserts.py command (let's not forget to run pipenv
shell first) in the main directory of our database. Running it will create five tables in the PostgreSQL
database and populate these tables with the data that we created. In the next section, we will learn how to query
these tables.
Querying Data with SQLAlchemy ORM
As we will see, querying data with SQLAlchemy ORM is quite simple. This library provides an intuitive, fluent API that
enables developers to write queries that are easy to read and to maintain. On SQLAlchemy ORM, all queries start with
a Query Object that is extracted from the current session and that is associated with a particular mapped class. To see
this API in action, let's create a file called queries.py and add to it the following source code:
# coding=utf-
8
# 1 -
imports
fro actorimpor
m t Actor
fro baseimpor
m t Session
fro contact_details
impor
m t ContactDetails
fro movieimpor
m t Movie
# 2 - extract a
session
session
= ()
Session
# 3 - extract all
movies
movies
= .quer(Movi).al ()
sessiony e l
# 4 - print movies'
details
prin('\n### All )
t movies:'
fo movie i :
r n movies
prin(f {movi.titl} was released
{movi
on .release_dat
}')
t ' e e e e
prin(' )
t '
To explore the usage of some of these functions, let's append the following code to queries
the .py script:
# 1 - imports
from datetimeimport date
.all()
.all()
print('')
The fifth section of the updated script uses the filter() function to fetch only movies that were released after
January the first, 2015. The sixth section shows how to use join() to fetch instances of Movie that the Actor
Dwayne Johnson participated in. The seventh and last section, shows the usage of join() and ilike() functions to
retrieve actors that have houses in Glendale.
Running the new version of the script (python queries.py ) now will result in the following output:
As we can see, using the API is straightforward and generates a code that is readable. To see other functions
supported by the Query API, and their description, take a look at the official documentation.
For example, to secure Python APIs written with Flask, we can simply create arequires_auth decorator:
def get_token_auth_header
():
"""Obtains the access token from the Authorization Header
"""
auth = request.headers.get("Authorization"
, None)
if not auth:
raise AuthError({"code": "authorization_header_missing"
,
"description"
:
"Authorization header is expected"
}, 401)
parts = auth.split()
if parts[0].lower() != "bearer":
raise AuthError({"code": "invalid_header"
,
"description"
:
"Authorization header must start with"
"description"
:
"incorrect claims,"
"please check the audience and issuer"
}, 401)
except Exception:
raise AuthError({"code": "invalid_header"
,
"description"
:
"Unable to parse authentication"
" token."}, 400)
_app_ctx_stack
.top.current_user= payload
return f(*args, **kwargs)
# Controllers API
To learn more about securing Python APIs with Auth0, take a look at this tutorial . Alongside with tutorials for
backend technologies (like Python, Java, and PHP), the Auth0 Docs webpage also provides tutorials for
Mobile/Native apps and Single-Page applications.