100% found this document useful (8 votes)
76 views

Full Download (Ebook) Python Descriptors: Understanding and Using the Descriptor Protocol by Jacob Zimmerman ISBN 9781484237267, 1484237269 PDF DOCX

The document is a promotional material for the ebook 'Python Descriptors: Understanding and Using the Descriptor Protocol' by Jacob Zimmerman, which is available for download in various formats. It includes details about the book's content, such as chapters on descriptors, their protocol, and practical applications. Additionally, it provides links to other related ebooks and information about the author and technical reviewer.

Uploaded by

hiedarhameb
Copyright
© © All Rights Reserved
Available Formats
Download as PDF, TXT or read online on Scribd
100% found this document useful (8 votes)
76 views

Full Download (Ebook) Python Descriptors: Understanding and Using the Descriptor Protocol by Jacob Zimmerman ISBN 9781484237267, 1484237269 PDF DOCX

The document is a promotional material for the ebook 'Python Descriptors: Understanding and Using the Descriptor Protocol' by Jacob Zimmerman, which is available for download in various formats. It includes details about the book's content, such as chapters on descriptors, their protocol, and practical applications. Additionally, it provides links to other related ebooks and information about the author and technical reviewer.

Uploaded by

hiedarhameb
Copyright
© © All Rights Reserved
Available Formats
Download as PDF, TXT or read online on Scribd
You are on page 1/ 41

Download Full Version ebook - Visit ebooknice.

com

(Ebook) Python Descriptors: Understanding and


Using the Descriptor Protocol by Jacob Zimmerman
ISBN 9781484237267, 1484237269

https://ebooknice.com/product/python-descriptors-
understanding-and-using-the-descriptor-protocol-7210836

Click the button below to download

DOWLOAD EBOOK

Discover More Ebook - Explore Now at ebooknice.com


Instant digital products (PDF, ePub, MOBI) ready for you
Download now and discover formats that fit your needs...

Start reading on any device today!

(Ebook) Python Descriptors: Understanding and Using the


Descriptor Protocol by Jacob Zimmerman ISBN 9781484237267,
9781484237274, 1484237269, 1484237277
https://ebooknice.com/product/python-descriptors-understanding-and-
using-the-descriptor-protocol-43138602

ebooknice.com

(Ebook) Python Descriptors by Jacob Zimmerman ISBN


9781484225042, 9781484225059, 148422504X, 1484225058

https://ebooknice.com/product/python-descriptors-5684810

ebooknice.com

(Ebook) SIP: Understanding the Session Initiation Protocol


by Alan B. Johnston ISBN 9781607839958, 1607839954

https://ebooknice.com/product/sip-understanding-the-session-
initiation-protocol-1378072

ebooknice.com

(Ebook) SIP: Understanding the Session Initiation Protocol


by Alan B. Johnston ISBN 9781580536554, 1580536557

https://ebooknice.com/product/sip-understanding-the-session-
initiation-protocol-977908

ebooknice.com
(Ebook) Using Asyncio in Python: Understanding Python's
Asynchronous Programming Features by Caleb Hattingh ISBN
9781492075332, 1492075337, B084D653HW
https://ebooknice.com/product/using-asyncio-in-python-understanding-
python-s-asynchronous-programming-features-10698978

ebooknice.com

(Ebook) Introduction to Computation and Programming Using


Python With Application to Understanding Data by John V.
Guttag ISBN 9780262529624, 0262529629
https://ebooknice.com/product/introduction-to-computation-and-
programming-using-python-with-application-to-understanding-
data-5852392
ebooknice.com

(Ebook) Introduction to Computation and Programming Using


Python with Application to Understanding Data by Guttag,
John V ISBN 9780262337397, 9780262529624, 0262337398,
0262529629
https://ebooknice.com/product/introduction-to-computation-and-
programming-using-python-with-application-to-understanding-
data-11821064
ebooknice.com

(Ebook) Sip: Understanding the Session Initiation Protocol


by Alan B. Johnston ISBN 9781580531689, 9781580534130,
1580531687, 1580534139
https://ebooknice.com/product/sip-understanding-the-session-
initiation-protocol-1888062

ebooknice.com

(Ebook) Biota Grow 2C gather 2C cook by Loucas, Jason;


Viles, James ISBN 9781459699816, 9781743365571,
9781925268492, 1459699815, 1743365578, 1925268497
https://ebooknice.com/product/biota-grow-2c-gather-2c-cook-6661374

ebooknice.com
Python
Descriptors
Understanding and Using the
Descriptor Protocol

Second Edition

Jacob Zimmerman
Python Descriptors
Understanding and Using
the Descriptor Protocol
Second Edition

Jacob Zimmerman
Python Descriptors: Understanding and Using the Descriptor Protocol
Jacob Zimmerman
New York, USA

ISBN-13 (pbk): 978-1-4842-3726-7 ISBN-13 (electronic): 978-1-4842-3727-4


https://doi.org/10.1007/978-1-4842-3727-4
Library of Congress Control Number: 2018960194
Copyright © 2018 by Jacob Zimmerman
This work is subject to copyright. All rights are reserved by the Publisher, whether the whole or
part of the material is concerned, specifically the rights of translation, reprinting, reuse of
illustrations, recitation, broadcasting, reproduction on microfilms or in any other physical way,
and transmission or information storage and retrieval, electronic adaptation, computer software,
or by similar or dissimilar methodology now known or hereafter developed.
Trademarked names, logos, and images may appear in this book. Rather than use a trademark
symbol with every occurrence of a trademarked name, logo, or image we use the names, logos,
and images only in an editorial fashion and to the benefit of the trademark owner, with no
intention of infringement of the trademark.
The use in this publication of trade names, trademarks, service marks, and similar terms, even if
they are not identified as such, is not to be taken as an expression of opinion as to whether or not
they are subject to proprietary rights.
While the advice and information in this book are believed to be true and accurate at the date of
publication, neither the authors nor the editors nor the publisher can accept any legal
responsibility for any errors or omissions that may be made. The publisher makes no warranty,
express or implied, with respect to the material contained herein.
Managing Director, Apress Media LLC: Welmoed Spahr
Acquisitions Editor: Steve Anglin
Development Editor: Matthew Moodie
Coordinating Editor: Mark Powers
Cover designed by eStudioCalamar
Cover image designed by Freepik (www.freepik.com)
Distributed to the book trade worldwide by Springer Science+Business Media New York, 233
Spring Street, 6th Floor, New York, NY 10013. Phone 1-800-SPRINGER, fax (201) 348-4505, e-mail
[email protected], or visit www.springeronline.com. Apress Media, LLC is a
California LLC and the sole member (owner) is Springer Science + Business Media Finance Inc
(SSBM Finance Inc). SSBM Finance Inc is a Delaware corporation.
For information on translations, please e-mail [email protected]; for reprint, paperback, or
audio rights, please email [email protected].
Apress titles may be purchased in bulk for academic, corporate, or promotional use. eBook
versions and licenses are also available for most titles. For more information, reference our Print
and eBook Bulk Sales web page at http://www.apress.com/bulk-sales.
Any source code or other supplementary material referenced by the author in this book is
available to readers on GitHub via the book’s product page, located at www.apress.com/
9781484237267. For more detailed information, please visit http://www.apress.com/
source-code.
Printed on acid-free paper
Table of Contents
About the Author��������������������������������������������������������������������������������vii

About the Technical Reviewer�������������������������������������������������������������ix


Acknowledgments�������������������������������������������������������������������������������xi
Introduction���������������������������������������������������������������������������������������xiii

Part I: About Descriptors�������������������������������������������������������������1


Chapter 1: What Is a Descriptor?����������������������������������������������������������3
Data Descriptors versus Non-Data Descriptors����������������������������������������������������4
The Use of Descriptors by Python�������������������������������������������������������������������������4
Summary��������������������������������������������������������������������������������������������������������������5

Chapter 2: The Descriptor Protocol������������������������������������������������������7


The __get__(self, instance, owner) Method����������������������������������������������������������7
The __set__(self, instance, value) Method��������������������������������������������������������� 8
The __delete__(self, instance) Method�����������������������������������������������������������������9
Summary������������������������������������������������������������������������������������������������������������10

Chapter 3: What Are Descriptors Good For?���������������������������������������11


Pros of Python Descriptors���������������������������������������������������������������������������������11
Encapsulation������������������������������������������������������������������������������������������������11
Reuse of Read/Write Patterns�����������������������������������������������������������������������12
Writing for the Class Level�����������������������������������������������������������������������������13

iii
Table of Contents

Cons of Python Descriptors���������������������������������������������������������������������������������13


Encapsulation������������������������������������������������������������������������������������������������13
Can Be Difficult To Write��������������������������������������������������������������������������������14
Additional Objects�����������������������������������������������������������������������������������������14
Summary������������������������������������������������������������������������������������������������������������14

Chapter 4: Descriptors in the Standard Library����������������������������������15


The property Class����������������������������������������������������������������������������������������������16
The classmethod Descriptor�������������������������������������������������������������������������������18
The staticmethod Descriptor�������������������������������������������������������������������������������19
Regular Methods�������������������������������������������������������������������������������������������������19
Summary������������������������������������������������������������������������������������������������������������20

Chapter 5: Attribute Access and Descriptors�������������������������������������21


Instance Access��������������������������������������������������������������������������������������������������22
Set and Delete Calls��������������������������������������������������������������������������������������������27
The Reasoning Behind Data versus Non-Data Descriptors��������������������������������28
Summary������������������������������������������������������������������������������������������������������������28

Part II: Making Descriptors�������������������������������������������������������29


Chapter 6: Which Methods Are Needed?��������������������������������������������31
When __get__() Is Called Without an instance Argument����������������������������������32
Raise Exception or Return self����������������������������������������������������������������������32
“Unbound” Attributes������������������������������������������������������������������������������������33
Summary������������������������������������������������������������������������������������������������������������37

Chapter 7: Storing the Attributes�������������������������������������������������������39


Class-Level Storage��������������������������������������������������������������������������������������������39
Storing Data on the Descriptor����������������������������������������������������������������������������40
Storing on the Instance Dictionary����������������������������������������������������������������46

iv
Table of Contents

Asking for the Location����������������������������������������������������������������������������������48


Indirectly Asking for the Location�����������������������������������������������������������������������50
Name Mangling���������������������������������������������������������������������������������������������51
Store the Original and the Mangled���������������������������������������������������������������54
Summary������������������������������������������������������������������������������������������������������������57

Chapter 8: Read-Only Descriptors������������������������������������������������������59


Set-Once Descriptors������������������������������������������������������������������������������������������60
Secret-Set Descriptors����������������������������������������������������������������������������������������61
Forced-Set Descriptors���������������������������������������������������������������������������������������62
Class Constants��������������������������������������������������������������������������������������������������63
Summary������������������������������������������������������������������������������������������������������������65

Chapter 9: Writing__delete__()�����������������������������������������������������������67
Summary������������������������������������������������������������������������������������������������������������68

Chapter 10: Descriptors Are Classes Too��������������������������������������������69


Inheritance����������������������������������������������������������������������������������������������������������69
More Methods�����������������������������������������������������������������������������������������������������70
Optional/Default Parameters�������������������������������������������������������������������������������70
Descriptors on Descriptors���������������������������������������������������������������������������������70
Passing an Instance Around��������������������������������������������������������������������������������71
Descriptors Just Abstract Method Calls��������������������������������������������������������������72
Summary������������������������������������������������������������������������������������������������������������72

Chapter 11: Reusing the Wheel����������������������������������������������������������73


Storage Solutions������������������������������������������������������������������������������������������������73
Read-Only Solutions��������������������������������������������������������������������������������������������76
Simple Unbound Attributes���������������������������������������������������������������������������������78
Summary������������������������������������������������������������������������������������������������������������80

v
Table of Contents

Chapter 12: Instance-Level Descriptors���������������������������������������������81


Properties in Other Languages���������������������������������������������������������������������������81
Back to Python����������������������������������������������������������������������������������������������������82
Attempt 1�������������������������������������������������������������������������������������������������������82
Attempt 2�������������������������������������������������������������������������������������������������������83
Attempt 3�������������������������������������������������������������������������������������������������������85
Attempt 4�������������������������������������������������������������������������������������������������������85
Example��������������������������������������������������������������������������������������������������������������87
Go Nuts���������������������������������������������������������������������������������������������������������������88

Chapter 13: Other Uses of Descriptors In the World���������������������������89


SQLAlchemy��������������������������������������������������������������������������������������������������������89
Jigna�������������������������������������������������������������������������������������������������������������������90
Elk�����������������������������������������������������������������������������������������������������������������������90
Validators������������������������������������������������������������������������������������������������������������91
Summary������������������������������������������������������������������������������������������������������������91

Bibliography���������������������������������������������������������������������������������������93

Index���������������������������������������������������������������������������������������������������95

vi
About the Author
Jacob Zimmerman is a blogger, gamer (tabletop more so than video
games), and programmer who was born and raised in Wisconsin. He has a
twin brother who could also be considered to have all those traits.
Jacob has his own programming blog that focuses on Java, Kotlin,
and Python programming, called “Programming Ideas with Jake”. He also
writes for a gaming blog with his brother-in-law called the “Ramblings of
Jacob and Delos”.
His brother writes a JavaScript blog called JoeZimJS and works with
his best friend on a gaming YouTube channel called “Bork & Zim Gaming,”
which Jacob helps out with on occasion.
Programming Ideas with Jake
http://programmingideaswithjake.wordpress.com/
Ramblings of Jacob and Delos
http://www.ramblingsofjacobanddelos.com/
JoeZimJS
http://www.joezimjs.com

vii
About the Technical Reviewer
Michael Thomas has worked in software development for more than 20
years as an individual contributor, team lead, program manager, and vice
president of engineering. Michael has more than 10 years of experience
working with mobile devices. His current focus is in the medical sector,
using mobile devices to accelerate information transfer between patients
and health care providers.

ix
Acknowledgments
In order to be sure that I got everything right—it would really suck for a
“comprehensive guide” to be missing a big chunk of functionality or to get
anything wrong—I enlisted the help of some Python experts on the first
edition. In return for their help, I let them introduce themselves to you
here. That’s not all I did in return, but it’s all you’re going to see :)
Emanuel Barry is a self-taught Python programmer who loves pushing
the language to its limits as well as exploring its darkest corners. He has to
do a lot of proofreading and editing for a local non-for-profit organization,
and decided to combine his love of Python and knowledge sharing with
his background in proofreading to help make this book even better. He can
often be found in the shadows of the mailing lists or the issue tracker, as
well as the Python IRC channel, as Vgr.
Chris Angelico has played around with Python since the late 90s, getting
more serious with the language in the mid 2000s. As a PEP Editor and active
participant in the various mailing lists, he keeps well up to date with what’s
new and upcoming in the language and also shares that knowledge with
fledgling students in the Thinkful tutoring/mentoring program. When not
coding in Python, he is often found wordsmithing for a Dungeons & Dragons
campaign, or exploring the linguistic delights of Alice in Wonderland and
similar works. If you find a subtle Alice reference in this text, blame him!

https://github.com/Rosuav

Kevin Mackay is a software engineer who has been programming in


Python since 2010 and is currently working at BBC, improving the Taster
platform. He is enthusiastic about open source software and occasionally
contributes to the 3D graphics application, Blender. He can be found on the
Python IRC channel as yakca or hiking on a mountain somewhere in Scotland.

xi
Visit https://ebooknice.com to
discover a wide range of
eBooks across various genres.
Enjoy exclusive deals and
discounts to enhance your
reading experience. Start your
digital reading journey today!
Introduction
Python is a remarkable language with many surprisingly powerful features
baked into it. Generators, metaclasses, and decorators are some of those,
but this book is all about descriptors.

Code Samples
All code samples are written in Python 3, since that is the most recent
version, but all the ideas and principles taught in this book apply to Python
2 as well, as long as you’re using new style classes.

The Descriptor Tools Library


Written alongside this book was a library, called descriptor-tools, which
can be installed with pip. It contains the fruition of a lot of the ideas and
helpers to make it easier to implement them all. It’s an open source project
with a public GitHub repository.1

Note Superscript letters like the one at the end of the previous line
are in reference to the bibliography at the back of the book, which
includes URLs to the referenced site.

xiii
Introduction

Conventions in This Book


When the text mentions “class” and “instance” in a general sense, they
refer to a class that has a descriptor attribute and to instances of such
classes, respectively. All other classes and instances will be referred to
more specifically.

New in the 2nd Edition


The 2nd edition is an update including new features of Python as well as
new ideas to learn. One of the new things is incredibly important if this
book wants to maintain the status of “comprehensive” guide that it strives
for. This important addition is about the addition of __set_name__() to the
descriptor protocol in Python 3.6. You can read about this in Chapter 7,
“Storing the Attributes”.
Another addition is an idea that was inspired by looking into the
__set_name__() addition to the protocol, which you’ll see just after the
section on that addition. Also, I added a chapter on creating instance-level
descriptors, which were added to descriptor-tools well before this edition
really got started.
The next thing is actually a change, not an addition. Since writing
the first book, I found out about the built-in function vars(). Calling
vars(obj) is equivalent to obj.__dict__, but is more Pythonic. Kind of
like calling len(obj) instead of obj.__len__(). So the code examples
have been updated to use vars(). Any remaining references to __dict__
are purposeful.
Pretty much everything else new in this edition is just cleaning up the
language to be more legible.

xiv
PART I

About Descriptors
Part I is a deep explanation of what descriptors are, how they work, and
how they’re used. It gives enough information that you should be able to
look at any descriptor and understand how it works and why it works that
way, assuming the writer of the code made the code legible enough.
Creating your own descriptors isn’t difficult once you have the
information from Part I, but little to no guidance is given to help with
it. Instead, Part II covers that with a bunch of options for creating new
descriptors, as well as tips for avoiding common mistakes.
CHAPTER 1

What Is a Descriptor?
Put very simply, a descriptor is a class that can be used to call a method
with simple attribute access, but there’s obviously more to it than that. It’s
difficult to explain beyond that without digging a little into how descriptors
implemented. So, here’s a high-level view of the descriptor protocol.
A descriptor implements at least one of these three methods:
__get__(), __set__(), or __delete__(). Each of those methods has a list
of parameters needed, which will be discussed a little later, and each is
called by a different sort of access of the attribute the descriptor represents.
Doing simple a.x access will call the __get__() method of x; setting the
attribute using a.x = value will call the __set__() method of x; and using
del a.x will call, as expected, the __delete__() method of x.

Note Since version 3.6, there’s another method that descriptors


can take advantage of, called __set_name__(), but using just
that method doesn’t make it a descriptor the way any of the other
three will. This method will be mostly ignored for a while, since it
doesn’t have as big a role into how descriptors work. It will only be
mentioned where most relevant.

As stated, only one of the methods needs to be implemented in


order to be considered a descriptor, but any number of them can be
implemented. And, depending on descriptor type and on which methods

© Jacob Zimmerman 2018 3


J. Zimmerman, Python Descriptors, https://doi.org/10.1007/978-1-4842-3727-4_1
Chapter 1 What Is a Descriptor?

are implemented, not implementing certain methods can restrict certain


types of attribute access or provide interesting alternative behaviors for
them. There are two types of descriptors based on which sets of these
methods are implemented: data and non-data.

 ata Descriptors versus Non-Data


D
Descriptors
A data descriptor implements at least __set__() or __delete__(), but
can include both. Data descriptors also often include __get__() since it’s
rare to want to set something without also being able to get it too. You can
get the value, even if the descriptor doesn’t include __get__(), but it’s
either roundabout or the descriptor writes it to the instance. That will be
discussed more later.
A non-data descriptor only implements __get__(). If it adds a __set__()
or __delete__() method, it becomes a data descriptor.
Unfortunately, the PyPy interpreter (up to version 2.4.0) gets this a little
bit wrong. It doesn’t take __delete__() into consideration until it knows
that it’s a data descriptor, and PyPy doesn’t believe something is a data
descriptor unless __set__() is implemented. Luckily, since a huge majority
of data descriptors implement __set__(), this rarely becomes a problem.
It may seem like the distinction is pointless, but it is not. It comes into
play upon attribute lookup. This will be discussed more later, but basically,
the distinction is the types of uses it provides.

The Use of Descriptors by Python


It is worth noting that descriptors are an inherent part of how Python
works. Python is known to be a multi-paradigm language, and as such
supports paradigms such as functional programming, imperative

4
Chapter 1 What Is a Descriptor?

programming, and object-oriented programming. This book does not


attempt to go into depth about the different paradigms; only the object-­
oriented programming paradigm will be observed. Descriptors are used
implicitly in Python for the language’s object-oriented mechanisms. As will
be explained shortly, methods are implemented using descriptors. As you
may guess from reading this, it is thanks to descriptors that object-oriented
programming is possible in Python. Descriptors are very powerful and
advanced, and this book aims to teach Python programmers how to use
them fully.

Summary
As you have seen, descriptors occupy a large part of the Python language, as
they can replace attribute access with method calls, and even restrict which
types of attribute access is allowed. Now that you have a broad idea of how
descriptors are implemented as well as their use by the language, we will
dig a little deeper yet, gaining a better understanding of how they work.

5
CHAPTER 2

The Descriptor
Protocol
In order to get a better idea of what descriptors are good for, let’s finish
showing the full descriptor protocol. It’s time to see the full signatures of
the protocol’s methods and what the parameters are.

The __get__(self, instance, owner) Method


This method is clearly the method for retrieving whatever data or object the
descriptor is meant to maintain. Obviously, self is a parameter, since it’s a
method. Also, it receives instance and/or owner. We’ll start with owner.
owner is the class that the descriptor is accessed from, or else the
class of the instance it’s being accessed from. When you make the call
A.x (A being a class), and x is a descriptor object with __get__(), it’s called
with an owner with the instance set to None. So the lookup gets effectively
transformed into A.__dict__['x'].__get__(None, A). This lets the
descriptor know that __get__() is being called from a class, not an instance.
owner is also often written to have a default value of None, but that’s largely
an optimization that only built-in descriptors take advantage of.
Now, onto the last parameters. instance is the instance that the
descriptor is being accessed from, if it is being accessed from an instance.
As previously stated, if None is passed into instance, the descriptor knows

© Jacob Zimmerman 2018 7


J. Zimmerman, Python Descriptors, https://doi.org/10.1007/978-1-4842-3727-4_2
Chapter 2 The Descriptor Protocol

that it’s being called from the class level. But, if instance is not None, then it
tells the descriptor which instance it’s being called from. So an a.x call will
be effectively translated to type(a).__dict__['x'].__get__(a, type(a)).
Notice that it still receives the instance’s class. Notice also that the call still
starts with type(a), not just a, because descriptors are stored on classes.
In order to be able to apply per-instance as well as per-class functionality,
descriptors are given instance and owner (the class of the instance). How
this translation and application happens will be discussed later.
Remember—and this applies to __set__() and __delete__() as
well—self is an instance of the descriptor itself. It is not the instance that
the descriptor is being called from; the instance parameter is the instance
the descriptor is being called from. This may sound confusing at first, but
don’t worry if you don’t understand for now—everything will be explained
further.
The __get__() method is the only one that bothers to get the class
separately. That’s because it’s the only method on non-data descriptors,
which are generally made at a class level. The built-in decorator
classmethod is implemented using descriptors and the __get__()
method. In that case, it will use the owner parameter alone.

The __set__(self, instance, value) Method


As mentioned, __set__() does not have an owner parameter that accepts
a class. __set__() does not need it, since data descriptors are generally
designed for storing per-instance data. Even if the data is being stored on a
per-class level, it should be stored internally without needing to reference
the class.
self should be self-explanatory now; the next parameter is instance.
This is the same as it is in the __get__() method. In this case, though,
your initial call is a.x = someValue, which is then translated into
type(a).__dict__['x'].__set__(a, someValue).

8
Chapter 2 The Descriptor Protocol

The last parameter is value, which is the value the attribute is being
assigned.
One thing to note: when setting an attribute that is currently a
descriptor from the class level, it will replace the descriptor with whatever
is being set. For example, A.x = someValue does not get translated to
anything; someValue replaces the descriptor object stored in x. To act on
the class, see the following note.

The __delete__(self, instance) Method


After having learned about the __get__() and __set__() methods,
__delete__() should be easy to figure out. self and instance are the
same as in the other methods, but this method is invoked when del a.x is
called and is translated to type(a).__dict__['x'].__delete__(a).
Do not accidentally name it __del__(), as that won’t work as intended.
__del__() would be the destructor of the descriptor instance, not of the
attribute stored within.
It must be noted that, again, that __delete__() does not work from the
class level, just like __set__(). Using del from the class level will remove
the descriptor from the class’ dictionary rather than calling the descriptor’s
__delete__() method.

Note If you want a descriptor’s __set__() or __delete__()


methods to work from the class level, that means that the descriptor
must be created on the class’ metaclass. When doing so, everything
that refers to owner is referring to the metaclass, while a reference
to instance refers to the class. After all, classes are just instances
of metaclasses. The section on metadescriptors will explain that in
greater detail.

9
Chapter 2 The Descriptor Protocol

Summary
That’s the sum total of the descriptor protocol. Having a basic idea of how
it works, you’ll now get a high-level view of the types of things that can be
done with descriptors.

10
Visit https://ebooknice.com to
discover a wide range of
eBooks across various genres.
Enjoy exclusive deals and
discounts to enhance your
reading experience. Start your
digital reading journey today!
CHAPTER 3

What Are Descriptors


Good For?
Nothing is perfect in this world, and Python’s descriptors are no exception.
Descriptors allow you to do some pretty cool things, but those cool things
come at a cost. Here, we discuss the good and the bad.

Pros of Python Descriptors


Obviously we’re going to go over the good things about descriptors.
Would there be an entire book about them if they couldn’t be considered a
good thing?

E ncapsulation
One of the most useful aspects of descriptors is that they encapsulate
data so well. With descriptors, you can access an attribute the simple way
using attribute access notation (a.x) while having more complex actions
happen in the background. For example, a Circle class might have radius,
diameter, circumference, and area all available as if they were attributes,
but since they’re all linked, you only need to store one (we’ll use the radius
for the example) and calculate the others based on it. But from the outside,
they all look like attributes stored on the object.

© Jacob Zimmerman 2018 11


J. Zimmerman, Python Descriptors, https://doi.org/10.1007/978-1-4842-3727-4_3
Chapter 3 What Are Descriptors Good For?

Reuse of Read/Write Patterns


Using specialized descriptors, you can reuse code that you used with
reading and/or writing of attributes. These can be used for repetitious
attributes within the same class or attribute types shared by other classes
as well. Some examples of reusable patterns are described in the following
sections.

Lazy Instantiation
You can use descriptors to define a really simple syntax for lazily
instantiating an attribute. There will be code provided for a nice lazy
attribute implementation later in the book.
In the Circle example, the non-radius attributes, after having their
caches invalidated, don’t need to calculate their values right away; they
could wait until they’re needed. That’s laziness.

Validation
Many descriptors are written simply to make sure that data being passed
in conforms to the class’ or attribute’s invariants. Such descriptors can
usually be designed as handy decorators, too.
Again with the Circle example: all of those attributes should be
positive, so all the descriptors could also make sure the value being set is
positive.

Triggering Actions
Descriptors can be used to trigger certain actions when the attribute is
accessed. For example, the observer pattern can be implemented in a
per-attribute sense to trigger calls to the observer whenever an attribute is
changed.

12
Chapter 3 What Are Descriptors Good For?

Last Circle example: all the “attributes” are based on the radius
calculated lazily. In order to keep from having to calculate them every
time, you could cache the result. Then, whenever one of them changes,
it could trigger invalidating all the others’ caches.

Writing for the Class Level


Because descriptors are stored at the class scope instead of the instance
scope, it allows you to do more robust things at the class level. For instance,
descriptors make classmethod and staticmethod work, which will be
explained in the next chapter.

Cons of Python Descriptors


As great as descriptors are, they come at a cost, just like just about
everything else in programming.

Encapsulation
Wait… encapsulation was a pro. How can it also be a con? The problem
is that you can hide incredible amounts of complexity behind something
that just looks like attribute use. With getters and setters, the user at least
sees that there’s a function being called, and plenty can happen in a single
function call. But the user won’t necessarily expect that what is seemingly
attribute access is causing something else to happen, too. Most of the time,
this isn’t a problem, but it can get in the user’s way of trying to debug any
problems, since clearly that code can’t be a problem.

13
Chapter 3 What Are Descriptors Good For?

Can Be Difficult To Write


It can be easy for the mind to get all twisted up when it comes to thinking
about the fact that descriptors are stored at the class level, but are usually
for dealing with attributes at the instance level. Besides that, there are
a lot of considerations and common pitfalls to deal with when deciding
how to save the represented attribute, whether you decide to do it on the
descriptor or on the the object that the attribute is for. The descriptor-tools
library was created specifically because of this.

Additional Objects
Because descriptors add another layer of indirection/abstraction to the
mix, they also add at least one additional object in memory, along with at
least one additional call stack level. In most cases, it’ll be more than one of
each. This adds bloat that could at least be partially mitigated using getters
and setters.

Summary
Descriptors are awesome, allowing for a variety of nice features that are
good at hiding their complexity from users of your code, but you should
definitely be aware that the power comes with cost.

14
CHAPTER 4

Descriptors in the
Standard Library
There are three basic, well-known descriptors that come with Python:
property, classmethod, and staticmethod. There’s also a fourth one that
you use all the time, but are less likely to know is a descriptor.
Of all the descriptors being shown in this chapter, it’s possible that
you only knew of property as a descriptor. Plenty of people even learn
the basics of descriptors from it, but a lot of people don’t know that
classmethod and staticmethod are descriptors. They feel like super
magical constructs built into the language that no one could reproduce in
pure Python. Once someone has an understanding of descriptors, though,
their basic implementation becomes relatively obvious. In fact, example
code will be provided for all three in simplified, pure Python code.
Lastly, it will be shown that all methods are actually implemented
with descriptors. Normal methods are actually done “magically,” since the
descriptor creation is implicit, but it’s still not entirely magical because it’s
done using a language construct the anyone could create.
What I find really interesting is that the first three are all function
decorators, which are another really awesome feature of Python that
deserves its own book, even though they’re way simpler.

© Jacob Zimmerman 2018 15


J. Zimmerman, Python Descriptors, https://doi.org/10.1007/978-1-4842-3727-4_4
Chapter 4 Descriptors in the Standard Library

The property Class


This book doesn’t include instructions for how to use the property class
and decorator; it is focused on understanding and creating descriptors.
The official documentation for using property can be found in Python’s
documentation2.
Of all the descriptors out there, property is likely the most versatile.
This is because it doesn’t really do anything on its own, but rather allows
the users to inject their wanted functionality into it by providing their own
getters, setters, and deleters.
To get a better idea of how it works, here is a simplified pure Python
implementation of property.

class property:
    def __init__(self, fget=None, fset=None, fdel=None):
        self.fget = fget
        self.fset = fset
        self.fdel = fdel

    def __get__(self, instance, owner):


        if instance is None:
            return self
        elif self.fget is None:
            raise AttributeError("unreadable attribute")
        else:
            return self.fget(instance)

    def __set__(self, instance, value):


        if self.fset is None:
            raise AttributeError("can't set attribute")
        else:
            self.fset(instance, value)

16
Chapter 4 Descriptors in the Standard Library

    def __delete__(self, instance):


        if self.fdel is None:
            raise AttributeError("can't delete attribute")
        else:
            self.fdel(instance)

    def getter(self, fget):


        return type(self)(fget, self.fset, self.fdel)

    def setter(self, fset):


        return type(self)(self.fget, fset, self.fdel)

    def deleter(self, fdel):


        return type(self)(self.fget, self.fset, fdel)

As you can now see, the property class has almost no real functionality
of its own; it simply delegates to the functions given to it. When a function
is not provided for a certain method to delegate to, property assumes that
it is a forbidden action and raises an AttributeError with an appropriate
message.
A nice thing about the property class is that it largely just accepts
methods. Even its constructor, which can be given all three methods at
once, is capable of being called with just one, or even none. Because of
this, the constructor and other methods can be used as decorators in a
very convenient syntax. Check out the documentation2 to learn more
about it.
Omitted from this code example is the doc functionality, where it sets
its own __doc__ property based on what is passed in through __init__()’s
doc parameter or using __doc__ from fget if nothing is given. Also omitted
is the code that sets other attributes on property, such as __name__, in
order to help it appear even more like a simple attribute. They did not
seem important enough to worry about, since the focus was more on the
main functionality.

17
Chapter 4 Descriptors in the Standard Library

The classmethod Descriptor


classmethod is another descriptor that can be used as a decorator, but,
unlike property, there’s no good reason not to use it as one. classmethod
is an interesting concept that doesn’t exist in many other languages
(if any). Python’s type system, which uses classes as objects, makes
classmethods easy and worthwhile to make.
Here’s the Python code for classmethod.

class classmethod:
    def __init__(self, func):
        self.func = func

    def __get__(self, instance, owner):


        return functools.partial(self.func, owner)

That’s all there is to it. classmethod is a non-data descriptor, so it only


implements __get__(). This __get__() method completely ignores the
instance parameter because, as “class” in the name implies, the method
has nothing to do with an instance of the class and only deals with the
class itself. What’s really nice is the fact that this can still be called from an
instance without any issues.
Why does the __get__() method return a functools.partial object
with the owner passed in, though? To understand this, think about the
parameter list of a function marked as a classmethod. The first parameter
is the class parameter, usually named cls. This class parameter is filled in
the call to partial so that the returned function can be called with just the
arguments the user wants to explicitly provide. The true implementation
doesn’t use partial, but works similarly.
Again, the code that sets __name__, __doc__, etc. is omitted to show
only how the main functionality works.

18
Chapter 4 Descriptors in the Standard Library

The staticmethod Descriptor


A method marked with staticmethod is strange in that it’s a method that
is really just a function, but it is “attached” to a class. Being part of the class
doesn’t do anything other than show users that it is associated with that
class and giving it a more specific namespace. Also, interestingly, because
staticmethod and classmethod are implemented using descriptors,
they’re inherited by subclasses.
The implementation of staticmethod is even simpler than that of
classmethod; it just accepts a function and then returns it when __get__()
is called.

class staticmethod:
    def __init__(self, func):
        self.func = func

    def __get__(self, instance, owner):


        return self.func

Regular Methods
Remember that it was stated earlier that regular methods implicitly use
descriptors as well. In fact, all functions can be used as methods. This is
because functions are non-data descriptors as well as callables.
Here is a Python implementation that roughly shows how a function
looks.

class function:
    def __call__(self, *args, **kwargs):
        # do something

19
Exploring the Variety of Random
Documents with Different Content
INCLUDING BUT NOT LIMITED TO WARRANTIES OF
MERCHANTABILITY OR FITNESS FOR ANY PURPOSE.

1.F.5. Some states do not allow disclaimers of certain implied


warranties or the exclusion or limitation of certain types of damages.
If any disclaimer or limitation set forth in this agreement violates the
law of the state applicable to this agreement, the agreement shall be
interpreted to make the maximum disclaimer or limitation permitted
by the applicable state law. The invalidity or unenforceability of any
provision of this agreement shall not void the remaining provisions.

1.F.6. INDEMNITY - You agree to indemnify and hold the Foundation,


the trademark owner, any agent or employee of the Foundation,
anyone providing copies of Project Gutenberg™ electronic works in
accordance with this agreement, and any volunteers associated with
the production, promotion and distribution of Project Gutenberg™
electronic works, harmless from all liability, costs and expenses,
including legal fees, that arise directly or indirectly from any of the
following which you do or cause to occur: (a) distribution of this or
any Project Gutenberg™ work, (b) alteration, modification, or
additions or deletions to any Project Gutenberg™ work, and (c) any
Defect you cause.

Section 2. Information about the Mission


of Project Gutenberg™
Project Gutenberg™ is synonymous with the free distribution of
electronic works in formats readable by the widest variety of
computers including obsolete, old, middle-aged and new computers.
It exists because of the efforts of hundreds of volunteers and
donations from people in all walks of life.

Volunteers and financial support to provide volunteers with the


assistance they need are critical to reaching Project Gutenberg™’s
goals and ensuring that the Project Gutenberg™ collection will
remain freely available for generations to come. In 2001, the Project
Gutenberg Literary Archive Foundation was created to provide a
secure and permanent future for Project Gutenberg™ and future
generations. To learn more about the Project Gutenberg Literary
Archive Foundation and how your efforts and donations can help,
see Sections 3 and 4 and the Foundation information page at
www.gutenberg.org.

Section 3. Information about the Project


Gutenberg Literary Archive Foundation
The Project Gutenberg Literary Archive Foundation is a non-profit
501(c)(3) educational corporation organized under the laws of the
state of Mississippi and granted tax exempt status by the Internal
Revenue Service. The Foundation’s EIN or federal tax identification
number is 64-6221541. Contributions to the Project Gutenberg
Literary Archive Foundation are tax deductible to the full extent
permitted by U.S. federal laws and your state’s laws.

The Foundation’s business office is located at 809 North 1500 West,


Salt Lake City, UT 84116, (801) 596-1887. Email contact links and up
to date contact information can be found at the Foundation’s website
and official page at www.gutenberg.org/contact

Section 4. Information about Donations to


the Project Gutenberg Literary Archive
Foundation
Project Gutenberg™ depends upon and cannot survive without
widespread public support and donations to carry out its mission of
increasing the number of public domain and licensed works that can
be freely distributed in machine-readable form accessible by the
widest array of equipment including outdated equipment. Many
small donations ($1 to $5,000) are particularly important to
maintaining tax exempt status with the IRS.

The Foundation is committed to complying with the laws regulating


charities and charitable donations in all 50 states of the United
States. Compliance requirements are not uniform and it takes a
considerable effort, much paperwork and many fees to meet and
keep up with these requirements. We do not solicit donations in
locations where we have not received written confirmation of
compliance. To SEND DONATIONS or determine the status of
compliance for any particular state visit www.gutenberg.org/donate.

While we cannot and do not solicit contributions from states where


we have not met the solicitation requirements, we know of no
prohibition against accepting unsolicited donations from donors in
such states who approach us with offers to donate.

International donations are gratefully accepted, but we cannot make


any statements concerning tax treatment of donations received from
outside the United States. U.S. laws alone swamp our small staff.

Please check the Project Gutenberg web pages for current donation
methods and addresses. Donations are accepted in a number of
other ways including checks, online payments and credit card
donations. To donate, please visit: www.gutenberg.org/donate.

Section 5. General Information About


Project Gutenberg™ electronic works
Professor Michael S. Hart was the originator of the Project
Gutenberg™ concept of a library of electronic works that could be
freely shared with anyone. For forty years, he produced and
distributed Project Gutenberg™ eBooks with only a loose network of
volunteer support.
Project Gutenberg™ eBooks are often created from several printed
editions, all of which are confirmed as not protected by copyright in
the U.S. unless a copyright notice is included. Thus, we do not
necessarily keep eBooks in compliance with any particular paper
edition.

Most people start at our website which has the main PG search
facility: www.gutenberg.org.

This website includes information about Project Gutenberg™,


including how to make donations to the Project Gutenberg Literary
Archive Foundation, how to help produce our new eBooks, and how
to subscribe to our email newsletter to hear about new eBooks.
Welcome to our website – the ideal destination for book lovers and
knowledge seekers. With a mission to inspire endlessly, we offer a
vast collection of books, ranging from classic literary works to
specialized publications, self-development books, and children's
literature. Each book is a new journey of discovery, expanding
knowledge and enriching the soul of the reade

Our website is not just a platform for buying books, but a bridge
connecting readers to the timeless values of culture and wisdom. With
an elegant, user-friendly interface and an intelligent search system,
we are committed to providing a quick and convenient shopping
experience. Additionally, our special promotions and home delivery
services ensure that you save time and fully enjoy the joy of reading.

Let us accompany you on the journey of exploring knowledge and


personal growth!

ebooknice.com

You might also like