Table of Contents

Introduction 1.1
01. Variables 1.2
02 Data Type 1.3
01 Data Type 1.3.1
02 Slices 1.3.2
03 Conditional Statements 1.4
04 Functions 1.5
05. OOPs 1.6
06. Operators 1.7
07. Generators and Iterators 1.8
08. IO 1.9
09. Exception 1.10
10 Modules 1.11
99. Easter Eggs 1.12
Hidden Treasures: Python
This ebook contains few lesser known Python gems which
developers can use to enhance their knowledge in Python and get
more productive.

As usual, I will try to keep them updated and will continue to

expand. If you wish to add any new, send them to me at
(funmayank @ yahoo . co . in).
Variables

Unicode identifier
Python 3 allows to have unicode identifier's, which allows non-
english speaking users to code.

िह दी = 10
print(िह दी)

10

Multi Assignation

Multiple Assignation
Multiple variables can be assigned different values in a single line

a, b, c = "1001", "Python", True

print(a, b, c)

1001 Python True

 a, b, c = range(3)
print(a, b, c)

0 1 2

Multiple variables assigned to same data

a = b = c = "1001"
print(a, b, c)
print(id(a), id(b), id(c))

1001 1001 1001

139707904162816 139707904162816 139707904162816

In-place value swapping

a = 10
b = "TEST"
a, b = b, a
print(a, b)

TEST 10

Memory Usage Of An variable

import sys

a = 10
b = "TEST"
print(sys.getsizeof(a))
print(sys.getsizeof(b))
print(sys.getsizeof([a, b]))

28
53
80

class X:
b = 10
x = X
print(sys.getsizeof(x))
x.a = list(range(10000))
print(sys.getsizeof(x))

print(sys.getsizeof(x.a))
print("Funny, x is smaller than x.a")

1056
1056
90112
Funny, x is smaller than x.a
Integer

Negative round
round is a function to round off the numbers and its normal usage
is as follows

num = round(283746.32321, 1)
print(num)

283746.3

The second parameter defines the decimal number to which the

number to rounded of. But if we provide a -ve number to it then it
starts rounding of the number itself instead of decimal digit as
shown in the below example

num = round(283746.32321, -2)

print(num)
num = round(283746.32321, -1)
print(num)
num = round(283746.32321, -4)
print(num)
 283700.0
283750.0
280000.0

pow power - pow() can calculate (x ** y) %

z

x, y, z = 1019292929191, 1029228322, 222224

pow(x, y, z)

115681

# Do not run this, please. it will take forever.

##### (x ** y) % z
String

Multi line strings

In python we can have multiple ways to achieve multi line strings.

Using triple quotes

txt = """The Supreme Lord said: The indestructible,

transcendental living
entity is called Brahman and his eternal nature is called
the
self. Action pertaining to the development of these material
bodies is called karma, or fruitive activities."""

print(txt)

The Supreme Lord said: The indestructible, transcendental

living
entity is called Brahman and his eternal nature is called
the
self. Action pertaining to the development of these material
bodies is called karma, or fruitive activities.

Using brackets "( )"

txt = ("The Supreme Lord said: The indestructible,
transcendental living"
" entity is called Brahman and his eternal nature is
called the "
"self. Action pertaining to the development of these
material "
"bodies is called karma, or fruitive activities.")
print(txt)

The Supreme Lord said: The indestructible, transcendental

living entity is called Brahman and his eternal nature is
called the self. Action pertaining to the development of
these material bodies is called karma, or fruitive
activities.

Using '\'

txt = "The Supreme Lord said: The indestructible,

transcendental living " \
"entity is called Brahman and his eternal nature is
called the" \
"self. Action pertaining to the development of these
material " \
"bodies is called karma, or fruitive activities."

print(txt)

The Supreme Lord said: The indestructible, transcendental

living entity is called Brahman and his eternal nature is
called theself. Action pertaining to the development of these
 material bodies is called karma, or fruitive activities.

Print String multiple times

using string multiply with int results in concatenating string
that number of times. Lets print a line on console using - .

print("-^*" * 20)

-^*-^*-^*-^*-^*-^*-^*-^*-^*-^*-^*-^*-^*-^*-^*-^*-^*-^*-^*-^*

Search substring in string

print("ash" in "ashwini")

True

print("ash" is ['a', 's', 'h'])

False

print("ash" is 'ash')
 True

Search multiple substring in string using

startswith

names = ["Mr Subhash Chandra Bose Ji", "Mrs Durgawati Devi

Ji", "Swami Dayananda Saraswati Ji", "Mahatama Ghandhi Ji"]

for name in names:

print(name.startswith(("Mr", "Mrs", "Swami")))

True
True
True
False

Implicit concatenation without "+"

operator

name = "Mayank" " " "Johri"

print(name)

Mayank Johri

Join list of strings

 list_cities = ["Bhopal", "New Delhi", "Agra", "Mumbai",
"Aligarh", "Hyderabad"]

# Lets join the list of string in string using `join`

str_cities = ", ".join(list_cities)
print(str_cities)

Bhopal, New Delhi, Agra, Mumbai, Aligarh, Hyderabad

Reverse the string

There are few methods to reverse the string, but two are most
common

using slices

txt = "The Mother Earth"

print(txt[::-1])

htraE rehtoM ehT

txt = "The Mother Earth"

print("".join(list(reversed(txt))))

htraE rehtoM ehT

List / Tuple

tuple / list unpacking

a, b, *remaining = (1, 2, 3, 4, 5, "tst")

print(a, b)
print(remaining)

1 2
[3, 4, 5, 'tst']

a, b, *remaining = [1, 2, 3, 4, 5, "tst"]

print(a, b)
print(remaining)

1 2
[3, 4, 5, 'tst']

first,*middle,last = (1,2,3,4,5,6,7,8)

print(first, last)
print(middle)

1 8
[2, 3, 4, 5, 6, 7]
 first,*middle,last = [1,2,3,4,5,6,7,8]

print(first, last)
print(middle)

1 8
[2, 3, 4, 5, 6, 7]

List/tuple multiplication ;)
similar to String we can literally multiply string and tuples with
integer as shown below

lst = [1, 2, 3]
print(lst*3)

[1, 2, 3, 1, 2, 3, 1, 2, 3]

lst = (1, 2, 3)
print(lst*3)

(1, 2, 3, 1, 2, 3, 1, 2, 3)

Array Transpose using zip

 a = [(1,2), (3,4), (5,6)]
print(list(zip(a)))
print("*"*33)
print(list(zip(*a)))

[((1, 2),), ((3, 4),), ((5, 6),)]

*********************************
[(1, 3, 5), (2, 4, 6)]

enumerate with predefined starting index

lst = ["Ashwini", "Banti", "Bhaiya", "Mayank", "Shashank",

"Rahul" ]
list(enumerate(lst))

[(0, 'Ashwini'),
(1, 'Banti'),
(2, 'Bhaiya'),
(3, 'Mayank'),
(4, 'Shashank'),
(5, 'Rahul')]

Now, lets change the starting index to 10

print(list(enumerate(lst, 10)))

[(10, 'Ashwini'), (11, 'Banti'), (12, 'Bhaiya'), (13,

 'Mayank'), (14, 'Shashank'), (15, 'Rahul')]

Reverse the list

built-in keyword reversed allows the list to be reversed.

print(list(reversed([1, 2, 3, 4, 53])))

[53, 4, 3, 2, 1]

Flattening of list

l = [[1,2], [3], [4,5], [6], [7, 8, 9]]

l1 = [[1,2], 3, [4,5], [6], [7, 8, 9]]

Method 1:

from itertools import chain

flattened_list = list(chain(*l))
print(flattened_list)

[1, 2, 3, 4, 5, 6, 7, 8, 9]

NOTE: this method will fail if any of the element is non list
item as shown in the below example
from itertools import chain
try:
flattened_list = list(chain(*l1))
print(flattened_list)
except Exception as e:
print(e)

'int' object is not iterable

Method 2:

flattened_list = [y for x in l for y in x]

print(flattened_list)

[1, 2, 3, 4, 5, 6, 7, 8, 9]

NOTE: this method will fail if any of the element is non list
item as shown in the below example

try:
flattened_list = [y for x in l1 for y in x]
print(flattened_list)
except Exception as e:
print(e)

'int' object is not iterable

Lets update code to handle this situation

flattened_list = [si for i in l1 for si in (i if

isinstance(i, list) else [i])]
print(flattened_list)

[1, 2, 3, 4, 5, 6, 7, 8, 9]

Method 3:

flattened_list = sum(l, [])

print(flattened_list)

[1, 2, 3, 4, 5, 6, 7, 8, 9]

NOTE: this method will fail if any of the element is non list
item as shown in the below example

try:
sum(l1, [])
except Exception as e:
print(e)

can only concatenate list (not "int") to list

Method 4:
flattened_list = []
for x in l:
for y in x:
flattened_list.append(y)
print(flattened_list)

[1, 2, 3, 4, 5, 6, 7, 8, 9]

NOTE: this method will fail if any of the element is non list
item as shown in the below example

try:
flattened_list = []
for x in l1:
for y in x:
flattened_list.append(y)
print(flattened_list)
except Exception as e:
print(e)

'int' object is not iterable

Method 5:

from functools import reduce

flattened_list = reduce(lambda x, y: x + y, l)
print(flattened_list)
[1, 2, 3, 4, 5, 6, 7, 8, 9]

NOTE: this method will fail if any of the element is non list
item as shown in the below example

try:
flattened_list = reduce(lambda x, y: x + y, l1)
print(flattened_list)
except Exception as e:
print(e)

can only concatenate list (not "int") to list

Method 6:

import operator
flattened_list = reduce(operator.add, l)
print(flattened_list)

[1, 2, 3, 4, 5, 6, 7, 8, 9]

NOTE: this method will fail if any of the element is non list
item as shown in the below example

import operator
try:
flattened_list = reduce(operator.add, l1)
 print(flattened_list)
except Exception as e:
print(e)

can only concatenate list (not "int") to list

Infinite Recursion

lst = [1, 2]
lst.append(lst)
print(lst)

[1, 2, [...]]

lets check if really we have infinite recursion, with the following

code. We should get RuntimeError: maximum recursion depth

exceeded in comparison error later in the execution.

def test(lst):
for a in lst:
if isinstance(a, list):
print("A", a)
test(a)
print(a)

test(lst)
printing elements of list

brothers = ["Ashwini", "Banti", "Bhaiya", "Mayank",

"Shashank", "Rahul" ]

print(brothers) # not good

print("Brothers:", end=" ")
print(*brothers, sep=", ") # better solution

['Ashwini', 'Banti', 'Bhaiya', 'Mayank', 'Shashank', 'Rahul']

Brothers: Ashwini, Banti, Bhaiya, Mayank, Shashank, Rahul

Copy a list

### printing elements of list

brothers = ["Ashwini", "Banti", "Bhaiya", "Mayank",

"Shashank", "Rahul" ]

print(brothers) # not good

print("Brothers:", end=" ")
print(*brothers, sep=", ") # better solutionori = [1, 2, 3,
4, 5, 6]
dup = brothers
print(id(brothers))
print(id(dup))

['Ashwini', 'Banti', 'Bhaiya', 'Mayank', 'Shashank', 'Rahul']

Brothers: Ashwini, Banti, Bhaiya, Mayank, Shashank, Rahul
 140307524551688
140307524551688

Both the variables are still pointing to same list, thus change in one
will change another also.

dup.insert(0, 29)
print(brothers)
print(dup)

[29, 'Ashwini', 'Banti', 'Bhaiya', 'Mayank', 'Shashank',

'Rahul']
[29, 'Ashwini', 'Banti', 'Bhaiya', 'Mayank', 'Shashank',
'Rahul']

Deepcopy a list

ori = [1, 2, 3, 4, 5, 6]
dup = ori[:]
print(id(ori))
print(id(dup))

140307524228424
140307524230408

dup.insert(0, 29)
print(ori)
 print(dup)

[1, 2, 3, 4, 5, 6]
[29, 1, 2, 3, 4, 5, 6]

Most common elements in an iterable

ori = [1, 2, 3, 3, 4, 5, 5, 4, 5, 6, 4, 5, 6, ]

from collections import Counter

counter = Counter(ori)

print(counter.most_common())

[(5, 4), (4, 3), (3, 2), (6, 2), (1, 1), (2, 1)]

Dictionaries

converting dictionry keys to tuple

tuple({'aa': 0, 'bb': 1, 'cc': 2, 'dd': 3, 'ee': 4})

('aa', 'bb', 'cc', 'dd', 'ee')

Inverting a dictionary

states_capitals = {'MP': 'Bhopal', 'UP': 'Lucknow',

'Rajasthan': 'Jaipur'}

Method 1:

capitals_states =
dict(zip(*list(zip(*states_capitals.items()))[::-1]))
print(capitals_states)

{'Bhopal': 'MP', 'Lucknow': 'UP', 'Jaipur': 'Rajasthan'}

Method 2:

capitals_states = dict([v,k] for k,v in

states_capitals.items())
print(capitals_states)

{'Bhopal': 'MP', 'Lucknow': 'UP', 'Jaipur': 'Rajasthan'}

Method 3:

capitals_states = dict(zip(states_capitals.values(),
states_capitals.keys()))
print(capitals_states)
 {'Bhopal': 'MP', 'Lucknow': 'UP', 'Jaipur': 'Rajasthan'}

or

capitals_states = dict(zip(states_capitals.values(),
states_capitals))
print(capitals_states)

{'Bhopal': 'MP', 'Lucknow': 'UP', 'Jaipur': 'Rajasthan'}

Method 4: using a dictionary comprehension

capitals_states = {states_capitals[k] : k for k in

states_capitals}
print(capitals_states)

{'Bhopal': 'MP', 'Lucknow': 'UP', 'Jaipur': 'Rajasthan'}

or

states_capitals = {'MP': 'Bhopal', 'UP': 'Lucknow',

'Rajasthan': 'Jaipur'}
capitals_states = {v: k for k, v in states_capitals.items()}
print(capitals_states)

{'Bhopal': 'MP', 'Lucknow': 'UP', 'Jaipur': 'Rajasthan'}

Creating dictionaries
Multiple methods can be used to create a dictionary. We are going
to cover few of the cool ones.

Using two lists

states = ["MP", "UP", "Rajasthan"]

capitals = ["Bhopal", "Lucknow", "Jaipur"]

states_capitals = dict(zip(states, capitals))

print(states_capitals)

{'MP': 'Bhopal', 'UP': 'Lucknow', 'Rajasthan': 'Jaipur'}

Using arguments

states_capitals = dict(MP='Bhopal', Rajasthan='Jaipur',

UP='Lucknow')
print(states_capitals)

{'MP': 'Bhopal', 'Rajasthan': 'Jaipur', 'UP': 'Lucknow'}

list of tuples

states_capitals = dict([('MP', 'Bhopal'), ('UP', 'Lucknow'),

('Rajasthan', 'Jaipur')])
print(states_capitals)
{'MP': 'Bhopal', 'UP': 'Lucknow', 'Rajasthan': 'Jaipur'}

By adding two dictionary using copy and update

a = {'MP': 'Bhopal', 'UP': 'Lucknow', 'Rajasthan': 'Jaipur'}

b = {'Jaipur': 'Rajasthan', 'Bhopal': 'MP', 'Lucknow': 'UP'}
c = a.copy()
c.update(b)
print(c)

{'MP': 'Bhopal', 'UP': 'Lucknow', 'Rajasthan': 'Jaipur',

'Jaipur': 'Rajasthan', 'Bhopal': 'MP', 'Lucknow': 'UP'}

# for Python >= 3.5: https://www.python.org/dev/peps/pep-0448

c = {**b, **a}
print(c)

Using dictionary comprehension

lst = list(range(10))

d = {k : k * k for k in lst}
print(d)

{0: 0, 1: 1, 2: 4, 3: 9, 4: 16, 5: 25, 6: 36, 7: 49, 8: 64,

9: 81}
 {chr(97 + i) * 2 : i for i in range(5)}

{'aa': 0, 'bb': 1, 'cc': 2, 'dd': 3, 'ee': 4}

Using dictionary as switch construct of

C Language

In the below example, we are using functions as first class

objects and taking the benefit that value to key is a function.

calculator = {
'add': lambda x, y: x + y,
'subtract': lambda x, y: x - y,
'multiple': lambda x, y: x * y,
'div': lambda x, y: x % y
}

print(calculator['add'](15, 2))
print(calculator['subtract'](12, 11))
print(calculator['multiple'](15, 2))
print(calculator['div'](12, 2))

17
1
30
0
Slices

Named slices

lst = list(range(1, 20, 2))

print(lst)

[1, 3, 5, 7, 9, 11, 13, 15, 17, 19]

odd = slice(0, -1, 2)

print(lst[odd])

[1, 5, 9, 13, 17]

help(slice)

Help on class slice in module builtins:

class slice(object)
| slice(stop)
| slice(start, stop[, step])
|
| Create a slice object. This is used for extended slicing
(e.g. a[0:10:2]).
|
| Methods defined here:
|
| __eq__(self, value, /)
| Return self==value.
|
| __ge__(self, value, /)
| Return self>=value.
|
| __getattribute__(self, name, /)
| Return getattr(self, name).
|
| __gt__(self, value, /)
| Return self>value.
|
| __le__(self, value, /)
| Return self<=value.
|
| __lt__(self, value, /)
| Return self<value.
|
| __ne__(self, value, /)
| Return self!=value.
|
| __new__(*args, **kwargs) from builtins.type
| Create and return a new object. See help(type) for
accurate signature.
|
| __reduce__(...)
| Return state information for pickling.
|
| __repr__(self, /)
| Return repr(self).
|
| indices(...)
| S.indices(len) -> (start, stop, stride)
|
| Assuming a sequence of length len, calculate the
start and stop
| indices, and the stride length of the extended slice
described by
| S. Out of bounds indices are clipped in a manner
consistent with the
| handling of normal slices.
|
| ---------------------------------------------------------
-------------
| Data descriptors defined here:
|
| start
|
| step
|
| stop
|
| ---------------------------------------------------------
-------------
| Data and other attributes defined here:
|
| __hash__ = None
if

Conditional Assignment

y = 10
x = 3 if (y == 1) else 2
print(x)

x = 3 if (y == 1) else 2 if (y == -1) else 1

print(x)

Using and and or instead of if/else

x = 2
x == 2 and 10

10

x = 2
x == 2 and 10 or 5

10

x = 4
x == 2 and 10 or 5

5
Functions

default arguments

Dangerous mutable default arguments

def foo(x=[]):
x.append(1)
print(x)

foo()
foo()
foo()

[1]
[1, 1]
[1, 1, 1]

# instead use:
def fun(x=None):
if x is None:
x = []
x.append(1)
print(x)

fun()
fun()
 fun()

[1]
[1]
[1]

TODO: Add more examples

Function argument unpacking

def draw_point(x, y):

"""You can unpack a list or a dictionary as
function arguments using * and **."""
print(x, y)

point_foo = (3, 4)
point_bar = {'y': 3, 'x': 2}

draw_point(*point_foo)
draw_point(**point_bar)

3 4
2 3

Function arguments

def letsEcho():
 test = "Hello"
print(test)

letsEcho.test = "Welcome"
print(letsEcho.test)
letsEcho()

Welcome
Hello

Finally returns the ultimate return

def dum_dum():
try:
return '`dum dum` returning from try'
finally:
return '`dum dum` returning from finally'

print(dum_dum())

`dum dum` returning from finally

just saving one return value from a multi

value return function.

def my_list(x):
return x, x - 1
print(my_list(10))
result, _ = my_list(10)

(10, 9)
OOPS

Attributes

Dynamically added attributes

class Test():
def __getattribute__(self, name):
f = lambda: " ".join([name, name[::-1]])
return f

t = Test()
# New attribute created at runtime
t.rev()

'rev ver'
Operators

Chaining comparison operators

x = 5

1 < x < 100

True

10 < x < 20

False

x < 10 < x*10 < 100

True

10 > x <= 9

True
5 == x > 4

True
enumerate
Wrap an iterable with enumerate and it will yield the item along
with its index.

a = ['a', 'b', 'c', 'd', 'e']

for index, item in enumerate(a): print (index, item)

0 a
1 b
2 c
3 d
4 e
Generators
Sending values into generator functions
https://www.python.org/dev/peps/pep-0342/, also please reaad
http://www.dabeaz.com/coroutines/

def mygen():
"""Yield 5 until something else is passed back via
send()"""
a = 5
while True:
f = (yield a) #yield a and possibly get f in return
if f is not None:
a = f #store the new value

g = mygen()
print(next(g))
print(next(g))
g.send(7)
print(next(g))
print(next(g))
g.send(17)
print(next(g))
print(next(g))

5
5
7
 7
17
17

Descriptor
http://users.rcn.com/python/download/Descriptor.htm

Iterators

iter() can take a callable argument

def seek_next_line(f):
"""
The iter(callable, until_value) function repeatedly calls
callable and yields its result until until_value is
returned.
"""
for c in iter(lambda: f.read(1),'\n'):
pass
I/O

with

open multiple files in a single with .

try:
with open('a', 'w') as a, open('b', 'w') as b:
pass
except IOError as e:
print ('Operation failed: %s' % e.strerror)

write file using print

with open("outfile.txt" , "w+") as outFile:

print('Modern Standard Hindi is a standardised and
sanskritised register of the Hindustani language.',
file=outFile)
Exception

Re-raising exceptions:

def some_operation():
raise Exception

def is_fatal(e):
return True

# Python 3 syntax
try:
some_operation()
except Exception as e:
if is_fatal(e):
raise
handle_nonfatal(e)

-------------------------------------------------------------
--------------
Exception Traceback (most
recent call last)
<ipython-input-4-18cc95aa93e6> in <module>()
7 # Python 3 syntax
8 try:
----> 9 some_operation()
10 except Exception as e:
11 if is_fatal(e):

<ipython-input-4-18cc95aa93e6> in some_operation()
 1 def some_operation():
----> 2 raise Exception
3
4 def is_fatal(e):
5 return True

Exception:
Modules

Finding Path Of Imported Modules

import os

print(os)

<module 'os' from '/usr/lib64/python3.6/os.py'>

Lets check with the local module. I have created a echo.py file and
now importing and testing on it.

import echo

print(echo)

<module 'echo' from

'/home/mayank/code/mj/ebooks/python/htcp/ipy/10
Modules/echo.py'>
!!! Easter Eggs !!!
from __future__ import braces

File "<ipython-input-3-6d5c5b2f0daf>", line 1

from __future__ import braces
SyntaxError: not a chance

import __hello__

Hello world!

Lets encrypt our code using cot13

import codecs
s = 'The Zen of Python, by Tim Peters'
enc = codecs.getencoder( "rot-13" )
dec = codecs.getdecoder("rot-13")
os = enc( s )[0]
print(os)
print(dec(os)[0])

Gur Mra bs Clguba, ol Gvz Crgref

The Zen of Python, by Tim Peters
import this

The Zen of Python, by Tim Peters

Beautiful is better than ugly.

Explicit is better than implicit.
Simple is better than complex.
Complex is better than complicated.
Flat is better than nested.
Sparse is better than dense.
Readability counts.
Special cases aren't special enough to break the rules.
Although practicality beats purity.
Errors should never pass silently.
Unless explicitly silenced.
In the face of ambiguity, refuse the temptation to guess.
There should be one-- and preferably only one --obvious way
to do it.
Although that way may not be obvious at first unless you're
Dutch.
Now is better than never.
Although never is often better than *right* now.
If the implementation is hard to explain, it's a bad idea.
If the implementation is easy to explain, it may be a good
idea.
Namespaces are one honking great idea -- let's do more of
those!