SlideShare a Scribd company logo
Relational Database Access
  with Python „sans‟ ORM
           Mark Rees
              CTO
 Century Software (M) Sdn. Bhd.
Your Current Relational Database Access Style?

# Django ORM
>>> from ip2country.models import Ip2Country

>>> Ip2Country.objects.all()
[<Ip2Country: Ip2Country object>, <Ip2Country: Ip2Country object>, '...(remaining
elements truncated)...']

>>> sgp = Ip2Country.objects.filter(assigned__year=2012)
... .filter(countrycode2='SG')

>>> sgp[0].ipfrom
1729580032.0
Your Current Relational Database Access Style?
# SQLAlchemy ORM
>>> from sqlalchemy import create_engine, extract
>>> from sqlalchemy.orm import sessionmaker
>>> from models import Ip2Country

>>> engine =
create_engine('postgresql://ip2country_rw:secret@localhost/ip2country')
>>> Session = sessionmaker(bind=engine)
>>> session = Session()

>>> all_data = session.query(Ip2Country).all()

>>> sgp = session.query(Ip2Country).
... filter(extract('year',Ip2Country.assigned) == 2012).
... filter(Ip2Country.countrycode2 == 'SG')

print sgp[0].ipfrom
1729580032.0
SQL Relational Database Access
SELECT * FROM ip2country;

"ipfrom";"ipto";"registry";"assigned";"countrycode2";"countrycode3";"countryname"
1729522688;1729523711;"apnic";"2011-08-05";"CN";"CHN";"China"
1729523712;1729524735;"apnic";"2011-08-05";"CN";"CHN";"China”
...

SELECT * FROM ip2country
WHERE date_part('year', assigned) = 2012
AND countrycode2 = 'SG';

"ipfrom";"ipto";"registry";"assigned";"countrycode2";"countrycode3";"countryname"
1729580032;1729581055;"apnic";"2012-01-16";"SG";"SGP";"Singapore"
1729941504;1729942527;"apnic";"2012-01-10";"SG";"SGP";"Singapore”
...

SELECT ipfrom FROM ip2country
WHERE date_part('year', assigned) = 2012
AND countrycode2 = 'SG';

"ipfrom"
1729580032
1729941504
...
Python + SQL == Python DB-API 2.0

• The Python standard for a consistent
  interface to relational databases is the
  Python DB-API (PEP 249)
• The majority of Python database interfaces
  adhere to this standard
Python DB-API UML Diagram
Python DB-API Connection Object
Access the database via the connection object
• Use connect constructor to create a
  connection with database
     conn = psycopg2.connect(parameters…)
• Create cursor via the connection
     cur = conn.cursor()
• Transaction management (implicit begin)
     conn.commit()
     conn.rollback()
• Close connection (will rollback current
  transaction)
     conn.close()
• Check module capabilities by globals
  psycopg2.apilevel psycopg2.threadsafety psycopg2.paramstyle
Python DB-API Cursor Object
A cursor object is used to represent a database
cursor, which is used to manage the context of
fetch operations.
• Cursors created from the same connection
   are not isolated
     cur = conn.cursor()
     cur2 = conn.cursor()
• Cursor methods
     cur.execute(operation, parameters)
     cur.executemany(op,seq_of_parameters)
     cur.fetchone()
     cur.fetchmany([size=cursor.arraysize])
     cur.fetchall()
     cur.close()
Python DB-API Cursor Object
• Optional cursor methods
     cur.scroll(value[,mode='relative']) cur.next()
     cur.callproc(procname[,parameters])
     cur.__iter__()
• Results of an operation
     cur.description
     cur.rowcount
     cur.lastrowid
• DB adaptor specific “proprietary” cursor
  methods
Python DB-API Parameter Styles
Allows you to keep SQL separate from parameters

Improves performance & security


Warning Never, never, NEVER use Python string
concatenation (+) or string parameters
interpolation (%) to pass variables to a SQL query
string. Not even at gunpoint.

From http://initd.org/psycopg/docs/usage.html#query-parameters
Python DB-API Parameter Styles
Global paramstyle gives supported style for the
adaptor

  qmark Question mark style
     WHERE countrycode2 = ?
  numeric Numeric positional style
     WHERE countrycode2 = :1
  named Named style
     WHERE countrycode2 = :code
  format ANSI C printf format style
     WHERE countrycode2 = %s
  pyformat Python format style
     WHERE countrycode2 = %(name)s
Python + SQL: INSERT
import csv, datetime, psycopg2
conn = psycopg2.connect("dbname=ip2country user=ip2country_rw password=secret”)
cur = conn.cursor()
with open("IpToCountry.csv", "rb") as f:
   reader = csv.reader(f)
   try:
       for row in reader:
          print row
          if row[0][0] != "#":
              row[3] = datetime.datetime.utcfromtimestamp(float(row[3]))
              cur.execute("""INSERT INTO ip2country(
                ipfrom, ipto, registry, assigned,
                countrycode2, countrycode3, countryname)
                VALUES (%s, %s, %s, %s, %s, %s, %s)""", row)
   except:
       conn.rollback()
   else:
       conn.commit()
   finally:
      cur.close()
      conn.close()
Python + SQL: SELECT
# Find ipv4 address ranges assigned to Singapore
import psycopg2, socket, struct

def num_to_dotted_quad(n):
  """convert long int to dotted quad string
     http://code.activestate.com/recipes/66517/"""
  return socket.inet_ntoa(struct.pack('!L',n))

conn = psycopg2.connect("dbname=ip2country user=ip2country_rw password=secret")

cur = conn.cursor()

cur.execute("""SELECT * FROM ip2country
          WHERE countrycode2 = 'SG'
          ORDER BY ipfrom""")

for row in cur:
   print "%s - %s" % (num_to_dotted_quad(int(row[0])),
                num_to_dotted_quad(int(row[1])))
SQLite

• sqlite3
  • CPython 2.5 & 3
  • DB-API 2.0
  • Part of CPython distribution since 2.5
PostgreSQL

• psycopg
  • CPython 2 & 3
  • DB-API 2.0, level 2 thread safe
  • Appears to be most popular
  • http://initd.org/psycopg/

• py-postgresql
  • CPython 3
  • DB-API 2.0
  • Written in Python with optional C
    optimizations
  • pg_python - console
  • http://python.projects.postgresql.org/
PostgreSQL

• PyGreSQL
  • CPython 2.3+
  • Classic & DB-API 2.0 interfaces
  • http://www.pygresql.org/
  • Last release 2009

• pyPgSQL
  • CPython 2
  • Classic & DB-API 2.0 interfaces
  • http://www.pygresql.org/
  • Last release 2006
PostgreSQL

• pypq
  • CPython 2.7 & pypy 1.7+
  • Uses ctypes
  • DB-API 2.0 interface
  • psycopg2-like extension API
  • https://bitbucket.org/descent/pypq

• psycopg2ct
  • CPython 2.6+ & pypy 1.6+
  • Uses ctypes
  • DB-API 2.0 interface
  • psycopg2 compat layer
  • http://github.com/mvantellingen/psyco
    pg2-ctypes
MySQL
• MySQL-python
  • CPython 2.3+
  • DB-API 2.0 interface
  • http://sourceforge.net/projects/mysql-
    python/
• PyMySQL
  • CPython 2.4+ & 3
  • Pure Python DB-API 2.0 interface
  • http://www.pymysql.org/
• MySQL-Connector
  • CPython 2.4+ & 3
  • Pure Python DB-API 2.0 interface
  • https://launchpad.net/myconnpy
Other “Enterprise” Databases

• cx_Oracle
  • CPython 2 & 3
  • DB-API 2.0 interface
  • http://cx-oracle.sourceforge.net/
• informixda
  • CPython 2
  • DB-API 2.0 interface
  • http://informixdb.sourceforge.net/
  • Last release 2007
• Ibm-db
  • CPython 2
  • DB-API 2.0 for DB2 & Informix
  • http://code.google.com/p/ibm-db/
ODBC

• mxODBC
  • CPython 2.3+
  • DB-API 2.0 interfaces
  • http://www.egenix.com/products/pytho
    n/mxODBC/doc
  • Commercial product

• PyODBC
  • CPython 2 & 3
  • DB-API 2.0 interfaces with extensions
  • http://code.google.com/p/pyodbc/

• ODBC interfaces not limited to Windows
  thanks to iODBC and unixODBC
Jython + SQL

• zxJDBC
  • DB-API 2.0 Written in Java using JDBC API
     so can utilize JDBC drivers
  • Support for connection pools and JNDI
     lookup
  • Included with standard Jython
     installation http://www.jython.org/
• jyjdbc
  • DB-API 2.0 compliant
  • Written in Python/Jython so can utilize
     JDBC drivers
  • Decimal data type support
  • http://code.google.com/p/jyjdbc/
IronPython + SQL

• adodbapi
  • IronPython 2+
  • Also works with CPython 2.3+ with
    pywin32
  • http://adodbapi.sourceforge.net/
Gerald, the half a schema

     • Database schema toolkit
       • via DB-API currently supports
          • PostgreSQL
          • MySQL
          • Oracle
     • http://halfcooked.com/code/gerald/
import gerald
s1 = gerald.PostgresSchema(’public',
       'postgres://ip2country_rw:secret@localhost/ip2country')
s2 = gerald.PostgresSchema(’public',
     'postgres://ip2country_rw:secret@localhost/ip2countryv4')

print s1.schema['ip2country'].compare(s2.schema['ip2country'])
DIFF: Definition of assigned is different
DIFF: Column countryname not in ip2country
DIFF: Definition of registry is different
DIFF: Column countrycode3 not in ip2country
DIFF: Definition of countrycode2 is different
SQLPython

      • A command-line interface to relational
        databases
        • via DB-API currently supports
           • PostgreSQL
           • MySQL
           • Oracle
      • http://packages.python.org/sqlpython/
$ sqlpython --postgresql ip2country ip2country_rw
Password:
0:ip2country_rw@ip2country> select * from ip2country where countrycode2='SG';
...
1728830464.0 1728830719.0 apnic 2011-11-02 SG             SGP         Singapore
551 rows selected.
0:ip2country_rw@ip2country> select * from ip2country where countrycode2='SG'j
[...
{"ipfrom": 1728830464.0, "ipto": 1728830719.0, "registry": "apnic”,"assigned": "2011-11-02",
"countrycode2": "SG", "countrycode3": "SGP", "countryname": "Singapore"}]
SQLPython, batteries included
0:ip2country_rw@ip2country> select * from ip2country where countrycode2 ='SG’;
...
1728830464.0 1728830719.0 apnic 2011-11-02 SG          SGP       Singapore
551 rows selected.
0:ip2country_rw@ip2country> py
Python 2.6.6 (r266:84292, May 20 2011, 16:42:25)
[GCC 4.4.5 20110214 (Red Hat 4.4.5-6)] on linux2

    py <command>: Executes a Python command.
    py: Enters interactive Python mode.
    End with `Ctrl-D` (Unix) / `Ctrl-Z` (Windows), `quit()`, 'exit()`.
    Past SELECT results are exposed as list `r`;
      most recent resultset is `r[-1]`.
    SQL bind, substitution variables are exposed as `binds`, `substs`.
    Run python code from external files with ``run("filename.py")``

>>> r[-1][-1]
(1728830464.0, 1728830719.0, 'apnic', datetime.date(2011, 11, 2), 'SG', 'SGP', 'Singapore')
>>> import socket, struct
>>> def num_to_dotted_quad(n):
... return socket.inet_ntoa(struct.pack('!L',n))
...
>>> num_to_dotted_quad(int(r[-1][-1].ipfrom))
'103.11.220.0'
SpringPython – Database Templates
# Find ipv4 address ranges assigned to Singapore
# using SpringPython DatabaseTemplate & DictionaryRowMapper

from springpython.database.core import *
from springpython.database.factory import *

conn_factory = PgdbConnectionFactory(
            user="ip2country_rw", password="secret",
            host="localhost", database="ip2country")
dt = DatabaseTemplate(conn_factory)


results = dt.query(
           "SELECT * FROM ip2country WHERE countrycode2=%s",
           ("SG",), DictionaryRowMapper())

for row in results:
   print "%s - %s" % (num_to_dotted_quad(int(row['ipfrom'])),
               num_to_dotted_quad(int(row['ipto'])))
Attributions

DB-API 2.0 PEP
 http://www.python.org/dev/peps/pep-0249/

Travis Spencer‟s DB-API UML Diagram
 http://travisspencer.com/

Andrew Kuchling's introduction to the DB-API
 http://www.amk.ca/python/writing/DB-API.html
Attributions

Andy Todd‟s OSDC paper

http://halfcooked.com/presentations/osdc2006/p
ython_databases.html

Source of csv data used in examples from
WebNet77 licensed under GPLv3
 http://software77.net/geo-ip/
Contact Details




             Mark Rees
mark at centurysoftware dot com dot my
              +Mark Rees
             @hexdump42
        hex-dump.blogspot.com
Ad

More Related Content

What's hot (20)

Building a near real time search engine & analytics for logs using solr
Building a near real time search engine & analytics for logs using solrBuilding a near real time search engine & analytics for logs using solr
Building a near real time search engine & analytics for logs using solr
lucenerevolution
 
Monitoring Spark Applications
Monitoring Spark ApplicationsMonitoring Spark Applications
Monitoring Spark Applications
Tzach Zohar
 
Large Scale Log Analytics with Solr: Presented by Rafał Kuć & Radu Gheorghe, ...
Large Scale Log Analytics with Solr: Presented by Rafał Kuć & Radu Gheorghe, ...Large Scale Log Analytics with Solr: Presented by Rafał Kuć & Radu Gheorghe, ...
Large Scale Log Analytics with Solr: Presented by Rafał Kuć & Radu Gheorghe, ...
Lucidworks
 
Beyond shuffling global big data tech conference 2015 sj
Beyond shuffling   global big data tech conference 2015 sjBeyond shuffling   global big data tech conference 2015 sj
Beyond shuffling global big data tech conference 2015 sj
Holden Karau
 
Why your Spark job is failing
Why your Spark job is failingWhy your Spark job is failing
Why your Spark job is failing
Sandy Ryza
 
Beyond parallelize and collect - Spark Summit East 2016
Beyond parallelize and collect - Spark Summit East 2016Beyond parallelize and collect - Spark Summit East 2016
Beyond parallelize and collect - Spark Summit East 2016
Holden Karau
 
Mongo db
Mongo dbMongo db
Mongo db
Athira Mukundan
 
Effective testing for spark programs Strata NY 2015
Effective testing for spark programs   Strata NY 2015Effective testing for spark programs   Strata NY 2015
Effective testing for spark programs Strata NY 2015
Holden Karau
 
Beyond Shuffling - Effective Tips and Tricks for Scaling Spark (Vancouver Sp...
Beyond Shuffling  - Effective Tips and Tricks for Scaling Spark (Vancouver Sp...Beyond Shuffling  - Effective Tips and Tricks for Scaling Spark (Vancouver Sp...
Beyond Shuffling - Effective Tips and Tricks for Scaling Spark (Vancouver Sp...
Holden Karau
 
Algebird : Abstract Algebra for big data analytics. Devoxx 2014
Algebird : Abstract Algebra for big data analytics. Devoxx 2014Algebird : Abstract Algebra for big data analytics. Devoxx 2014
Algebird : Abstract Algebra for big data analytics. Devoxx 2014
Samir Bessalah
 
Effective testing for spark programs scala bay preview (pre-strata ny 2015)
Effective testing for spark programs scala bay preview (pre-strata ny 2015)Effective testing for spark programs scala bay preview (pre-strata ny 2015)
Effective testing for spark programs scala bay preview (pre-strata ny 2015)
Holden Karau
 
DataSource V2 and Cassandra – A Whole New World
DataSource V2 and Cassandra – A Whole New WorldDataSource V2 and Cassandra – A Whole New World
DataSource V2 and Cassandra – A Whole New World
Databricks
 
Requery overview
Requery overviewRequery overview
Requery overview
Sunghyouk Bae
 
Debugging PySpark: Spark Summit East talk by Holden Karau
Debugging PySpark: Spark Summit East talk by Holden KarauDebugging PySpark: Spark Summit East talk by Holden Karau
Debugging PySpark: Spark Summit East talk by Holden Karau
Spark Summit
 
Apache Spark Structured Streaming for Machine Learning - StrataConf 2016
Apache Spark Structured Streaming for Machine Learning - StrataConf 2016Apache Spark Structured Streaming for Machine Learning - StrataConf 2016
Apache Spark Structured Streaming for Machine Learning - StrataConf 2016
Holden Karau
 
A deeper-understanding-of-spark-internals
A deeper-understanding-of-spark-internalsA deeper-understanding-of-spark-internals
A deeper-understanding-of-spark-internals
Cheng Min Chi
 
Norikra: SQL Stream Processing In Ruby
Norikra: SQL Stream Processing In RubyNorikra: SQL Stream Processing In Ruby
Norikra: SQL Stream Processing In Ruby
SATOSHI TAGOMORI
 
Kotlin @ Coupang Backed - JetBrains Day seoul 2018
Kotlin @ Coupang Backed - JetBrains Day seoul 2018Kotlin @ Coupang Backed - JetBrains Day seoul 2018
Kotlin @ Coupang Backed - JetBrains Day seoul 2018
Sunghyouk Bae
 
Large Scale Log Analytics with Solr (from Lucene Revolution 2015)
Large Scale Log Analytics with Solr (from Lucene Revolution 2015)Large Scale Log Analytics with Solr (from Lucene Revolution 2015)
Large Scale Log Analytics with Solr (from Lucene Revolution 2015)
Sematext Group, Inc.
 
JS Fest 2019/Autumn. Maciej Treder. Angular Schematics - Develop for developers
JS Fest 2019/Autumn. Maciej Treder. Angular Schematics - Develop for developersJS Fest 2019/Autumn. Maciej Treder. Angular Schematics - Develop for developers
JS Fest 2019/Autumn. Maciej Treder. Angular Schematics - Develop for developers
JSFestUA
 
Building a near real time search engine & analytics for logs using solr
Building a near real time search engine & analytics for logs using solrBuilding a near real time search engine & analytics for logs using solr
Building a near real time search engine & analytics for logs using solr
lucenerevolution
 
Monitoring Spark Applications
Monitoring Spark ApplicationsMonitoring Spark Applications
Monitoring Spark Applications
Tzach Zohar
 
Large Scale Log Analytics with Solr: Presented by Rafał Kuć & Radu Gheorghe, ...
Large Scale Log Analytics with Solr: Presented by Rafał Kuć & Radu Gheorghe, ...Large Scale Log Analytics with Solr: Presented by Rafał Kuć & Radu Gheorghe, ...
Large Scale Log Analytics with Solr: Presented by Rafał Kuć & Radu Gheorghe, ...
Lucidworks
 
Beyond shuffling global big data tech conference 2015 sj
Beyond shuffling   global big data tech conference 2015 sjBeyond shuffling   global big data tech conference 2015 sj
Beyond shuffling global big data tech conference 2015 sj
Holden Karau
 
Why your Spark job is failing
Why your Spark job is failingWhy your Spark job is failing
Why your Spark job is failing
Sandy Ryza
 
Beyond parallelize and collect - Spark Summit East 2016
Beyond parallelize and collect - Spark Summit East 2016Beyond parallelize and collect - Spark Summit East 2016
Beyond parallelize and collect - Spark Summit East 2016
Holden Karau
 
Effective testing for spark programs Strata NY 2015
Effective testing for spark programs   Strata NY 2015Effective testing for spark programs   Strata NY 2015
Effective testing for spark programs Strata NY 2015
Holden Karau
 
Beyond Shuffling - Effective Tips and Tricks for Scaling Spark (Vancouver Sp...
Beyond Shuffling  - Effective Tips and Tricks for Scaling Spark (Vancouver Sp...Beyond Shuffling  - Effective Tips and Tricks for Scaling Spark (Vancouver Sp...
Beyond Shuffling - Effective Tips and Tricks for Scaling Spark (Vancouver Sp...
Holden Karau
 
Algebird : Abstract Algebra for big data analytics. Devoxx 2014
Algebird : Abstract Algebra for big data analytics. Devoxx 2014Algebird : Abstract Algebra for big data analytics. Devoxx 2014
Algebird : Abstract Algebra for big data analytics. Devoxx 2014
Samir Bessalah
 
Effective testing for spark programs scala bay preview (pre-strata ny 2015)
Effective testing for spark programs scala bay preview (pre-strata ny 2015)Effective testing for spark programs scala bay preview (pre-strata ny 2015)
Effective testing for spark programs scala bay preview (pre-strata ny 2015)
Holden Karau
 
DataSource V2 and Cassandra – A Whole New World
DataSource V2 and Cassandra – A Whole New WorldDataSource V2 and Cassandra – A Whole New World
DataSource V2 and Cassandra – A Whole New World
Databricks
 
Debugging PySpark: Spark Summit East talk by Holden Karau
Debugging PySpark: Spark Summit East talk by Holden KarauDebugging PySpark: Spark Summit East talk by Holden Karau
Debugging PySpark: Spark Summit East talk by Holden Karau
Spark Summit
 
Apache Spark Structured Streaming for Machine Learning - StrataConf 2016
Apache Spark Structured Streaming for Machine Learning - StrataConf 2016Apache Spark Structured Streaming for Machine Learning - StrataConf 2016
Apache Spark Structured Streaming for Machine Learning - StrataConf 2016
Holden Karau
 
A deeper-understanding-of-spark-internals
A deeper-understanding-of-spark-internalsA deeper-understanding-of-spark-internals
A deeper-understanding-of-spark-internals
Cheng Min Chi
 
Norikra: SQL Stream Processing In Ruby
Norikra: SQL Stream Processing In RubyNorikra: SQL Stream Processing In Ruby
Norikra: SQL Stream Processing In Ruby
SATOSHI TAGOMORI
 
Kotlin @ Coupang Backed - JetBrains Day seoul 2018
Kotlin @ Coupang Backed - JetBrains Day seoul 2018Kotlin @ Coupang Backed - JetBrains Day seoul 2018
Kotlin @ Coupang Backed - JetBrains Day seoul 2018
Sunghyouk Bae
 
Large Scale Log Analytics with Solr (from Lucene Revolution 2015)
Large Scale Log Analytics with Solr (from Lucene Revolution 2015)Large Scale Log Analytics with Solr (from Lucene Revolution 2015)
Large Scale Log Analytics with Solr (from Lucene Revolution 2015)
Sematext Group, Inc.
 
JS Fest 2019/Autumn. Maciej Treder. Angular Schematics - Develop for developers
JS Fest 2019/Autumn. Maciej Treder. Angular Schematics - Develop for developersJS Fest 2019/Autumn. Maciej Treder. Angular Schematics - Develop for developers
JS Fest 2019/Autumn. Maciej Treder. Angular Schematics - Develop for developers
JSFestUA
 

Viewers also liked (20)

Www Kitebird Com Articles Pydbapi Html Toc 1
Www Kitebird Com Articles Pydbapi Html Toc 1Www Kitebird Com Articles Pydbapi Html Toc 1
Www Kitebird Com Articles Pydbapi Html Toc 1
AkramWaseem
 
Rethink db with Python
Rethink db with PythonRethink db with Python
Rethink db with Python
Prabhu Raghav
 
Succumbing to the Python in Financial Markets
Succumbing to the Python in Financial MarketsSuccumbing to the Python in Financial Markets
Succumbing to the Python in Financial Markets
dcerezo
 
"PostgreSQL and Python" Lightning Talk @EuroPython2014
"PostgreSQL and Python" Lightning Talk @EuroPython2014"PostgreSQL and Python" Lightning Talk @EuroPython2014
"PostgreSQL and Python" Lightning Talk @EuroPython2014
Henning Jacobs
 
MySQL User Conference 2009: Python and MySQL
MySQL User Conference 2009: Python and MySQLMySQL User Conference 2009: Python and MySQL
MySQL User Conference 2009: Python and MySQL
Ted Leung
 
Test Tech 2011 Overview
Test Tech 2011 OverviewTest Tech 2011 Overview
Test Tech 2011 Overview
jamono
 
User Manual Joomla2006
User Manual Joomla2006User Manual Joomla2006
User Manual Joomla2006
defg gf
 
Database concepts
Database conceptsDatabase concepts
Database concepts
FLYMAN TECHNOLOGY LIMITED
 
Presentation latin american_librarians
Presentation latin american_librariansPresentation latin american_librarians
Presentation latin american_librarians
Aquiles Alencar Brayner
 
Scaling mysql with python (and Docker).
Scaling mysql with python (and Docker).Scaling mysql with python (and Docker).
Scaling mysql with python (and Docker).
Roberto Polli
 
Python for Derivative Analytics
Python for Derivative AnalyticsPython for Derivative Analytics
Python for Derivative Analytics
Alicia G
 
Lesson Two Exploring An Access Database
Lesson Two   Exploring An Access DatabaseLesson Two   Exploring An Access Database
Lesson Two Exploring An Access Database
guevarra_2000
 
MS Access Training
MS Access TrainingMS Access Training
MS Access Training
Michael Sheyahshe
 
Programming with Python and PostgreSQL
Programming with Python and PostgreSQLProgramming with Python and PostgreSQL
Programming with Python and PostgreSQL
Peter Eisentraut
 
Python Utilities for Managing MySQL Databases
Python Utilities for Managing MySQL DatabasesPython Utilities for Managing MySQL Databases
Python Utilities for Managing MySQL Databases
Mats Kindahl
 
PostgreSQLとPythonとSQL
PostgreSQLとPythonとSQLPostgreSQLとPythonとSQL
PostgreSQLとPythonとSQL
Satoshi Yamada
 
Python for Big Data Analytics
Python for Big Data AnalyticsPython for Big Data Analytics
Python for Big Data Analytics
Edureka!
 
Python for Big Data Analytics
Python for Big Data AnalyticsPython for Big Data Analytics
Python for Big Data Analytics
Edureka!
 
Sample training manual
Sample training manualSample training manual
Sample training manual
Sherri Orwick Ogden
 
Ten Reasons Why You Should Prefer PostgreSQL to MySQL
Ten Reasons Why You Should Prefer PostgreSQL to MySQLTen Reasons Why You Should Prefer PostgreSQL to MySQL
Ten Reasons Why You Should Prefer PostgreSQL to MySQL
anandology
 
Www Kitebird Com Articles Pydbapi Html Toc 1
Www Kitebird Com Articles Pydbapi Html Toc 1Www Kitebird Com Articles Pydbapi Html Toc 1
Www Kitebird Com Articles Pydbapi Html Toc 1
AkramWaseem
 
Rethink db with Python
Rethink db with PythonRethink db with Python
Rethink db with Python
Prabhu Raghav
 
Succumbing to the Python in Financial Markets
Succumbing to the Python in Financial MarketsSuccumbing to the Python in Financial Markets
Succumbing to the Python in Financial Markets
dcerezo
 
"PostgreSQL and Python" Lightning Talk @EuroPython2014
"PostgreSQL and Python" Lightning Talk @EuroPython2014"PostgreSQL and Python" Lightning Talk @EuroPython2014
"PostgreSQL and Python" Lightning Talk @EuroPython2014
Henning Jacobs
 
MySQL User Conference 2009: Python and MySQL
MySQL User Conference 2009: Python and MySQLMySQL User Conference 2009: Python and MySQL
MySQL User Conference 2009: Python and MySQL
Ted Leung
 
Test Tech 2011 Overview
Test Tech 2011 OverviewTest Tech 2011 Overview
Test Tech 2011 Overview
jamono
 
User Manual Joomla2006
User Manual Joomla2006User Manual Joomla2006
User Manual Joomla2006
defg gf
 
Scaling mysql with python (and Docker).
Scaling mysql with python (and Docker).Scaling mysql with python (and Docker).
Scaling mysql with python (and Docker).
Roberto Polli
 
Python for Derivative Analytics
Python for Derivative AnalyticsPython for Derivative Analytics
Python for Derivative Analytics
Alicia G
 
Lesson Two Exploring An Access Database
Lesson Two   Exploring An Access DatabaseLesson Two   Exploring An Access Database
Lesson Two Exploring An Access Database
guevarra_2000
 
Programming with Python and PostgreSQL
Programming with Python and PostgreSQLProgramming with Python and PostgreSQL
Programming with Python and PostgreSQL
Peter Eisentraut
 
Python Utilities for Managing MySQL Databases
Python Utilities for Managing MySQL DatabasesPython Utilities for Managing MySQL Databases
Python Utilities for Managing MySQL Databases
Mats Kindahl
 
PostgreSQLとPythonとSQL
PostgreSQLとPythonとSQLPostgreSQLとPythonとSQL
PostgreSQLとPythonとSQL
Satoshi Yamada
 
Python for Big Data Analytics
Python for Big Data AnalyticsPython for Big Data Analytics
Python for Big Data Analytics
Edureka!
 
Python for Big Data Analytics
Python for Big Data AnalyticsPython for Big Data Analytics
Python for Big Data Analytics
Edureka!
 
Ten Reasons Why You Should Prefer PostgreSQL to MySQL
Ten Reasons Why You Should Prefer PostgreSQL to MySQLTen Reasons Why You Should Prefer PostgreSQL to MySQL
Ten Reasons Why You Should Prefer PostgreSQL to MySQL
anandology
 
Ad

Similar to Relational Database Access with Python ‘sans’ ORM (20)

Angular2 inter3
Angular2 inter3Angular2 inter3
Angular2 inter3
Oswald Campesato
 
Icinga 2010 at Nagios Workshop
Icinga 2010 at Nagios WorkshopIcinga 2010 at Nagios Workshop
Icinga 2010 at Nagios Workshop
Icinga
 
ql.io at NodePDX
ql.io at NodePDXql.io at NodePDX
ql.io at NodePDX
Subbu Allamaraju
 
Automating with NX-OS: Let's Get Started!
Automating with NX-OS: Let's Get Started!Automating with NX-OS: Let's Get Started!
Automating with NX-OS: Let's Get Started!
Cisco DevNet
 
ELK stack at weibo.com
ELK stack at weibo.comELK stack at weibo.com
ELK stack at weibo.com
琛琳 饶
 
ql.io: Consuming HTTP at Scale
ql.io: Consuming HTTP at Scale ql.io: Consuming HTTP at Scale
ql.io: Consuming HTTP at Scale
Subbu Allamaraju
 
Letswift19-clean-architecture
Letswift19-clean-architectureLetswift19-clean-architecture
Letswift19-clean-architecture
Jung Kim
 
Introduction to cloudforecast
Introduction to cloudforecastIntroduction to cloudforecast
Introduction to cloudforecast
Masahiro Nagano
 
Developing with the Go client for Apache Kafka
Developing with the Go client for Apache KafkaDeveloping with the Go client for Apache Kafka
Developing with the Go client for Apache Kafka
Joe Stein
 
Icinga 2009 at OSMC
Icinga 2009 at OSMCIcinga 2009 at OSMC
Icinga 2009 at OSMC
Icinga
 
DBA だってもっと効率化したい!〜最近の自動化事情とOracle Database〜
DBA だってもっと効率化したい!〜最近の自動化事情とOracle Database〜DBA だってもっと効率化したい!〜最近の自動化事情とOracle Database〜
DBA だってもっと効率化したい!〜最近の自動化事情とOracle Database〜
Michitoshi Yoshida
 
Pycon - Python for ethical hackers
Pycon - Python for ethical hackers Pycon - Python for ethical hackers
Pycon - Python for ethical hackers
Mohammad Reza Kamalifard
 
Python para equipos de ciberseguridad
Python para equipos de ciberseguridad Python para equipos de ciberseguridad
Python para equipos de ciberseguridad
Jose Manuel Ortega Candel
 
2024 Dec 05 - PyData Global - Tutorial Its In The Air Tonight
2024 Dec 05 - PyData Global - Tutorial Its In The Air Tonight2024 Dec 05 - PyData Global - Tutorial Its In The Air Tonight
2024 Dec 05 - PyData Global - Tutorial Its In The Air Tonight
Timothy Spann
 
Our Puppet Story (GUUG FFG 2015)
Our Puppet Story (GUUG FFG 2015)Our Puppet Story (GUUG FFG 2015)
Our Puppet Story (GUUG FFG 2015)
DECK36
 
Serving Deep Learning Models At Scale With RedisAI: Luca Antiga
Serving Deep Learning Models At Scale With RedisAI: Luca AntigaServing Deep Learning Models At Scale With RedisAI: Luca Antiga
Serving Deep Learning Models At Scale With RedisAI: Luca Antiga
Redis Labs
 
OSMC 2009 | Icinga by Icinga Team
OSMC 2009 | Icinga by Icinga TeamOSMC 2009 | Icinga by Icinga Team
OSMC 2009 | Icinga by Icinga Team
NETWAYS
 
Oleksandr Tarasenko "ORM vs GraphQL"
Oleksandr Tarasenko "ORM vs GraphQL"Oleksandr Tarasenko "ORM vs GraphQL"
Oleksandr Tarasenko "ORM vs GraphQL"
Fwdays
 
ORM vs GraphQL - Python fwdays 2019
ORM vs GraphQL - Python fwdays 2019ORM vs GraphQL - Python fwdays 2019
ORM vs GraphQL - Python fwdays 2019
Oleksandr Tarasenko
 
OpenStack API's and WSGI
OpenStack API's and WSGIOpenStack API's and WSGI
OpenStack API's and WSGI
Mike Pittaro
 
Icinga 2010 at Nagios Workshop
Icinga 2010 at Nagios WorkshopIcinga 2010 at Nagios Workshop
Icinga 2010 at Nagios Workshop
Icinga
 
Automating with NX-OS: Let's Get Started!
Automating with NX-OS: Let's Get Started!Automating with NX-OS: Let's Get Started!
Automating with NX-OS: Let's Get Started!
Cisco DevNet
 
ELK stack at weibo.com
ELK stack at weibo.comELK stack at weibo.com
ELK stack at weibo.com
琛琳 饶
 
ql.io: Consuming HTTP at Scale
ql.io: Consuming HTTP at Scale ql.io: Consuming HTTP at Scale
ql.io: Consuming HTTP at Scale
Subbu Allamaraju
 
Letswift19-clean-architecture
Letswift19-clean-architectureLetswift19-clean-architecture
Letswift19-clean-architecture
Jung Kim
 
Introduction to cloudforecast
Introduction to cloudforecastIntroduction to cloudforecast
Introduction to cloudforecast
Masahiro Nagano
 
Developing with the Go client for Apache Kafka
Developing with the Go client for Apache KafkaDeveloping with the Go client for Apache Kafka
Developing with the Go client for Apache Kafka
Joe Stein
 
Icinga 2009 at OSMC
Icinga 2009 at OSMCIcinga 2009 at OSMC
Icinga 2009 at OSMC
Icinga
 
DBA だってもっと効率化したい!〜最近の自動化事情とOracle Database〜
DBA だってもっと効率化したい!〜最近の自動化事情とOracle Database〜DBA だってもっと効率化したい!〜最近の自動化事情とOracle Database〜
DBA だってもっと効率化したい!〜最近の自動化事情とOracle Database〜
Michitoshi Yoshida
 
2024 Dec 05 - PyData Global - Tutorial Its In The Air Tonight
2024 Dec 05 - PyData Global - Tutorial Its In The Air Tonight2024 Dec 05 - PyData Global - Tutorial Its In The Air Tonight
2024 Dec 05 - PyData Global - Tutorial Its In The Air Tonight
Timothy Spann
 
Our Puppet Story (GUUG FFG 2015)
Our Puppet Story (GUUG FFG 2015)Our Puppet Story (GUUG FFG 2015)
Our Puppet Story (GUUG FFG 2015)
DECK36
 
Serving Deep Learning Models At Scale With RedisAI: Luca Antiga
Serving Deep Learning Models At Scale With RedisAI: Luca AntigaServing Deep Learning Models At Scale With RedisAI: Luca Antiga
Serving Deep Learning Models At Scale With RedisAI: Luca Antiga
Redis Labs
 
OSMC 2009 | Icinga by Icinga Team
OSMC 2009 | Icinga by Icinga TeamOSMC 2009 | Icinga by Icinga Team
OSMC 2009 | Icinga by Icinga Team
NETWAYS
 
Oleksandr Tarasenko "ORM vs GraphQL"
Oleksandr Tarasenko "ORM vs GraphQL"Oleksandr Tarasenko "ORM vs GraphQL"
Oleksandr Tarasenko "ORM vs GraphQL"
Fwdays
 
ORM vs GraphQL - Python fwdays 2019
ORM vs GraphQL - Python fwdays 2019ORM vs GraphQL - Python fwdays 2019
ORM vs GraphQL - Python fwdays 2019
Oleksandr Tarasenko
 
OpenStack API's and WSGI
OpenStack API's and WSGIOpenStack API's and WSGI
OpenStack API's and WSGI
Mike Pittaro
 
Ad

More from Mark Rees (6)

Porting a legacy app to python 3
Porting a legacy app to python 3Porting a legacy app to python 3
Porting a legacy app to python 3
Mark Rees
 
Seeing with Python presented at PyCon AU 2014
Seeing with Python presented at PyCon AU 2014Seeing with Python presented at PyCon AU 2014
Seeing with Python presented at PyCon AU 2014
Mark Rees
 
Seeing with Python - Pycon SG 2014
Seeing with Python - Pycon SG 2014Seeing with Python - Pycon SG 2014
Seeing with Python - Pycon SG 2014
Mark Rees
 
Pypy is-it-ready-for-production-the-sequel
Pypy is-it-ready-for-production-the-sequelPypy is-it-ready-for-production-the-sequel
Pypy is-it-ready-for-production-the-sequel
Mark Rees
 
PyPy - is it ready for production
PyPy - is it ready for productionPyPy - is it ready for production
PyPy - is it ready for production
Mark Rees
 
What do you mean it needs to be Java based? How jython saved the day.
What do you mean it needs to be Java based? How jython saved the day.What do you mean it needs to be Java based? How jython saved the day.
What do you mean it needs to be Java based? How jython saved the day.
Mark Rees
 
Porting a legacy app to python 3
Porting a legacy app to python 3Porting a legacy app to python 3
Porting a legacy app to python 3
Mark Rees
 
Seeing with Python presented at PyCon AU 2014
Seeing with Python presented at PyCon AU 2014Seeing with Python presented at PyCon AU 2014
Seeing with Python presented at PyCon AU 2014
Mark Rees
 
Seeing with Python - Pycon SG 2014
Seeing with Python - Pycon SG 2014Seeing with Python - Pycon SG 2014
Seeing with Python - Pycon SG 2014
Mark Rees
 
Pypy is-it-ready-for-production-the-sequel
Pypy is-it-ready-for-production-the-sequelPypy is-it-ready-for-production-the-sequel
Pypy is-it-ready-for-production-the-sequel
Mark Rees
 
PyPy - is it ready for production
PyPy - is it ready for productionPyPy - is it ready for production
PyPy - is it ready for production
Mark Rees
 
What do you mean it needs to be Java based? How jython saved the day.
What do you mean it needs to be Java based? How jython saved the day.What do you mean it needs to be Java based? How jython saved the day.
What do you mean it needs to be Java based? How jython saved the day.
Mark Rees
 

Recently uploaded (20)

TrustArc Webinar: Consumer Expectations vs Corporate Realities on Data Broker...
TrustArc Webinar: Consumer Expectations vs Corporate Realities on Data Broker...TrustArc Webinar: Consumer Expectations vs Corporate Realities on Data Broker...
TrustArc Webinar: Consumer Expectations vs Corporate Realities on Data Broker...
TrustArc
 
Dev Dives: Automate and orchestrate your processes with UiPath Maestro
Dev Dives: Automate and orchestrate your processes with UiPath MaestroDev Dives: Automate and orchestrate your processes with UiPath Maestro
Dev Dives: Automate and orchestrate your processes with UiPath Maestro
UiPathCommunity
 
DevOpsDays Atlanta 2025 - Building 10x Development Organizations.pptx
DevOpsDays Atlanta 2025 - Building 10x Development Organizations.pptxDevOpsDays Atlanta 2025 - Building 10x Development Organizations.pptx
DevOpsDays Atlanta 2025 - Building 10x Development Organizations.pptx
Justin Reock
 
Linux Support for SMARC: How Toradex Empowers Embedded Developers
Linux Support for SMARC: How Toradex Empowers Embedded DevelopersLinux Support for SMARC: How Toradex Empowers Embedded Developers
Linux Support for SMARC: How Toradex Empowers Embedded Developers
Toradex
 
Build Your Own Copilot & Agents For Devs
Build Your Own Copilot & Agents For DevsBuild Your Own Copilot & Agents For Devs
Build Your Own Copilot & Agents For Devs
Brian McKeiver
 
TrsLabs - Fintech Product & Business Consulting
TrsLabs - Fintech Product & Business ConsultingTrsLabs - Fintech Product & Business Consulting
TrsLabs - Fintech Product & Business Consulting
Trs Labs
 
Procurement Insights Cost To Value Guide.pptx
Procurement Insights Cost To Value Guide.pptxProcurement Insights Cost To Value Guide.pptx
Procurement Insights Cost To Value Guide.pptx
Jon Hansen
 
Into The Box Conference Keynote Day 1 (ITB2025)
Into The Box Conference Keynote Day 1 (ITB2025)Into The Box Conference Keynote Day 1 (ITB2025)
Into The Box Conference Keynote Day 1 (ITB2025)
Ortus Solutions, Corp
 
Andrew Marnell: Transforming Business Strategy Through Data-Driven Insights
Andrew Marnell: Transforming Business Strategy Through Data-Driven InsightsAndrew Marnell: Transforming Business Strategy Through Data-Driven Insights
Andrew Marnell: Transforming Business Strategy Through Data-Driven Insights
Andrew Marnell
 
Quantum Computing Quick Research Guide by Arthur Morgan
Quantum Computing Quick Research Guide by Arthur MorganQuantum Computing Quick Research Guide by Arthur Morgan
Quantum Computing Quick Research Guide by Arthur Morgan
Arthur Morgan
 
The Evolution of Meme Coins A New Era for Digital Currency ppt.pdf
The Evolution of Meme Coins A New Era for Digital Currency ppt.pdfThe Evolution of Meme Coins A New Era for Digital Currency ppt.pdf
The Evolution of Meme Coins A New Era for Digital Currency ppt.pdf
Abi john
 
Semantic Cultivators : The Critical Future Role to Enable AI
Semantic Cultivators : The Critical Future Role to Enable AISemantic Cultivators : The Critical Future Role to Enable AI
Semantic Cultivators : The Critical Future Role to Enable AI
artmondano
 
AI Changes Everything – Talk at Cardiff Metropolitan University, 29th April 2...
AI Changes Everything – Talk at Cardiff Metropolitan University, 29th April 2...AI Changes Everything – Talk at Cardiff Metropolitan University, 29th April 2...
AI Changes Everything – Talk at Cardiff Metropolitan University, 29th April 2...
Alan Dix
 
tecnologias de las primeras civilizaciones.pdf
tecnologias de las primeras civilizaciones.pdftecnologias de las primeras civilizaciones.pdf
tecnologias de las primeras civilizaciones.pdf
fjgm517
 
Complete Guide to Advanced Logistics Management Software in Riyadh.pdf
Complete Guide to Advanced Logistics Management Software in Riyadh.pdfComplete Guide to Advanced Logistics Management Software in Riyadh.pdf
Complete Guide to Advanced Logistics Management Software in Riyadh.pdf
Software Company
 
HCL Nomad Web – Best Practices and Managing Multiuser Environments
HCL Nomad Web – Best Practices and Managing Multiuser EnvironmentsHCL Nomad Web – Best Practices and Managing Multiuser Environments
HCL Nomad Web – Best Practices and Managing Multiuser Environments
panagenda
 
Big Data Analytics Quick Research Guide by Arthur Morgan
Big Data Analytics Quick Research Guide by Arthur MorganBig Data Analytics Quick Research Guide by Arthur Morgan
Big Data Analytics Quick Research Guide by Arthur Morgan
Arthur Morgan
 
Designing Low-Latency Systems with Rust and ScyllaDB: An Architectural Deep Dive
Designing Low-Latency Systems with Rust and ScyllaDB: An Architectural Deep DiveDesigning Low-Latency Systems with Rust and ScyllaDB: An Architectural Deep Dive
Designing Low-Latency Systems with Rust and ScyllaDB: An Architectural Deep Dive
ScyllaDB
 
Noah Loul Shares 5 Steps to Implement AI Agents for Maximum Business Efficien...
Noah Loul Shares 5 Steps to Implement AI Agents for Maximum Business Efficien...Noah Loul Shares 5 Steps to Implement AI Agents for Maximum Business Efficien...
Noah Loul Shares 5 Steps to Implement AI Agents for Maximum Business Efficien...
Noah Loul
 
Technology Trends in 2025: AI and Big Data Analytics
Technology Trends in 2025: AI and Big Data AnalyticsTechnology Trends in 2025: AI and Big Data Analytics
Technology Trends in 2025: AI and Big Data Analytics
InData Labs
 
TrustArc Webinar: Consumer Expectations vs Corporate Realities on Data Broker...
TrustArc Webinar: Consumer Expectations vs Corporate Realities on Data Broker...TrustArc Webinar: Consumer Expectations vs Corporate Realities on Data Broker...
TrustArc Webinar: Consumer Expectations vs Corporate Realities on Data Broker...
TrustArc
 
Dev Dives: Automate and orchestrate your processes with UiPath Maestro
Dev Dives: Automate and orchestrate your processes with UiPath MaestroDev Dives: Automate and orchestrate your processes with UiPath Maestro
Dev Dives: Automate and orchestrate your processes with UiPath Maestro
UiPathCommunity
 
DevOpsDays Atlanta 2025 - Building 10x Development Organizations.pptx
DevOpsDays Atlanta 2025 - Building 10x Development Organizations.pptxDevOpsDays Atlanta 2025 - Building 10x Development Organizations.pptx
DevOpsDays Atlanta 2025 - Building 10x Development Organizations.pptx
Justin Reock
 
Linux Support for SMARC: How Toradex Empowers Embedded Developers
Linux Support for SMARC: How Toradex Empowers Embedded DevelopersLinux Support for SMARC: How Toradex Empowers Embedded Developers
Linux Support for SMARC: How Toradex Empowers Embedded Developers
Toradex
 
Build Your Own Copilot & Agents For Devs
Build Your Own Copilot & Agents For DevsBuild Your Own Copilot & Agents For Devs
Build Your Own Copilot & Agents For Devs
Brian McKeiver
 
TrsLabs - Fintech Product & Business Consulting
TrsLabs - Fintech Product & Business ConsultingTrsLabs - Fintech Product & Business Consulting
TrsLabs - Fintech Product & Business Consulting
Trs Labs
 
Procurement Insights Cost To Value Guide.pptx
Procurement Insights Cost To Value Guide.pptxProcurement Insights Cost To Value Guide.pptx
Procurement Insights Cost To Value Guide.pptx
Jon Hansen
 
Into The Box Conference Keynote Day 1 (ITB2025)
Into The Box Conference Keynote Day 1 (ITB2025)Into The Box Conference Keynote Day 1 (ITB2025)
Into The Box Conference Keynote Day 1 (ITB2025)
Ortus Solutions, Corp
 
Andrew Marnell: Transforming Business Strategy Through Data-Driven Insights
Andrew Marnell: Transforming Business Strategy Through Data-Driven InsightsAndrew Marnell: Transforming Business Strategy Through Data-Driven Insights
Andrew Marnell: Transforming Business Strategy Through Data-Driven Insights
Andrew Marnell
 
Quantum Computing Quick Research Guide by Arthur Morgan
Quantum Computing Quick Research Guide by Arthur MorganQuantum Computing Quick Research Guide by Arthur Morgan
Quantum Computing Quick Research Guide by Arthur Morgan
Arthur Morgan
 
The Evolution of Meme Coins A New Era for Digital Currency ppt.pdf
The Evolution of Meme Coins A New Era for Digital Currency ppt.pdfThe Evolution of Meme Coins A New Era for Digital Currency ppt.pdf
The Evolution of Meme Coins A New Era for Digital Currency ppt.pdf
Abi john
 
Semantic Cultivators : The Critical Future Role to Enable AI
Semantic Cultivators : The Critical Future Role to Enable AISemantic Cultivators : The Critical Future Role to Enable AI
Semantic Cultivators : The Critical Future Role to Enable AI
artmondano
 
AI Changes Everything – Talk at Cardiff Metropolitan University, 29th April 2...
AI Changes Everything – Talk at Cardiff Metropolitan University, 29th April 2...AI Changes Everything – Talk at Cardiff Metropolitan University, 29th April 2...
AI Changes Everything – Talk at Cardiff Metropolitan University, 29th April 2...
Alan Dix
 
tecnologias de las primeras civilizaciones.pdf
tecnologias de las primeras civilizaciones.pdftecnologias de las primeras civilizaciones.pdf
tecnologias de las primeras civilizaciones.pdf
fjgm517
 
Complete Guide to Advanced Logistics Management Software in Riyadh.pdf
Complete Guide to Advanced Logistics Management Software in Riyadh.pdfComplete Guide to Advanced Logistics Management Software in Riyadh.pdf
Complete Guide to Advanced Logistics Management Software in Riyadh.pdf
Software Company
 
HCL Nomad Web – Best Practices and Managing Multiuser Environments
HCL Nomad Web – Best Practices and Managing Multiuser EnvironmentsHCL Nomad Web – Best Practices and Managing Multiuser Environments
HCL Nomad Web – Best Practices and Managing Multiuser Environments
panagenda
 
Big Data Analytics Quick Research Guide by Arthur Morgan
Big Data Analytics Quick Research Guide by Arthur MorganBig Data Analytics Quick Research Guide by Arthur Morgan
Big Data Analytics Quick Research Guide by Arthur Morgan
Arthur Morgan
 
Designing Low-Latency Systems with Rust and ScyllaDB: An Architectural Deep Dive
Designing Low-Latency Systems with Rust and ScyllaDB: An Architectural Deep DiveDesigning Low-Latency Systems with Rust and ScyllaDB: An Architectural Deep Dive
Designing Low-Latency Systems with Rust and ScyllaDB: An Architectural Deep Dive
ScyllaDB
 
Noah Loul Shares 5 Steps to Implement AI Agents for Maximum Business Efficien...
Noah Loul Shares 5 Steps to Implement AI Agents for Maximum Business Efficien...Noah Loul Shares 5 Steps to Implement AI Agents for Maximum Business Efficien...
Noah Loul Shares 5 Steps to Implement AI Agents for Maximum Business Efficien...
Noah Loul
 
Technology Trends in 2025: AI and Big Data Analytics
Technology Trends in 2025: AI and Big Data AnalyticsTechnology Trends in 2025: AI and Big Data Analytics
Technology Trends in 2025: AI and Big Data Analytics
InData Labs
 

Relational Database Access with Python ‘sans’ ORM

  • 1. Relational Database Access with Python „sans‟ ORM Mark Rees CTO Century Software (M) Sdn. Bhd.
  • 2. Your Current Relational Database Access Style? # Django ORM >>> from ip2country.models import Ip2Country >>> Ip2Country.objects.all() [<Ip2Country: Ip2Country object>, <Ip2Country: Ip2Country object>, '...(remaining elements truncated)...'] >>> sgp = Ip2Country.objects.filter(assigned__year=2012) ... .filter(countrycode2='SG') >>> sgp[0].ipfrom 1729580032.0
  • 3. Your Current Relational Database Access Style? # SQLAlchemy ORM >>> from sqlalchemy import create_engine, extract >>> from sqlalchemy.orm import sessionmaker >>> from models import Ip2Country >>> engine = create_engine('postgresql://ip2country_rw:secret@localhost/ip2country') >>> Session = sessionmaker(bind=engine) >>> session = Session() >>> all_data = session.query(Ip2Country).all() >>> sgp = session.query(Ip2Country). ... filter(extract('year',Ip2Country.assigned) == 2012). ... filter(Ip2Country.countrycode2 == 'SG') print sgp[0].ipfrom 1729580032.0
  • 4. SQL Relational Database Access SELECT * FROM ip2country; "ipfrom";"ipto";"registry";"assigned";"countrycode2";"countrycode3";"countryname" 1729522688;1729523711;"apnic";"2011-08-05";"CN";"CHN";"China" 1729523712;1729524735;"apnic";"2011-08-05";"CN";"CHN";"China” ... SELECT * FROM ip2country WHERE date_part('year', assigned) = 2012 AND countrycode2 = 'SG'; "ipfrom";"ipto";"registry";"assigned";"countrycode2";"countrycode3";"countryname" 1729580032;1729581055;"apnic";"2012-01-16";"SG";"SGP";"Singapore" 1729941504;1729942527;"apnic";"2012-01-10";"SG";"SGP";"Singapore” ... SELECT ipfrom FROM ip2country WHERE date_part('year', assigned) = 2012 AND countrycode2 = 'SG'; "ipfrom" 1729580032 1729941504 ...
  • 5. Python + SQL == Python DB-API 2.0 • The Python standard for a consistent interface to relational databases is the Python DB-API (PEP 249) • The majority of Python database interfaces adhere to this standard
  • 7. Python DB-API Connection Object Access the database via the connection object • Use connect constructor to create a connection with database conn = psycopg2.connect(parameters…) • Create cursor via the connection cur = conn.cursor() • Transaction management (implicit begin) conn.commit() conn.rollback() • Close connection (will rollback current transaction) conn.close() • Check module capabilities by globals psycopg2.apilevel psycopg2.threadsafety psycopg2.paramstyle
  • 8. Python DB-API Cursor Object A cursor object is used to represent a database cursor, which is used to manage the context of fetch operations. • Cursors created from the same connection are not isolated cur = conn.cursor() cur2 = conn.cursor() • Cursor methods cur.execute(operation, parameters) cur.executemany(op,seq_of_parameters) cur.fetchone() cur.fetchmany([size=cursor.arraysize]) cur.fetchall() cur.close()
  • 9. Python DB-API Cursor Object • Optional cursor methods cur.scroll(value[,mode='relative']) cur.next() cur.callproc(procname[,parameters]) cur.__iter__() • Results of an operation cur.description cur.rowcount cur.lastrowid • DB adaptor specific “proprietary” cursor methods
  • 10. Python DB-API Parameter Styles Allows you to keep SQL separate from parameters Improves performance & security Warning Never, never, NEVER use Python string concatenation (+) or string parameters interpolation (%) to pass variables to a SQL query string. Not even at gunpoint. From http://initd.org/psycopg/docs/usage.html#query-parameters
  • 11. Python DB-API Parameter Styles Global paramstyle gives supported style for the adaptor qmark Question mark style WHERE countrycode2 = ? numeric Numeric positional style WHERE countrycode2 = :1 named Named style WHERE countrycode2 = :code format ANSI C printf format style WHERE countrycode2 = %s pyformat Python format style WHERE countrycode2 = %(name)s
  • 12. Python + SQL: INSERT import csv, datetime, psycopg2 conn = psycopg2.connect("dbname=ip2country user=ip2country_rw password=secret”) cur = conn.cursor() with open("IpToCountry.csv", "rb") as f: reader = csv.reader(f) try: for row in reader: print row if row[0][0] != "#": row[3] = datetime.datetime.utcfromtimestamp(float(row[3])) cur.execute("""INSERT INTO ip2country( ipfrom, ipto, registry, assigned, countrycode2, countrycode3, countryname) VALUES (%s, %s, %s, %s, %s, %s, %s)""", row) except: conn.rollback() else: conn.commit() finally: cur.close() conn.close()
  • 13. Python + SQL: SELECT # Find ipv4 address ranges assigned to Singapore import psycopg2, socket, struct def num_to_dotted_quad(n): """convert long int to dotted quad string http://code.activestate.com/recipes/66517/""" return socket.inet_ntoa(struct.pack('!L',n)) conn = psycopg2.connect("dbname=ip2country user=ip2country_rw password=secret") cur = conn.cursor() cur.execute("""SELECT * FROM ip2country WHERE countrycode2 = 'SG' ORDER BY ipfrom""") for row in cur: print "%s - %s" % (num_to_dotted_quad(int(row[0])), num_to_dotted_quad(int(row[1])))
  • 14. SQLite • sqlite3 • CPython 2.5 & 3 • DB-API 2.0 • Part of CPython distribution since 2.5
  • 15. PostgreSQL • psycopg • CPython 2 & 3 • DB-API 2.0, level 2 thread safe • Appears to be most popular • http://initd.org/psycopg/ • py-postgresql • CPython 3 • DB-API 2.0 • Written in Python with optional C optimizations • pg_python - console • http://python.projects.postgresql.org/
  • 16. PostgreSQL • PyGreSQL • CPython 2.3+ • Classic & DB-API 2.0 interfaces • http://www.pygresql.org/ • Last release 2009 • pyPgSQL • CPython 2 • Classic & DB-API 2.0 interfaces • http://www.pygresql.org/ • Last release 2006
  • 17. PostgreSQL • pypq • CPython 2.7 & pypy 1.7+ • Uses ctypes • DB-API 2.0 interface • psycopg2-like extension API • https://bitbucket.org/descent/pypq • psycopg2ct • CPython 2.6+ & pypy 1.6+ • Uses ctypes • DB-API 2.0 interface • psycopg2 compat layer • http://github.com/mvantellingen/psyco pg2-ctypes
  • 18. MySQL • MySQL-python • CPython 2.3+ • DB-API 2.0 interface • http://sourceforge.net/projects/mysql- python/ • PyMySQL • CPython 2.4+ & 3 • Pure Python DB-API 2.0 interface • http://www.pymysql.org/ • MySQL-Connector • CPython 2.4+ & 3 • Pure Python DB-API 2.0 interface • https://launchpad.net/myconnpy
  • 19. Other “Enterprise” Databases • cx_Oracle • CPython 2 & 3 • DB-API 2.0 interface • http://cx-oracle.sourceforge.net/ • informixda • CPython 2 • DB-API 2.0 interface • http://informixdb.sourceforge.net/ • Last release 2007 • Ibm-db • CPython 2 • DB-API 2.0 for DB2 & Informix • http://code.google.com/p/ibm-db/
  • 20. ODBC • mxODBC • CPython 2.3+ • DB-API 2.0 interfaces • http://www.egenix.com/products/pytho n/mxODBC/doc • Commercial product • PyODBC • CPython 2 & 3 • DB-API 2.0 interfaces with extensions • http://code.google.com/p/pyodbc/ • ODBC interfaces not limited to Windows thanks to iODBC and unixODBC
  • 21. Jython + SQL • zxJDBC • DB-API 2.0 Written in Java using JDBC API so can utilize JDBC drivers • Support for connection pools and JNDI lookup • Included with standard Jython installation http://www.jython.org/ • jyjdbc • DB-API 2.0 compliant • Written in Python/Jython so can utilize JDBC drivers • Decimal data type support • http://code.google.com/p/jyjdbc/
  • 22. IronPython + SQL • adodbapi • IronPython 2+ • Also works with CPython 2.3+ with pywin32 • http://adodbapi.sourceforge.net/
  • 23. Gerald, the half a schema • Database schema toolkit • via DB-API currently supports • PostgreSQL • MySQL • Oracle • http://halfcooked.com/code/gerald/ import gerald s1 = gerald.PostgresSchema(’public', 'postgres://ip2country_rw:secret@localhost/ip2country') s2 = gerald.PostgresSchema(’public', 'postgres://ip2country_rw:secret@localhost/ip2countryv4') print s1.schema['ip2country'].compare(s2.schema['ip2country']) DIFF: Definition of assigned is different DIFF: Column countryname not in ip2country DIFF: Definition of registry is different DIFF: Column countrycode3 not in ip2country DIFF: Definition of countrycode2 is different
  • 24. SQLPython • A command-line interface to relational databases • via DB-API currently supports • PostgreSQL • MySQL • Oracle • http://packages.python.org/sqlpython/ $ sqlpython --postgresql ip2country ip2country_rw Password: 0:ip2country_rw@ip2country> select * from ip2country where countrycode2='SG'; ... 1728830464.0 1728830719.0 apnic 2011-11-02 SG SGP Singapore 551 rows selected. 0:ip2country_rw@ip2country> select * from ip2country where countrycode2='SG'j [... {"ipfrom": 1728830464.0, "ipto": 1728830719.0, "registry": "apnic”,"assigned": "2011-11-02", "countrycode2": "SG", "countrycode3": "SGP", "countryname": "Singapore"}]
  • 25. SQLPython, batteries included 0:ip2country_rw@ip2country> select * from ip2country where countrycode2 ='SG’; ... 1728830464.0 1728830719.0 apnic 2011-11-02 SG SGP Singapore 551 rows selected. 0:ip2country_rw@ip2country> py Python 2.6.6 (r266:84292, May 20 2011, 16:42:25) [GCC 4.4.5 20110214 (Red Hat 4.4.5-6)] on linux2 py <command>: Executes a Python command. py: Enters interactive Python mode. End with `Ctrl-D` (Unix) / `Ctrl-Z` (Windows), `quit()`, 'exit()`. Past SELECT results are exposed as list `r`; most recent resultset is `r[-1]`. SQL bind, substitution variables are exposed as `binds`, `substs`. Run python code from external files with ``run("filename.py")`` >>> r[-1][-1] (1728830464.0, 1728830719.0, 'apnic', datetime.date(2011, 11, 2), 'SG', 'SGP', 'Singapore') >>> import socket, struct >>> def num_to_dotted_quad(n): ... return socket.inet_ntoa(struct.pack('!L',n)) ... >>> num_to_dotted_quad(int(r[-1][-1].ipfrom)) '103.11.220.0'
  • 26. SpringPython – Database Templates # Find ipv4 address ranges assigned to Singapore # using SpringPython DatabaseTemplate & DictionaryRowMapper from springpython.database.core import * from springpython.database.factory import * conn_factory = PgdbConnectionFactory( user="ip2country_rw", password="secret", host="localhost", database="ip2country") dt = DatabaseTemplate(conn_factory) results = dt.query( "SELECT * FROM ip2country WHERE countrycode2=%s", ("SG",), DictionaryRowMapper()) for row in results: print "%s - %s" % (num_to_dotted_quad(int(row['ipfrom'])), num_to_dotted_quad(int(row['ipto'])))
  • 27. Attributions DB-API 2.0 PEP http://www.python.org/dev/peps/pep-0249/ Travis Spencer‟s DB-API UML Diagram http://travisspencer.com/ Andrew Kuchling's introduction to the DB-API http://www.amk.ca/python/writing/DB-API.html
  • 28. Attributions Andy Todd‟s OSDC paper http://halfcooked.com/presentations/osdc2006/p ython_databases.html Source of csv data used in examples from WebNet77 licensed under GPLv3 http://software77.net/geo-ip/
  • 29. Contact Details Mark Rees mark at centurysoftware dot com dot my +Mark Rees @hexdump42 hex-dump.blogspot.com

Editor's Notes

  • #3: For some Python programmers, their only exposure to accessing relational data is via a object relational mapper (ORM). As powerful is the concept of mapping objects to data, sometimes it is much simpler to manipulate your relational data using SQL. This talk will be about using the DB-API, Python’s standard mechanism for accessing relational databases.
  • #4: Or maybe you prefer sqlalchemy to abstract away the database. This talk will be about using the DB-API, Python’s standard mechanism for accessing relational databases.
  • #5: SQL (Structured Query Language) is a DSL and we can achieve the same results as the previous two slides. This what DBA’s program in. 
  • #7: This diagram no longer seems toexist on Travis’s site
  • #11: Always use parameter binding. Why? * you normally get better performance from some database engines due to to SQL query caching * reduce the chance of SQL injection
  • #12: Always use parameter binding. Why? * you normally get better performance from some database engines due to to SQL query caching * reduce the chance of SQL injection
  • #24: Gerald is a general purpose database schema toolkit written in Python. It can be used for cataloguing, managing and deploying database schemas. It is designed to allow you to easily identify the differences between databases.
  • #25: SQLPython is a command-line interface to relational databases written in Python. It was created as an alternative to Oracle’s SQL\\*Plus, and can likewise be used instead of postgres’ psql or mysql’smysql text clients. In addition, it offers several extra features inspired by other command-line clients: Neatened output, smart prompt, tab completion, history, scripting, output to file, paste buffer &amp; os command, unix like commands – ls cat grep, data dictionary exploration. Another feature is special output formats. By replacing the ; that terminates a SELECT statement with a backslash-character sequence, you can get output in a number of useful formats like xml, json, csvetc
  • #26: One of the most powerful features is the py command. The py command allows the user to execute Python commands, either one-at-a-time (with py {command}) or in an interactive environment (beginning with a bare py statement, and continuing until Ctrl-D, quit(), or exit() is entered). A history of result sets from each query is exposed to the python session as the list r; the most recent result set is r[-1]. Each row can be references as a tuple, or as an object with an attribute for each column.
  • #27: Spring Python takes the concepts of the Spring Framework and Spring Security, and brings them to the world of Python. It isn&apos;t a simple line-by-line port of the code. Instead, it takes some powerful ideas that were discovered in the realm of Java, and pragmatically applies them in the world of Python.One of these paradigms is a Portable Service Abstraction called DatabaseTemplate. * It is portable because it uses Python&apos;s standardized API, not tying us to any database vendor. Instead, in our example, we injected in an instance of Sqlite3ConnectionFactory* It provides the useful service of easily accessing information stored in a relational database, but letting us focus on the query, not the plumbing code* It offers a nice abstraction over Python&apos;s low level database API with reduced code noise. This allows us to avoid the cost and risk of writing code to manage cursors and exception handlingDatabaseTemplate handles exceptions by catching and holding them, then properly closing the cursor. It then raises it wrapped inside a Spring Python DataAccessException. This way, database resources are properly disposed of without losing the exception stack trace.The Database Template can be used in isolation from the SpringPython framework.