PHP to Python with No Regrets
- 1. PHP to Python: No Regrets The Grass Is Always Greener Alex Ezell Emma July 31, 2011
- 2. the on e on th e left. Hi, I’m
- 3. The bloodiest wars in history have been religious wars. -Richard M. Nixon
- 4. Why did we change? How did we change? Would I do it all again? What did we learn?
- 5. Why did we change?
- 6. Change Is What We Do
- 7. Big, Hairy Ass Goals Rewrite the platform incrementally Provide a world-class customer API Reexamine every tool Quadruple the tech team Lead instead of follow
- 8. Opportunity Major push from the business for new features more often and the freedom to make that happen Team growth Stability and agreement in other areas: infrastructure, strategic planning, etc. Developer desire - we simply wanted more. Necessity with regards to current codebase
- 9. Code Unity / Management Lots of background, server processes managing parts of our application. Desire for code in background processes to more closely resemble application code Threading, decoupling, and queueing. Third-party mailer integration. Several small teams working on a single project.
- 10. Modernizing Our Toolset Homegrown PHP Web Framework -> Django PEAR_DB -> SQLAlchemy Apache -> WSGI No Testing/Docs -> nose and Sphinx CVS Update -> Fabric Deploys of Git Repo
- 11. Why Python? Maturity and capability Ease of use / Powerful / Low learning curve Rapid development Future of the technology Ecosystem of tools
- 12. How did we change?
- 13. Revolution or Evolution? Stop for a full rewrite and not have something compelling when you reboot No new features while the rewrite occurs Inability to really beta test new stuff Fairly entrenched customer base Competition is innovating quickly
- 14. Service API Scalability, Speed, Fault-tolerant, Easy Deployment Collection of Python WSGI Apps Delayed processes, log readers, and queue consumers New structure for all business logic Smaller, tighter codebase - Easier to build on Public API is just a wrapper Heavy use of Redis
- 15. Module Fiesta accounts tantuclient tantu audience queueworkers jumper dbsession logparser graffiti
- 16. Learn the Python Ecosystem Fabric==1.1.2 meld3==0.6.7 Jinja2==2.5.5 nose==1.0.0 Pygments==1.4 paramiko==1.7.7.1 SQLAlchemy==0.7.0 psycopg2==2.4.1 Sphinx==1.0.7 pycrypto==2.3 Tempita==0.5.1 redis==2.0.0 WebOb==1.0 requests==0.5.0 WebTest==1.2.2 setproctitle==1.1 coverage==3.4 supervisor==3.0a10 decorator==3.3.1 wsgiref==0.1.2 distribute==0.6.16 yoyo-migrations==4.1.0 docutils==0.7
- 17. Testing! Testing! Testing! nose, WebOb, WebTest Rapid iteration Distributed team No testing of the Django app piece
- 18. Session Handling Let PHP continue to handle session authentication Teach Python how to read/write PHP serialization using Scott Hurring’s code: http://hurring.com/scott/code/ python/serialize/ Django has built-in memcache support, so we make PHP save sessions in memcache
- 19. Gotta Get That Data PEAR_DB handled all queries and the database connection with hand-written SQL Moving to multi-database, sharded model - handle connections seamlessly Handle different types of encoding seamlessly Rewrite all queries in SQLAlchemy from straight SQL
- 22. The Campaign Edit Example 2600 lines of code to 700 Exact same functionality including some additional stuff Number of database queries stayed close to the same Very Javascript heavy, so changes might not be as extensive as other sections
- 25. What did we learn?
- 26. What We Learned We don’t really need a web framework Django is awesome, so is Flask, so is WSGI git is invaluable for distributed teams on big projects Easy deployment is vital Data encoding sucks and is hard Timezone handling sucks and is hard
- 27. What We Learned PHP frameworks (CakePHP and Symfony) have come a long way and are on par with Django The shift to sqlalchemy has provided lots of challenges Django template parsing can be slow in certain situations Hiring Python developers can be tough, especially in Nashville Juggling multiple systems is fraught with danger
- 28. What We Learned Incremental rewrites are technically more difficult Starting from scratch would have left us lost in the marketplace Many of our problems had nothing to do with language choice or tools or frameworks
- 29. Learning a New Language Porting is bad Bad habits are easy to break in Python, though not unavoidable Python’s interactive interpreter makes tinkering easy Every developer learns in unique ways and at different rates Reading great code
- 30. Would I do it all again?
- 31. www.wordle.net
- 32. Positives Codebase is becoming more standardized Developer energy is high Features we have implemented have turned out well and been easy to add functionality to Python’s standard library offers powerful, quality add- on modules for free Developers we have hired have been able to contribute quickly and be impactful
- 33. Many of us crucify ourselves between two thieves - regret for the past and fear of the future. - Fulton Oursler
- 34. We’re Launching It Sept. 1
- 35. Contact Me Email: [email protected] Twitter: @aezellisdead or @emmaemailtech
- 36. We Are Hiring at Emma Happy Hour Tonight at Eddie George’s Grill from 6-8pm http://myemma.com/tech-jobs/ Email: [email protected] Twitter: @aezellisdead or @emmaemailtech
- 37. Photo Credits Nixon - BusinessWeek: http://images.businessweek.com/ss/07/04/0426_dow/ source/5.htm Change - http://www.flickr.com/photos/spursfan_ace/2328879637/ Kofi Annan - http://www.gambianow.com/news/Open-Forum/Have-Your-Say/ Many_Give_Annan_High_Marks_as_UN_Chief_Despite_Flaws.html Butterflies - http://21stcenturylearning.typepad.com/blog/2009/01/why- change.html Sad boy - http://www.psychologytoday.com/blog/ulterior-motives/200806/will-i- regret-it-in-the-morning-how-about-when-im-64
Editor's Notes
- #2: - while there will be some comparison of languages, this isn’t a talk about which language is better, it’s more about why we chose to switch from PHP to Python\n\n
- #3: Name, Title, Experience?\n
- #4: - because knowledge is power, I know more about this than you know about that\n- because success begets success, why change what you’re good at?\n- because language, even computer language, is a very personal human construct\n
- #5: 4 topics for the preso\n
- #6: \n
- #7: - software development, especially web, is centered around change\n- business health demands change, mostly growth, but also strategic\n- personal growth demands change, not necessarily language, but new challenges\n
- #8: \n
- #9: - Looking at a complete rewrite of code for PHP 5/6 compliance\n- Current framework was showing its age: older templating system, older database access layer, maintenance was becoming a chore, adding new features was really difficult\n
- #10: - yes, php can run from the command line -- environment, database connection, etc.\n- python is more general purpose from its inception\n- reuse code in scripts\n- threading in mailer processes, yay!\n
- #11: - evaluation really began with looking at available Frameworks\n- What’s the most mature and can be used right now?\n- What has some capabilities to change where we might need change?\n- Who has a favorite framework? Name the thing you like most about it.\n
- #12: - maturity/capability - who’s using it now? will it integrate with our other tools (database)?\n- use/learning - can we learn it? do we want to?\n- django’s user base is growing exponentially\n- python’s standard library - lots of quality modules\n\n
- #13: Talk about timeline\n
- #14: - tipping point where retrofit changes sides\n
- #15: \n
- #16: \n
- #17: \n
- #18: \n
- #19: - added some code to the session stuff\n- wanted to pave the way for Django user management, because it’s nice\n
- #20: - multi-database needs to shift on the fly\n- special encoding types: Latin 1, ASCII, Unicode\n- speed of sqlalchemy or the Django ORM - objects vs. arrays\n- writing queries was a big learning curve\n
- #21: PHP response. Lots of tables. Many queries. Bad structure forcing bad code.\n
- #22: sqlalchemy version of the same query. Better database structure begets better code.\n
- #23: - code includes html, db access, javascript, everything\n- queries are interesting because they don’t change much and might actually be more code\n
- #24: \n
- #25: \n
- #26: \n
- #27: - do something: internal tools, one-off fun project\n- direction: concurrency, speed, etc.\n
- #28: \n
- #29: \n
- #30: - bad habits: brute force loops vs. generators\n
- #31: \n
- #32: \n
- #33: \n
- #34: \n
- #35: \n
- #36: \n
- #37: \n
- #38: \n