The Great Consolidation - Entertainment Weekly Migration Case Study - SANDcamp 2015
1 like833 views
The migration project involved moving content from Entertainment Weekly's WordPress and Vignette systems to Drupal. A team of 4 developers from Four Kitchens and 1 from Time Inc worked on the migration over 17 sprints from April 2014 to January 2015. Key aspects of the project included theming the site with Aurora, implementing JavaScript standards, migrating over 100,000 posts, images and terms from WordPress and Vignette, improving performance, and collaborating with Time Inc on custom content types and workflows. Testing was done to ensure the migrated site met performance standards.
The Great Consolidation - Entertainment Weekly Migration Case Study - SANDcamp 2015
- 1. SANDcamp, San Diego, CA - February 27, 2015 The Great Consolidation Entertainment Weekly migration case study
- 2. Who are we? • Four Kitchens Web Chefs • Jon Peck - drupal.org/u/fluxsauce • Matt Grill - drupal.org/u/drpal • Patrick Coffey - drupal.org/u/pcoffey
- 3. Project overview What did we do & who did it?
- 4. Entertainment Weekly • Magazine that covers film, TV, music, Broadway theater, books, and pop culture • 40th largest in the US • ew.com • 13.7 million consumers / week
- 5. Migration scope •WordPress • 10 blogs • 51,616 terms • 101,581 posts • 90,435 images •Vignette V6 • 117,326 articles • 201,986 images • 97,611 terms • 4,924 galleries
- 6. Development Teams • Four Kitchens - 3 developers to 6 • Time Inc. - 1 developer to 4 • Zoom, HipChat for regular communication • Video really helps • As does GitHub selfies :-)
- 7. Timeline • First commit: April 29th, 2014 • Launch: January 29th, 2015 • 17 sprints
- 8. Delegation of responsibilities •Four Kitchens • Project Management • Data structures • Migration • Implement design, advertising •Time Inc. • Product Owner • Design • Workflow • Infrastructure, build process
- 10. Definition of Readiness and Completion • Stakeholders • Context necessary for development • Developers • Code reviews, PO acceptance, demo, docs • Evolving framework
- 11. Code and style standards • Algorithmically enforced • PHP_CodeSniffer (rules from coder) • jshint - syntax • jscs - code style • EditorConfig settings
- 12. Peer code reviews • Functional - does it… • Fulfill the intent of the story? • Use best practices? • Avoid technical debt?
- 13. Development Environments • Documented project-specific process • Superset of existing instructions • Greatly reduced on-boarding time • Provided plentiful feedback to DCMS • Beta tested new VMs
- 14. Branching standard • Reduces clutter • Improves communication, navigation • Best practices • sprint-xx (kept for one sprint) • SEGEWCMSM-YY (deleted after merge) • Tags added when deleting sprint branch
- 15. Front-end Theming, performance and advertising
- 16. THEMING
- 17. https://www.drupal.org/project/aurora Aurora, mobile first theming • Used Aurora as base theme • HTML5, Sass & Compass • Mobile theme is sub-theme of Aurora • Desktop is a sub-theme of Mobile
- 18. Gulp task management • Compiles Sass • Runs JavaScript checkers • jshint • jscs
- 19. JAVASCRIPT
- 20. JavaScript global scoping • Created global JS object for all EW functions • Best practice • Centralized point of entry • Easy access
- 21. Atomic JavaScript functionality • Break related functions into individual files • Only load what’s needed • No monolithic structure • Improves front-end performance
- 22. PREPROCESSING
- 23. Balancing performance with business logic • Node template data built in pre-processing • Does not use views, panels • Clean & reusable • Intuitive structure • Helper and abstractions
- 24. BEACONS
- 25. Global metadata object • Useful for integrations requiring metadata • Avoids repetition • Facilitates data collection by crawlers
- 26. Beacon performance • Delay loading of beacons (non-blocking) • Use defer for injecting JS tags
- 27. ADVERTISING
- 28. Collaborating with DCMS • TGX improvements • Single-request mode • More native GPT functionality • Ad Manager • Patches, recommendations
- 29. Custom ad rendering to maximise performance • No logic in templates • Data attributes contain values • Avoids inline JS • Improves rendering performance • Ad renders in footer (after page load)
- 30. Editorial UI & Workflow
- 31. Packages and Channels • Automatic dynamic content • Editors define rules for selection • Optional manual curation
- 32. Custom content types • Used existing DCMS features to begin • Rounds of PO, editor feedback • Usability key to adoption
- 33. Collaboration with DCMS • Provided feedback on tools • Search interfaces • Transient messages • Crop tool • Auto save
- 34. Dynamic Entity References • Lots of aggregate views in EW • Editorial control of views, but with overrides • Custom module that “fills in” empty parts of a view based on context • View lists 15, editor defines 5, DER populates remaining
- 35. Dynamic ER challenges • Modular, extensible, reusable • Accurate dynamically curated nodes • High performance • Editoral expectations for caching • State Machine integration
- 36. Content migrations WordPress, Vignette to Drupal
- 37. WORDPRESS
- 38. Challenges • 10 different blogs • Different structures • Multiple taxonomies • Invalid markup • Custom shortcodes, filters • WordPress Migrate module not enough
- 39. Solutions • Interviewed other teams, shared code • Pre-processing • Strip comments • Combine into one monolithic file • Separates authors, tags, images
- 40. Shortcodes and Filters • Custom markup • Created custom short code system • Most will not be migrated • Pre-processing solution • Render as HTML • Used WordPress, existing code
- 41. WordPress Migrate • Can’t use directly… • Can extend! • Helps normalize WXR structure • Focus on custom logic
- 42. Mapping terms on import • Separates monolithic taxonomy • Specific vocabularies • New content types (people, creative works) • CSV Spreadsheet used for defining rules • Curated by Editors, PO
- 43. VIGNETTE
- 44. Exporting from Vignette • Coordinated with TI team • Defined XML structure • Gave feedback on export UI • Resolve edge cases
- 45. Importing • XML content is sanitized and validated • Transforms overloaded taxonomies to content • People, Creative Works • Articles, Packages, Galleries
- 47. Improving migration performance • Validate all XML before ingestion • Eliminate all PHP errors, warnings • Avoid redundant migrations • Separate in pre-import • Reduce overall number of migrations • Implement cache_counts
- 48. Reducing migration overhead • Disabled solr, pathauto, metatag, others during migration • Project requirement was 2.5; you should use 2.7 • Ran migrations in parallel • Ensure HW resources are sufficient
- 49. Migration redirects • Minimize Drupal performing redirects • Redirect farm (rules, 1:1 redirects) • Exported if source, target known • DNS changes • Redirects from migrated servers
- 51. Frontend performance • CSS • Sass and Compass optimized • Targeted - only load what’s needed • JavaScript • Linted, strict standards • In the footer (non-blocking) • Global objects (less is more)
- 52. Frontend caching strategy • Editorial 1 minute publish to live • Ensure cacheable headers are set • Shorter TTL on Akamai than Varnish • Ex: 5 minutes on Akamai, 10 on Varnish • Send purges on publishing
- 53. Backend performance • Eliminate PHP errors • Cache, EXPLAIN custom queries • Minimize unnecessary overhead • Module count • Benchmarking
- 54. Performance testing • Load test production with migrated data • blitz.io • New Relic for introspection • Exercised different content types, exceeded TTLs • Front-end testing • WebPageTest.org
- 55. Questions?
- 56. Thank you!