Razorfish - Amazon EMR usecase
- 1. Edge in the cloudSalim Hemdani VP, Experiences and Platforms@shemdani
- 3. 1,000
- 4. 100
- 5. 25
- 6. 13
- 7. What are these numbers?
- 8. Numbers500,000,000,000 records1,000 clients100 markets25 data sources13 terabytes per day
- 9. Agenda
- 10. Time for a change
- 11. Transition Service AgreementMove from AtlasTraditional hosting environment
- 12. Heavy on CAPEX
- 13. Managed by Atlas/MSFT networking teams
- 14. To be completed by October 2010; no interruption in SLAMove away from PVM
- 15. Ad Serving Event LogRequesthash(key) mod RFS01FS03FS029810198104981159820198203980049800798065
- 16. MapReduce (divide and concur)HDFSDistributed data storage
- 18. Language agnostic Any LanguageJob trackerTask tracker
- 19. AWS
- 20. Aggregate Ad Serving data Log FilesFile ExportAPIsInternetClient Provided DataData SourcesPresentation LayerTalend Data Flow ManagerDirect Analytics Processing via EMRWeb Application LayerODBCEdge Provisioning DBOLAPCacheCloud Storage S3 HBase/SDB15Elastic MapReduce
- 21. Name Brand Retailer Case StudyBusiness challengeChanging competitive landscape
- 22. Decreasing web marketing effectiveness
- 23. Monetization of their web assetsBring it all togetherProduct interestAffinityGeneration++In market GamerSport Enthusiast Purchaser Home Theater( 1 of 36 “Personalization” segments )
- 24. Drive a personalized messageUser recently purchased a home theater system and is now looking for sports gamesTarget Ad( 1.7 million per day )
- 25. We import Atlas transaction level data24 serversS3 file storageCompress and upload 200 + GB of data per day( 180 days = ½ Trillion ICA records )
- 26. We use EMR to process and segmentEMRS3100 Machinecluster created on demand( 3.5 Billion records, 71 million unique cookies a day)
- 27. Process and CostThis all happens in about 8 hours every day and is fully automated (previously 2+ days)And increased ROAS by 500% (to $74)
- 28. Why AWSEfficientElastic infrastructure from AWS allows capacity to be provisioned as needed based on load, reducing cost and the risk of processing delaysEase of integrationAmazon Elastic MapReduce with Cascading allows data processing in the cloud without any changes to the underlying algorithmsFlexibleHadoop with Cascading is flexible enough to allow “agile” implementation and unit testing of sophisticated algorithms.AdaptableCascading simplifies the integration of Hadoop with external ad systemScalableAWS infrastructure helps reliably store and process huge (Petabytes) data setss
- 29. Learning
- 30. Thank you.
Editor's Notes
- #22: Return on advertising spend (ROAS)