AWS Cloud for HPC and Big Data
8 likes3,274 views
In this video from the 2014 HPC User Forum in Seattle, David Pellerin from Amazon presents: Update on HPC Use on AWS. "High Performance Computing (HPC) allows scientists and engineers to solve complex science, engineering, and business problems using applications that require high bandwidth, low latency networking, and very high compute capabilities. AWS allows you to increase the speed of research by running high performance computing in the cloud and to reduce costs by providing Cluster Compute or Cluster GPU servers on-demand without large capital investments. You have access to a full-bisection, high bandwidth network for tightly-coupled, IO-intensive workloads, which enables you to scale out across thousands of cores for throughput-oriented applications." Watch the video presentation: http://wp.me/p3RLHQ-d0n
Ad
More Related Content
Similar to AWS Cloud for HPC and Big Data (18)
More from inside-BigData.com (20)
Ad
Recently uploaded (20)
Ad
AWS Cloud for HPC and Big Data
- 1. AWS Cloud for HPC and Big Data David Pellerin, Business Development Principal IDC HPC User Forum September 16, 2014
- 2. US West (Oregon) US West (Northern California) US East (Northern Virginia) EU (Ireland) Asia Pacific (Singapore) Asia Pacific (Tokyo) AWS Regions (10) AWS Edge Locations (52) South America (Sao Paulo) AWS Regions GovCloud (ITAR Compliance) Asia Pacific (Sydney) China (Beijing)
- 3. Human Interac8on Support AWS Globa l Infras truct ure API Customer Applica8ons Development & Deployment AWS Global Infrastructure IAM Federa8on AWS Global Infrastructure AWS Global Infrastructure AWS Global Infrastructure Applica'on Services Founda'on Services Applica8ons Interac8on Libraries, SDK’s Networking VPC Direct Connect ELB Route53 Databases RDS Dynamo Elas8Cache RedShiK Content Delivery CloudFront SES SNS SQS Elas8c Transcoder CloudSearch SWF Iden8ty & Access Web Console Regions AWS GloAbvaailla bIilnityf Zroanesst ructure Edge Loca'ons Analy8cs EMR DataPipeline Kinesis Compute EC2 WorkSpaces AppStream Storage S3 EBS Glacier Storage Gateway Monitoring CloudWatch Deployment & Management BeanStalk Cloud OpsWork Forma8on CloudTrail Command Line
- 4. EC2 Instance Type History Increasing customer choice… m1.small new existing m1.xlarge m1.large m1.small m2.2xlarge m2.4xlarge c1.medium c1.xlarge m1.xlarge m1.large m1.small cc2.8xlarge cc1.4xlarge cg1.4xlarge t1.micro m2.xlarge m2.2xlarge m2.4xlarge c1.medium c1.xlarge m1.xlarge m1.large m1.small cr1.8xlarge hs1.8xlarge m3.xlarge m3.2xlarge hi1.4xlarge m1.medium cc2.8xlarge cg1.4xlarge t1.micro m2.xlarge m2.2xlarge m2.4xlarge c1.medium c1.xlarge m1.xlarge m1.large m1.small cc1.4xlarge cg1.4xlarge t1.micro m2.xlarge m2.2xlarge m2.4xlarge c1.medium c1.xlarge m1.xlarge m1.large m1.small t2.micro t2.small t2.medium t1.micro hs1.8xlarge m3.xlarge m3.2xlarge hi1.4xlarge m1.medium cc2.8xlarge cr1.8xlarge cg1.4xlarge m2.xlarge m2.2xlarge m2.4xlarge c1.medium c1.xlarge m1.xlarge m1.large m1.small c1.medium c1.xlarge m1.xlarge m1.large m1.small g2.2xlarge hs1.xlarge hs1.2xlarge hs1.4xlarge c3.large c3.xlarge c3.2xlarge c3.4xlarge c3.8xlarge m3.medium m3.large i2.large i2.xlarge i2.4xlarge i2.8xlarge r3.large r3.xlarge r3.2xlarge r3.4xlarge r3.8xlarge 2006 2007 2008 2009 2010 2011 2012-2013 September, 2014
- 5. Multiple Purchase Models On-Demand Pay for compute capacity by the hour with no long-term commitments For spiky workloads, or to define needs Reserved Make a low, one-time payment and receive a significant discount on the hourly charge For committed utilization Spot Bid for unused capacity, charged at a Spot Price which fluctuates based on supply and demand For time-insensitive or transient workloads AWS Spot is a game-changer for HPC
- 6. Motivators for HPC in the Cloud Cloud for HPC Scalability Cloud for Secure Global Collaboration Cloud for Big Data
- 7. “HGST is using AWS for a higher performance, lower cost, faster deployed solution vs buying a huge on-site cluster.” - Steve Philpott, CIO HGST application roadmap: ü Molecular dynamics ü CAD, CFD, EDA ü Collaboration tools for engineering ü Big data for manufacturing yield analysis, including Amazon Redshift
- 8. On AWS, deploy multiple clusters running at the same time and match the architectures to the jobs
- 9. Use automation to manage cluster sizing and monitor jobs and costs AWS Auto Scaling can work with existing HPC scheduling software
- 10. Many HPC Deployment Methods • Tradi'onal HPC schedulers and cluster managers • “Born in the cloud” tools • MIT StarCluster • Cycle Compu'ng CycleServer • AWS-‐provided tools and APIs • Cloudforma'on, Auto Scaling • cfncluster (github.com/awslabs/cfncluster) • For many use-‐cases…
- 11. Touch-Sensor Modeling on AWS for TRUETOUCH® Touchscreen Controllers Courtesy of Cypress Semiconductor
- 12. Reservoir Simulation on AWS
- 13. Bristol-Myers Squibb Clinical Trials on AWS On-Premises Cloud # of Simulations # of Servers Total Run Time (hr) 2000 2 60 2000 256 1.2 Clinical trial simulations took 98% less time More efficient and iterative simulations results in fewer human trials 64% savings on clinical trial costs We’re using fewer subjects in these trials, and needing fewer blood samples. Russell Towell Senior Solutions Specialist
- 14. Collaboration and Design in the Cloud Industrial manufacturing Cross-functional collaboration app Helps design around manufacturing Allows users to define how they work Users can spin-up their own environments This could change the way manufacturing is architected. people Joe Salvo Manager, Business Integration Technologies Laboratory devices software design
- 15. Enabling Global Collaboration Bring the users to the data, don’t send the data to the users
- 16. Thin Client Remote Collaboration Calgary Scien;fic PureWeb™ www.calgaryscien'fic.com/resolu'onmd/web/ demos.getpureweb.com/
- 17. Amazon AppStream • Applica'on Streaming • Remote visualiza'on • Thin client 3D applica'ons
- 18. Big Data Plus Cloud Equals Awesome
- 19. AWS Has Always Been About Big Data
- 20. Big Data for Financial Market Analysis
- 21. Big Data for Genome Analysis Baylor College of Medicine, Amazon Web Services, and DNAnexus: cloud-based analysis of genomic data from over 14,000 patients
- 22. Big Data is Everywhere!
- 23. Summary Cloud for Scalability Cloud for Global Collabora8on Cloud for Big Data
- 24. Thank You