HPC HUB - Virtual Supercomputer on Demand
- 2. Vendor of software solutions in HPC virtualization area, provide cloud computational services for high-tech business areas
- 3. PROBLEM Installation of own computational clusters by large companies. Renting in large supercomputer research centres. Cloud HPC services: General purpose: Penguin on demand, Bull, Amazon EC2, Rescale, Nimbix, Cycle Computing. Focused on bioinformatics tasks: Illumina BaseSpace, DNANexus, iPlant, BGI EasyGenomics, Knome, Quagen/CLCBio. High costs of a hardware and servicing. Complexity of setting up and running of the new equipment. One-off short access unprofitable because of preliminary agreements. Problem of task stopping and restoring. Problem of migrating configured environment at change or upgrade of supercomputer. Difficulties in providing software configurations being convenient for all users. Not all cloud vendors support low latency network. All services are hosted abroad that restrict data transfer. Unpredictable risks of sanctions Commonly used schemes of HPC resources have a number of disadvantages both technical and legal restricting HPC application by all groups of users. Rapidly growing HPC market requires a new level of service and it should be streamlined for a commercial use in order to engage new customers. 02 Traditional scheme Disadvantages
- 4. SOLUTION Automated provisioning of vSC on-demand with an access to be granted through the web interface (for the B2B и B2C segments): • A wide spectrum of vSC configuration • Pay per use • Ability to get vSC with preinstalled comuting software • A flexible management system of vSC • Saving a copy of vSC, vSC migration, checkpointing (coming soon) HPC cloud on the client's infrastructure: • Higher average utilization level • An ease of HPC resources management • A simple utilization of computing software: independent environments for various tasks within a single physical cluster • Rapid auto-deployment of flexible and always “clean” HPC environment with required capacity • Effective intermediate level between the hardware and HPC users of HPC-resources • Scalability of vSC and flexibility in reserving the time for computation • Rapid HPC resources redistribution • Ease of migration between physical supercomputers • Ability to save the state of tasks and their subsequent restoration • Convenient platform for monetization of third-party computing software (SaaS) PRODUCT №1: HPC ENVIRONMENT ON-DEMAND PRODUCT №2: PRIVATE HPC CLOUD FACILITIES OBTAINED BY CLIENTS 03
- 5. SUPERCOMPUTER CLOUD An association of computational clusters into a single HPC-platform, which is accessible via the Internet and is being controlled by a special software. vSC A virtual supercomputer, that is being created on the HPC- platform within a single homogeneous computational cluster. Front-end System A software layer for the management of HPC-platform, the creation of vSC and end-user service. 04 PRODUCT STRUCTURE
- 6. Architecture description • On the first stage KVM has been chosen as a virtualization backend, because it sufficiently meets current virtual cluster requirements. • LXC is considered for late stages to reduce virtualization overhead, but still maintain virtual cluster isolation capability. • It's considered to use InfiniBand low-latency networking over TCP/IP for robustness and out of the box checkpoint-restoring of vSC containers. Technical requirements • Each vSC is isolated among each other and contains a full set of preinstalled clustering software, but still retain an ability for fine tuning. • vSC replicates application environment and user interface of modern clusters, that accelerates integration and allows to use regular cluster applications. • vSC is highly scalable and allows to manage number of worker nodes. • vSC has minimal performance overhead over real cluster hardware. Advantages • Based on open technologies: Openstack, KVM and Ceph. • Checkpoint and snapshoting on late stages. • Computing environment migration between physical supercomputers. • Highly scalable. 05 CLOUD ARCHITECTURE
- 7. FRONT-END SYSTEM Provides user interface to manage customer requests. FS PHYSICAL HPC CLUSTER Cloud infrastructure deployed on HPC clusters and managed by Openstack and Ceph.HW VIRTUAL SUPERCOMPUTER A number of virtual machines (containers) with shared block storage, that is combined into isolated HPC environment with SGE scheduler. vSC API OPENSTACK & CEPH The architecture of our cloud infrastructure confidently allow to cover over 80% of computing tasks and applications demanded by the market. 06 CLOUD ARCHITECTURE
- 8. Shared file storage Ceph: - distributed file system, - almost linear scalability, - high performance. HPC Hub team use the best practices in designing cloud infrastructure achieving high elasticity, reliability and efficiency of cloud HPC services. Secure computations: - L2 network isolation, - KVM virtualization. Minimal overhead of virtualization by special tunning (NUMA tunnig, vCPU tunning, huge pages). Reliability of data storage - data replication on Ceph level. Ability to transfer vSC image between users. Pure OpenStack based architecture. 07 ADVANTAGES
- 9. COMPANY HPC HUB Penguin on demand Bull/eXreme Factory Amazon EC2 Rescale Nimbix Cycle Computing HPC environment on-demand with shared fil storage. Bare metal HPC infrastructure. No full control on HPC environment. Prepaid HPC resources. Professional service and consulting. Based on XEN virtualization. No low latency network. Infrastructure of AWS, Microsoft, Azure, IBM Softlayer, Rackspace. HPC services with GPU based on own data centers. Provision of different HPC services over Amazon EC2. Russia USA France USA USA USA USA 0,08-0,12 >0,12 0,20-0,34 0,08-0,25 0,15-0,35 >0,40 0,25 BRIEF COUNTRY PRICE COMPUTING (USD/CORE*HOUR) COMPUTING VIRTUALIZATION REAL TIME PROVISIONING HPC ENVIRONMENT LOW LATENTLY NETWORK CHECKPOINTING 08 COMPETITORS
- 10. VILGELM BITNER CEO, PhD in Computer Science HPC HUB mob.: +7 999 815 7218 tel.: +7 495 774 1937 e-mail: [email protected] www.hpchub.ru HPC HUB. ALL RIGHTS RESERVED © 2015.