Сервисы Azure для научных исследований
Download as PPTX, PDF•1 like•246 views
Документ представляет обзор технологий высокопроизводительных вычислений (HPC) и их интеграции с облачными сервисами Microsoft Azure. Он описывает инструментарий HPC Pack, его возможности масштабирования и развертывания HPC-кластеров, а также применение машинного обучения в различных областях. В тексте также затрагиваются примеры использования и производительность облачных ресурсов для задач машинного обучения и аналитики данных.
Сервисы Azure для научных исследований
- 1. Эксперт по стратегическим технологиям, Cloud/Open Source/HPC [email protected] @ahriman_ru
- 2. Введение в проблематику Big Compute/HPC Сервисы и продукты Машинное обучение и аналитика
- 4. Embarrasingly Parallel Parametric Sweep Batch И еще много других названий, суть которых, в принципе, одна Purely parallel Message Passing Interface
- 8. • HPC Pack Burst в Azure
- 9. Что такое HPC Pack Бесплатное дополнение к Windows Server для создание HPC-кластера HPC Pack: Планировщик задач и ресурсов Распределенная среда выполнения: Sweep, MPI, SOA, Excel UDT Cluster Manager API Поддержка разных языков через парадигму SOA Поддержка параллельных алгоритмов через MPI (напр., mpi4py)
- 10. • HPC Pack Burst в Azure
- 11. A-series • 1-16 cores • 0.75-112GB RAM • 20-605 GB HDD • Up to InfiniBand 40Gbit/s RDMA network (MPI) D-series • 1-16 cores • 3.5-112 GB RAM • Up to 800GB SSD G-series • 32 cores • 468 GB RAM • 6.5 TB SSD
- 12. HPC: • CPU: 2x8 core processors per node, Sandybridge E-2670 at 2.6 GHz • High Memory: 128 GB, 1600MHz DDR3 • Fast Interconnect: QDR InfiniBand for intra deployment traffic, 10gigE for standard Azure traffic and internet access • : 2 TB per node • Available in 8 core/56 GB and 16 core/112 GB instances • RDMA for Linux and Windows Bare Metal Equivalent Performance • ~2.5-3.1 microsecond latency • >3GB/sec non blocking • 90% efficiency on Linpack • Example: linear scaling on NAMD
- 13. Size/Component N1 N2 N10 N11 N12 N21 CPU Cores (E5-2690v3) 6 24 6 12 24 24 RAM 64 GB 256 GB 64 GB 128 GB 256 GB 256 GB SSD ~0.5 TB ~2.0 TB ~0.5 TB ~1.0 TB ~2.0 TB ~2.0 TB Network Azure Network Azure Network Azure Network Azure Network Azure Network Azure Network + Dedicated RDMA Backend GPU Resources 1 x M60 GPU (1/2 Physical Card) 4 x M60 GPU (2 Physical Cards) 1 x K80 GPU (1/2 Physical Card) 2 x K80 GPUs (1 Physical Card) 4 x K80 GPUs (2 Physical Cards) 4 x K80 GPUs (2 Physical Cards)
- 14. • HPC Pack Burst в Azure
- 17. Облако даёт большие возможности по масштабированию Снимает множество задач, но в некоторых сценариях требует переосмысления 1 2
- 18. Технологический евангелист, Cloud/Open Source [email protected] @ahriman_ru
- 19. Причина -05k 05k 15k 25k 35k 0 50 100 150 2011/02/01 2011/02/02 2011/02/03 2011/02/04 2011/02/05 2011/02/06 2011/02/07 EnergyPointsEarned TimeSpent(Minutes) Student Detailed View Exercise Minutes Video Minutes Energy Points Student Lesson 1 Lesson 2 Lesson 3 Lesson 4 Lesson 5 Lesson 6 Cindy Ravi David Zach Bill Dylan Student Status Azure ML Классификация Данные о студентах Предсказания 0 2 4 6 Category 1 Category 2 Category 3 Category 4 Customer All Customers Определе ние лучших учителей и т.д. Оценка знаний Предсказание Отслеживание
- 20. Студенты под рискомРезюме по студенту Отслеживание результатов Предсказание вероятности Агрегирование и действия Student Hours in class Comments online Logons Dylan 0 15 3 David 6 Bill 10 70 5 Cindy 12 4 6 Zach 6 3 8 Online Classroom Social media Databases User interface 0 20 40 60 80 100 1 1.5 2 2.5 3 3.5 4 Performance At-Risk Score (5) Risk Level Intervention Recommended 4.2 High Risk Yes 0% 2% 4% 6% 8% 10% Class School District Low Risk Medium Risk High Risk Average
- 21. Запуск Bing maps Как лучше добраться? Создание Microsoft Research Запуск Kinect Распознавание «смысла» движения Azure Machine Learning Что будет дальше? Запуск Hotmail Спам или нет? Запуск Bing search Какой результат самый релевантный? Запуск Skype Translator Что говорит пользователь? Microsoft & Machine Learning 1991 201420091997 201520102008 Machine learning проходит через все продукты Microsoft.
- 22. APIML STUDIO
- 23. Скоринг Action True Positive True Negative False Positive False Negative
- 24. Предсказание будущих негативных событий Предсказание энергопотребления в Нью-Йорке Microsoft Azure Machine Learning в действии:
- 25. Social network analysis Weather forecasting Healthcare outcomes Predictive maintenance Targeted advertising Natural resource exploration Fraud detection Telemetry data analysis Buyer propensity models Churn analysis Life sciences research Web app optimization Network intrusion detection Smart meter monitoring
- 26. «Определение возраста и эмоций» -как-сервис Microsoft Azure Machine Learning в действии:
- 27. Планка для использования решений ИИ сейчас на удивление низкая 1 2 Машинное обучение прекрасно работает в облаке, которое позволяет значительно повысить сложность решаемых проблем 3
- 29. Эксперт по стратегическим технологиям, Cloud/Open Source/HPC [email protected] @ahriman_ru
- 30. © 2016 Microsoft Corporation. All rights reserved. Microsoft, Windows, Windows Vista and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries. The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.
Editor's Notes
- #20: Let’s look at how Microsoft can help you enable student achievement. You start by assessing what each student currently knows and is able to achieve, by measuring the student’s understanding of lessons through homework and test scores, as well as the student’s ability to express herself or himself during class participation. Then, you feed that data into a prediction model using Azure Machine Learning to identify which students possess adequate knowledge of teaching materials and ability to explain those concepts. The model also identifies struggling students who need more help, enabling instructors to tailor education in ways that best help each student. Then, you give instructors the ability to track each student’s progress in every lesson over time through detailed and easily understandable, user-friendly reports. Instructors can monitor student knowledge levels and activity completion. They also gain valuable feedback on lesson effectiveness, which arms them with the knowledge needed to adjust lessons to maximize their effectiveness and help students achieve their potential.
- #21: Next, let’s look at how you might identify at-risk students. You begin by collecting information about each student’s current performance and interactions in the classroom and online. Here, we’re tracking how often they attend class in person, how they perform based on grades, and how much they participate in online discussions for the class. Any student can be individually selected for further analysis, such as Dylan, who has significant problems with attendance. Next, you put Dylan’s data into a dropout threat predictor. This predicts which students are at high risk of dropping out of school based on either sudden changes in performance or consistent signs of struggle. A machine learning algorithm is used to classify students by risk level: Low, Medium, and High. As you can see, Dylan is flagged as at high risk, and intervention is recommended. Now, Dylan’s instructors and other institution personnel can take proactive actions to help Dylan succeed and graduate. You can also aggregate at-risk student statistics throughout the education system, by summing up and generalizing the number of at-risk students at the classroom, school, and district levels. The helps administrators quantify the extent of the problem at a higher level, and, when viewed over time, measure the effectiveness of policy changes in the school system.
- #29: https://notebooks.azure.com/library/LIGOOpenScienceCenter