SlideShare a Scribd company logo
November 10, 2016
Adrian Bowles, PhD
Founder, STORM Insights, Inc.
info@storminsights.com
Emerging Hardware Choices for #ModernAI
Copyright (c) 2016 by STORM Insights Inc. All Rights reserved.
Hardware - The Final Frontier for Workload Optimization
Performance Challenges for #ModernAI
Optimizing Workloads Through Parallel Execution
Three Architectural Paths
Neuromorphic
GPU/Advanced Memory
Quantum
Market Overview & Recommendations
Agenda
Copyright (c) 2016 by STORM Insights Inc. All Rights reserved.
Value Migrates to Hardware
Optimize
Commoditize
Standardize
Conventional
AI
Machine
Learning
Big
Data
#ModernAI Scope
Copyright (c) 2016 by STORM Insights Inc. All Rights reserved.
Emerging AI Hardware Trends and Options
A Role for Hardware Optimization
Cognitive
Machine Learning
Reasoning
Understanding
Planning
Human Input
Language
Vision
Aural
Human-Oriented Output
Machine Input
IOT
Machine-Oriented Output
Emerging AI Hardware Trends and Options
Human
Machine
Input Output
Narrative Generation
Copyright (c) 2016 by STORM Insights Inc. All Rights reserved.
Data Mgmt
Learn Model
Reason
Understand
Plan
Taste
Smell
Touch
Hear
See
Gestures
Emotions
Language
Visualization
Reports
Haptics
IoT IoT
Cognitive Systems: Communication & Control
Sensors
Systems
Controls
Copyright (c) 2016 by STORM Insights Inc. All Rights reserved.
Hearing (audioception)
~12,000 outer hair cells/ear
~3,500 inner hair cells Vision (ophthalmoception)
Photoreceptors - Per Eye
~120,000,000 rod cells
(triggered by single photon)
~6,000,000 cone cells
(require more photons to trigger)
~ 60,000 photosensitive ganglion cells
Touch (tactioception)
Thermoreceptors, mechanoreceptors,
chemoreceptors and nociceptors for touch, pressure, pain,
temperature, vibration
Smell (olfacoception)
Chemoreception
Taste (gustaoception)
Chemoreception
Neurosynaptic Problem Solving Scope
Copyright (c) 2016 by STORM Insights Inc. All Rights reserved.
Hearing (audioception)
~12,000 outer hair cells/ear
~3,500 inner hair cells Vision (ophthalmoception)
Photoreceptors - Per Eye
~120,000,000 rod cells
(triggered by single photon)
~6,000,000 cone cells
(require more photons to trigger)
~ 60,000 photosensitive ganglion cells
Touch (tactioception)
Thermoreceptors, mechanoreceptors,
chemoreceptors and nociceptors for touch, pressure, pain,
temperature, vibration
Smell (olfacoception)
Chemoreception
Taste (gustaoception)
Chemoreception
Human Cognition
~100,000,000,000 (100B) Neurons
~100-500,000,000,000,000 (100-500T) Synapses
Neurosynaptic Problem Solving Scope
Learn
ModelReason
Understand
Plan
Copyright (c) 2015 by STORM Insights Inc. All Rights reserved.
deep
learning
Deep learning refers to a biologically-inspired approach to machine
learning that leverages a collection of simple processing units - analogous
to neurosynaptic elements - that collaborate to solve complex problems at
multiple levels of abstraction.
These modern neural networks can support supervised, reinforcement, or
unsupervised learning systems.
In general, deep learning solutions require a high degree of parallelism,
which may be implemented in hardware and/or software.
Deep Learning is Inherently Parallel
Copyright (c) 2016 by STORM Insights Inc. All Rights reserved.
Memory
(Instructions & Data)
Central Processing Unit
(CPU)
Control Unit
Arithmetic/Logic Unit
(ALU)
Input
Device(s)
Output
Device(s)
Operating System
The von Neumann Architecture
Copyright (c) 2016 by STORM Insights Inc. All Rights reserved.
Memory
(Instructions & Data)
Central Processing Unit
(CPU)
Control Unit
Arithmetic/Logic Unit
(ALU)
Input
Device(s)
Output
Device(s)
Operating System
“Speed”/Throughput Constraints
Copyright (c) 2016 by STORM Insights Inc. All Rights reserved.
Memory
(Instructions & Data)
Central Processing Unit
(CPU)
Control Unit
Arithmetic/Logic Unit
(ALU)
Input
Device(s)
Output
Device(s)
Operating System
Control Unit
Arithmetic/Logic Unit
(ALU)
Parallelism With Multi-Cores
Copyright (c) 2016 by STORM Insights Inc. All Rights Reserved. 9/28/2011
IBM Power 750
90 servers, 32 cores/server,
2880 Cores in 10 racks
16Tb RAM
~80TeraFLOPS
80,000,000,000,000FLOPS
IBM Watson - Parallelism for Deep QA
Copyright (c) 2016 by STORM Insights Inc. All Rights reserved.Source: https://www.top500.org/system/177999
Amdahl’s Law: The theoretical performance improvement resulting from
a resource improvement for a fixed workload is limited by that part of the
workload that cannot benefit from the resource improvement.
Limits to Parallelism
Copyright (c) 2015 by STORM Insights Inc. All Rights reserved.
Research Examples:
The European Commission FACETS (Fast Analog Computing with Emergent Transient States)
and BrainScaleS (Brain-inspired multi scale computation in neuromorphic hybrid systems)
UK SpiNNaker (Spiking Neural Network Architecture)
DARPA - SyNAPSE (Systems of Neuromorphic Adaptive Plastic Scalable Electronics)
Computer, device/component -level systems modeled after biological
systems or components, such as neurons and synapses. These may be
implemented in analog, digital or hybrid hardware. Typically designed to learn
by experience over time, rather than by programming.
Neuromorphic Architectures (“Brain-Inspired”)
Massively interconnected networks of very simple processors.
Copyright (c) 2016 by STORM Insights Inc. All Rights reserved.
Synapse 16 chip board
Neuromorphic Architectures
IBM - SyNAPSE board
“TrueNorth chips can be seamlessly tiled to create vast, scalable neuromorphic systems.”
Already demonstrated 16 million neurons and 4 billion synapses.
Goal is to integrate 4,096 chips in a single rack with 4 billion neurons
and 1 trillion synapses while consuming ~4kW of power.
Source: Qualcomm
Copyright (c) 2015 by STORM Insights Inc. All Rights reserved.
Neuromorphic Architectures
MAY 2, 2016: Qualcomm Incorporated (NASDAQ: QCOM) today announced at the Embedded Vision Summit in Santa Clara, Calif., that its subsidiary,
Qualcomm Technologies, Inc., is offering the first deep learning software development kit (SDK) for devices powered by Qualcomm® Snapdragon™ 820
processors. The SDK, called the Qualcomm Snapdragon Neural Processing Engine, is powered by the Qualcomm® Zeroth™ Machine Intelligence
Platform
Copyright (c) 2016 by STORM Insights Inc. All Rights reserved.
The Nvidia M40 processor for training neural networks.
Nvidia
NVIDIA Maxwell™ architecture
Up to 7 Teraflops of single-precision performance with NVIDIA GPU Boost™
3072 NVIDIA CUDA® cores
24 GB of GDDR5 memory
288 GB/sec memory bandwidth
Qualified to deliver maximum uptime in the datacenter
GPU/Advanced Memory Architectures
Copyright (c) 2016 by STORM Insights Inc. All Rights reserved.
GPU/Advanced Memory Architectures
Copyright (c) 2016 by STORM Insights Inc. All Rights reserved.
Server racks with TPUs used in the
AlphaGo matches with Lee Sedol
GPU/Advanced Memory Architectures
Copyright (c) 2016 by STORM Insights Inc. All Rights reserved.
At Facebook, we've made great progress thus far with off-the-shelf infrastructure components
and design. We've developed software that can read stories, answer questions about
scenes, play games and even learn unspecified tasks through observing some examples.
But we realized that truly tackling these problems at scale would require us to design our own
systems. Today, we're unveiling our next-generation GPU-based systems for training neural
networks, which we've code-named “Big Sur.”
• FAIR is more than tripling its investment in GPU hardware as we focus even more on
research and enable other teams across the company to use neural networks in our
products and services.
• As part of our ongoing commitment to open source and open standards, we plan to
contribute our innovations in GPU hardware to the Open Compute Project so others
can benefit from them.
Facebook Open-source AI hardware design
https://code.facebook.com/posts/1687861518126048/facebook-to-open-source-ai-hardware-design/
GPU/Advanced Memory Architectures
Copyright (c) 2016 by STORM Insights Inc. All Rights reserved.
Source: https://www.micron.com/about/emerging-technologies/automata-processing
GPU/Advanced Memory Architectures
Copyright (c) 2016 by STORM Insights Inc. All Rights reserved.
GPU/Advanced Memory Architectures
Copyright (c) 2016 by STORM Insights Inc. All Rights reserved.
http://www.research.ibm.com/quantum/
Quantum Architectures
Copyright (c) 2016 by STORM Insights Inc. All Rights reserved.
Source: https://arxiv.org/abs/1608.00263
Quantum Architectures
Copyright (c) 2016 by STORM Insights Inc. All Rights reserved.
Probabalistic Architecture?
Copyright (c) 2016 by STORM Insights Inc. All Rights reserved.
Neuromorphic
GPU/
Memory Acceleration
Quantum
Market/Technology Positions & Maturity
Ready Now
Much More in the Pipeline
Promising -
Ready Now At Handset Level
Promising -
Watch But Don’t Wait
Proven approach for ||ism
Easy interoperability
with conventional systems
+Natural behavioral process model
+Lower power requirements
- Requires new software model
& skills
+Incredible compute power potential
- Requires new software model
& skills
- Requires interface to
conventional system for
pre-processing
- Requires extremely cold
(big, expensive) environment
Copyright (c) 2016 by STORM Insights Inc. All Rights reserved.
IBM
Qualcomm
Brain Corporation
(hosted by Qualcomm)
Knupath
Tenstorrent
Cirrascale
Neurogrid (Stanford)
Tensilica - Cadence
1026 Labs
Cerebras
Artificial Learning
HRL Laboratories
Isocline
Nvidia
Intel
AMD
Facebook (FAIR)
Nervana Systems/Intel
Movidius - Intel (Vision processing)
Google TPU
IBM
D-Wave
Google
Neuromorphic
GPU/
Memory Acceleration
Quantum
Ones to Watch
On the Horizon
Ready Now
Much More in the Pipeline
Promising -
Ready Now At Handset Level
Promising -
Watch But Don’t Wait
Copyright (c) 2016 by STORM Insights Inc. All Rights reserved.
adrian@storminsights.com
Twitter @ajbowles
Skype ajbowles
Upcoming Webinar Dates & Topics
December 8 Leverage the IOT to Build a Smart Data Ecosystem
January #Modern AI and Cognitive Computing: Boundaries and Opportunities
February Artificial General Intelligence: When I Can I Get It?
March Data Science and Business Analysis: A Look at Best Practices for Roles, Skills, and Processes
April Machine Learning: Moving Beyond Discovery to Understanding
May Streaming Analytics for Agile IoT-Oriented Applications
June Machine Learning Case Studies
July Advances in Natural Language Processing I: Understanding
August Organizing Data and Knowledge: The Role of Taxonomies and Ontologies
September Advances in Natural Language Processing II: NL Generation
October Choosing the Right Data Management Architecture for Cognitive Computing
November See Me, Feel Me, Touch Me, Heal Me: The Rise of the Cognitive Interface
December The Road to Autonomous Applications
For More Information…
Copyright (c) 2016 by STORM Insights Inc. All Rights reserved.
Basilar membrane. (2016, October 28). In Wikipedia, The Free Encyclopedia. Retrieved 01:58, October 28, 2016, from https://en.wikipedia.org/w/index.php?title=Basilar_membrane&oldid=746543229
Somatosensory system. (2016, October 9). In Wikipedia, The Free Encyclopedia. Retrieved 04:59, October 9, 2016, from https://en.wikipedia.org/w/index.php?title=Somatosensory_system&oldid=743336883
Photoreceptor cell. (2016, September 19). In Wikipedia, The Free Encyclopedia. Retrieved 03:07, September 19, 2016, from https://en.wikipedia.org/w/index.php?title=Photoreceptor_cell&oldid=740108113
Copyright (c) 2016 by STORM Insights Inc. All Rights reserved.
Hardware - The Final Frontier for Workload Optimization
#ModernAI Defined
Performance Challenges
Optimizing Workloads Through Parallel Execution
Three Architecture Paths
Neuromorphic
GPU/Advanced Memory
Quantum
Agenda
A Role for Hardware
Cognitive
Machine Learning
Reasoning
Understanding
Planning
Human Input
Language
Vision
Aural
Human-Oriented Output
Machine Input
IOT
Machine-Oriented Output
Copyright (c) 2016 by STORM Insights Inc. All Rights reserved.
Copyright (c) 2015 by STORM Insights Inc. All Rights reserved.

More Related Content

What's hot (20)

PDF
FPGAs and Machine Learning
inside-BigData.com
 
PDF
Vertex Perspectives | AI Optimized Chipsets | Part II
Vertex Holdings
 
PPTX
Introduction to PowerAI - The Enterprise AI Platform
Indrajit Poddar
 
PDF
Vertex Perspectives | AI Optimized Chipsets | Part IV
Vertex Holdings
 
PPT
Presentation
butest
 
PDF
A Survey of Machine Learning Methods Applied to Computer ...
butest
 
PDF
NIPS - Deep learning @ Edge using Intel's NCS
geetachauhan
 
PDF
Distributed deep learning optimizations for Finance
geetachauhan
 
PDF
Vertex perspectives artificial intelligence
Yanai Oron
 
PDF
Best Practices for On-Demand HPC in Enterprises
geetachauhan
 
PDF
On-Device AI
LGCNSairesearch
 
PDF
Transparent Hardware Acceleration for Deep Learning
Indrajit Poddar
 
PDF
Intel 2020 Labs Day Keynote Slides
DESMOND YUEN
 
PPTX
PowerAI Deep Dive ( key points )
Paulo Sergio Lemes Queiroz
 
PDF
OpenPOWER/POWER9 AI webinar
Ganesan Narayanasamy
 
PDF
AI and Deep Learning
Subrat Panda, PhD
 
PDF
Deep learning with FPGA
Ayush Singh, MS
 
PDF
08 Supercomputer Fugaku
RCCSRENKEI
 
DOC
Adaptive Computing Seminar Report - Suyog Potdar
Suyog Potdar
 
PPTX
PowerAI Deep dive
Ganesan Narayanasamy
 
FPGAs and Machine Learning
inside-BigData.com
 
Vertex Perspectives | AI Optimized Chipsets | Part II
Vertex Holdings
 
Introduction to PowerAI - The Enterprise AI Platform
Indrajit Poddar
 
Vertex Perspectives | AI Optimized Chipsets | Part IV
Vertex Holdings
 
Presentation
butest
 
A Survey of Machine Learning Methods Applied to Computer ...
butest
 
NIPS - Deep learning @ Edge using Intel's NCS
geetachauhan
 
Distributed deep learning optimizations for Finance
geetachauhan
 
Vertex perspectives artificial intelligence
Yanai Oron
 
Best Practices for On-Demand HPC in Enterprises
geetachauhan
 
On-Device AI
LGCNSairesearch
 
Transparent Hardware Acceleration for Deep Learning
Indrajit Poddar
 
Intel 2020 Labs Day Keynote Slides
DESMOND YUEN
 
PowerAI Deep Dive ( key points )
Paulo Sergio Lemes Queiroz
 
OpenPOWER/POWER9 AI webinar
Ganesan Narayanasamy
 
AI and Deep Learning
Subrat Panda, PhD
 
Deep learning with FPGA
Ayush Singh, MS
 
08 Supercomputer Fugaku
RCCSRENKEI
 
Adaptive Computing Seminar Report - Suyog Potdar
Suyog Potdar
 
PowerAI Deep dive
Ganesan Narayanasamy
 

Viewers also liked (7)

PDF
System On Chip
Dr. A. B. Shinde
 
PPTX
Kim Solez Singularity explained and promoted fall 2016
Kim Solez ,
 
PPT
Flacso Mn Kn Singularity Pp 18 June 07
John Moravec
 
PPTX
Uses of Artificial Intelligence in Bioinformatics
Pragya Pai
 
PDF
Smart Data Slides: Leverage the IOT to Build a Smart Data Ecosystem
DATAVERSITY
 
PDF
Pi ai landscape
Manish Singhal
 
PPTX
BootstrapLabs - Tracxn Report - artificial intelligence for the Applied Arti...
BootstrapLabs
 
System On Chip
Dr. A. B. Shinde
 
Kim Solez Singularity explained and promoted fall 2016
Kim Solez ,
 
Flacso Mn Kn Singularity Pp 18 June 07
John Moravec
 
Uses of Artificial Intelligence in Bioinformatics
Pragya Pai
 
Smart Data Slides: Leverage the IOT to Build a Smart Data Ecosystem
DATAVERSITY
 
Pi ai landscape
Manish Singhal
 
BootstrapLabs - Tracxn Report - artificial intelligence for the Applied Arti...
BootstrapLabs
 
Ad

Similar to Smart Data Slides: Emerging Hardware Choices for Modern AI Data Management (20)

PDF
China AI Summit talk 2017
Dileep Bhandarkar
 
PDF
Deep Learning: Convergence of HPC and Hyperscale
inside-BigData.com
 
PPTX
AI Hardware Landscape 2021
Grigory Sapunov
 
PDF
AI Chip Trends and Forecast
CastLabKAIST
 
PDF
Enabling Artificial Intelligence - Alison B. Lowndes
WithTheBest
 
PDF
Infrastructure and Tooling - Full Stack Deep Learning
Sergey Karayev
 
PDF
GTC Taiwan 2017 企業端深度學習與人工智慧應用
NVIDIA Taiwan
 
PPTX
Chapter 4 - Pioneering Specialized Hardware.pptx
TngNguynSn19
 
PDF
GTC 2017: Powering the AI Revolution
NVIDIA
 
PDF
TECHNICAL OVERVIEW NVIDIA DEEP LEARNING PLATFORM Giant Leaps in Performance ...
Willy Marroquin (WillyDevNET)
 
PDF
May 2025 - Top 10 Read Articles in Artificial Intelligence and Applications (...
gerogepatton
 
PDF
HPC DAY 2017 | NVIDIA Volta Architecture. Performance. Efficiency. Availability
HPC DAY
 
PDF
Vertex Perspectives | AI-optimized Chipsets | Part I
Vertex Holdings
 
PPTX
19-7960-01.pptx
Sourabh97054
 
PPTX
19-7960-01.pptx
survivesurviving
 
PDF
The Convergence of HPC and Deep Learning
inside-BigData.com
 
PDF
infoShare AI Roadshow 2018 - Tomasz Kopacz (Microsoft) - jakie możliwości daj...
Infoshare
 
PDF
NVIDIA DGX User Group 1st Meet Up_30 Apr 2021.pdf
MuhammadAbdullah311866
 
PDF
Ai Forum at Computex 2017 - Keynote Slides by Jensen Huang
NVIDIA Taiwan
 
PDF
Deep learning: Hardware Landscape
Grigory Sapunov
 
China AI Summit talk 2017
Dileep Bhandarkar
 
Deep Learning: Convergence of HPC and Hyperscale
inside-BigData.com
 
AI Hardware Landscape 2021
Grigory Sapunov
 
AI Chip Trends and Forecast
CastLabKAIST
 
Enabling Artificial Intelligence - Alison B. Lowndes
WithTheBest
 
Infrastructure and Tooling - Full Stack Deep Learning
Sergey Karayev
 
GTC Taiwan 2017 企業端深度學習與人工智慧應用
NVIDIA Taiwan
 
Chapter 4 - Pioneering Specialized Hardware.pptx
TngNguynSn19
 
GTC 2017: Powering the AI Revolution
NVIDIA
 
TECHNICAL OVERVIEW NVIDIA DEEP LEARNING PLATFORM Giant Leaps in Performance ...
Willy Marroquin (WillyDevNET)
 
May 2025 - Top 10 Read Articles in Artificial Intelligence and Applications (...
gerogepatton
 
HPC DAY 2017 | NVIDIA Volta Architecture. Performance. Efficiency. Availability
HPC DAY
 
Vertex Perspectives | AI-optimized Chipsets | Part I
Vertex Holdings
 
19-7960-01.pptx
Sourabh97054
 
19-7960-01.pptx
survivesurviving
 
The Convergence of HPC and Deep Learning
inside-BigData.com
 
infoShare AI Roadshow 2018 - Tomasz Kopacz (Microsoft) - jakie możliwości daj...
Infoshare
 
NVIDIA DGX User Group 1st Meet Up_30 Apr 2021.pdf
MuhammadAbdullah311866
 
Ai Forum at Computex 2017 - Keynote Slides by Jensen Huang
NVIDIA Taiwan
 
Deep learning: Hardware Landscape
Grigory Sapunov
 
Ad

More from DATAVERSITY (20)

PDF
Architecture, Products, and Total Cost of Ownership of the Leading Machine Le...
DATAVERSITY
 
PDF
Data at the Speed of Business with Data Mastering and Governance
DATAVERSITY
 
PDF
Exploring Levels of Data Literacy
DATAVERSITY
 
PDF
Building a Data Strategy – Practical Steps for Aligning with Business Goals
DATAVERSITY
 
PDF
Make Data Work for You
DATAVERSITY
 
PDF
Data Catalogs Are the Answer – What is the Question?
DATAVERSITY
 
PDF
Data Catalogs Are the Answer – What Is the Question?
DATAVERSITY
 
PDF
Data Modeling Fundamentals
DATAVERSITY
 
PDF
Showing ROI for Your Analytic Project
DATAVERSITY
 
PDF
How a Semantic Layer Makes Data Mesh Work at Scale
DATAVERSITY
 
PDF
Is Enterprise Data Literacy Possible?
DATAVERSITY
 
PDF
The Data Trifecta – Privacy, Security & Governance Race from Reactivity to Re...
DATAVERSITY
 
PDF
Emerging Trends in Data Architecture – What’s the Next Big Thing?
DATAVERSITY
 
PDF
Data Governance Trends - A Look Backwards and Forwards
DATAVERSITY
 
PDF
Data Governance Trends and Best Practices To Implement Today
DATAVERSITY
 
PDF
2023 Trends in Enterprise Analytics
DATAVERSITY
 
PDF
Data Strategy Best Practices
DATAVERSITY
 
PDF
Who Should Own Data Governance – IT or Business?
DATAVERSITY
 
PDF
Data Management Best Practices
DATAVERSITY
 
PDF
MLOps – Applying DevOps to Competitive Advantage
DATAVERSITY
 
Architecture, Products, and Total Cost of Ownership of the Leading Machine Le...
DATAVERSITY
 
Data at the Speed of Business with Data Mastering and Governance
DATAVERSITY
 
Exploring Levels of Data Literacy
DATAVERSITY
 
Building a Data Strategy – Practical Steps for Aligning with Business Goals
DATAVERSITY
 
Make Data Work for You
DATAVERSITY
 
Data Catalogs Are the Answer – What is the Question?
DATAVERSITY
 
Data Catalogs Are the Answer – What Is the Question?
DATAVERSITY
 
Data Modeling Fundamentals
DATAVERSITY
 
Showing ROI for Your Analytic Project
DATAVERSITY
 
How a Semantic Layer Makes Data Mesh Work at Scale
DATAVERSITY
 
Is Enterprise Data Literacy Possible?
DATAVERSITY
 
The Data Trifecta – Privacy, Security & Governance Race from Reactivity to Re...
DATAVERSITY
 
Emerging Trends in Data Architecture – What’s the Next Big Thing?
DATAVERSITY
 
Data Governance Trends - A Look Backwards and Forwards
DATAVERSITY
 
Data Governance Trends and Best Practices To Implement Today
DATAVERSITY
 
2023 Trends in Enterprise Analytics
DATAVERSITY
 
Data Strategy Best Practices
DATAVERSITY
 
Who Should Own Data Governance – IT or Business?
DATAVERSITY
 
Data Management Best Practices
DATAVERSITY
 
MLOps – Applying DevOps to Competitive Advantage
DATAVERSITY
 

Recently uploaded (20)

PDF
How do you fast track Agentic automation use cases discovery?
DianaGray10
 
PDF
The 2025 InfraRed Report - Redpoint Ventures
Razin Mustafiz
 
PDF
Transcript: Book industry state of the nation 2025 - Tech Forum 2025
BookNet Canada
 
PDF
Kit-Works Team Study_20250627_한달만에만든사내서비스키링(양다윗).pdf
Wonjun Hwang
 
PDF
Newgen 2022-Forrester Newgen TEI_13 05 2022-The-Total-Economic-Impact-Newgen-...
darshakparmar
 
PDF
Automating Feature Enrichment and Station Creation in Natural Gas Utility Net...
Safe Software
 
PPTX
MuleSoft MCP Support (Model Context Protocol) and Use Case Demo
shyamraj55
 
PDF
UPDF - AI PDF Editor & Converter Key Features
DealFuel
 
PDF
Future-Proof or Fall Behind? 10 Tech Trends You Can’t Afford to Ignore in 2025
DIGITALCONFEX
 
PDF
“NPU IP Hardware Shaped Through Software and Use-case Analysis,” a Presentati...
Edge AI and Vision Alliance
 
PDF
NASA A Researcher’s Guide to International Space Station : Physical Sciences ...
Dr. PANKAJ DHUSSA
 
PDF
Mastering Financial Management in Direct Selling
Epixel MLM Software
 
PDF
Newgen Beyond Frankenstein_Build vs Buy_Digital_version.pdf
darshakparmar
 
PPTX
Seamless Tech Experiences Showcasing Cross-Platform App Design.pptx
presentifyai
 
PDF
Peak of Data & AI Encore AI-Enhanced Workflows for the Real World
Safe Software
 
PPTX
Future Tech Innovations 2025 – A TechLists Insight
TechLists
 
PDF
Transforming Utility Networks: Large-scale Data Migrations with FME
Safe Software
 
PDF
UiPath DevConnect 2025: Agentic Automation Community User Group Meeting
DianaGray10
 
PDF
Agentic AI lifecycle for Enterprise Hyper-Automation
Debmalya Biswas
 
PDF
What’s my job again? Slides from Mark Simos talk at 2025 Tampa BSides
Mark Simos
 
How do you fast track Agentic automation use cases discovery?
DianaGray10
 
The 2025 InfraRed Report - Redpoint Ventures
Razin Mustafiz
 
Transcript: Book industry state of the nation 2025 - Tech Forum 2025
BookNet Canada
 
Kit-Works Team Study_20250627_한달만에만든사내서비스키링(양다윗).pdf
Wonjun Hwang
 
Newgen 2022-Forrester Newgen TEI_13 05 2022-The-Total-Economic-Impact-Newgen-...
darshakparmar
 
Automating Feature Enrichment and Station Creation in Natural Gas Utility Net...
Safe Software
 
MuleSoft MCP Support (Model Context Protocol) and Use Case Demo
shyamraj55
 
UPDF - AI PDF Editor & Converter Key Features
DealFuel
 
Future-Proof or Fall Behind? 10 Tech Trends You Can’t Afford to Ignore in 2025
DIGITALCONFEX
 
“NPU IP Hardware Shaped Through Software and Use-case Analysis,” a Presentati...
Edge AI and Vision Alliance
 
NASA A Researcher’s Guide to International Space Station : Physical Sciences ...
Dr. PANKAJ DHUSSA
 
Mastering Financial Management in Direct Selling
Epixel MLM Software
 
Newgen Beyond Frankenstein_Build vs Buy_Digital_version.pdf
darshakparmar
 
Seamless Tech Experiences Showcasing Cross-Platform App Design.pptx
presentifyai
 
Peak of Data & AI Encore AI-Enhanced Workflows for the Real World
Safe Software
 
Future Tech Innovations 2025 – A TechLists Insight
TechLists
 
Transforming Utility Networks: Large-scale Data Migrations with FME
Safe Software
 
UiPath DevConnect 2025: Agentic Automation Community User Group Meeting
DianaGray10
 
Agentic AI lifecycle for Enterprise Hyper-Automation
Debmalya Biswas
 
What’s my job again? Slides from Mark Simos talk at 2025 Tampa BSides
Mark Simos
 

Smart Data Slides: Emerging Hardware Choices for Modern AI Data Management

  • 1. November 10, 2016 Adrian Bowles, PhD Founder, STORM Insights, Inc. [email protected] Emerging Hardware Choices for #ModernAI
  • 2. Copyright (c) 2016 by STORM Insights Inc. All Rights reserved. Hardware - The Final Frontier for Workload Optimization Performance Challenges for #ModernAI Optimizing Workloads Through Parallel Execution Three Architectural Paths Neuromorphic GPU/Advanced Memory Quantum Market Overview & Recommendations Agenda
  • 3. Copyright (c) 2016 by STORM Insights Inc. All Rights reserved. Value Migrates to Hardware Optimize Commoditize Standardize
  • 5. Copyright (c) 2016 by STORM Insights Inc. All Rights reserved. Emerging AI Hardware Trends and Options A Role for Hardware Optimization Cognitive Machine Learning Reasoning Understanding Planning Human Input Language Vision Aural Human-Oriented Output Machine Input IOT Machine-Oriented Output Emerging AI Hardware Trends and Options
  • 6. Human Machine Input Output Narrative Generation Copyright (c) 2016 by STORM Insights Inc. All Rights reserved. Data Mgmt Learn Model Reason Understand Plan Taste Smell Touch Hear See Gestures Emotions Language Visualization Reports Haptics IoT IoT Cognitive Systems: Communication & Control Sensors Systems Controls
  • 7. Copyright (c) 2016 by STORM Insights Inc. All Rights reserved. Hearing (audioception) ~12,000 outer hair cells/ear ~3,500 inner hair cells Vision (ophthalmoception) Photoreceptors - Per Eye ~120,000,000 rod cells (triggered by single photon) ~6,000,000 cone cells (require more photons to trigger) ~ 60,000 photosensitive ganglion cells Touch (tactioception) Thermoreceptors, mechanoreceptors, chemoreceptors and nociceptors for touch, pressure, pain, temperature, vibration Smell (olfacoception) Chemoreception Taste (gustaoception) Chemoreception Neurosynaptic Problem Solving Scope
  • 8. Copyright (c) 2016 by STORM Insights Inc. All Rights reserved. Hearing (audioception) ~12,000 outer hair cells/ear ~3,500 inner hair cells Vision (ophthalmoception) Photoreceptors - Per Eye ~120,000,000 rod cells (triggered by single photon) ~6,000,000 cone cells (require more photons to trigger) ~ 60,000 photosensitive ganglion cells Touch (tactioception) Thermoreceptors, mechanoreceptors, chemoreceptors and nociceptors for touch, pressure, pain, temperature, vibration Smell (olfacoception) Chemoreception Taste (gustaoception) Chemoreception Human Cognition ~100,000,000,000 (100B) Neurons ~100-500,000,000,000,000 (100-500T) Synapses Neurosynaptic Problem Solving Scope Learn ModelReason Understand Plan
  • 9. Copyright (c) 2015 by STORM Insights Inc. All Rights reserved. deep learning Deep learning refers to a biologically-inspired approach to machine learning that leverages a collection of simple processing units - analogous to neurosynaptic elements - that collaborate to solve complex problems at multiple levels of abstraction. These modern neural networks can support supervised, reinforcement, or unsupervised learning systems. In general, deep learning solutions require a high degree of parallelism, which may be implemented in hardware and/or software. Deep Learning is Inherently Parallel
  • 10. Copyright (c) 2016 by STORM Insights Inc. All Rights reserved. Memory (Instructions & Data) Central Processing Unit (CPU) Control Unit Arithmetic/Logic Unit (ALU) Input Device(s) Output Device(s) Operating System The von Neumann Architecture
  • 11. Copyright (c) 2016 by STORM Insights Inc. All Rights reserved. Memory (Instructions & Data) Central Processing Unit (CPU) Control Unit Arithmetic/Logic Unit (ALU) Input Device(s) Output Device(s) Operating System “Speed”/Throughput Constraints
  • 12. Copyright (c) 2016 by STORM Insights Inc. All Rights reserved. Memory (Instructions & Data) Central Processing Unit (CPU) Control Unit Arithmetic/Logic Unit (ALU) Input Device(s) Output Device(s) Operating System Control Unit Arithmetic/Logic Unit (ALU) Parallelism With Multi-Cores
  • 13. Copyright (c) 2016 by STORM Insights Inc. All Rights Reserved. 9/28/2011 IBM Power 750 90 servers, 32 cores/server, 2880 Cores in 10 racks 16Tb RAM ~80TeraFLOPS 80,000,000,000,000FLOPS IBM Watson - Parallelism for Deep QA
  • 14. Copyright (c) 2016 by STORM Insights Inc. All Rights reserved.Source: https://www.top500.org/system/177999 Amdahl’s Law: The theoretical performance improvement resulting from a resource improvement for a fixed workload is limited by that part of the workload that cannot benefit from the resource improvement. Limits to Parallelism
  • 15. Copyright (c) 2015 by STORM Insights Inc. All Rights reserved. Research Examples: The European Commission FACETS (Fast Analog Computing with Emergent Transient States) and BrainScaleS (Brain-inspired multi scale computation in neuromorphic hybrid systems) UK SpiNNaker (Spiking Neural Network Architecture) DARPA - SyNAPSE (Systems of Neuromorphic Adaptive Plastic Scalable Electronics) Computer, device/component -level systems modeled after biological systems or components, such as neurons and synapses. These may be implemented in analog, digital or hybrid hardware. Typically designed to learn by experience over time, rather than by programming. Neuromorphic Architectures (“Brain-Inspired”) Massively interconnected networks of very simple processors.
  • 16. Copyright (c) 2016 by STORM Insights Inc. All Rights reserved. Synapse 16 chip board Neuromorphic Architectures IBM - SyNAPSE board “TrueNorth chips can be seamlessly tiled to create vast, scalable neuromorphic systems.” Already demonstrated 16 million neurons and 4 billion synapses. Goal is to integrate 4,096 chips in a single rack with 4 billion neurons and 1 trillion synapses while consuming ~4kW of power.
  • 17. Source: Qualcomm Copyright (c) 2015 by STORM Insights Inc. All Rights reserved. Neuromorphic Architectures MAY 2, 2016: Qualcomm Incorporated (NASDAQ: QCOM) today announced at the Embedded Vision Summit in Santa Clara, Calif., that its subsidiary, Qualcomm Technologies, Inc., is offering the first deep learning software development kit (SDK) for devices powered by Qualcomm® Snapdragon™ 820 processors. The SDK, called the Qualcomm Snapdragon Neural Processing Engine, is powered by the Qualcomm® Zeroth™ Machine Intelligence Platform
  • 18. Copyright (c) 2016 by STORM Insights Inc. All Rights reserved. The Nvidia M40 processor for training neural networks. Nvidia NVIDIA Maxwell™ architecture Up to 7 Teraflops of single-precision performance with NVIDIA GPU Boost™ 3072 NVIDIA CUDA® cores 24 GB of GDDR5 memory 288 GB/sec memory bandwidth Qualified to deliver maximum uptime in the datacenter GPU/Advanced Memory Architectures
  • 19. Copyright (c) 2016 by STORM Insights Inc. All Rights reserved. GPU/Advanced Memory Architectures
  • 20. Copyright (c) 2016 by STORM Insights Inc. All Rights reserved. Server racks with TPUs used in the AlphaGo matches with Lee Sedol GPU/Advanced Memory Architectures
  • 21. Copyright (c) 2016 by STORM Insights Inc. All Rights reserved. At Facebook, we've made great progress thus far with off-the-shelf infrastructure components and design. We've developed software that can read stories, answer questions about scenes, play games and even learn unspecified tasks through observing some examples. But we realized that truly tackling these problems at scale would require us to design our own systems. Today, we're unveiling our next-generation GPU-based systems for training neural networks, which we've code-named “Big Sur.” • FAIR is more than tripling its investment in GPU hardware as we focus even more on research and enable other teams across the company to use neural networks in our products and services. • As part of our ongoing commitment to open source and open standards, we plan to contribute our innovations in GPU hardware to the Open Compute Project so others can benefit from them. Facebook Open-source AI hardware design https://code.facebook.com/posts/1687861518126048/facebook-to-open-source-ai-hardware-design/ GPU/Advanced Memory Architectures
  • 22. Copyright (c) 2016 by STORM Insights Inc. All Rights reserved. Source: https://www.micron.com/about/emerging-technologies/automata-processing GPU/Advanced Memory Architectures
  • 23. Copyright (c) 2016 by STORM Insights Inc. All Rights reserved. GPU/Advanced Memory Architectures
  • 24. Copyright (c) 2016 by STORM Insights Inc. All Rights reserved. http://www.research.ibm.com/quantum/ Quantum Architectures
  • 25. Copyright (c) 2016 by STORM Insights Inc. All Rights reserved. Source: https://arxiv.org/abs/1608.00263 Quantum Architectures
  • 26. Copyright (c) 2016 by STORM Insights Inc. All Rights reserved. Probabalistic Architecture?
  • 27. Copyright (c) 2016 by STORM Insights Inc. All Rights reserved. Neuromorphic GPU/ Memory Acceleration Quantum Market/Technology Positions & Maturity Ready Now Much More in the Pipeline Promising - Ready Now At Handset Level Promising - Watch But Don’t Wait Proven approach for ||ism Easy interoperability with conventional systems +Natural behavioral process model +Lower power requirements - Requires new software model & skills +Incredible compute power potential - Requires new software model & skills - Requires interface to conventional system for pre-processing - Requires extremely cold (big, expensive) environment
  • 28. Copyright (c) 2016 by STORM Insights Inc. All Rights reserved. IBM Qualcomm Brain Corporation (hosted by Qualcomm) Knupath Tenstorrent Cirrascale Neurogrid (Stanford) Tensilica - Cadence 1026 Labs Cerebras Artificial Learning HRL Laboratories Isocline Nvidia Intel AMD Facebook (FAIR) Nervana Systems/Intel Movidius - Intel (Vision processing) Google TPU IBM D-Wave Google Neuromorphic GPU/ Memory Acceleration Quantum Ones to Watch On the Horizon Ready Now Much More in the Pipeline Promising - Ready Now At Handset Level Promising - Watch But Don’t Wait
  • 29. Copyright (c) 2016 by STORM Insights Inc. All Rights reserved. [email protected] Twitter @ajbowles Skype ajbowles Upcoming Webinar Dates & Topics December 8 Leverage the IOT to Build a Smart Data Ecosystem January #Modern AI and Cognitive Computing: Boundaries and Opportunities February Artificial General Intelligence: When I Can I Get It? March Data Science and Business Analysis: A Look at Best Practices for Roles, Skills, and Processes April Machine Learning: Moving Beyond Discovery to Understanding May Streaming Analytics for Agile IoT-Oriented Applications June Machine Learning Case Studies July Advances in Natural Language Processing I: Understanding August Organizing Data and Knowledge: The Role of Taxonomies and Ontologies September Advances in Natural Language Processing II: NL Generation October Choosing the Right Data Management Architecture for Cognitive Computing November See Me, Feel Me, Touch Me, Heal Me: The Rise of the Cognitive Interface December The Road to Autonomous Applications For More Information…
  • 30. Copyright (c) 2016 by STORM Insights Inc. All Rights reserved. Basilar membrane. (2016, October 28). In Wikipedia, The Free Encyclopedia. Retrieved 01:58, October 28, 2016, from https://en.wikipedia.org/w/index.php?title=Basilar_membrane&oldid=746543229 Somatosensory system. (2016, October 9). In Wikipedia, The Free Encyclopedia. Retrieved 04:59, October 9, 2016, from https://en.wikipedia.org/w/index.php?title=Somatosensory_system&oldid=743336883 Photoreceptor cell. (2016, September 19). In Wikipedia, The Free Encyclopedia. Retrieved 03:07, September 19, 2016, from https://en.wikipedia.org/w/index.php?title=Photoreceptor_cell&oldid=740108113
  • 31. Copyright (c) 2016 by STORM Insights Inc. All Rights reserved. Hardware - The Final Frontier for Workload Optimization #ModernAI Defined Performance Challenges Optimizing Workloads Through Parallel Execution Three Architecture Paths Neuromorphic GPU/Advanced Memory Quantum Agenda A Role for Hardware Cognitive Machine Learning Reasoning Understanding Planning Human Input Language Vision Aural Human-Oriented Output Machine Input IOT Machine-Oriented Output
  • 32. Copyright (c) 2016 by STORM Insights Inc. All Rights reserved.
  • 33. Copyright (c) 2015 by STORM Insights Inc. All Rights reserved.