AI Overview

Foreword

⚫ Mankind is welcoming the fourth industrial revolution represented by intelligent technology. New
technologies such as AI, IoT, 5G and bioengineering are integrated into all aspects of human society;
driving changes in global macro trends, such as sustainable social development and economic growth.
New kinetic energy, smart city upgrading, industrial digital transformation, consumer experience, etc.
⚫ As the world‘s leading provider of ICT (information and communications) infrastructure and smart
terminals, Huawei actively participates in the transformation of artificial intelligence and proposes
Huawei’s full-stack full-scenario AI strategy. This chapter will mainly introduce AI Overview, Technical
Fields and Application Fields of AI, Huawei's AI Development Strategy, AI Disputes, Future Prospects of
AI.

2 Huawei Confidential
 Objectives

Upon completion of this course, you will be able to:

 Understand basic concepts of AI.
 Understand AI technologies and their development history.
 Understand the application technologies and application fields of AI.
 Know Huawei's AI development strategy.
 Know the development trends of AI.

3 Huawei Confidential
 Contents

1. AI Overview

2. Technical Fields and Application Fields of AI

3. Huawei's AI Development Strategy

4. AI Disputes

5. Future Prospects of AI

4 Huawei Confidential
AI in the Eyes of the Society
⚫ People get to know AI through news, movies, and actual applications in daily life. What is
AI in the eyes of the public?

Haidian Park: First AI-themed Park in the World StarCraft The Terminator Self-service security check
II: AlphaStar Beat Professional Players 2001: A Space Odyssey Spoken language evaluation
AI-created Edmond de Belamy Sold at US$430,000 The Matrix Music/Movie recommendation
Demand for AI Programmers:↑ 35 Times! Salary: Top 1! I, Robot Smart speaker
50% Jobs Will be Replaced by AI in the future Blade Runner …
Winter is Coming? AI Faces Challenges Elle
… Bicentennial Man
…

News Movies Applications in daily life

AI Applications AI Control over human beings Security protection
AI industry outlook Fall in love with AI Entertainment
Challenges faced by AI Self-awareness of AI Smart Home
… … Finance
…

5 Huawei Confidential
AI in the Eyes of Researchers
"I propose to consider the question, 'Can machines think?'"

— Alan Turing 1950

The branch of computer science concerned with making computers behave like humans.

— John McCarthy 1956

The science of making machines do things that would require intelligence if done by men.

— Marvin Minsky

6 Huawei Confidential
What Are Intelligences?
⚫ Howard Gardner's Multiple Intelligences
⚫ Human intelligences can be divided into seven categories:
 Verbal/Linguistic
 Logical/Mathematical
 Visual/Spatial
 Bodily/Kinesthetic
 Musical/Rhythmic
 Inter-personal/Social
 Intra-personal/Introspective

7 Huawei Confidential
What Is AI?
⚫ Artificial Intelligence (AI) is a new technical science that studies and develops theories, methods, techniques,
and application systems for simulating and extending human intelligence. In 1956, the concept of AI was first
proposed by John McCarthy, who defined the subject as "science and engineering of making intelligent
machines, especially intelligent computer program". AI is concerned with making machines work in an
intelligent way, similar to the way that the human mind works. At present, AI has become an interdisciplinary
course that involves various fields.

Brain
science Cognitive
science
Computer
science

AI Psychology
Philosophy

Linguistics
Logic

Identification of concepts related to AI and machine learning

AI Development Report 2020

9 Huawei Confidential
Relationship of AI, Machine Learning, and Deep Learning

10 Huawei Confidential
Relationship of AI, Machine Learning and Deep Learning
⚫ AI: A new technical science that focuses on the research and development of theories, methods,
techniques, and application systems for simulating and extending human intelligence.
⚫ Machine learning: A core research field of AI. It focuses on the study of how computers can obtain
new knowledge or skills by simulating or performing learning behavior of human beings, and
reorganize existing knowledge architecture to improve its performance. It is one of the core
research fields of AI.
⚫ Deep learning: A new field of machine learning. The concept of deep learning originates from the
research on artificial neural networks. The multi-layer perceptron (MLP) is a type a deep learning
architecture. Deep learning aims to simulate the human brain to interpret data such as images,
sounds, and texts.

11 Huawei Confidential
Three Major Schools of Thought: Symbolism
⚫ Basic thoughts
 The cognitive process of human beings is the process of inference and operation of various
symbols.
 A human being is a physical symbol system, and so is a computer. Computers, therefore, can be
used to simulate intelligent behavior of human beings.
 The core of AI lies in knowledge representation, knowledge inference, and knowledge
application. Knowledge and concepts can be represented with symbols. Cognition is the process
of symbol processing while inference refers to the process of solving problems by using heuristic
knowledge and search.

⚫ Representative of symbolism: inference, including symbolic inference and machine

inference
12 Huawei Confidential
Three Major Schools of Thought: Connectionism
⚫ Basic thoughts
 The basis of thinking is neurons rather than the process of symbol processing.
 Human brains vary from computers. A computer working mode based on connectionism is proposed to
replace the computer working mode based on symbolic operation.

• Representative of connectionism: neural networks and

deep learning

13 Huawei Confidential
Three Major Schools of Thought: Behaviorism
⚫ Basic thoughts:
 Intelligence depends on perception and action. The perception-action mode of intelligent
behavior is proposed.
 Intelligence requires no knowledge, representation, or inference. AI can evolve like human
intelligence. Intelligent behavior can only be demonstrated in the real world through the
constant interaction with the surrounding environment.

⚫ Representative of behaviorism: behavior control, adaptation, and evolutionary computing

14 Huawei Confidential
Brief Development History of AI
2016 March: AlphaG o
defeated t he world
2014: Microsoft champion Go player Lee
released the first Sedol by 4-1.
1997: Deep Blue individual intelligent
1985: Decision-
defeated t he world assistant Microsf t
1956: AI was proposed at making tree
1976: Due to failure chess champion Cortana in the world.
the Dartmouth Conference. models with
of projects such as Garry Kasparov.
better
machine t ranslat ion
visualization 2006: Hinton and his
and negat ive impact
ef fect and multi- students st arted deep
of some academic
layer ANNs learning.
reports, the f und for
which broke
AI was decreased 2017 October: The Deep
through t he limit
in general. Mind team released
of early
percept ron. AlphaG o Zero, the
1987: The 2010: The
1959: Arthur Samuel strongest version of
market of LISP era of big
proposed machine AlphaG o.
machines data
learning.
collapsed. came.

1950s 1960s 1970s 1980s 1990s 2000s 2010s 2020s

1956-1976 1997-2010
First period of boom Period of recovery
The concept and development target 1976-1982 1982-1987 Computing perf ormance
2010-
of AI were determined at t he First period of Second period was improved and Internet
1987-1997 Period of rapid growth
Dartmouth conf erence. low ebb of boom technologies got
Second period of low New-generation
AI suff ered from Expert syst em popularized quickly.
ebb information technologies
quest ioning and capable of logic Technical f ields f aced triggered transformation of
crit icism due to rule inference bottlenecks, people information environment
insufficient and answering on longer focused on and dat a basis. Multi-
computing quest ions of abstract inference, model data such as
capabilities, high specific fields and models based on massive images, voices,
computing went popular symbol processing and texts emerged
complexity, and and fifth- were rejected. continuously. Computing
great difficulty of generation capabilities were improved.
inference computers
realization. developed.

15 Huawei Confidential
Overview of AI Technologies
⚫ AI technologies are multi-layered, covering the application, algorithm mechanism,
toolchain, device, chip, process, and material layers.

Application

Algorithm

Device

Chip

Process

16 Huawei Confidential
Types of AI
⚫ Strong AI
 The strong AI view holds that it is possible to create intelligent machines that can really reason
and solve problems. Such machines are considered to be conscious and self-aware, can
independently think about problems and work out optimal solutions to problems, have their
own system of values and world views, and have all the same instincts as living things, such as
survival and security needs. It can be regarded as a new civilization in a certain sense.

⚫ Weak AI
 The weak AI view holds that intelligent machines cannot really reason and solve problems.
These machines only look intelligent, but do not have real intelligence or self-awareness.

18 Huawei Confidential
Classification of Intelligent Robots
⚫ Currently, there is no unified definition of AI research. Intelligent robots are generally
classified into the following four types:
 "Thinking like human beings": weak AI, such as Watson and AlphaGo
 "Acting like human beings": weak AI, such as humanoid robot, iRobot, and Atlas of Boston
Dynamics
 "Thinking rationally": strong AI (Currently, no intelligent robots of this type have been created
due to the bottleneck in brain science.)
 "Acting rationally": strong AI

19 Huawei Confidential
AI Industry Ecosystem
⚫ The four elements of AI are data, algorithm, computing power, and scenario. To meet requirements of these
four elements, we need to combine AI with cloud computing, big data, and IoT to build an intelligent society.

20 Huawei Confidential
Sub-fields of AI

AI Development Report 2020

21 Huawei Confidential
 Contents

1. AI Overview

2. Technical Fields and Application Fields of AI

3. Huawei's AI Development Strategy

4. AI Disputes

5. Future Prospects of AI

22 Huawei Confidential
Technical Fields and Application Fields of AI

Global AI Development White

Paper 2020
23 Huawei Confidential
Distribution of AI Application Technologies in Enterprises Inside
and Outside China
⚫ At present, application directions of AI technologies mainly
include:
 Computer vision: a science of how to make computers "see"
 Speech processing: a general term for various processing
technologies used to research the voicing process, statistical
features of speech signals, speech recognition, machine-based
speech synthesis, and speech perception
 Natural language processing (NLP): a subject that use computer
technologies to understand and use natural language Distribution of AI application technologies in
enterprises inside and outside China

China AI Development Report 2018

24 Huawei Confidential
Voice Processing Application Scenario (1)
⚫ The main topics of voice processing research include voice recognition, voice synthesis, voice wakeup,
voiceprint recognition, and audio-based incident detection. Among them, the most mature technology is voice
recognition. As for near field recognition in a quite indoor environment, the recognition accuracy can reach
96%.
⚫ Application scenarios:
Question Answering Bot (QABot) Voice navigation

25 Huawei Confidential
Voice Processing Application Scenario (2)

Intelligent
education Real-time conference
records

⚫ Other applications:
 Spoken language evaluation
 Diagnostic robot
 Voiceprint recognition
 Smart sound box
 ...

26 Huawei Confidential
NLP Application Scenario (1)
⚫ The main topics of NLP research include machine translation, text mining, and sentiment analysis. NLP imposes high requirements
on technologies but confronts low technology maturity. Due to high complexity of semantics, it is hard to reach the human
understanding level using parallel computing based on big data and parallel computing only.

⚫ In future, NLP will achieve more growth: understanding of shallow semantics → automatic extraction of features and understanding
of deep semantics; single-purpose intelligence (ML) → hybrid intelligence (ML, DL, and RL)

⚫ Application scenarios:

Theme Trend
Public opinion mining analysis Evaluation
analysis analysis

Emotional
Public opinion
analysis
analysis

Hotspot
event Information
distribution

27 Huawei Confidential
NLP Application Scenario (2)

Text
Machine
classification
translation

⚫ Other applications:
 Knowledge graph
 Intelligent copywriting
 Video subtitle
 ...
28 Huawei Confidential
AI Application Field - Intelligent Healthcare

Medicine mining: quick development of personalized medicines by AI assistants

Health management: nutrition, and physical/mental health management

Hospital management: structured services concerning medical records (focus)

Assistance for medical research: assistance for biomedical researchers in research

Virtual assistant: electronic voice medical records, intelligent guidance, intelligent diagnosis, and
medicine recommendation

Medical image: medical image recognition, image marking, and 3D image reconstruction

Assistance for diagnosis and treatment: diagnostic robot

Disease risk forecast: disease risk forecast based on gene sequencing

29 Huawei Confidential
AI Application Field - Smart Home
⚫ Based on IoT technologies, a smart home ecosystem is formed with hardware, software, and cloud
platforms, providing users personalized life services and making home life more convenient,
comfortable, and safe.

Okay, the
temperature's set.
Set the temperature to
26 degrees.

Control smart home products with voice Develop user profiles and recommend
processing such as air conditioning content to users with the help of machine
temperature adjustment, curtain switch learning and deep learning technologies
control, and voice control on the lighting and based on historical records of smart
system. speakers and smart TVs.

30 Huawei Confidential
AI Application Field - Retail
⚫ AI will bring revolutionary changes to the retail industry. A typical symptom is unmanned supermarkets. For example, Amazon Go, unmanned
supermarket of Amazon, uses sensors, cameras, computer vision, and deep learning algorithms to completely cancel the checkout process,
allowing customers to pick up goods and "just walk out".

⚫ One of the biggest challenges for unmanned supermarket is how to charge the right fees to the right customers. So far, Amazon Go is the only
successful business case and even this case involves many controlled factors. For example, only Prime members can enter Amazon Go. Other
enterprises, to follow the example of Amazon, have to build their membership system first.

31 Huawei Confidential
AI Application Field - Autonomous Driving
⚫ The Society of Automotive Engineers (SAE) in the U.S. defines 6 levels of driving automation ranging from 0
(fully manual) to 5 (fully autonomous). L0 indicates that the driving of a vehicle completely depends on the
driver's operation. The system above L3 can implement the driver's hand-off operation in specific cases, L5
depends on the system when vehicles are driving in all scenarios.
⚫ Currently, only some commercial passenger vehicle models, such as Audi A8, Tesla, and Cadillac, support L2
and L3 Advanced driver-assistance systems (ADAS). It is estimated that more L3 vehicle models will emerge
with the further improvement of sensors and vehicle-mounted processors. L4 and L5 autonomous driving is
expected to be first implemented on commercial vehicles in closed campuses. A wider range of passenger
vehicles require advanced autonomous driving, which requires further improvement of technologies, policies,
and infrastructure. It is estimated that L4 and L5 autonomous driving will be supported by common roads in
2025–2030.

32 Huawei Confidential
 Contents

1. AI Overview

2. Technical Fields and Application Fields of AI

3. Huawei's AI Development Strategy

4. AI Disputes

5. Future Prospects of AI

34 Huawei Confidential
Huawei's Full-Stack, All-Scenario AI Portfolio
AI Applications Application enablement: provides end-to-end
Application
services (ModelArts), layered APIs, and pre-
HiAI Engine ModelArts Enablement integrated solutions.
MindSpore: supports the unified training and
TensorFlow PyTorch PaddlePaddle MindSpore Framework
inference framework that is independent of the
device, edge, and cloud.
Chip
Full Stack CANN Enablement CANN: a chip operator library and highly automated
operator development tool.
IP & Chip
Ascend-Nano Ascend-Tiny Ascend-Lite Ascend Ascend-Mini Ascend-Max IP and Chip
Ascend: provides a series of NPU IPs and chips based
on a unified, scalable architecture.

Atlas: enables an all-scenario AI infrastructure solution

Atlas that is oriented to the device, edge, and cloud based
on the Ascend series AI processors and various product
forms.
All Scenarios

Consumer Device Public Cloud Private Cloud Edge Computing Industrial IoT Device

Huawei's "all AI scenarios" indicate different deployment scenarios for AI, including public clouds, private
clouds, edge computing in all forms, industrial IoT devices, and consumer devices.

35 Huawei Confidential
Full Stack - ModelArts Full-Cycle AI Workflow
EI Intelligent Twins

EI Cognition Service AI Service

AI data Algorithm
Training Deployment Market
framework development
Efficient filtering and Out-of-the-box Distributed training, One-click deployment AI sharing platform
semi-automated development shortening training on device, edge, and helps enterprises build
labeling, data environment compatible period from weeks to cloud internal and external
preprocessing with mainstream minutes All-scenario AI ecosystems
frameworks
Data Efficiency improved by deployment
MoXing library, Wizard-based
100 times simplifying model Inference on the
AutoLearning, code-
development Ascend AI processor AI applications
free development,
Built-in model algorithms, enabling model
improving development
training from scratch
efficiency

Visualized Workflow Management

Version management, traceable and worry-free development

ModelArts

AI data framework Visualized workflow Distributed training One-click deployment on Automatic learning AI sharing platform
accelerates data processing management shortens training from device, edge, and cloud enables you to start builds internal and external
by 100 folds. makes development weeks to minutes. supports various deployment from scratch. AI ecosystems for
worry-free. scenarios. enterprises.

36 Huawei Confidential
Full Stack — MindSpore (Huawei AI Computing Framework)

⚫ MindSpore provides automatic parallel capabilities. With MindSpore, senior algorithm engineers and data scientists who
focus on data modeling and problem solving can run algorithms on dozens or even thousands of AI computing nodes with
only a few lines of description.
⚫ The MindSpore framework supports both large-scale and small-scale deployment, adapting to independent deployment in
all scenarios. In addition to the Ascend AI processors, MindSpore also supports other processors such as GPUs and CPUs.

AI application ecosystem for all scenarios

MindSpore
Unified APIs for all scenarios

MindSpore intermediate representation (IR) for computational

graph

On-demand collaborative distributed architecture across device-edge-cloud

(deployment, scheduling, and communications)

Processors: Ascend, GPU, and CPU

38 Huawei Confidential
Full Stack — CANN
CANN:
A chip operators library and highly automated operator
development toolkit
Optimal development efficiency, in-depth optimization of the
AI applications common operator library, and abundant APIs
Operator convergence, best matching the performance of the
Ascend chip
HiAI Service General APIs Advanced APIs Pre-integrated Solutions
Application
HiAI Engine ModelArts enablement
CANN
Compute Architecture for Neural Networks
Full MindSpore TensorFlow PyTorch PaddlePaddle … Framework
stack
FusionEngine
Processor
CANN enablement
TBE operator CCE Operator
development tool Library
Ascend- Ascend- Ascend- Ascend- Ascend-
Nano Tiny Lite Ascend Mini Max
IP and Chip
CCE Compiler
Public Private Edge Industrial
Consumer device
cloud cloud computing devices

All scenarios

39 Huawei Confidential
Full Stack — Ascend 310 AI Processor and Da Vinci Core

40 Huawei Confidential
Ascend AI Processors: Infusing Superior Intelligence for
Computing
FLOPS
256T
4

3
125T
Ascend 310 Ascend 910 2 90T
45T
AI SoC with ultimate Most powerful AI 1
energy efficiency processor
Ascend-Mini Ascend 910
Architecture: Da Vinci Ascend-Max
Architecture: Da Vinci
Half-precision (FP16): 8 TFLOPS
Integer precision (INT8): 16 TOPS Half-precision (FP16): 256 TFLOPS
16-channel full-HD video decoder: H.264/265 Integer precision (INT8): 512 TOPS
1-channel full-HD video encoder: H.264/265 128-channel full HD video decoder: H.264/265
Max. power: 8 W Max. power: 310 W

41 Huawei Confidential
Atlas AI Computing Platform Portfolio
Internet, security, finance, transportation, power, etc.
Atlas intelligent edge platform Atlas deep learning platform
Application
Enablement Industry SDK/Container Cluster management/Model
engine/Basic service repository management/Data pre-processing

TensorFlow/PyTorch/Caffe/MxNet Common
Framework MindSpore components
Framework
Framework Adapter
Adapret

AscendCL

Unified O&M and configuration management

AXE toolchain (log/profiling/Mind Studio)

Graph engine for graph optimization
Operator/Acceleration/Communication libraries (BLAS, FFT, DNN, Rand, Solver, Sparse, HCCL)
CANN
Runtime

Safety subsystem
Driver

subsystem
Atlas 800 AI inference server
Atlas 300 inference
Atlas 200 512 TOPS INT8
accelerator card
16 TOPS INT8 64 TOPS INT8
Atlas 500 Atlas 900
Chips & 16 TOPS INT8 256–1024 PFLOPS FP16
Hardware Atlas 800 AI training server
Atlas 300 training card
Atlas 200 developer kit 256 TFLOPS FP16 2 PFLOPS FP16

Da Vinci
Ascend 310 Architecture Ascend 910

42 Huawei Confidential
Huawei Atlas Computational Reasoning Platform

43 Huawei Confidential
HUAWEI CLOUD AI and HUAWEI Mobile Phones Help RFCx
Protect the Rainforest

44 Huawei Confidential
 Contents

1. AI Overview

2. Technical Fields and Application Fields of AI

3. Huawei's AI Development Strategy

4. AI Disputes

5. Future Prospects of AI

45 Huawei Confidential
Seeing = Believing?
⚫ With the development of computer vision technologies, reliability of images and videos is decreasing. Fake
images can be produced with technologies such as PS and generative adversarial networks (GAN), making it
hard to identify whether images are true or not.
⚫ Example:
 A suspect provided fake evidence by forging an image in which the suspect is in a place where he has never been to or
with someone he has never seen using PS technologies.
 In advertisements for diet pills, people's appearances before and after weight loss can be changed with PS technologies
to exaggerate the effect of the pills.
 Lyrebird, a tool for simulating voice of human beings based on recording samples of minutes, may be used by criminals.
 Household images released on rent and hotel booking platforms may be generated through GAN.

46 Huawei Confidential
AI Development = Rising Unemployment?
⚫ Looking back, human beings have always been seeking ways to improve efficiency, that is, obtain more with
less resources. We used sharp stones to hunt and collect food more efficiently. We used steam engines to
reduce the need for horses. Every step in achieving automation will change our life and work. In the era of AI,
what jobs will be replaced by AI?
⚫ The answer is repetitive jobs that involve little creativity and social interaction.

Jobs Most Likely to Be Replaced by AI Jobs Most Unlikely to Be Replaced by AI

Courier Writer
Taxi driver Management personnel
Soldier Software engineers
Accounting HR manager
Telesales personnel Designer
Customer service Activity planner
... ...
47 Huawei Confidential
Problems to Be Solved
⚫ Are AI-created works protected by copyright laws?
⚫ Who gives authority to robots?
⚫ What rights shall be authorized to robots?
⚫ ...

48 Huawei Confidential
 Contents

1. AI Overview

2. Technical Fields and Application Fields of AI

3. Huawei's AI Development Strategy

4. AI Disputes

5. Future Prospects of AI

49 Huawei Confidential
Development Trends of AI Technologies
⚫ Framework: easier-to-use development framework
⚫ Algorithm: algorithm models with better performance and smaller size
⚫ Computing power: comprehensive development of device-edge-cloud computing
⚫ Data: more comprehensive basic data service industry and more secure data sharing
⚫ Scenario: continuous breakthroughs in industry applications

50 Huawei Confidential
Easier-to-Use Development Framework
⚫ Various AI development frameworks are evolving towards ease-of-use and omnipotent,
continuously lowering the threshold for AI development.

51 Huawei Confidential
Tensorflow 2.0
⚫ TensorFlow 2.0 has been officially released. It integrates Keras as its high-level API, greatly
improving usability.

52 Huawei Confidential
Pytorch vs Tensorflow
⚫ PyTorch is widely recognized by academia for its ease of use.

Comparison between PyTorch and TensorFlow usage

statistics of top academic conferences

53 Huawei Confidential
Smaller Deep Learning Models
⚫ A model with better performance usually has a larger quantity of parameters, and a large model
has lower running efficiency in industrial applications. More and more model compression
technologies are proposed to further compress the model size while ensuring the model
performance, meeting the requirements of industrial applications.
 Low rank approximation
Network
architecture
 Network pruning design

 Network quantification Low rank Network

approximation
pruning
 Knowledge distillation Model
compression
 Compact network design
Network Knowledge
quantification distillation

54 Huawei Confidential
Computing Power with Comprehensive Device-Edge-Cloud
Development
⚫ The scale of AI chips applied to the cloud, edge devices, and mobile devices keeps increasing,
further meeting the computing power demand of AI.

Sales revenue (CNY100 million) Growth rate

China AI Chip Industry Development White Paper 2020

Market Scale and Growth Prediction of AI Chips in China from 2020 to 2021

55 Huawei Confidential
More Secure Data Sharing
⚫ Federated learning uses different data sources to train models, further breaking data bottlenecks
while ensuring data privacy and security.

Federated Learning White Paper V1.0

56 Huawei Confidential
Continuous Breakthroughs in Application Scenarios
⚫ With the continuous exploration of AI in various verticals, the application scenarios of AI
will be continuously broken through.
 Mitigating psychological problems
 Automatic vehicle insurance and loss assessment
 Office automation
 ...

57 Huawei Confidential
Mitigating Psychological Problems
⚫ AI chat robots help alleviate mental health problems such as autism by combining psychological
knowledge.

58 Huawei Confidential
Automatic Vehicle Insurance and Loss Assessment
⚫ AI technologies help insurance companies optimize vehicle insurance claims and complete vehicle
insurance loss assessment using deep learning algorithms such as image recognition.

59 Huawei Confidential
Office Automation
⚫ AI is automating management, but the different nature and format of data makes it a challenging
task. While each industry and application has its own unique challenges, different industries are
gradually adopting machine learning-based workflow solutions.

61 Huawei Confidential
 Summary

⚫ This chapter introduces the definition and development history of AI, describes the
technical fields and application fields of AI, briefly introduces Huawei's AI
development strategy, and finally discusses the disputes and the development trends
of AI.

62 Huawei Confidential
 Quiz

1. (Multiple-answer question) Which of the following are AI application fields?

A. Smart household

B. Smart healthcare

C. Smart city

D. Smart education

2. (True or False) By "all AI scenarios", Huawei means different deployment scenarios for AI, including
public clouds, private clouds, edge computing in all forms, industrial IoT devices, and consumer
devices.
A. True

B. False

63 Huawei Confidential
 More Information

Online learning website

 https://e.huawei.com/en/talent/#/home

Huawei Knowledge Base

 https://support.huawei.com/enterprise/en/knowledge?lang=en

64 Huawei Confidential
Thank you. 把数字世界带入每个人、每个家庭、
每个组织，构建万物互联的智能世界。
Bring digital to every person, home, and
organization for a fully connected,
intelligent world.

Copyright©2020 Huawei Technologies Co., Ltd.

All Rights Reserved.

The information in this document may contain predictive

statements including, without limitation, statements regarding
the future financial and operating results, future product
portfolio, new technology, etc. There are a number of factors that
could cause actual results and developments to differ materially
from those expressed or implied in the predictive statements.
Therefore, such information is provided for reference purpose
only and constitutes neither an offer nor an acceptance. Huawei
may change the information at any time without notice.
Machine Learning Overview
 Foreword

⚫ Machine learning is a core research field of AI, and it is also a necessary knowledge
for deep learning. Therefore, this chapter mainly introduces the main concepts of
machine learning, the classification of machine learning, the overall process of
machine learning, and the common algorithms of machine learning.

2 Huawei Confidential
 Objectives

Upon completion of this course, you will be able to:

 Master the learning algorithm definition and machine learning process.
 Know common machine learning algorithms.
 Understand concepts such as hyperparameters, gradient descent, and cross validation.

3 Huawei Confidential
 Contents

1. Machine Learning Definition

2. Machine Learning Types

3. Machine Learning Process

4. Other Key Machine Learning Methods

5. Common Machine Learning Algorithms

6. Case Study

4 Huawei Confidential
Machine Learning Algorithms (1)
⚫ Machine learning (including deep learning) is a study of learning algorithms. A computer
program is said to learn from experience 𝐸 with respect to some class of tasks 𝑇 and
performance measure 𝑃 if its performance at tasks in 𝑇, as measured by 𝑃, improves with
experience 𝐸.

Learning Basic
Data
algorithms understanding
(Experience E)
(Task T) (Measure P)

5 Huawei Confidential
Machine Learning Algorithms (2)

Experience Historical data

Induction Training

Input Prediction Input Prediction

New New Future
Regularity Future Model
problems data attributes

6 Huawei Confidential
 Created by: Jim Liang

Differences Between Machine Learning Algorithms and

Traditional Rule-Based Algorithms
Rule-based algorithms Machine learning

Training
data

Machine
learning

New data Model Prediction

• Samples are used for training.

• Explicit programming is used to solve problems. • The decision-making rules are complex or difficult
to describe.
• Rules can be manually specified.
• Rules are automatically learned by machines.

7 Huawei Confidential
Application Scenarios of Machine Learning (1)
⚫ The solution to a problem is complex, or the problem may involve a large amount of data
without a clear data distribution function.
⚫ Machine learning can be used in the following scenarios:

Task rules change over time. For Data distribution changes over
Rules are complex or cannot be example, in the part-of-speech time, requiring constant
described, such as voice tagging task, new words or readaptation of programs, such as
recognition. meanings are generated at any predicting the trend of commodity
time. sales.

8 Huawei Confidential
Application Scenarios of Machine Learning (2)

Complex
Machine learning
Manual rules
Rule complexity

Simple algorithms

Rule-based
Simple problems
algorithms

Small Large

Scale of the problem

9 Huawei Confidential
Rational Understanding of Machine Learning Algorithms

Target function
𝑓: 𝑋 → 𝑌

Ideal

Actual
Training data Hypothesis function
Learning algorithms
𝐷: {(𝑥1 , 𝑦1 ) ⋯ , (𝑥𝑛 , 𝑦𝑛 )} 𝑔≈𝑓

⚫ Target function f is unknown. Learning algorithms cannot obtain a perfect function f.

⚫ Assume that hypothesis function g approximates function f, but may be different from function f.

10 Huawei Confidential
Main Problems Solved by Machine Learning
⚫ Machine learning can deal with many types of tasks. The following describes the most typical and common types of tasks.
 Classification: A computer program needs to specify which of the k categories some input belongs to. To accomplish this task, learning
algorithms usually output a function 𝑓: 𝑅𝑛 → (1,2, … , 𝑘). For example, the image classification algorithm in computer vision is developed
to handle classification tasks.

 Regression: For this type of task, a computer program predicts the output for the given input. Learning algorithms typically output a
function 𝑓: 𝑅𝑛 → 𝑅. An example of this task type is to predict the claim amount of an insured person (to set the insurance premium) or
predict the security price.

 Clustering: A large amount of data from an unlabeled dataset is divided into multiple categories according to internal similarity of the
data. Data in the same category is more similar than that in different categories. This feature can be used in scenarios such as image
retrieval and user profile management.

⚫ Classification and regression are two main types of prediction, accounting from 80% to 90%. The output of classification is
discrete category values, and the output of regression is continuous numbers.

11 Huawei Confidential
 Contents

1. Machine Learning Definition

2. Machine Learning Types

3. Machine Learning Process

4. Other Key Machine Learning Methods

5. Common Machine Learning Algorithms

6. Case study

12 Huawei Confidential
Machine Learning Classification
⚫ Supervised learning: Obtain an optimal model with required performance through training and learning based on the
samples of known categories. Then, use the model to map all inputs to outputs and check the output for the purpose of
classifying unknown data.

⚫ Unsupervised learning: For unlabeled samples, the learning algorithms directly model the input datasets. Clustering is a
common form of unsupervised learning. We only need to put highly similar samples together, calculate the similarity
between new samples and existing ones, and classify them by similarity.

⚫ Semi-supervised learning: In one task, a machine learning model that automatically uses a large amount of unlabeled data
to assist learning directly of a small amount of labeled data.

⚫ Reinforcement learning: It is an area of machine learning concerned with how agents ought to take actions in an
environment to maximize some notion of cumulative reward. The difference between reinforcement learning and
supervised learning is the teacher signal. The reinforcement signal provided by the environment in reinforcement learning is
used to evaluate the action (scalar signal) rather than telling the learning system how to perform correct actions.

13 Huawei Confidential
Supervised Learning
Data feature Label

Feature 1 ... Feature n Goal

Supervised learning
Feature 1 ... Feature n Goal
algorithm

Feature 1 ... Feature n Goal

Wind Enjoy
Weather Temperature
Speed Sports
Sunny Warm Strong Yes
Rainy Cold Fair No
Sunny Cold Weak Yes
15 Huawei Confidential
Supervised Learning - Regression Questions
⚫ Regression: reflects the features of attribute values of samples in a sample dataset. The
dependency between attribute values is discovered by expressing the relationship of sample
mapping through functions.
 How much will I benefit from the stock next week?
 What's the temperature on Tuesday?

16 Huawei Confidential
Supervised Learning - Classification Questions
⚫ Classification: maps samples in a sample dataset to a specified category by using a
classification model.
 Will there be a traffic jam on XX road during
the morning rush hour tomorrow?
 Which method is more attractive to customers:
5 yuan voucher or 25% off?

17 Huawei Confidential
Unsupervised Learning

Data Feature

Feature 1 ... Feature n

Unsupervised learning Internal

Feature 1 ... Feature n similarity
algorithm

Feature 1 ... Feature n

Monthly Consumption
Commodity
Consumption Time Category
Badminton Cluster 1
1000–2000 6:00–12:00
racket
Cluster 2
500–1000 Basketball 18:00–24:00
1000–2000 Game console 00:00–6:00

18 Huawei Confidential
Unsupervised Learning - Clustering Questions
⚫ Clustering: classifies samples in a sample dataset into several categories based on the
clustering model. The similarity of samples belonging to the same category is high.
 Which audiences like to watch movies
of the same subject?
 Which of these components are
damaged in a similar way?

19 Huawei Confidential
Semi-Supervised Learning
Data Feature Label

Feature 1 ... Feature n Goal

Semi-supervised
Feature 1 ... Feature n Unknown
learning algorithms

Feature 1 ... Feature n Unknown

Wind Enjoy
Weather Temperature
Speed Sports
Sunny Warm Strong Yes
Rainy Cold Fair /
Sunny Cold Weak /

20 Huawei Confidential
Reinforcement Learning
⚫ The model perceives the environment, takes actions, and makes adjustments and choices
based on the status and award or punishment.

Model

Reward or Action 𝑎𝑡
Status 𝑠𝑡
punishment 𝑟𝑡

𝑟𝑡+1

𝑠𝑡+1 Environment

21 Huawei Confidential
Reinforcement Learning - Best Behavior
⚫ Reinforcement learning: always looks for best behaviors. Reinforcement learning is
targeted at machines or robots.
 Autopilot: Should it brake or accelerate when the yellow light starts to flash?
 Cleaning robot: Should it keep working or go back for charging?

22 Huawei Confidential
 Contents

1. Machine learning algorithm

2. Machine Learning Classification

3. Machine Learning Process

4. Other Key Machine Learning Methods

5. Common Machine Learning Algorithms

6. Case study

23 Huawei Confidential
Machine Learning Process

Feature Model
Data Model Model
Data cleansing extraction and deployment and
collection training evaluation
selection integration

Feedback and iteration

24 Huawei Confidential
Basic Machine Learning Concept — Dataset
⚫ Dataset: a collection of data used in machine learning tasks. Each data record is called a
sample. Events or attributes that reflect the performance or nature of a sample in a
particular aspect are called features.
⚫ Training set: a dataset used in the training process, where each sample is referred to as a
training sample. The process of creating a model from data is called learning (training).
⚫ Test set: Testing refers to the process of using the model obtained after learning for
prediction. The dataset used is called a test set, and each sample is called a test sample.

25 Huawei Confidential
Checking Data Overview
⚫ Typical dataset form

Feature 1 Feature 2 Feature 3 Label

No. Area School Districts Direction House Price

1 100 8 South 1000

2 120 9 Southwest 1300

Training
set
3 60 6 North 700

4 80 9 Southeast 1100

Test set 5 95 3 South 850

26 Huawei Confidential
Importance of Data Processing
⚫ Data is crucial to models. It is the ceiling of model capabilities. Without good data, there is
no good model.

Data
Data cleansing
preprocessing Data normalization

Fill in missing values, Normalize data to reduce

and detect and noise and improve model
eliminate causes of accuracy.
dataset exceptions.

Data dimension
reduction
Simplify data attributes to
avoid dimension
explosion.

27 Huawei Confidential
Workload of Data Cleansing
⚫ Statistics on data scientists' work in machine learning

3% Remodeling training datasets

5% Others
4% Optimizing models

9% Mining modes from data

19% Collecting datasets

60% Cleansing and sorting data

CrowdFlower Data Science Report 2016

28 Huawei Confidential
Data Cleansing
⚫ Most machine learning models process features, which are usually numeric
representations of input variables that can be used in the model.
⚫ In most cases, the collected data can be used by algorithms only after being preprocessed.
The preprocessing operations include the following:
 Data filtering
 Processing of lost data
 Processing of possible exceptions, errors, or abnormal values
 Combination of data from multiple data sources
 Data consolidation

29 Huawei Confidential
Dirty Data (1)
⚫ Generally, real data may have some quality problems.
 Incompleteness: contains missing values or the data that lacks attributes
 Noise: contains incorrect records or exceptions.
 Inconsistency: contains inconsistent records.

30 Huawei Confidential
Dirty Data (2)
#Stu
IsTea
# Id Name Birthday Gender dent Country City
cher
s

1 111 John 31/12/1990 M 0 0 Ireland Dublin

2 222 Mery 15/10/1978 F 1 15 Iceland Missing value

3 333 Alice 19/04/2000 F 0 0 Spain Madrid

4 444 Mark 01/11/1997 M 0 0 France Paris

5 555 Alex 15/03/2000 A 1 23 Germany Berlin Invalid value

6 555 Peter 1983-12-01 M 1 10 Italy Rome

7 777 Calvin 05/05/1995 M 0 0 Italy Italy Value that should

be in another
8 888 Roxane 03/08/1948 F 0 0 Portugal Lisbon
column
Genev
9 999 Anne 05/09/1992 F 0 5 Switzerland
Invalid duplicate item a

10 101010 Paul 14/11/1992 M 1 26 Ytali Rome Misspelling

Incorrect format Attribute dependency

31 Huawei Confidential
Data Conversion
⚫ After being preprocessed, the data needs to be converted into a representation form suitable for the machine
learning model. Common data conversion forms include the following:
 With respect to classification, category data is encoded into a corresponding numerical representation.
 Value data is converted to category data to reduce the value of variables (for age segmentation).
 Other data
◼ In the text, the word is converted into a word vector through word embedding (generally using the word2vec model, BERT model,
etc).
◼ Process image data (color space, grayscale, geometric change, Haar feature, and image enhancement)

 Feature engineering
◼ Normalize features to ensure the same value ranges for input variables of the same model.
◼ Feature expansion: Combine or convert existing variables to generate new features, such as the average.

32 Huawei Confidential
Necessity of Feature Selection
⚫ Generally, a dataset has many features, some of which may be redundant or irrelevant to
the value to be predicted.
⚫ Feature selection is necessary in the following aspects:

Simplify
models to
Reduce the
make them
training time
easy for users
to interpret

Improve
Avoid model
dimension generalization
explosion and avoid
overfitting

33 Huawei Confidential
Feature Selection Methods - Filter
⚫ Filter methods are independent of the model during feature selection.
By evaluating the correlation between each feature
and the target attribute, these methods use a
statistical measure to assign a value to each feature.
Features are then sorted by score, which is helpful for
preserving or eliminating specific features.

Common methods
Select the
• Pearson correlation coefficient
Traverse all Train models Evaluate the
features optimal feature performance • Chi-square coefficient
subset
• Mutual information

Procedure of a filter method Limitations

• The filter method tends to select redundant
variables as the relationship between features is not
considered.

34 Huawei Confidential
Feature Selection Methods - Wrapper
⚫ Wrapper methods use a prediction model to score feature subsets.

Wrapper methods consider feature selection as a search

issue for which different combinations are evaluated
and compared. A predictive model is used to evaluate a
combination of features and assign a score based on
Select the optimal
model accuracy.
feature subset
Common methods
Generate a
Traverse all
feature
Train models
Evaluate • Recursive feature elimination (RFE)
features
subset models
Limitations
Procedure of a • Wrapper methods train a new model for each subset,
wrapper method resulting in a huge number of computations.
• A feature set with the best performance is usually
provided for a specific type of model.

35 Huawei Confidential
Feature Selection Methods - Embedded
⚫ Embedded methods consider feature selection as a part of model construction.

The most common type of embedded feature selection

method is the regularization method.
Regularization methods are also called penalization
Select the optimal feature subset methods that introduce additional constraints into the
optimization of a predictive algorithm that bias the model
toward lower complexity and reduce the number of
Traverse all Generate a Train models
features feature subset + Evaluate the effect features.

Common methods
Procedure of an embedded method
• Lasso regression
• Ridge regression

36 Huawei Confidential
Overall Procedure of Building a Model
Model Building Procedure

1 2 3

Data splitting: Model training: Model verification:

Divide data into training Use data that has been cleaned Use validation sets to
sets, test sets, and up and feature engineering to validate the model
validation sets. train a model. validity.

6 5 4

Model fine-tuning: Model deployment: Model test:

Continuously tune the Deploy the model in Use test data to evaluate the
model based on the actual an actual production generalization capability of
data of a service scenario. scenario. the model in a real
environment.

37 Huawei Confidential
Examples of Supervised Learning - Learning Phase
⚫ Use the classification model to predict whether a person is a basketball player.
Feature (attribute)
Target

Service Name City Age Label

Training set
data Mike Miami 42 yes The model searches
Jerry New York 32 no for the relationship
(Cleansed features and tags)
between features and
Splitting Bryan Orlando 18 no
targets.
Task: Use a classification model to predict Patricia Miami 45 yes
whether a person is a basketball player under a
specific feature. Elodie Phoenix 35 no Test set
Remy Chicago 72 yes Use new data to verify
the model validity.
John New York 48 yes
Model
training
Each feature or a combination of several features can
provide a basis for a model to make a judgment.

38 Huawei Confidential
Examples of Supervised Learning - Prediction Phase
Name City Age Label
Marine Miami 45 ?
Julien Miami 52 ? Unknown data
Recent data, it is not
New Fred Orlando 20 ?
known whether the
data
Michelle Boston 34 ? people are basketball
Nicolas Phoenix 90 ? players.

IF city = Miami → Probability = +0.7

IF city= Orlando → Probability = +0.2
IF age > 42 → Probability = +0.05*age + 0.06
Application IF age ≤ 42 → Probability = +0.01*age + 0.02
model
Name City Age Prediction
Marine Miami 45 0.3
New Possibility prediction
Julien Miami 52 0.9
data Apply the model to the
Fred Orlando 20 0.6 new data to predict
Prediction whether the customer will
data Michelle Boston 34 0.5
change the supplier.
Nicolas Phoenix 90 0.4

39 Huawei Confidential
What Is a Good Model?

• Generalization capability
Can it accurately predict the actual service data?

• Interpretability
Is the prediction result easy to interpret?

• Prediction speed
How long does it take to predict each piece of data?

• Practicability
Is the prediction rate still acceptable when the service
volume increases with a huge data volume?

40 Huawei Confidential
Model Validity (1)
⚫ Generalization capability: The goal of machine learning is that the model obtained after learning should
perform well on new samples, not just on samples used for training. The capability of applying a model to new
samples is called generalization or robustness.
⚫ Error: difference between the sample result predicted by the model obtained after learning and the actual
sample result.
 Training error: error that you get when you run the model on the training data.
 Generalization error: error that you get when you run the model on new samples. Obviously, we prefer a model with a
smaller generalization error.

⚫ Underfitting: occurs when the model or the algorithm does not fit the data well enough.
⚫ Overfitting: occurs when the training error of the model obtained after learning is small but the generalization
error is large (poor generalization capability).

41 Huawei Confidential
Model Validity (2)
⚫ Model capacity: model's capability of fitting functions, which is also called model complexity.
 When the capacity suits the task complexity and the amount of training data provided, the algorithm effect is usually
optimal.
 Models with insufficient capacity cannot solve complex tasks and underfitting may occur.
 A high-capacity model can solve complex tasks, but overfitting may occur if the capacity is higher than that required by a
task.

Underfitting Good fitting Overfitting

Not all features are learned. Noises are learned.
42 Huawei Confidential
Overfitting Cause — Error
⚫ Total error of final prediction = Bias2 + Variance + Irreducible error
⚫ Generally, the prediction error can be divided into two types:
 Error caused by "bias"
Variance
 Error caused by "variance"

⚫ Variance: Bias

 Offset of the prediction result from the average value

 Error caused by the model's sensitivity to small fluctuations
in the training set

⚫ Bias:
 Difference between the expected (or average) prediction value and the correct value we
are trying to predict.

43 Huawei Confidential
Variance and Bias
⚫ Combinations of variance and bias are as follows:
 Low bias & low variance –> Good model
 Low bias & high variance
 High bias & low variance
 High bias & high variance –> Poor model

⚫ Ideally, we want a model that can accurately capture

the rules in the training data and summarize the
invisible data (new data). However, it is usually
impossible for the model to complete both tasks at
the same time.

44 Huawei Confidential
Model Complexity and Error
⚫ As the model complexity increases, the training error decreases.
⚫ As the model complexity increases, the test error decreases to a certain point and then
increases in the reverse direction, forming a convex curve.

High bias & low Low bias & high

variance variance

Testing error
Error

Training error

Model Complexity

45 Huawei Confidential
Machine Learning Performance Evaluation - Regression
⚫ The closer the Mean Absolute Error (MAE) is to 0, the better the model can fit the training data.

𝑚
1
𝑀𝐴𝐸 = ෍ 𝑦𝑖 − 𝑦ො𝑖
m
𝑖=1

⚫ Mean Square Error (MSE)

m
1 2
𝑀𝑆𝐸 = ෍ 𝑦𝑖 − 𝑦ො𝑖
m
𝑖=1

⚫ The value range of R2 is (–∞, 1]. A larger value indicates that the model can better fit the training data. TSS
indicates the difference between samples. RSS indicates the difference between the predicted value and
sample value.
𝑚 2
2
𝑅𝑆𝑆 σ𝑖=1 𝑦𝑖 − 𝑦ො𝑖
𝑅 =1− =1− 𝑚 2
𝑇𝑆𝑆 σ𝑖=1 𝑦𝑖 − 𝑦ത𝑖

46 Huawei Confidential
Machine Learning Performance Evaluation - Classification (1)
⚫ Terms and definitions: Estimated
amount
yes no Total
 𝑃: positive, indicating the number of real positive cases
Actual amount
in the data.
yes 𝑇𝑃 𝐹𝑁 𝑃
 𝑁: negative, indicating the number of real negative cases
no 𝐹𝑃 𝑇𝑁 𝑁
in the data.
Total 𝑃′ 𝑁′ 𝑃+𝑁
 𝑇P : true positive, indicating the number of positive cases that are correctly
classified by the classifier. Confusion matrix
 𝑇𝑁: true negative, indicating the number of negative cases that are correctly classified by the classifier.
 𝐹𝑃: false positive, indicating the number of positive cases that are incorrectly classified by the classifier.
 𝐹𝑁: false negative, indicating the number of negative cases that are incorrectly classified by the classifier.

⚫ Confusion matrix: at least an 𝑚 × 𝑚 table. 𝐶𝑀𝑖,𝑗 of the first 𝑚 rows and 𝑚 columns indicates the number of cases that
actually belong to class 𝑖 but are classified into class 𝑗 by the classifier.
 Ideally, for a high accuracy classifier, most prediction values should be located in the diagonal from 𝐶𝑀1,1 to 𝐶𝑀𝑚,𝑚 of the table while
values outside the diagonal are 0 or close to 0. That is, 𝐹𝑃 and 𝐹𝑃 are close to 0.

47 Huawei Confidential
Machine Learning Performance Evaluation - Classification (2)
Measurement Ratio
𝑇𝑃 + 𝑇𝑁
Accuracy and recognition rate
𝑃+𝑁
𝐹𝑃 + 𝐹𝑁
Error rate and misclassification rate
𝑃+𝑁
𝑇𝑃
Sensitivity, true positive rate, and recall
𝑃
𝑇𝑁
Specificity and true negative rate
𝑁
𝑇𝑃
Precision
𝑇𝑃 + 𝐹𝑃
𝐹1 , harmonic mean of the recall rate and 2 × 𝑝𝑟𝑒𝑐𝑖𝑠𝑖𝑜𝑛 × 𝑟𝑒𝑐𝑎𝑙𝑙
precision 𝑝𝑟𝑒𝑐𝑖𝑠𝑖𝑜𝑛 + 𝑟𝑒𝑐𝑎𝑙𝑙

𝐹𝛽 , where 𝛽 is a non-negative real (1 + 𝛽 2 ) × 𝑝𝑟𝑒𝑐𝑖𝑠𝑖𝑜𝑛 × 𝑟𝑒𝑐𝑎𝑙𝑙

number 𝛽 2 × 𝑝𝑟𝑒𝑐𝑖𝑠𝑖𝑜𝑛 + 𝑟𝑒𝑐𝑎𝑙𝑙

48 Huawei Confidential
Example of Machine Learning Performance Evaluation
⚫ We have trained a machine learning model to identify whether the object in an image is a cat. Now
we use 200 pictures to verify the model performance. Among the 200 images, objects in 170
images are cats, while others are not. The identification result of the model is that objects in 160
images are cats, while others are not.
𝑇𝑃 140
Precision: 𝑃 = 𝑇𝑃+𝐹𝑃 = 140+20 = 87.5% Estimated amount
Actual 𝒚𝒆𝒔 𝒏𝒐 Total
amount
𝑇𝑃 140
Recall: 𝑅 = 𝑃
=
170
= 82.4%
𝑦𝑒𝑠 140 30 170

𝑇𝑃+𝑇𝑁 140+10
Accuracy: 𝐴𝐶𝐶 = 𝑃+𝑁
=
170+30
= 75% 𝑛𝑜 20 10 30

Total 160 40 200

49 Huawei Confidential
 Contents

1. Machine Learning Definition

2. Machine Learning Types

3. Machine Learning Process

4. Other Key Machine Learning Methods

5. Common Machine Learning Algorithms

6. Case study

50 Huawei Confidential
Machine Learning Training Method - Gradient Descent (1)
⚫ The gradient descent method uses the negative gradient Cost surface
direction of the current position as the search direction, which
is the steepest direction. The formula is as follows:

wk +1 = wk − f wk ( x )
i

⚫ In the formula, 𝜂 indicates the learning rate and 𝑖 indicates the

data record number 𝑖. The weight parameter w indicates the
change in each iteration.
⚫ Convergence: The value of the objective function changes very
little, or the maximum number of iterations is reached.

51 Huawei Confidential
Machine Learning Training Method - Gradient Descent (2)
⚫ Batch Gradient Descent (BGD) uses the samples (m in total) in all datasets to update the
weight parameter based on the gradient value at the current point.
1 m
wk +1 = wk −   f wk ( x i )
m i =1
⚫ Stochastic Gradient Descent (SGD) randomly selects a sample in a dataset to update the
weight parameter based on the gradient value at the current point.
wk +1 = wk − f wk ( x i )
⚫ Mini-Batch Gradient Descent (MBGD) combines the features of BGD and SGD and selects
the gradients of n samples in a dataset to update the weight parameter.
1 t + n −1
wk +1 = wk −   f wk ( x i )
n i=t
52 Huawei Confidential
Machine Learning Training Method - Gradient Descent (3)
⚫ Comparison of three gradient descent methods
 In the SGD, samples selected for each training are stochastic. Such instability causes the loss function to be unstable or
even causes reverse displacement when the loss function decreases to the lowest point.
 BGD has the highest stability but consumes too many computing resources. MBGD is a method that balances SGD and
BGD.

BGD
Uses all training samples for training each time.

SGD
Uses one training sample for training each time.

MBGD
Uses a certain number of training samples for training
each time.

53 Huawei Confidential
Parameters and Hyperparameters in Models
⚫ The model contains not only parameters but also hyperparameters. The purpose is to
enable the model to learn the optimal parameters.
 Parameters are automatically learned by models.
 Hyperparameters are manually set.

Model parameters are

"distilled" from data.

Model

Training
Use hyperparameters to
control training.
54 Huawei Confidential
Hyperparameters of a Model

• λ during Lasso/Ridge regression

• Often used in model parameter
• Learning rate for training a neural
estimation processes. network, number of iterations, batch
size, activation function, and number
• Often specified by the practitioner. of neurons
• Can often be set using heuristics. • 𝐶 and 𝜎 in support vector machines
(SVM)
• Often tuned for a given predictive • K in k-nearest neighbor (KNN)
modeling problem. • Number of trees in a random forest

Model hyperparameters are

Common model hyperparameters
external configurations of models.

55 Huawei Confidential
Hyperparameter Search Procedure and Method

1. Dividing a dataset into a training set, validation set, and test set.
2. Optimizing the model parameters using the training set based on the model performance
indicators.
3. Searching for the model hyper-parameters using the validation set based on the model
Procedure for performance indicators.
searching 4. Perform step 2 and step 3 alternately. Finally, determine the model parameters and
hyperparameters hyperparameters and assess the model using the test set.

• Grid search
• Random search
• Heuristic intelligent search
Search algorithm (step • Bayesian search
3)

56 Huawei Confidential
Hyperparameter Searching Method - Grid Search
⚫ Grid search attempts to exhaustively search all possible
hyperparameter combinations to form a hyperparameter value
grid. Grid search
5
⚫ In practice, the range of hyperparameter values to search is

Hyperparameter 1
4
specified manually.
3
⚫ Grid search is an expensive and time-consuming method.
2
 This method works well when the number of hyperparameters
1
is relatively small. Therefore, it is applicable to generally
machine learning algorithms but inapplicable to neural networks 0 1 2 3 4 5

(see the deep learning part). Hyperparameter 2

57 Huawei Confidential
Hyperparameter Searching Method - Random Search
⚫ When the hyperparameter search space is large, random search
is better than grid search. Random search
⚫ In random search, each setting is sampled from the distribution
of possible parameter values, in an attempt to find the best
subset of hyperparameters.

Parameter 1
⚫ Note:
 Search is performed within a coarse range, which then will be
narrowed based on where the best result appears.
 Some hyperparameters are more important than others, and the
Parameter 2
search deviation will be affected during random search.

58 Huawei Confidential
Cross Validation (1)
⚫ Cross validation: It is a statistical analysis method used to validate the performance of a classifier. The basic
idea is to divide the original dataset into two parts: training set and validation set. Train the classifier using the
training set and test the model using the validation set to check the classifier performance.
⚫ k-fold cross validation (𝑲 − 𝑪𝑽):
 Divide the raw data into 𝑘 groups (generally, evenly divided).
 Use each subset as a validation set, and use the other 𝑘 − 1 subsets as the training set. A total of 𝑘 models can be
obtained.
 Use the mean classification accuracy of the final validation sets of 𝑘 models as the performance indicator of the 𝐾 − 𝐶𝑉
classifier.

59 Huawei Confidential
Cross Validation (2)

Entire dataset

Training set Test set

Training set Validation set Test set

⚫ Note: The K value in K-fold cross validation is also a hyperparameter.

60 Huawei Confidential
 Contents

1. Machine Learning Definition

2. Machine Learning Types

3. Machine Learning Process

4. Other Key Machine Learning Methods

5. Common Machine Learning Algorithms

6. Case study

61 Huawei Confidential
Machine Learning Algorithm Overview

Machine learning

Supervised learning Unsupervised learning

Classification Regression Clustering Others

Logistic regression Linear regression K-means Correlation rule

Hierarchical Principal component
SVM SVM
clustering analysis (PCA)
Neural network Neural network Density-based Gaussian mixture
clustering model (GMM)
Decision tree Decision tree

Random forest Random forest

GBDT GBDT

KNN

Naive Bayes

62 Huawei Confidential
Linear Regression (1)
⚫ Linear regression: a statistical analysis method to determine the quantitative relationships between
two or more variables through regression analysis in mathematical statistics.
⚫ Linear regression is a type of supervised learning.

Unary linear regression Multi-dimensional linear regression

63 Huawei Confidential
Linear Regression (2)
⚫ The model function of linear regression is as follows, where 𝑤 indicates the weight parameter, 𝑏 indicates the bias, and 𝑥
indicates the sample attribute.

hw ( x) = wT x + b
⚫ The relationship between the value predicted by the model and actual value is as follows, where 𝑦 indicates the actual
value, and 𝜀 indicates the error.
y = w x+b+
T

⚫ The error 𝜀 is influenced by many factors independently. According to the central limit theorem, the error 𝜀 follows normal
distribution. According to the normal distribution function and maximum likelihood estimation, the loss function of linear
regression is as follows:
1
J ( w) =  ( hw ( x) − y )
2

2m
⚫ To make the predicted value close to the actual value, we need to minimize the loss value. We can use the gradient descent
method to calculate the weight parameter 𝑤 when the loss function reaches the minimum, and then complete model
building.

64 Huawei Confidential
Linear Regression Extension - Polynomial Regression
⚫ Polynomial regression is an extension of linear regression. Generally, the complexity of a dataset
exceeds the possibility of fitting by a straight line. That is, obvious underfitting occurs if the original
linear regression model is used. The solution is to use polynomial regression.

hw ( x ) = w1 x + w2 x 2 + + wn x n + b
⚫ where, the nth power is a polynomial regression
dimension (degree).
⚫ Polynomial regression belongs to linear regression as
the relationship between its weight parameters 𝑤 is still
linear while its nonlinearity is reflected in the feature
Comparison between linear regression and
dimension. polynomial regression

65 Huawei Confidential
Linear Regression and Overfitting Prevention
⚫ Regularization terms can be used to reduce overfitting. The value of 𝑤 cannot be too large or too
small in the sample space. You can add a square sum loss on the target function.

1
J ( w) =  ( w − )  2
2 2
h ( x ) y + w
⚫ Regularization terms (norm): The2regularization
m term here is called L2-norm. Linear regression that
uses this loss function is also called Ridge regression.

1
J ( w) =  ( w − ) +  w 1
2
h ( x ) y
2m
⚫ Linear regression with absolute loss is called Lasso regression.

66 Huawei Confidential
Logistic Regression (1)
⚫ Logistic regression: The logistic regression model is used to solve classification problems. The
model is defined as follows:
𝑒 𝑤𝑥+𝑏
𝑃 𝑌=1𝑥 =
1 + 𝑒 𝑤𝑥+𝑏
1
𝑃 𝑌=0𝑥 =
1 + 𝑒 𝑤𝑥+𝑏
where 𝑤 indicates the weight, 𝑏 indicates the bias, and 𝑤𝑥 + 𝑏 is regarded as the linear function of 𝑥. Compare the
preceding two probability values. The class with a higher probability value is the class of 𝑥.

67 Huawei Confidential
Logistic Regression (2)
⚫ Both the logistic regression model and linear regression model are generalized linear models.
Logistic regression introduces nonlinear factors (the sigmoid function) based on linear regression
and sets thresholds, so it can deal with binary classification problems.
⚫ According to the model function of logistic regression, the loss function of logistic regression can be
estimated as follows by using the maximum likelihood estimation:
1
J ( w) = −  ( y ln hw ( x) + (1 − y ) ln(1 − hw ( x)) )
m
⚫ where 𝑤 indicates the weight parameter, 𝑚 indicates the number of samples, 𝑥 indicates the
sample, and 𝑦 indicates the real value. The values of all the weight parameters 𝑤 can also be
obtained through the gradient descent algorithm.

68 Huawei Confidential
Logistic Regression Extension - Softmax Function (1)
⚫ Logistic regression applies only to binary classification problems. For multi-class
classification problems, use the Softmax function.

Binary classification problem Multi-class classification problem

Grape?

Male? Orange?

Apple?

Female? Banana?

69 Huawei Confidential
Logistic Regression Extension - Softmax Function (2)
⚫ Softmax regression is a generalization of logistic regression that we can use for K-class
classification.
⚫ The Softmax function is used to map a K-dimensional vector of arbitrary real values to
another K-dimensional vector of real values, where each vector element is in the interval
(0, 1).
⚫ The regression probability function of Softmax is as follows:
wkT x
e
p ( y = k | x; w) = K
, k = 1, 2 ,K
e
l =1
wlT x

70 Huawei Confidential
Logistic Regression Extension - Softmax Function (3)
⚫ Softmax assigns a probability to each class in a multi-class problem. These probabilities must add
up to 1.
 Softmax may produce a form belonging to a particular class. Example:
Category Probability

Grape? 0.09

• Sum of all probabilities:

Orange? 0.22 • 0.09 + 0.22 + 0.68 + 0.01 =1
• Most probably, this picture is an
Apple? apple.
0.68

Banana? 0.01

71 Huawei Confidential
Decision Tree
⚫ A decision tree is a tree structure (a binary tree or a non-binary tree). Each non-leaf node represents a test on a feature
attribute. Each branch represents the output of a feature attribute in a certain value range, and each leaf node stores a
category. To use the decision tree, start from the root node, test the feature attributes of the items to be classified, select
the output branches, and use the category stored on the leaf node as the final result.

Root

Short Tall

Cannot Can Short Long

squeak squeak neck neck

Short Long
Might be a Might be a
Might be a nose nose
squirrel giraffe
rat

On land In water Might be an

elephant

Might be a Might be a
rhinoceros hippo
72 Huawei Confidential
Decision Tree Structure
Root Node

Internal Internal
Node Node

Internal
Leaf Node Leaf Node Node Leaf Node

Leaf Node Leaf Node Leaf Node

73 Huawei Confidential
Key Points of Decision Tree Construction
⚫ To create a decision tree, we need to select attributes and determine the tree structure between
feature attributes. The key step of constructing a decision tree is to divide data of all feature
attributes, compare the result sets in terms of 'purity', and select the attribute with the highest
'purity' as the data point for dataset division.
⚫ The metrics to quantify the 'purity' include the information entropy and GINI Index. The formula is
as follows:
K K
H ( X )= - pk log 2 ( pk ) Gini = 1 −  pk2
k =1 k =1

⚫ where 𝑝𝑘 indicates the probability that the sample belongs to class k (there are K classes in total). A
greater difference between purity before segmentation and that after segmentation indicates a
better decision tree.
⚫ Common decision tree algorithms include ID3, C4.5, and CART.
74 Huawei Confidential
Decision Tree Construction Process
⚫ Feature selection: Select a feature from the features of the training data as the split
standard of the current node. (Different standards generate different decision tree
algorithms.)
⚫ Decision tree generation: Generate internal node upside down based on the selected
features and stop until the dataset can no longer be split.
⚫ Pruning: The decision tree may easily become overfitting unless necessary pruning
(including pre-pruning and post-pruning) is performed to reduce the tree size and optimize
its node structure.

75 Huawei Confidential
Decision Tree Example
⚫ The following figure shows a classification when a decision tree is used. The classification result is impacted by
three attributes: Refund, Marital Status, and Taxable Income.

Marital Taxable
Tid Refund Cheat
Status Income
1 Yes Single 125,000 No
Refund
2 No Married 100,000 No
3 No Single 70,000 No Marital
No Status
4 Yes Married 120,000 No
5 No Divorced 95,000 Yes
Taxable
6 No Married 60,000 No Income No
7 Yes Divorced 220,000 No
8 No Single 85,000 Yes No Yes
9 No Married 75,000 No
10 No Single 90,000 Yes

76 Huawei Confidential
SVM
⚫ SVM is a binary classification model whose basic model is a linear classifier defined in the eigenspace with the
largest interval. SVMs also include kernel tricks that make them nonlinear classifiers. The SVM learning
algorithm is the optimal solution to convex quadratic programming.

weight
Projection

Complex segmentation Easy segmentation in

height
in low-dimensional high-dimensional space
space

77 Huawei Confidential
Linear SVM (1)
⚫ How do we split the red and blue datasets by a straight line?

or

With binary classification Both the left and right methods can be used to divide
Two-dimensional dataset datasets. Which of them is correct?

78 Huawei Confidential
Linear SVM (2)
⚫ Straight lines are used to divide data into different classes. Actually, we can use multiple straight lines to divide
data. The core idea of the SVM is to find a straight line and keep the point close to the straight line as far as
possible from the straight line. This can enable strong generalization capability of the model. These points are
called support vectors.
⚫ In two-dimensional space, we use straight lines for segmentation. In high-dimensional space, we use
hyperplanes for segmentation.

Distance between
support vectors
is as far as possible

79 Huawei Confidential
Nonlinear SVM (1)
⚫ How do we classify a nonlinear separable dataset?

Linear SVM can function well for Nonlinear datasets cannot be split
linear separable datasets. with straight lines.

80 Huawei Confidential
Nonlinear SVM (2)
⚫ Kernel functions are used to construct nonlinear SVMs.
⚫ Kernel functions allow algorithms to fit the largest hyperplane in a transformed high-dimensional
feature space.
Common kernel functions

Linear Polynomial
kernel kernel
function function

Gaussian Sigmoid
kernel kernel
function function Input space High-dimensional
feature space

81 Huawei Confidential
k-NN Algorithm (1)
⚫ The k-NN classification algorithm is a theoretically
mature method and one of the simplest machine
learning algorithms. According to this method, if
the majority of k samples most similar to one
sample (nearest neighbors in the eigenspace) ?
belong to a specific category, this sample also
belongs to this category.

The target category of point ? varies with the

number of the most adjacent nodes.

82 Huawei Confidential
k-NN Algorithm (2)
⚫ As the prediction result is determined based on the
number and weights of neighbors in the training set, the
k-NN algorithm has a simple logic.
⚫ k-NN is a non-parametric method which is usually used
in datasets with irregular decision boundaries.
 The k-NN algorithm generally adopts the majority voting
method for classification prediction and the average value
method for regression prediction.

⚫ k-NN requires a huge number of computations.

83 Huawei Confidential
k-NN Algorithm (3)
⚫ Generally, a larger k value reduces the impact of noise on classification, but obfuscates the boundary between
classes.
 A larger k value means a higher probability of underfitting because the segmentation is too rough. A smaller k value
means a higher probability of overfitting because the segmentation is too refined.

• The boundary becomes smoother as the

value of k increases.
• As the k value increases to infinity, all data
points will eventually become all blue or
all red.

84 Huawei Confidential
Naive Bayes (1)
⚫ Naive Bayes algorithm: a simple multi-class classification algorithm based on the Bayes theorem. It assumes
that features are independent of each other. For a given sample feature 𝑋, the probability that a sample
belongs to a category 𝐻 is:

P ( X 1 ,  , X n | Ck ) P ( Ck )
P ( Ck | X 1 ,  , X n ) =
P ( X 1 , , X n )

 𝑋1 , … , 𝑋𝑛 are data features, which are usually described by measurement values of m attribute sets.
◼ For example, the color feature may have three attributes: red, yellow, and blue.

 𝐶𝑘 indicates that the data belongs to a specific category 𝐶

 𝑃 𝐶𝑘 |𝑋1 , … , 𝑋𝑛 is a posterior probability, or a posterior probability of under condition 𝐶𝑘 .
 𝑃 𝐶𝑘 is a prior probability that is independent of 𝑋1 , … , 𝑋𝑛
 𝑃 𝑋1 , … , 𝑋𝑛 is the priori probability of 𝑋.

85 Huawei Confidential
Naive Bayes (2)
⚫ Independent assumption of features.
 For example, if a fruit is red, round, and about 10 cm (3.94 in.) in diameter, it can be considered
an apple.
 A Naive Bayes classifier considers that each feature independently contributes to the probability
that the fruit is an apple, regardless of any possible correlation between the color, roundness,
and diameter.

86 Huawei Confidential
Ensemble Learning
⚫ Ensemble learning is a machine learning paradigm in which multiple learners are trained and combined to solve the same
problem. When multiple learners are used, the integrated generalization capability can be much stronger than that of a
single learner.

⚫ If you ask a complex question to thousands of people at random and then summarize their answers, the summarized answer
is better than an expert's answer in most cases. This is the wisdom of the masses.

Training set

Dataset 1 Dataset 2 Dataset m

Model 1 Model 2 Model m

Large
Model
model
synthesis

87 Huawei Confidential
Classification of Ensemble Learning

Bagging (Random Forest)

• Independently builds several basic learners and then averages
Bagging
their predictions.
• On average, a composite learner is usually better than a
single-base learner because of a smaller variance.
Ensemble learning

Boosting (Adaboost, GBDT, and XGboost)

Constructs basic learners in sequence to gradually
Boosting
reduce the bias of a composite learner. The composite
learner can fit data well, which may also cause
overfitting.

88 Huawei Confidential
Ensemble Methods in Machine Learning (1)
⚫ Random forest = Bagging + CART decision tree

⚫ Random forests build multiple decision trees and merge them together to make predictions more accurate
and stable.
 Random forests can be used for classification and regression problems.
Bootstrap sampling Decision tree building Aggregation prediction
result
Data subset 1 Prediction 1

Data subset 2 Prediction 2

• Category: majority
All training data voting Final prediction
• Regression:
Prediction average value

Data subset n Prediction n

89 Huawei Confidential
Ensemble Methods in Machine Learning (2)
⚫ GBDT is a type of boosting algorithm.
⚫ For an aggregative mode, the sum of the results of all the basic learners equals the predicted value. In
essence, the residual of the error function to the predicted value is fit by the next basic learner. (The residual
is the error between the predicted value and the actual value.)
⚫ During model training, GBDT requires that the sample loss for model prediction be as small as possible.

Prediction
30 years old 20 years old
Residual
calculation

Prediction
10 years old 9 years old
Residual
calculation

Prediction
1 year old 1 year old

90 Huawei Confidential
Unsupervised Learning - K-means
⚫ K-means clustering aims to partition n observations into k clusters in which each observation belongs to the
cluster with the nearest mean, serving as a prototype of the cluster.
⚫ For the k-means algorithm, specify the final number of clusters (k). Then, divide n data objects into k clusters.
The clusters obtained meet the following conditions: (1) Objects in the same cluster are highly similar. (2) The
similarity of objects in different clusters is small.

x1 x1

K-means clustering

The data is not tagged. K-means

clustering can automatically classify
datasets.
x2 x2

91 Huawei Confidential
Unsupervised Learning - Hierarchical Clustering
⚫ Hierarchical clustering divides a dataset at different layers and forms a tree-like clustering structure. The
dataset division may use a "bottom-up" aggregation policy, or a "top-down" splitting policy. The hierarchy of
clustering is represented in a tree graph. The root is the unique cluster of all samples, and the leaves are the
cluster of only a sample.

92 Huawei Confidential
 Contents

1. Machine Learning Definition

2. Machine Learning Types

3. Machine Learning Process

4. Other Key Machine Learning Methods

5. Common Machine Learning Algorithms

6. Case study

93 Huawei Confidential
Comprehensive Case
⚫ Assume that there is a dataset containing the house areas and prices of 21,613 housing
units sold in a city. Based on this data, we can predict the prices of other houses in the
city.
House Area Price
1,180 221,900
2,570 538,000
770 180,000
1,960 604,000
1,680 510,000
5,420 1,225,000 Dataset
1,715 257,500
1,060 291,850
1,160 468,000
1,430 310,000
1,370 400,000
1,810 530,000
… …

94 Huawei Confidential
Problem Analysis
⚫ This case contains a large amount of data, including input x (house area), and output y (price), which is a continuous value.
We can use regression of supervised learning. Draw a scatter chart based on the data and use linear regression.
⚫ Our goal is to build a model function h(x) that infinitely approximates the function that expresses true distribution of the
dataset.
⚫ Then, use the model to predict unknown price data.

x Unary linear regression function

Feature: house area
h( x) = wo + w1 x
Input

Price
Dataset Learning h(x)
algorithm

Output

y
Label: price
House area

95 Huawei Confidential
Goal of Linear Regression
⚫ Linear regression aims to find a straight line that best fits the dataset.
⚫ Linear regression is a parameter-based model. Here, we need learning parameters 𝑤0 and 𝑤1 .
When these two parameters are found, the best model appears.

Which line is the best parameter?

h( x) = wo + w1 x
Price

Price
House area House area

96 Huawei Confidential
Loss Function of Linear Regression
⚫ To find the optimal parameter, construct a loss function and find the parameter values
when the loss function becomes the minimum.

1
J ( w) =  ( − )
2
Loss function of linear h ( x ) y
regression: 2m

Error
Error
Error
Error
Goal:
Price

1
arg min J ( w) =  ( h( x ) − y )
2

w 2m
• where, m indicates the number of samples,
• h(x) indicates the predicted value, and y indicates
House area the actual value.

97 Huawei Confidential
Gradient Descent Method
⚫ The gradient descent algorithm finds the minimum value of a function through iteration.
⚫ It aims to randomize an initial point on the loss function, and then find the global minimum value of the loss
function based on the negative gradient direction. Such parameter value is the optimal parameter value.
 Point A: the position of 𝑤0 and 𝑤1 after random initialization.
𝑤0 and 𝑤1 are the required parameters.

 A-B connection line: a path formed based on descents in Cost surface

a negative gradient direction. Upon each descent, values 𝑤0

and 𝑤1 change, and the regression line also changes.

 Point B: global minimum value of the loss function.

Final values of 𝑤0 and 𝑤1 are also found.

98 Huawei Confidential
Iteration Example
⚫ The following is an example of a gradient descent iteration. We can see that as red points on the
loss function surface gradually approach a lowest point, fitting of the linear regression red line with
data becomes better and better. At this time, we can get the best parameters.

99 Huawei Confidential
Model Debugging and Application
⚫ After the model is trained, test it with the test set The final model result is as follows:
to ensure the generalization capability. h( x) = 280.62 x − 43581
⚫ If overfitting occurs, use Lasso regression or Ridge
regression with regularization terms and tune the
hyperparameters.

Price
⚫ If underfitting occurs, use a more complex
regression model, such as GBDT.
⚫ Note:
 For real data, pay attention to the functions of data
cleansing and feature engineering.
House area

100 Huawei Confidential

 Summary

⚫ First, this course describes the definition and classification of machine learning, as well as
problems machine learning solves. Then, it introduces key knowledge points of machine
learning, including the overall procedure (data collection, data cleansing, feature extraction,
model training and evaluation, and model deployment), common algorithms (linear
regression, logistic regression, decision tree, SVM, naive Bayes, KNN, ensemble learning, K-
means, etc.), gradient descent algorithm, parameters and hyper-parameters.
⚫ Finally, a complete machine learning process is presented by a case of using linear regression
to predict house prices.

101 Huawei Confidential

 Quiz

1. (True or false) Gradient descent iteration is the only method of machine learning algorithms. ( )
A. True

B. False

2. (Single-answer question) Which of the following algorithms is not supervised learning ? ( )

A. Linear regression

B. Decision tree

C. KNN

D. K-means

102 Huawei Confidential

 Recommendations

⚫ Online learning website

 https://e.huawei.com/en/talent/#/

⚫ Huawei Knowledge Base

 https://support.huawei.com/enterprise/en/knowledge?lang=en

103 Huawei Confidential

Thank you. 把数字世界带入每个人、每个家庭、
每个组织，构建万物互联的智能世界。
Bring digital to every person, home, and
organization for a fully connected,
intelligent world.

Copyright©2020 Huawei Technologies Co., Ltd.

All Rights Reserved.

The information in this document may contain predictive

statements including, without limitation, statements regarding
the future financial and operating results, future product
portfolio, new technology, etc. There are a number of factors that
could cause actual results and developments to differ materially
from those expressed or implied in the predictive statements.
Therefore, such information is provided for reference purpose
only and constitutes neither an offer nor an acceptance. Huawei
may change the information at any time without notice.
Deep Learning Overview
 Foreword

⚫ The chapter describes the basic knowledge of deep learning, including the
development history of deep learning, components and types of deep learning
neural networks, and common problems in deep learning projects.

2 Huawei Confidential
 Objectives

On completion of this course, you will be able to:

 Describe the definition and development of neural networks.
 Learn about the important components of deep learning neural networks.
 Understand training and optimization of neural networks.
 Describe common problems in deep learning.

3 Huawei Confidential
 Contents

1. Deep Learning Summary

2. Training Rules

3. Activation Function

4. Normalizer

5. Optimizer

6. Types of Neural Networks

7. Common Problems

4 Huawei Confidential
Traditional Machine Learning and Deep Learning
⚫ As a model based on unsupervised feature learning and feature hierarchy learning, deep learning
has great advantages in fields such as computer vision, speech recognition, and natural language
processing.

Traditional Machine Learning Deep Learning

Low hardware requirements on the computer: Given the Higher hardware requirements on the computer: To
limited computing amount, the computer does not need a execute matrix operations on massive data, the computer
GPU for parallel computing generally. needs a GPU to perform parallel computing.
Applicable to training under a small data amount and
The performance can be high when high-dimensional
whose performance cannot be improved continuously as
weight parameters and massive training data are provided.
the data amount increases.
Level-by-level problem breakdown E2E learning
Manual feature selection Algorithm-based automatic feature extraction

Easy-to-explain features Hard-to-explain features

5 Huawei Confidential
Traditional Machine Learning

Issue analysis
Problem locating

Data Feature Feature

cleansing extraction selection

Model
training

Question: Can we use an

algorithm to automatically Execute inference,
execute the procedure? prediction, and
identification

6 Huawei Confidential
Deep Learning
⚫ Generally, the deep learning architecture is a deep neural network. "Deep" in "deep
learning" refers to the number of layers of the neural network.

Dendrite Synapse Output layer

Nucleus
Hidden layer

Axon
Input layer

Human neural network Perceptron Deep neural network

7 Huawei Confidential
Neural Network
⚫ Currently, the definition of the neural network has not been determined yet. Hecht Nielsen, a neural network
researcher in the U.S., defines a neural network as a computer system composed of simple and highly
interconnected processing elements, which process information by dynamic response to external inputs.
⚫ A neural network can be simply expressed as an information processing system designed to imitate the human
brain structure and functions based on its source, features, and explanations.
⚫ Artificial neural network (neural network): Formed by artificial neurons connected to each other, the neural
network extracts and simplifies the human brain's microstructure and functions. It is an important approach to
simulate human intelligence and reflect several basic features of human brain functions, such as concurrent
information processing, learning, association, model classification, and memory.

8 Huawei Confidential
Development History of Neural Networks

Deep
SVM
XOR network
Perceptron MLP

Golden age AI winter

1958 1970 1986 1995 2006

9 Huawei Confidential
Single-Layer Perceptron
⚫ Input vector: 𝑋 = [𝑥0 , 𝑥1 , … , 𝑥𝑛 ]𝑇 𝑥1
⚫ Weight: 𝑊 = [𝜔0 , 𝜔1 , … , 𝜔𝑛 ]𝑇 , in which 𝜔0 is the offset. 𝑥2

𝑥𝑛
1, 𝑛𝑒𝑡 > 0, 𝑛
⚫ Activation function: 𝑂 = 𝑠𝑖𝑔𝑛 𝑛𝑒𝑡 = ቊ
−1, 𝑜𝑡ℎ𝑒𝑟𝑤𝑖𝑠𝑒. 𝑛𝑒𝑡 = ෍ 𝜔𝑖 𝑥𝑖 = 𝑾𝑻 𝐗
𝑖=0
⚫ The preceding perceptron is equivalent to a classifier. It uses the high-dimensional 𝑋 vector as the input and performs binary
classification on input samples in the high-dimensional space. When 𝑾𝑻 𝐗 > 0, O = 1. In this case, the samples are
classified into a type. Otherwise, O = −1. In this case, the samples are classified into the other type. The boundary of these
two types is 𝑾𝑻 𝐗 = 0, which is a high-dimensional hyperplane.

Classification point Classification line Classification plane Classification hyperplane

𝐴𝑥 + 𝐵 = 0 𝐴𝑥 + 𝐵𝑦 + 𝐶 = 0 𝐴𝑥 + 𝐵𝑦 + 𝐶𝑧 + 𝐷 = 0 𝑊𝑇X + 𝑏 = 0
10 Huawei Confidential
XOR Problem
⚫ In 1969, Minsky, an American mathematician and AI pioneer, proved that a perceptron is
essentially a linear model that can only deal with linear classification problems, but cannot
process non-linear data.

AND OR XOR

11 Huawei Confidential
 Feedforward Neural Network

Input layer Output layer

Hidden layer 1 Hidden layer 2

12 Huawei Confidential
Solution of XOR

w0

w1

w2

w3

w4
XOR w5
XOR

13 Huawei Confidential
Impacts of Hidden Layers on A Neural Network

0 hidden layers 3 hidden layers 20 hidden layers

14 Huawei Confidential
 Contents

1. Deep Learning Summary

2. Training Rules

3. Activation Function

4. Normalizer

5. Optimizer

6. Types of Neural Networks

7. Common Problems

15 Huawei Confidential
Gradient Descent and Loss Function
⚫ The gradient of the multivariate function 𝑜 = 𝑓 𝑥 = 𝑓 𝑥0 , 𝑥1 , … , 𝑥𝑛 at 𝑋 ′ = [𝑥0 ′ , 𝑥1 ′ , … , 𝑥𝑛 ′ ] 𝑇 is shown as follows:
𝜕𝑓 𝜕𝑓 𝜕𝑓 𝑇
𝛻𝑓 𝑥0 ′ , 𝑥1 ′ , … , 𝑥𝑛 ′ = [ , ,…, ] |𝑋=𝑋 ′ ,
𝜕𝑥0 𝜕𝑥1 𝜕𝑥𝑛

The direction of the gradient vector is the fastest growing direction of the function. As a result, the direction of the negative
gradient vector −𝛻𝑓 is the fastest descent direction of the function.
⚫ During the training of the deep learning network, target classification errors must be parameterized. A loss function (error
function) is used, which reflects the error between the target output and actual output of the perceptron. For a single
training sample x, the most common error function is the Quadratic cost function.
1
𝐸 𝑤 = σ𝑑∈𝐷 𝑡𝑑 − 𝑜𝑑 2 ,
2

In the preceding function, 𝑑 is one neuron in the output layer, D is all the neurons in the output layer, 𝑡𝑑 is the target output,
and 𝑜𝑑 is the actual output.
⚫ The gradient descent method enables the loss function to search along the negative gradient direction and update the
parameters iteratively, finally minimizing the loss function.

16 Huawei Confidential
Extrema of the Loss Function
⚫ Purpose: The loss function 𝐸(𝑊) is defined on the weight space. The objective is to search for the weight vector 𝑊 that can
minimize 𝐸(𝑊).

⚫ Limitation: No effective method can solve the extremum in mathematics on the complex high-dimensional surface of
1
𝐸 𝑊 = σ𝑑∈𝐷 𝑡𝑑 − 𝑜𝑑 2 .
2

Example of gradient descent of binary

paraboloid

17 Huawei Confidential
Common Loss Functions in Deep Learning
⚫ Quadratic cost function:

1 2
𝐸 𝑊 = ෍ 𝑡𝑑 − 𝑜𝑑
2
𝑑∈𝐷

⚫ Cross entropy error function:

𝐸 𝑊 = − ෍ 𝑡𝑑 ln 𝑜𝑑
𝑑∈𝐷

⚫ The cross entropy error function depicts the distance between two probability distributions, which
is a widely used loss function for classification problems.
⚫ Generally, the mean square error function is used to solve the regression problem, while the cross
entropy error function is used to solve the classification problem.

18 Huawei Confidential
Batch Gradient Descent Algorithm (BGD)
⚫ In the training sample set 𝑋, each sample is recorded as < x, 𝑡 >, in which 𝑋 is the input vector, 𝑡 the target
output, 𝑜 the actual output, and 𝜂 the learning rate.
 Initializes each 𝑤𝑖 to a random value with a smaller absolute value.
 Before the end condition is met:
◼ Initializes each ∆𝑤𝑖 to zero.
◼ For each iteration:
− Input all the 𝑥 to this unit and calculate the output 𝑜𝑋 .

1 𝜕C(𝑡𝑥 ,𝑜𝑥 )
− For each 𝑤𝑖 in this unit: ∆𝑤𝑖 += -η𝑛 σ𝑥∈𝑋 𝜕𝑤𝑖
.

◼ For each 𝑤𝑖 in this unit: 𝑤𝑖 += ∆𝑤𝑖 .

⚫ The gradient descent algorithm of this version is not commonly used because:
 The convergence process is very slow as all training samples need to be calculated every time the weight is updated.

19 Huawei Confidential
Stochastic Gradient Descent Algorithm (SGD)
⚫ To address the BGD algorithm defect, a common variant called Incremental Gradient Descent algorithm is used,
which is also called the Stochastic Gradient Descent (SGD) algorithm. One implementation is called Online
Learning, which updates the gradient based on each sample:

1 𝜕C(𝑡𝑥 ,𝑜𝑥 ) 𝜕C(𝑡𝑥 ,𝑜𝑥 )

∆𝑤𝑖 = −𝜂 𝑛 σx∈𝑋 ⟹ ∆𝑤𝑖 = −𝜂 .
𝜕𝑤𝑖 𝜕𝑤𝑖

⚫ ONLINE-GRADIENT-DESCENT
 Initializes each 𝑤𝑖 to a random value with a smaller absolute value.
 Before the end condition is met:
 Generates a random <x, t> from X and does the following calculation:
◼ Input X to this unit and calculate the output 𝑜𝑥 .

𝜕C(𝑡𝑥 ,𝑜𝑥 )
◼ For each 𝑤𝑖 in this unit: 𝑤𝑖 += −𝜂 𝜕𝑤𝑖
.

20 Huawei Confidential
Mini-Batch Gradient Descent Algorithm (MBGD)
⚫ To address the defects of the previous two gradient descent algorithms, the Mini-batch Gradient Descent
Algorithm (MBGD) was proposed and has been most widely used. A small number of Batch Size (BS) samples
are used at a time to calculate ∆𝑤𝑖 , and then the weight is updated accordingly.
⚫ Batch-gradient-descent
 Initializes each 𝑤𝑖 to a random value with a smaller absolute value.
 Before the end condition is met:
◼ Initializes each ∆𝑤𝑖 to zero.
◼ For each < x, 𝑡 > in the BS samples in the next batch in 𝐵:
− Input 𝑥 to this unit and calculate the output 𝑜𝑥 .

1 𝜕C(𝑡𝑥 ,𝑜𝑥 )
− For each 𝑤𝑖 in this unit: ∆𝑤𝑖 += -η𝑛 σ𝑥∈𝐵 𝜕𝑤𝑖

◼ For each 𝑤𝑖 in this unit: 𝑤𝑖 += ∆𝑤𝑖

◼ For the last batch, the training samples are mixed up in a random order.

21 Huawei Confidential
Backpropagation Algorithm (1)
⚫ Signals are propagated in forward direction, and
errors are propagated in backward direction. Forward propagation direction
⚫ In the training sample set D, each sample is recorded
as <X, t>, in which X is the input vector, t the target 𝑥1
output, o the actual output, and w the weight
𝑜1
coefficient. 𝑥2
𝑜2
⚫ Loss function:
𝑥3
Output layer
1
E ( w ) =  ( dD ) (td − od ) 2
Input layer
Hidden layer
2
Backpropagation direction

22 Huawei Confidential
Backpropagation Algorithm (2)
⚫ According to the following formulas, errors in the input, hidden, and output layers are accumulated to
generate the error in the loss function.
⚫ wc is the weight coefficient between the hidden layer and the output layer, while wb is the weight coefficient
between the input layer and the hidden layer. 𝑓 is the activation function, D is the output layer set, and C and
B are the hidden layer set and input layer set respectively. Assume that the loss function is a quadratic cost
function:
1
 Output layer error: E= 
2 ( dD )
(t d − od ) 2

2
1 1
E =  ( dD ) td − f (netd )  =  ( dD ) td − f ( ( cC ) wc yc ) 
2
 Expanded hidden
2 2  
layer error:

( )
2
1 
E =  ( dD ) td − f  ( cC ) wc f (netc )  =
 Expanded input layer 2  

( ( ) )
2
1 t − f
  (cC ) wc f  bB wb xb
error:
2 ( dD ) 
d

23 Huawei Confidential
Backpropagation Algorithm (3)
⚫ To minimize error E, the gradient descent iterative calculation can be used to solve 𝑊𝑐 and
𝑊𝑏 , that is, calculating 𝑊𝑐 and 𝑊𝑏 to minimize error E.
⚫ Formula:
E
wc = − ,c C
wc
E
wb = − ,b  B
wb
⚫ If there are multiple hidden layers, chain rules are used to take a derivative for each layer
to obtain the optimized parameters by iteration.

24 Huawei Confidential
Backpropagation Algorithm (4)
⚫ For a neural network with any number of layers, the arranged formula for training is as follows:

wljk = − kl +1 f j ( z lj )
 f j ' ( z lj )(t j − f j ( z lj )), l  outputs, (1)

 lj = 
 
 k
 l +1 l
k w f
jk j
'
( z l
j ), otherwise, (2)

⚫ The BP algorithm is used to train the network as follows:

 Takes out the next training sample <X, T>, inputs X to the network, and obtains the actual output o.
 Calculates output layer δ according to the output layer error formula (1).
 Calculates δ of each hidden layer from output to input by iteration according to the hidden layer error propagation
formula (2).
 According to the δ of each layer, the weight values of all the layer are updated.

25 Huawei Confidential
 Contents

1. Deep Learning Summary

2. Training Rules

3. Activation Function

4. Normalizer

5. Optimizer

6. Types of Neural Networks

7. Common Problems

26 Huawei Confidential
Activation Function
⚫ Activation functions are important for the neural network model to learn and understand
complex non-linear functions. They allow introduction of non-linear features to the
network.
⚫ Without activation functions, output signals are only simple linear functions. The
complexity of linear functions is limited, and the capability of learning complex function
mappings from data is low.

Activation Function

output = f ( w1 x1 + w2 x2 + w3 x3 ) = f (W • X )
t

27 Huawei Confidential
Sigmoid

1
𝑓 𝑥 =
1 + 𝑒 −𝑥

28 Huawei Confidential
Tanh

𝑒 𝑥 − 𝑒 −𝑥
tanh 𝑥 = 𝑥
𝑒 + 𝑒 −𝑥

29 Huawei Confidential
Softsign

𝑥
𝑓 𝑥 =
𝑥 +1

30 Huawei Confidential
Rectified Linear Unit (ReLU)
𝑥, 𝑥 ≥ 0
𝑦=ቊ
0, 𝑥 < 0

31 Huawei Confidential
Softplus

𝑓 𝑥 = ln 𝑒 𝑥 + 1

32 Huawei Confidential
Softmax
⚫ Softmax function:

𝑒 𝑧𝑗
σ(z)𝑗 =
σ 𝑘 𝑒 𝑧𝑘
⚫ The Softmax function is used to map a K-dimensional vector of arbitrary real values to
another K-dimensional vector of real values, where each vector element is in the interval
(0, 1). All the elements add up to 1.
⚫ The Softmax function is often used as the output layer of a multiclass classification task.

33 Huawei Confidential
 Contents

1. Deep Learning Summary

2. Training Rules

3. Activation Function

4. Normalizer

5. Optimizer

6. Types of Neural Networks

7. Common Problems

34 Huawei Confidential
Regularization
⚫ Regularization is an important and effective technology to reduce generalization errors in
machine learning. It is especially useful for deep learning models that tend to be over-fit
due to a large number of parameters. Therefore, researchers have proposed many
effective technologies to prevent over-fitting, including:
 Adding constraints to parameters, such as 𝐿1 and 𝐿2 norms
 Expanding the training set, such as adding noise and transforming data
 Dropout
 Early stopping

35 Huawei Confidential
Penalty Parameters
⚫ Many regularization methods restrict the learning capability of models by adding a penalty
parameter Ω(𝜃) to the objective function 𝐽. Assume that the target function after
regularization is 𝐽ሚ.

𝐽ሚ 𝜃; 𝑋, 𝑦 = 𝐽 𝜃; 𝑋, 𝑦 + 𝛼Ω(𝜃),
⚫ Where 𝛼𝜖[0, ∞) is a hyperparameter that weights the relative contribution of the norm
penalty term Ω and the standard objective function 𝐽(𝑋; 𝜃). If 𝛼 is set to 0, no
regularization is performed. The penalty in regularization increases with 𝛼.

36 Huawei Confidential
𝐿1 Regularization
⚫ Add 𝐿1 norm constraint to model parameters, that is,

𝐽ሚ 𝑤; 𝑋, 𝑦 = 𝐽 𝑤; 𝑋, 𝑦 + 𝛼 𝑤 1,

⚫ If a gradient method is used to resolve the value, the parameter gradient is

𝛻 𝐽ሚ 𝑤 =∝ 𝑠𝑖𝑔𝑛 𝑤 + 𝛻𝐽 𝑤 .

37 Huawei Confidential
𝐿2 Regularization
⚫ Add norm penalty term 𝐿2 to prevent overfitting.

ሚ𝐽 𝑤; 𝑋, 𝑦 = 𝐽 𝑤; 𝑋, 𝑦 + 1 𝛼 𝑤 2
2,
2

⚫ A parameter optimization method can be inferred using an optimization technology (such

as a gradient method):

𝑤 = 1 − 𝜀𝛼 𝜔 − 𝜀𝛻𝐽(𝑤),
⚫ where 𝜀 is the learning rate. Compared with a common gradient optimization formula, this
formula multiplies the parameter by a reduction factor.

38 Huawei Confidential
𝐿1 v.s. 𝐿2
⚫ The major differences between 𝐿2 and 𝐿1 :
 According to the preceding analysis, 𝐿1 can generate a more sparse model than 𝐿2 . When the value of parameter 𝑤 is small, 𝐿1
regularization can directly reduce the parameter value to 0, which can be used for feature selection.

 From the perspective of probability, many norm constraints are equivalent to adding prior probability distribution to parameters. In 𝐿2
regularization, the parameter value complies with the Gaussian distribution rule. In 𝐿1 regularization, the parameter value complies with
the Laplace distribution rule.

𝐿1 𝐿2
39 Huawei Confidential
Dataset Expansion
⚫ The most effective way to prevent over-fitting is to add a training set. A larger training set has a smaller over-
fitting probability. Dataset expansion is a time-saving method, but it varies in different fields.
 A common method in the object recognition field is to rotate or scale images. (The prerequisite to image transformation
is that the type of the image cannot be changed through transformation. For example, for handwriting digit recognition,
categories 6 and 9 can be easily changed after rotation).
 Random noise is added to the input data in speech recognition.
 A common practice of natural language processing (NLP) is replacing words with their synonyms.
 Noise injection can add noise to the input or to the hidden layer or output layer. For example, for Softmax classification,
noise can be added using the label smoothing technology. If noise is added to categories 0 and 1, the corresponding
𝜀 𝑘−1
probabilities are changed to 𝑘 and 1 − 𝜀 respectively.
𝑘

40 Huawei Confidential
Dropout
⚫ Dropout is a common and simple regularization method, which has been widely used since 2014. Simply put, dropout
randomly discards some inputs during the training process. In this case, the parameters corresponding to the discarded
inputs are not updated. As an integration method, Dropout combines all sub-network results and obtains sub-networks by
randomly dropping inputs. See the figures below:

Dropout in training Testing

41 Huawei Confidential
Early Stopping
⚫ A test on data of the validation set can be inserted during the training. When the data loss
of the verification set increases, perform early stopping.

Early stopping

42 Huawei Confidential
 Contents

1. Deep Learning Summary

2. Training Rules

3. Activation Function

4. Normalizer

5. Optimizer

6. Types of Neural Networks

7. Common Problems

43 Huawei Confidential
Optimizer
⚫ There are various optimized versions of gradient descent algorithms. In object-oriented
language implementation, different gradient descent algorithms are often encapsulated
into objects called optimizers.
⚫ Purposes of the algorithm optimization include but are not limited to:
 Accelerating algorithm convergence.
 Preventing or jumping out of local extreme values.
 Simplifying manual parameter setting, especially the learning rate (LR).

⚫ Common optimizers: common GD optimizer, momentum optimizer, Nesterov, AdaGrad,

AdaDelta, RMSProp, Adam, AdaMax, and Nadam.

44 Huawei Confidential
Momentum Optimizer
⚫ A most basic improvement is to add momentum terms for ∆𝑤𝑗𝑖 . Assume that the weight correction of the 𝑛-th iteration is ∆𝑤𝑗𝑖 (𝑛) . The
weight correction rule is:

⚫ ∆𝑤𝑗𝑖𝑙 𝑛 = −𝜂𝛿𝑖𝑙+1 𝑥𝑗𝑙 (𝑛) + 𝛼∆𝑤𝑗𝑖𝑙 𝑛 − 1 (In the MSE loss function).

⚫ where 𝛼 is a constant (0 ≤ 𝛼 < 1) called Momentum Coefficient and 𝛼∆𝑤𝑗𝑖 𝑛 − 1 is a momentum term.

⚫ Imagine a small ball rolls down from a random point on the error surface. The introduction of the momentum term is equivalent to giving the
small ball inertia.

−𝜂𝛿𝑖𝑙+1 𝑥𝑗𝑙 (𝑛)

45 Huawei Confidential
Advantages and Disadvantages of Momentum Optimizer
⚫ Advantages:
 Enhances the stability of the gradient correction direction and reduces mutations.
 In areas where the gradient direction is stable, the ball rolls faster and faster (there is a speed upper limit because 𝛼 <
1), which helps the ball quickly overshoot the flat area and accelerates convergence.
 A small ball with inertia is more likely to roll over some narrow local extrema.

⚫ Disadvantages:
 The learning rate 𝜂 and momentum 𝛼 need to be manually set, which often requires more experiments to determine
the appropriate value.

46 Huawei Confidential
AdaGrad Optimizer (1)
⚫ The common feature of the random gradient descent algorithm (SGD), small-batch gradient descent algorithm (MBGD), and
momentum optimizer is that each parameter is updated with the same LR.

⚫ According to the approach of AdaGrad, different learning rates need to be set for different parameters.
C (t , o)
gt = Gradient calculation
wt
rt = rt −1 + gt2 Square gradient accumulation

wt = − gt Computing update
 + rt
Application update
wt +1 =wt + wt
⚫ 𝑔𝑡 indicates the t-th gradient, 𝑟 is a gradient accumulation variable, and the initial value of 𝑟 is 0, which increases
continuously. 𝜂 indicates the global LR, which needs to be set manually. 𝜀 is a small constant, and is set to about 10-7 for
numerical stability.

47 Huawei Confidential
AdaGrad Optimizer (2)
⚫ The AdaGrad optimization algorithm shows that the 𝑟 continues increasing while the overall
learning rate keeps decreasing as the algorithm iterates. This is because we hope LR to decrease as
the number of updates increases. In the initial learning phase, we are far away from the optimal
solution to the loss function. As the number of updates increases, we are closer to the optimal
solution, and therefore LR can decrease.
⚫ Pros:
 The learning rate is automatically updated. As the number of updates increases, the learning rate
decreases.

⚫ Cons:
 The denominator keeps accumulating so that the learning rate will eventually become very small, and the
algorithm will become ineffective.

48 Huawei Confidential
RMSProp Optimizer
⚫ The RMSProp optimizer is an improved AdaGrad optimizer. It introduces an attenuation coefficient to ensure a certain
attenuation ratio for 𝑟 in each round.

⚫ The RMSProp optimizer solves the problem that the AdaGrad optimizer ends the optimization process too early. It is suitable
for non-stable target handling and has good effects on the RNN.
C (t , o)
gt =
wt Gradient calculation

rt = rt −1 + (1 −  ) gt2 Square gradient accumulation


wt = − gt Computing update
 + rt
wt +1 = wt + wt Application update
⚫ 𝑔𝑡 indicates the t-th gradient, 𝑟 is a gradient accumulation variable, and the initial value of 𝑟 is 0, which may not increase
and needs to be adjusted using a parameter. 𝛽 is the attenuation factor,𝜂 indicates the global LR, which needs to be set
manually. 𝜀 is a small constant, and is set to about 10-7 for numerical stability.

49 Huawei Confidential
Adam Optimizer (1)
⚫ Adaptive Moment Estimation (Adam): Developed based on AdaGrad and AdaDelta, Adam
maintains two additional variables 𝑚𝑡 and 𝑣𝑡 for each variable to be trained:
𝑚𝑡 = 𝛽1 𝑚𝑡−1 + (1 − 𝛽1 )𝑔𝑡
𝑣𝑡 = 𝛽2 𝑣𝑡−1 + (1 − 𝛽2 )𝑔𝑡2

⚫ Where 𝑡 represents the 𝑡-th iteration and 𝑔𝑡 is the calculated gradient. 𝑚𝑡 and 𝑣𝑡 are
moving averages of the gradient and square gradient. From the statistical perspective, 𝑚𝑡
and 𝑣𝑡 are estimates of the first moment (the average value) and the second moment (the
uncentered variance) of the gradients respectively, which also explains why the method is
so named.

50 Huawei Confidential
Adam Optimizer (2)
⚫ If 𝑚𝑡 and 𝑣𝑡 are initialized using the zero vector, 𝑚𝑡 and 𝑣𝑡 are close to 0 during the initial iterations, especially
when 𝛽1 and 𝛽2 are close to 1. To solve this problem, we use 𝑚
ෝ 𝑡 and 𝑣ො𝑡 :
𝑚𝑡
𝑚
ෝ𝑡 =
1 − 𝛽1𝑡
𝑣𝑡
𝑣ො𝑡 =
1 − 𝛽2𝑡

⚫ The weight update rule of Adam is as follows:

𝜂
𝑤𝑡+1 = 𝑤𝑡 − 𝑚
ෝ𝑡
𝑣ො𝑡 + 𝜖

⚫ Although the rule involves manual setting of 𝜂 , 𝛽1 , and 𝛽2 , the setting is much simpler. According to
experiments, the default settings are 𝛽1 = 0.9, 𝛽2 = 0.999, 𝜖 = 10−8 , and 𝜂 = 0.001. In practice, Adam will converge
quickly. When convergence saturation is reached, 𝜂 can be reduced. After several times of reduction, a
satisfying local extremum will be obtained. Other parameters do not need to be adjusted.

51 Huawei Confidential
Optimizer Performance Comparison

Comparison of optimization Comparison of optimization algorithms

algorithms in contour maps of loss at the saddle point
functions

52 Huawei Confidential
 Contents

1. Deep Learning Summary

2. Training Rules

3. Activation Function

4. Normalizer

5. Optimizer

6. Types of Neural Networks

7. Common Problems

53 Huawei Confidential
Convolutional Neural Network
⚫ A convolutional neural network (CNN) is a feedforward neural network. Its artificial neurons may
respond to surrounding units within the coverage range. CNN excels at image processing. It
includes a convolutional layer, a pooling layer, and a fully connected layer.
⚫ In the 1960s, Hubel and Wiesel studied cats' cortex neurons used for local sensitivity and direction
selection and found that their unique network structure could simplify feedback neural networks.
They then proposed the CNN.
⚫ Now, CNN has become one of the research hotspots in many scientific fields, especially in the
pattern classification field. The network is widely used because it can avoid complex pre-processing
of images and directly input original images.

54 Huawei Confidential
Main Concepts of CNN
⚫ Local receptive field: It is generally considered that human perception of the outside world is from
local to global. Spatial correlations among local pixels of an image are closer than those among
distant pixels. Therefore, each neuron does not need to know the global image. It only needs to
know the local image. The local information is combined at a higher level to generate global
information.
⚫ Parameter sharing: One or more filters/kernels may be used to scan input images. Parameters
carried by the filters are weights. In a layer scanned by filters, each filter uses the same parameters
during weighted computation. Weight sharing means that when each filter scans an entire image,
parameters of the filter are fixed.

55 Huawei Confidential
Architecture of Convolutional Neural Network
Input Three-feature Three-feature Five-feature Five-feature Output
image image image image image layer

Convolutional Pooling layer Convolutional Pooling Fully connected

layer layer layer layer

Bird Pbird

Sunset Psunset

Dog Pdog

Cat Pcat
Vectorization
Convolution + nonlinearity Max pooling

Multi-category
Convolution layers + pooling layers
Fully connected layer

56 Huawei Confidential
Single-Filter Calculation (1)
⚫ Description of convolution calculation

57 Huawei Confidential
Single-Filter Calculation (2)
⚫ Demonstration of the convolution calculation

Han Bingtao, 2017, Convolutional Neural Network

58 Huawei Confidential
Convolutional Layer
⚫ The basic architecture of a CNN is multi-channel convolution consisting of multiple single convolutions. The output of the
previous layer (or the original image of the first layer) is used as the input of the current layer. It is then convolved with the
filter in the layer and serves as the output of this layer. The convolution kernel of each layer is the weight to be learned.
Similar to FCN, after the convolution is complete, the result should be biased and activated through activation functions
before being input to the next layer.

Wn bn
Fn

Input Output
tensor tensor
F1
W2 b2 Activate
Output

W1 b1
Convolutional Bias
kernel

59 Huawei Confidential
Pooling Layer
⚫ Pooling combines nearby units to reduce the size of the input on the next layer, reducing dimensions. Common pooling
includes max pooling and average pooling. When max pooling is used, the maximum value in a small square area is selected
as the representative of this area, while the mean value is selected as the representative when average pooling is used. The
side of this small area is the pool window size. The following figure shows the max pooling operation whose pooling window
size is 2.

Sliding direction

60 Huawei Confidential
Fully Connected Layer
⚫ The fully connected layer is essentially a classifier. The features extracted on the
convolutional layer and pooling layer are flattened and placed at the fully connected layer
to output and classify results.
⚫ Generally, the Softmax function is used as the activation function of the final fully
connected output layer to combine all local features into global features and calculate the
score of each type.

𝑒 𝑧𝑗
σ(z)𝑗 =
σ 𝑘 𝑒 𝑧𝑘

61 Huawei Confidential
Recurrent Neural Network
⚫ The recurrent neural network (RNN) is a neural network that captures dynamic information in
sequential data through periodical connections of hidden layer nodes. It can classify sequential
data.
⚫ Unlike other forward neural networks, the RNN can keep a context state and even store, learn, and
express related information in context windows of any length. Different from traditional neural
networks, it is not limited to the space boundary, but also supports time sequences. In other
words, there is a side between the hidden layer of the current moment and the hidden layer of the
next moment.
⚫ The RNN is widely used in scenarios related to sequences, such as videos consisting of image
frames, audio consisting of clips, and sentences consisting of words.

62 Huawei Confidential
Recurrent Neural Network Architecture (1)
⚫ 𝑋𝑡 is the input of the input sequence at time t.
⚫ 𝑆𝑡 is the memory unit of the sequence at time t and caches previous
information.

𝑆𝑡 = 𝑡𝑎𝑛ℎ 𝑈𝑋𝑡 + 𝑊𝑆𝑡−1 .

⚫ 𝑂𝑡 is the output of the hidden layer of the sequence at time t.

𝑂𝑡 = 𝑡𝑎𝑛ℎ 𝑉𝑆𝑡
⚫ 𝑂𝑡 after through multiple hidden layers, it can get the final output of the
sequence at time t.

63 Huawei Confidential
Recurrent Neural Network Architecture (2)

LeCun, Bengio, and G. Hinton, 2015, A Recurrent Neural Network and the
Unfolding in Time of the Computation Involved in Its Forward Computation

64 Huawei Confidential
Types of Recurrent Neural Networks

Andrej Karpathy, 2015, The Unreasonable Effectiveness of Recurrent Neural Networks

65 Huawei Confidential
Backpropagation Through Time (BPTT)
⚫ BPTT:
 Traditional backpropagation is the extension on the time sequence.

 There are two sources of errors in the sequence at time of memory unit: first is from the hidden layer output error at t time sequence;
the second is the error from the memory cell at the next time sequence t + 1.
 The longer the time sequence, the more likely the loss of the last time sequence to the gradient of w in the first time sequence causes
the vanishing gradient or exploding gradient problem.
 The total gradient of weight w is the accumulation of the gradient of the weight at all time sequence.

⚫ Three steps of BPTT:

 Computing the output value of each neuron through forward propagation.

 Computing the error value of each neuron through backpropagation 𝛿𝑗 .

 Computing the gradient of each weight.

⚫ Updating weights using the SGD algorithm.

66 Huawei Confidential
Recurrent Neural Network Problem
⚫ 𝑆𝑡 = 𝜎 𝑈𝑋𝑡 + 𝑊𝑆𝑡−1 is extended on the time sequence.

⚫ 𝑆𝑡 = σ 𝑈𝑋𝑡 + 𝑊 𝜎 𝑈𝑋𝑡−1 + 𝑊 𝜎 𝑈𝑋𝑡−2 + 𝑊 …

⚫ Despite that the standard RNN structure solves the problem of information memory, the
information attenuates during long-term memory.
⚫ Information needs to be saved long time in many tasks. For example, a hint at the beginning of a
speculative fiction may not be answered until the end.
⚫ The RNN may not be able to save information for long due to the limited memory unit capacity.
⚫ We expect that memory units can remember key information.

67 Huawei Confidential
Long Short-term Memory Network

Colah, 2015, Understanding LSTMs Networks

68 Huawei Confidential
Gated Recurrent Unit (GRU)

69 Huawei Confidential
Generative Adversarial Network (GAN)
⚫ Generative Adversarial Network is a framework that trains generator G and discriminator D through the adversarial process.
Through the adversarial process, the discriminator can tell whether the sample from the generator is fake or real. GAN
adopts a mature BP algorithm.

⚫ (1) Generator G: The input is noise z, which complies with manually selected prior probability distribution, such as even
distribution and Gaussian distribution. The generator adopts the network structure of the multilayer perceptron (MLP), uses
maximum likelihood estimation (MLE) parameters to represent the derivable mapping G(z), and maps the input space to the
sample space.

⚫ (2) Discriminator D: The input is the real sample x and the fake sample G(z), which are tagged as real and fake respectively.
The network of the discriminator can use the MLP carrying parameters. The output is the probability D(G(z)) that determines
whether the sample is a real or fake sample.

⚫ GAN can be applied to scenarios such as image generation, text generation, speech enhancement, image super-resolution.

70 Huawei Confidential
GAN Architecture
⚫ Generator/Discriminator

71 Huawei Confidential
Generative Model and Discriminative Model
⚫ Generative network ⚫ Discriminator network
 Generates sample data  Determines whether sample data is real
◼ Input: Gaussian white noise vector z ◼ Input: real sample data 𝑥𝑟𝑒𝑎𝑙 and generated
◼ Output: sample data vector x sample data 𝑥 = 𝐺 𝑧
◼ Output: probability that determines whether
the sample is real

x = G ( z; )G
y = D( x; D )
𝑥𝑟𝑒𝑎𝑙
G
z x D y
x

72 Huawei Confidential
Training Rules of GAN
⚫ Optimization objective:
 Value function

min maxV ( D, G ) = Ex  pdata（x）[logD( x)] + Ez  pz( z ) [log (1 − D(G ( z )))]

G D

 In the early training stage, when the outcome of G is very poor, D determines that the
generated sample is fake with high confidence, because the sample is obviously different from
training data. In this case, log(1-D(G(z))) is saturated (where the gradient is 0, and iteration
cannot be performed). Therefore, we choose to train G only by minimizing [-log(D(G(z))].

73 Huawei Confidential
 Contents

1. Deep Learning Summary

2. Training Rules

3. Activation Function

4. Normalizer

5. Optimizer

6. Types of Neural Networks

7. Common Problems

74 Huawei Confidential
Data Imbalance (1)
⚫ Problem description: In the dataset consisting of various task categories, the number of samples
varies greatly from one category to another. One or more categories in the predicted categories
contain very few samples.
⚫ For example, in an image recognition experiment, more than 2,000 categories among a total of
4251 training images contain just one image each. Some of the others have 2-5 images.
⚫ Impacts:
 Due to the unbalanced number of samples, we cannot get the optimal real-time result because
model/algorithm never examines categories with very few samples adequately.
 Since few observation objects may not be representative for a class, we may fail to obtain adequate
samples for verification and test.

75 Huawei Confidential
Data Imbalance (2)

Random Random Synthetic

undersampling oversampling Minority
• Deleting redundant • Copying samples Oversampling
samples in a category Technique
• Sampling
• Merging samples

76 Huawei Confidential
Vanishing Gradient and Exploding Gradient Problem (1)
⚫ Vanishing gradient: As network layers increase, the derivative value of backpropagation
decreases, which causes a vanishing gradient problem.
⚫ Exploding gradient: As network layers increase, the derivative value of backpropagation
increases, which causes an exploding gradient problem.
⚫ Cause: y𝑖 = 𝜎(𝑧𝑖) = 𝜎 𝑤𝑖 𝑥𝑖 + 𝑏𝑖 Where σ is sigmoid function.

w2 w3 w4
b1 b2 b3 C

⚫ Backpropagation can be deduced as follows:

𝜕C 𝜕C 𝜕𝑦4 𝜕𝑧4 𝜕𝑥4 𝜕𝑧3 𝜕𝑥3 𝜕𝑧2 𝜕𝑥2 𝜕𝑧1
=
𝜕𝑏1 𝜕𝑦4 𝜕𝑧4 𝜕𝑥4 𝜕𝑧3 𝜕𝑥3 𝜕𝑧2 𝜕𝑥2 𝜕𝑧1 𝜕𝑏1
𝜕C
= 𝜕𝑦 𝜎 ′ 𝑧4 𝑤4 𝜎 ′ 𝑧3 𝑤3 𝜎 ′ 𝑧2 𝑤2 𝜎 ′ 𝑧1 𝑥
4

78 Huawei Confidential
Vanishing Gradient and Exploding Gradient Problem (2)

1
⚫ The maximum value of 𝜎 ′ (𝑥) is :
4

1
⚫ However, the network weight 𝑤 is usually smaller than 1. Therefore, 𝜎 ′ 𝑧 𝑤 ≤ . According to the chain rule, as layers
4
𝜕C
increase, the derivation result 𝜕𝑏 decreases, resulting in the vanishing gradient problem.
1

⚫ When the network weight 𝑤 is large, resulting in 𝜎 ′ 𝑧 𝑤 > 1, the exploding gradient problem occurs.

⚫ Solution: For example, gradient clipping is used to alleviate the exploding gradient problem, ReLU activation function and
LSTM are used to alleviate the vanishing gradient problem.

79 Huawei Confidential
Overfitting
⚫ Problem description: The model performs well in the training set, but badly in the test set.
⚫ Root cause: There are too many feature dimensions, model assumptions, and parameters,
too much noise, but very few training data. As a result, the fitting function perfectly
predicts the training set, while the prediction result of the test set of new data is poor.
Training data is over-fitted without considering generalization capabilities.
⚫ Solution: For example, data augmentation, regularization, early stopping, and dropout

80 Huawei Confidential
 Summary

⚫ This chapter describes the definition and development of neural networks,

perceptrons and their training rules, common types of neural networks (CNN, RNN,
and GAN), and the Common Problems of neural networks in AI engineering and
solutions.

82 Huawei Confidential
 Quiz

1. (True or false) Compared with the recurrent neural network, the convolutional neural
network is more suitable for image recognition. ( )
A. True

B. False

2. (True or false) GAN is a deep learning model, which is one of the most promising methods
for unsupervised learning of complex distribution in recent years. ( )
A. True

B. False

83 Huawei Confidential
 Quiz
3. (Single-choice) There are many types of deep learning neural networks. Which of the following is not a deep learning neural
network? ( )
A. CNN

B. RNN

C. LSTM

D. Logistic

4. (Multi-choice) There are many important "components" in the convolutional neural network architecture. Which of the following
are the convolutional neural network "components"? ( )
A. Activation function

B. Convolutional kernel

C. Pooling

D. Fully connected layer

84 Huawei Confidential
 Recommendations

⚫ Online learning website

 https://e.huawei.com/cn/talent/#/home

⚫ Huawei Knowledge Base

 https://support.huawei.com/enterprise/servicecenter?lang=zh

85 Huawei Confidential
Thank you. 把数字世界带入每个人、每个家庭、
每个组织，构建万物互联的智能世界。
Bring digital to every person, home, and
organization for a fully connected,
intelligent world.

Copyright©2020 Huawei Technologies Co., Ltd.

All Rights Reserved.

The information in this document may contain predictive

statements including, without limitation, statements regarding
the future financial and operating results, future product
portfolio, new technology, etc. There are a number of factors that
could cause actual results and developments to differ materially
from those expressed or implied in the predictive statements.
Therefore, such information is provided for reference purpose
only and constitutes neither an offer nor an acceptance. Huawei
may change the information at any time without notice.
Mainstream Development Frameworks in the Industry
 Foreword

⚫ This chapter describes:

 Definition of deep learning framework and its advantages, and two mainstream deep
learning frameworks PyTorch and TensorFlow
 Basic operations and common modules of TensorFlow 2.x (by focusing on code)
 MNIST handwritten digit recognition experiment performed based on TensorFlow for
deeply understanding and getting familiar with a deep learning modeling process

2 Huawei Confidential
 Objectives

On completion of this course, you will be able to:

 Describe a deep learning framework.
 Know mainstream deep learning frameworks.
 Know the features of PyTorch.
 Know the features of TensorFlow.
 Differentiate between TensorFlow 1.x and 2.x.
 Master the basic syntax and common modules of TensorFlow 2.x.
 Master the process of an MNIST handwritten digit recognition experiment.

3 Huawei Confidential
 Contents

1. Mainstream Development Frameworks

◼ Deep Learning Framework

▫ PyTorch

▫ TensorFlow

2. TensorFlow 2.x Basics

3. Common Modules of TensorFlow 2.x

4. Basic Steps of Deep Learning Development

4 Huawei Confidential
Deep Learning Framework
⚫ A deep learning framework is an interface, library or a tool which allows us
to build deep learning models more easily and quickly, without getting into
the details of underlying algorithms. A deep learning framework can be
regarded as a set of building blocks. Each component in the building blocks is
a model or algorithm. Therefore, developers can use components to
assemble models that meet requirements, and do not need to start from
scratch.
⚫ The emergence of deep learning frameworks lowers the requirements for
developers. Developers no longer need to compile code starting from
complex neural networks and back-propagation algorithms. Instead, they can
use existing models to configure parameters as required, where the model
parameters are automatically trained. Moreover, they can add self-defined
network layers to the existing models, or select required classifiers and
optimization algorithms directly by invoking existing code.

5 Huawei Confidential
 Contents

1. Mainstream Development Frameworks

▫ Deep Learning Framework
◼ PyTorch

▫ TensorFlow

2. TensorFlow 2.x Basics

3. Common Modules of TensorFlow 2.x

4. Basic Steps of Deep Learning Development

6 Huawei Confidential
PyTorch
⚫ PyTorch is a Python-based machine learning computing framework developed by Facebook. It is
developed based on Torch, a scientific computing framework supported by a large number of
machine learning algorithms. Torch is a tensor operation library similar to NumPy, featured by high
flexibility, but is less popular because it uses the programming language Lua. This is why PyTorch is
developed.
⚫ In addition to Facebook, institutes such as Twitter, GMU also use PyTorch.

Image source: http://PyTorch123.com/FirstSection/PyTorchIntro/

7 Huawei Confidential
Features of PyTorch
⚫ Python first: PyTorch does not simply bind Python to a C++ framework. PyTorch directly supports Python
access at a fine grain. Developers can use PyTorch as easily as using NumPy or SciPy. This not only lowers the
threshold for understanding Python, but also ensures that the code is basically consistent with the native
Python implementation.
⚫ Dynamic neural network: Many mainstream frameworks such as TensorFlow 1.x do not support this feature.
To run TensorFlow 1.x, developers must create static computational graphs in advance, and run the feed and
run commands to repeatedly execute the created graphs. In contrast, PyTorch with this feature is free from
such complexity, and PyTorch programs can dynamically build/adjust computational graphs during execution.
⚫ Easy to debug: PyTorch can generate dynamic graphs during execution. Developers can stop an interpreter in a
debugger and view output of a specific node.
⚫ PyTorch provides tensors that support CPUs and GPUs, greatly accelerating computing.

8 Huawei Confidential
 Contents

1. Mainstream Development Frameworks

▫ Deep Learning Framework

▫ PyTorch
◼ TensorFlow

2. TensorFlow 2.x Basics

3. Common Modules of TensorFlow 2.x

4. Basic Steps of Deep Learning Development

9 Huawei Confidential
TensorFlow
⚫ TensorFlow is Google's second-generation open-source software library for digital
computing. The TensorFlow computing framework supports various deep learning
algorithms and multiple computing platforms, ensuring high system stability.

Image source: https://www.TensorFlow.org/

10 Huawei Confidential
Features of TensorFlow

Scalability Multi-lingual

GPU Multi-platform

Powerful computing Distributed

11 Huawei Confidential
TensorFlow - Distributed
⚫ TensorFlow can run on different computers:
 From smartphones to computer clusters, to generate desired training models.

⚫ Currently, supported native distributed deep learning frameworks include only TensorFlow,
CNTK, Deeplearning4J, and MXNet.
⚫ When a single GPU is used, most deep learning frameworks rely on cuDNN, and therefore
support almost the same training speed, provided that the hardware computing
capabilities or allocated memories slightly differ. However, for large-scale deep learning,
massive data makes it difficult for the single GPU to complete training in a limited time. To
handle such cases, TensorFlow enables distributed training.

12 Huawei Confidential
Why TensorFlow?
⚫ TensorFlow is considered as one of the best libraries for
neural networks, and can reduce difficulty in deep
learning development. In addition, as it is open-source, it
can be conveniently maintained and updated, thus the
efficiency of development can be improved.
⚫ Keras, ranking third in the number of stars on GitHub, is
packaged into an advanced API of TensorFlow 2.0, which
makes TensorFlow 2.x more flexible, and easier to debug.

Demand on the
recruitment market

13 Huawei Confidential
TensorFlow 2.x vs. TensorFlow 1.x
⚫ Disadvantages of TensorFlow 1.0:
 After a tensor is created in TensorFlow 1.0, the result cannot be returned directly. To obtain the
result, the session mechanism needs to be created, which includes the concept of graph, and
code cannot run without session.run. This style is more like the hardware programming
language VHDL.
 Compared with some simple frameworks such as PyTorch, TensorFlow 1.0 adds the session and
graph concepts, which are inconvenient for users.
 It is complex to debug TensorFlow 1.0, and its APIs are disordered, making it difficult for
beginners. Learners will come across many difficulties in using TensorFlow 1.0 even after gaining
the basic knowledge. As a result, many researchers have turned to PyTorch.

14 Huawei Confidential
TensorFlow 2.x vs. TensorFlow 1.x
⚫ Features of TensorFlow 2.x:
 Advanced API Keras:
◼ Easy to use: The graph and session mechanisms are removed. What you see is what you get, just like Python and PyTorch.

 Major improvements:
◼ The core function of TensorFlow 2.x is the dynamic graph mechanism called eager execution. It allows users to compile and debug
models like normal programs, making TensorFlow easier to learn and use.
◼ Multiple platforms and languages are supported, and compatibility between components can be improved via standardization on
exchange formats and alignment of APIs.
◼ Deprecated APIs are deleted and duplicate APIs are reduced to avoid confusion.
◼ Compatibility and continuity: TensorFlow 2.x provides a module enabling compatibility with TensorFlow 1.x.
◼ The tf.contrib module is removed. Maintained modules are moved to separate repositories. Unused and unmaintained modules are
removed.

15 Huawei Confidential
 Contents

1. Mainstream Development Frameworks

2. TensorFlow 2.x Basics

3. Common Modules of TensorFlow 2.x

4. Basic Steps of Deep Learning Development

16 Huawei Confidential
Tensors
⚫ Tensors are the most basic data structures in
TensorFlow. All data is encapsulated in tensors.
⚫ Tensor: a multidimensional array
One- Two- Three-
 A scalar is a rank-0 tensor. A vector is a rank-1 tensor. A
dimensional dimensional dimensional
matrix is a rank-2 tensor. tensor tensor tensor
⚫ In TensorFlow, tensors are classified into:
 Constant tensors
 Variable tensors
Four- Five- Six-
dimensional dimensional dimensional
tensor tensor tensor

17 Huawei Confidential
Basic Operations of TensorFlow 2.x
⚫ The following describes common APIs in TensorFlow by focusing on code. The main
content is as follows:
 Methods for creating constants and variables
 Tensor slicing and indexing
 Dimension changes of tensors
 Arithmetic operations on tensors
 Tensor concatenation and splitting
 Tensor sorting

18 Huawei Confidential
Eager Execution Mode of TensorFlow 2.x
⚫ Static graph: TensorFlow 1.x using static graphs (graph mode) separates computation definition and
execution by using computational graphs. This is a declarative programming model. In graph mode,
developers need to build a computational graph, start a session, and then input data to obtain an
execution result.
⚫ Static graphs are advantageous in distributed training, performance optimization, and deployment,
but inconvenient for debugging. Executing a static graph is similar to invoking a compiled C
language program, and internal debugging cannot be performed in this case. Therefore, eager
execution based on dynamic computational graphs emerges.
⚫ Eager execution is a command-based programming method, which is the same as native Python. A
result is returned immediately after an operation is performed.

19 Huawei Confidential
AutoGraph
⚫ Eager execution is enabled in TensorFlow 2.x by default. Eager execution is intuitive and
flexible for users (easier and faster to run a one-time operation), but may compromise
performance and deployability.
⚫ To achieve optimal performance and make a model deployable anywhere, you can run
@tf.function to add a decorator to build a graph from a program, making Python code
more efficient.
⚫ tf.function can build a TensorFlow operation in the function into a graph. In this way, this
function can be executed in graph mode. Such practice can be considered as encapsulating
the function as a TensorFlow operation of a graph.

20 Huawei Confidential
 Contents

1. Mainstream Development Frameworks

2. TensorFlow 2.x Basics

3. Common Modules of TensorFlow 2.x

4. Basic Steps of Deep Learning Development

21 Huawei Confidential
Common Modules of TensorFlow 2.x (1)
⚫ tf: Functions in the tf module are used to perform common arithmetic operations, such as tf.abs
(calculating an absolute value), tf.add (adding elements one by one), and tf.concat (concatenating
tensors). Most operations in this module can be performed by NumPy.
⚫ tf.errors: error type module of TensorFlow
⚫ tf.data: implements operations on datasets.
 Input pipes created by tf.data are used to read training data. In addition, data can be easily input from
memories (such as NumPy).

⚫ tf.distributions: implements various statistical distributions.

 The functions in this module are used to implement various statistical distributions, such as Bernoulli
distribution, uniform distribution, and Gaussian distribution.

22 Huawei Confidential
Common Modules of TensorFlow 2.x (2)
⚫ tf.io.gfile: implements operations on files.
 Functions in this module can be used to perform file I/O operations, copy files, and rename files.

⚫ tf.image: implements operations on images.

 Functions in this module include image processing functions. This module is similar to OpenCV, and
provides functions related to image luminance, saturation, phase inversion, cropping, resizing, image
format conversion (RGB to HSV, YUV, YIQ, or gray), rotation, and sobel edge detection. This module is
equivalent to a small image processing package of OpenCV.

⚫ tf.keras: a Python API for invoking Keras tools.

 This is a large module that enables various network operations.

23 Huawei Confidential
Keras Interface
⚫ TensorFlow 2.x recommends Keras for network building. Common neural networks are included in
Keras.layers.
⚫ Keras is a high-level API used to build and train deep learning models. It can be used for rapid prototype
design, advanced research, and production. It has the following three advantages:
 Easy to use
Keras provides simple and consistent GUIs optimized for common cases. It provides practical and clear feedback on user
errors.
 Modular and composable
You can build Keras models by connecting configurable building blocks together, with little restriction.
 Easy to extend
You can customize building blocks to express new research ideas, create layers and loss functions, and develop advanced
models.

24 Huawei Confidential
Common Keras Methods and Interfaces
⚫ The following describes common methods and interfaces of tf.keras by focusing on code.
The main content is as follows:
 Dataset processing: datasets and preprocessing
 Neural network model creation: Sequential, Model, Layers...
 Network compilation: compile, Losses, Metrics, and Optimizers
 Network training and evaluation: fit, fit_generator, and evaluate

25 Huawei Confidential
 Contents

1. Mainstream Development Frameworks

2. TensorFlow 2.x Basics

3. Common Modules of TensorFlow 2.x

4. Basic Steps of Deep Learning Development

26 Huawei Confidential
TensorFlow Environment Setup in Windows 10
⚫ Environment setup in Windows 10:
 Operating system: Windows 10
 pip software built in Anaconda 3 (adapting to Python 3)
 TensorFlow installation:
◼ Open Anaconda Prompt and run the pip command to install TensorFlow.
◼ Run pip install TensorFlow in the command line interface.

27 Huawei Confidential
TensorFlow Environment Setup in Ubuntu/Linux
⚫ The simplest way for installing TensorFlow in Linux is to run the pip command.

⚫ pip command: pip install TensorFlow==2.1.0

28 Huawei Confidential
TensorFlow Development Process
⚫ Data preparation
 Data exploration Data preparation
Model Model Model deployment
 Data processing training verification and application
Model definition
⚫ Network construction
 Defining a network structure.
 Defining loss functions, selecting optimizers, and defining model evaluation indicators.

⚫ Model training and verification

⚫ Model saving
⚫ Model restoration and invoking

29 Huawei Confidential
Project Description
⚫ Handwritten digit recognition is a common image recognition task where computers recognize text in
handwriting images. Different from printed fonts, handwriting of different people has different sizes and
styles, making it difficult for computers to recognize handwriting. This project applies deep learning and
TensorFlow tools to train and build models based on the MNIST handwriting dataset.

1
Handwritten digit recognition

5
30 Huawei Confidential
Data Preparation
⚫ MNIST datasets
 Download the MNIST datasets from http://yann.lecun.com/exdb/mnist/.
 The MNIST datasets consist of a training set and a test set.
◼ Training set: 60,000 handwriting images and corresponding labels
◼ Test set: 10,000 handwriting images and corresponding labels

Examples

Corresponding labels
[0,0,0,0,0,1, [0,0,0,0,0, [0,0,0,0,0,0 [0,0,0,1,0, [0,0,0,0,1,0,
0,0,0,0] 0,0,0,0,1] ,0,1,0,0] 0,0,0,0,0] 0,0,0,0]

31 Huawei Confidential
Network Structure Definition (1)
⚫ Softmax regression model
evidencei = Wi , j x j + bi
j

y = soft max(evidence)
⚫ The softmax function is also called normalized exponential function. It is a generalization of the binary
classification function sigmoid in terms of multi-class classification. The following figure shows the calculation
method of softmax.

32 Huawei Confidential
Network Structure Definition (2)
⚫ The process of model establishment is the core process of network structure definition.
⚫ The network operation process defines how model output is calculated based on input.

⚫ Matrix multiplication and vector addition are used to express the calculation process of softmax.

33 Huawei Confidential
Network Structure Definition (3)
⚫ TensorFlow-based softmax regression model

## import tensorflow
import tensorflow as tf
##define input variables with operator symbol variables.
‘’’ we use a variable to feed data into the graph through the placeholders X. Each input image is
flattened into a 784-dimensional vector. In this case, the shape of the tensor is [None, 784], None
indicates can be of any length. ’’’
X = tf.placeholder(tf.float32,[None,784])
‘’’ The variable that can be modified is used to indicate the weight w and bias b. The initial values are
set to 0. ’’’
w = tf.Variable(tf.zeros([784,10]))
b = tf.Variable(tf.zeros([10]))
‘’’ If tf.matmul(x, w) is used to indicate that x is multiplied by w, the Soft regression equation is y =
softmax(wx+b)'‘’
y = tf.nn.softmax(tf.matmul(x,w)+b)

34 Huawei Confidential
Network Compilation
⚫ Model compilation involves the following two parts:
 Loss function selection

⚫ In machine learning/deep learning, an indicator needs to be defined to indicate whether a model is proper. This indicator is
called cost or loss, and is minimized as far as possible. In this project, the cross entropy loss function is used.
 Gradient descent method

⚫ A loss function is constructed for an original model needs to be optimized by using an optimization algorithm, to find
optimal parameters and further minimize a value of the loss function. Among optimization algorithms for solving machine
learning parameters, the gradient descent-based optimization algorithm (Gradient Descent) is usually used.

model.compile(optimizer=tf.train.AdamOptimizer(),
loss=tf.keras.losses.categorical_crossentropy,
metrics=[tf.keras.metrics.categorical_accuracy])

35 Huawei Confidential
Model Training
⚫ Training process:
 All training data is trained through batch iteration or full iteration. In the experiment, all data is
trained five times.
 In TensorFlow, model.fit is used for training, where epoch indicates the number of training
iterations.

36 Huawei Confidential
Model Evaluation
⚫ You can test the model using the test set, compare predicted results with actual ones, and
find correctly predicted labels, to calculate the accuracy of the test set.

Loss value Accuracy

37 Huawei Confidential
 Quiz

1. In TensorFlow 2.x, eager execution is enabled by default. ( )

A. True

B. False

2. Which of the following statements about tf.keras.Model and tf.keras.Sequential is incorrect when the
tf.keras interface is used to build a network model? ( )
A. tf.keras.Model supports network models with multiple inputs, while tf.keras.Sequential does not.

B. tf.keras.Model supports network models with multiple outputs, while tf.keras.Sequential does not.

C. tf.keras.Model is recommended for model building when a sharing layer exists on the network.

D. tf.keras.Sequential is recommended for model building when a sharing layer exists on the network.

38 Huawei Confidential
 Summary

⚫ This chapter describes the following content by focusing on code: Features of

common deep learning frameworks, including PyTorch and TensorFlow
Basic syntax and common modules of TensorFlow 2.x Development procedure of
TensorFlow.

39 Huawei Confidential
 More Information

Official TensorFlow website: https://tensorflow.google.cn

Official PyTorch website: https://PyTorch.org/

40 Huawei Confidential
Thank you. 把数字世界带入每个人、每个家庭、
每个组织，构建万物互联的智能世界。
Bring digital to every person, home, and
organization for a fully connected,
intelligent world.

Copyright©2020 Huawei Technologies Co., Ltd.

All Rights Reserved.

The information in this document may contain predictive

statements including, without limitation, statements regarding
the future financial and operating results, future product
portfolio, new technology, etc. There are a number of factors that
could cause actual results and developments to differ materially
from those expressed or implied in the predictive statements.
Therefore, such information is provided for reference purpose
only and constitutes neither an offer nor an acceptance. Huawei
may change the information at any time without notice.
Huawei AI Development Framework — MindSpore
 Foreword

⚫ This chapter describes Huawei AI development framework — MindSpore. It

introduces MindSpore architecture, design ideas, and then MindSpore features
through development challenges of AI computing frameworks. At last, it describes
the MindSpore development and application to help you further understand this
development framework.

2 Huawei Confidential
 Objectives

On completion of this course, you will be able to:

 Describe MindSpore.
 Understand the MindSpore framework.
 Understand MindSpore design ideas.
 Understand MindSpore features.
 Understand MindSpore environment setup process and development cases.

3 Huawei Confidential
 Contents

1. AI Framework Development Trends and Challenges

▫ Development Trends

▫ Seven Challenges

2. MindSpore Development Framework

3. MindSpore Development and Application

4 Huawei Confidential
AI Framework Development History

Lua-based Python-based Model Zoo Inheriting from Theano Imperative

Google:
Imperative Computational graph Performance and speed Declarative + computational Dynamic graph
JAX
graph
MLIR
Auto differentiation Deployment Swift for TF Jointly explore the
Distributed + deployment development direction at
TensorFlow 2.0
the third stage.
• User state
• Running state
• Deployment state
PyTorch:
TorchScript

2002 2007 2013 2015 2017

The AI framework technology has not been converged. Google has invested in four different directions to explore
technologies. It is estimated that the integration workload in the future is huge.

5 Huawei Confidential
 "ABCDE": Five Factors Driving the Evolution of the AI Framework

• Increasing model scale and complexity (GPT-3 parameter quantity reaches 175 billion.)
• Evolution from a single NN to general-purpose AI and scientific computing

Aplication+Bigdata
(AI model and data)

• New AI programming • Model deployment

language Developer Enterprise • Security and privacy
• Unification of dynamic and AI framework protection
(AI algorithm (AI deployment +
static graphs
engineer) responsible AI)

Chip
(AI chip and computing
power)
• Continuous improvement of chip/cluster performance (Atlas 900 cluster supports a maximum of exabyte-level computing power.)
• Diversified heterogeneous computing power for CPUs, GPUs, and NPUs
6 Huawei Confidential
 Contents

1. AI Framework Development Trends and Challenges

▫ Development Trends

▫ Seven Challenges

2. MindSpore Development Framework

3. MindSpore Development and Application

7 Huawei Confidential
Challenge 1: Increasing Model Scale and Complexity

• GPT-3:
Date Model Parameters Institution 1. Parameters: 175 billion (600 GB+)
2. Datasets (before processing): 45 TB
2018.4 ELMO 94m Ai2 3. Training cost: tens of millions of dollars; 1024 V100 GPUs; 127 days

• Technical challenges and trends:

2018.7 GPT 110m OpenAI 1. Performance (memory, communication, and computing usage)
• Challenges: The single-device memory is insufficient (32 GB).
2018.10 BERT-Large 340m Google The traffic volume varies greatly due to different parallel
partitioning. The computing usage of different parallel
2019.1 Transformer 465m Ai2 partitioning is different. The data preprocessing bottleneck
occurs.
ELMO • Trend: memory overcommitment, hybrid parallelism (data
parallelism, model parallelism, and pipeline parallelism), and
data acceleration.
2019.1 GPT-2 1.5b OpenAI 2. Efficiency
• Challenges: Manual partitioning is demanding. Parallel logic
2019.7 MegatronLM 8.3b NVDIA and algorithm logic are coupled.
• Trend: automatic parallelism.
2020.2 T-NLG 17.5b Microsoft 3. Accuracy
• Challenge: Optimizer for large batch sizes
• Trend: second-order optimization
2020.5 GPT-3 175b OpenAI

8 Huawei Confidential
Challenge 2: Evolution from Single NN to General-Purpose Tensor
Differentiable Computing

Deep probabilistic learning: Graph neural networks: AI modeling AI solution Framework resolution
Combine NN and probability Combine NN and graph Build AI-based Design new solutions with the Accelerate equation solving with the
models. structure data. computable models. help of neural networks. help of new frameworks.

Challenges: Challenges:
• Integrate NN models and probability models for modeling,• Equations as code. Users can quickly construct expressions, and the serial coding
reducing the learning difficulty. and parallel coding are consistent.
• Store, partition, and sample trillions of distributed graph • Support large-scale heterogeneous parallelism and mixed precision computing.
data. • Support high-performance higher-order differentiation (the volume of computing
• Support dynamic network structure and elastically higher-order differentiation increases exponentially with the order).
distributed training.

• Computing graphs (Taichi)

Challenges:
Differentiable physical • Sparse expression
engines
• Separation of data and computing
• Differentiable programming

9 Huawei Confidential
Challenge 3: Continuously Increasing Computing Power and
Complexity
Key challenges to AI framework software during AI chip
development:
hardware
Mainstream
GPU: NVIDIA commercial product
➢ Improve the coupling of optimization and hardware, and integrate graph
build and operator build.
• Fusion optimization at graph layer: Converge hardware-independent optimization, fully
NPU: Huawei commercial product
utilize the hardware computing power, and break the boundary between the subgraph
level and operator level for overall optimization.
• Optimization at operator layer: Consider hardware capabilities when using operators to
TPU: Google commercial product implement algorithms.
➢ Apply model execution modes to scenarios and hardware.
• Mix the graph sink mode and single-operator execution. Use different optimal mode
according to the hardware.
Hardware development trends • Use the data flow execution mode to better exert the computing power.
➢ Increase the computing density of a single core, improve the bandwidth • Use the SoC-level distributed parallel strategy for packaging more silicon chips.
and the process, increase the number of cores, and package more silicon • Use virtualization execution mode in SoC.
chips. ➢ Huge programmability challenges.
➢ Widely use SIMD and increase the tensor core processing scale (4 x 4 → • The effective computing power is close to the theoretical computing power and has
16 x 16). high requirements on the compiler;
➢ New data types (such as TF32 and BF16), high-speed interconnection • Sparse acceleration, image preprocessing acceleration module, and complex SIMD
between chips, and support for virtualization. acceleration instructions;
• SoC-level heterogeneous programming: CUBE core, Vector core, and ARM.
• Multi-chip, single-chip cross-generation, and cross-model compatibility requirements.

10 Huawei Confidential
Challenge 4: New Programming Languages Making Breakthroughs in
Python
Func f(_ x:Float) ->Float{ return tanh(log(x) }
#gradient(of: f)
FLUX

Julia Src Lowering

SIL Mandatory Passes
Julia AST Reverse-mode AD
Primal Generation
Zygote
Julia IR & optimization Auto differentiation Adjoint Generation

LLVM IR & optimization CUBlas XLA

Sil @f : $(Float) -> Float
Sil @f__primal : $(Float) -> f_Checkpoints
Executable program Sil @f__adjoint : $(Float, f__Checkpoint, Float) ->Float
Sil @f__gradient : $(Float) -> Float
• Julia enters the AI field based on the tensor native expression, IR openness,
• Swift for TensorFlow tries to find differentiated competitiveness
and high performance as well as the accumulation in the scientific computing
based on enterprise-class features such as static type, easy
and HPC fields.
deployment, and high performance.

11 Huawei Confidential
Challenge 5: Unification of Dynamic and Static Graphs

• Research phase: dynamic graph; Python affinity, • Production phase: static graph; performance,
flexibility, and usability. and deployment capability.

• Pain point: The representation of the dynamic graph is not • Industry frameworks use compilers such as accelerated
completely the same as that of the static graph. linear algebra (XLA) to work with chips for in-depth
• Trend: Optimize JIT to achieve the consistency of the two optimization.
representations. • Gradually improve the IR from the perspective of
• Challenge: It is difficult to fully use JIT to support Python optimization to form open AI infrastructure, such as
flexibility and dynamics. Relay/TVM and MLIR.

12 Huawei Confidential
Challenge 6: AI Deployment in All Scenarios
According to the 2019 CIO Agenda survey conducted by Gartner, the proportion of enterprises that have deployed AI increased from
4% to 14% from 2018 to 2019. The data is in sharp contrast to the industry's increasing awareness of the value of AI.

Trend 1: To reduce latency and improve user experience, on-

device language model deployment becomes a trend. The
challenge is how to reduce the model size and minimize the
Cloud AI
precision loss.

Trend 2: device-cloud synergy

Privacy and security High computing power 1. Mobile AI = On-Device AI + Smart services, better considering

Low latency Large model personalization, security, and privacy.

High reliability Big data 2. Single agent → multiple agent collaboration, implementing real-

Low bandwidth required High network bandwidth time perception and decision-making.

Trend 3: Ubiquitous AI is deployed in scenarios where IoT and

Tapping mode identification:
Deep learning (90%) vs. Traditional method (60%) smart devices have extremely limited resources.

13 Huawei Confidential
Challenge 7: Security, Privacy, and Protection

Security Privacy Fairness Transparency Explainability

Responsible
• Adversarial examples • Model inversion AI
• Model theft • Encrypted AI • Individual fairness • Comprehensibility
• Model backdoor • Federated learning • Group fairness • Accountability

Trend insights: Key challenges:

➢ In the future, in addition to accuracy and ➢ There is no general analysis method and measurement
performance, meeting responsible AI will be a key system for all aspects of responsible AI, and there is no
requirement for AI service success. automatic measurement method for scenario
➢ The AI framework bears AI services and must awareness.
have the capability of enabling responsible AI. ➢ AI model robustness, privacy protection technologies,
and encrypted AI have great impact on model
performance in actual scenarios.
➢ Responsible AI is deeply combined with AI explainability
and verifiability.

14 Huawei Confidential
 Contents

1. AI Framework Development Trends and Challenges

2. MindSpore Development Framework

◼ MindSpore Architecture
 MindSpore Key Features

3. MindSpore Development and Application

15 Huawei Confidential
MindSpore Open-source Deep Learning Framework
MindSpore MindSpore Lite
MindExpress Lightweight inference tool Ultra-lightweight On-device learning
inference Incremental
NN Operator Cangjie/Julia language
ModelZoo Data backup/transfer
Training Inference learning
Model format conversion
Federated learning

Quantization
Automatic Heterogeneous Memory Data processing
Type derivation Auto parallelism (int8/int4/mixed
differentiation scheduling overcommitment
precision) Generating graph
MindArmour operator code
Second-order Knowledge Online subgraph Custom fine-tune layer
Memory optimization Graph kernel fusion Kernel Select
optimization distillation/pruning partition

Integrated MindIR
MindData GE (accelerated by Ascend) MindAKG
Multiple Heterogeneous
Operator
High-performance
Large and
MindRT Lite synchronization/ small core
batches/sessions execution memory allocation
asynchronization scheduling
MindRT
Distributed DAG parallel execution
High-performance CPU/GPU operator library and Kirin/MTK AI acceleration library

Superior
performance Lightweight
All-scenario Efficient
support deployment

16 Huawei Confidential
MindSpore Vision and Value
⚫ Lower the barrier for AI development, maximize Ascend computing power, and empower
inclusive AI. Efficient
development
Profound expertise required
Algorithms
Programming

Flexible High
deployment performance
High deployment time and cost Diverse computing power and
Develop once models
Deploy everywhere CPU+NPU
Graph+Matrix

17 Huawei Confidential
MindSpore Logical Architecture
Design Objectives
MindSpore Extend ⚫ Beyond AI: NN applications general AI + numerical
GNN, deep probabilistic programming, reinforcement learning, differential equation, and more
computation
 Key feature: general-purpose tensor derivable computing
MindExpress ⚫ Distributed parallel native: supporting AI models to go
NN Data Operator Cangjie, Julia, 3rd beyond trillions of parameters
Model Zoo
frontend  Key features: automatic parallelism, memory-constrained
Training Inference programming, and second-order optimization
Mind
⚫ In-depth graph kernel fusion: capitalizing on the
Data MindCompiler computing power of AI chips
 Key features: joint graph and kernel optimization as well as
Data processing Type derivation
Automatic
Auto parallelism
Second-order automatic optimization based on Poly
differentiation optimization
Quantization, pruning,
and more
⚫ Enterprise-level capabilities in all scenarios: flexible
Mind Memory optimization Graph kernel fusion Pipeline execution deployment and collaboration, secure, reliable, and
Insight
explainable
Debug and tuning
MindIR  Key features: ultra-lightweight runtime, private training,
adaptive model generation, quantitative training, and
MindAKG (poly automatic optimization) explainable AI
Mind
Armour Design philosophy: AI "JDK"
Encrypted AI, model ⚫ Representation/optimization/operation decoupling: multi-
obfuscation, and MindRT frontend, cross-chip, and cross-platform
device-cloud synergy
for privacy MindRT MindRT Lite/Macro ⚫ Openness: opening the general graph compilation and running
protection Distributed heterogeneous parallel Operator Heterogeneous parallel Operator capabilities to third-party frameworks
⚫ Centralized architecture for all scenarios: integrated APIs and IRs,
enabling smooth AI applications
CANN (Ascend) CUDA Eigen... Android, Harmony, IoT

18 Huawei Confidential
Subsystem: MindExpress
Design objectives:
⚫ Design both high-level and low-level APIs for users, supporting
High-Level Python API
network building, entire graph execution, subgraph execution, and
Model Callback Quant single-operator execution.
⚫ Provide integrated APIs for model training, inference, and export,
Amp Parallel Other suitable for various scenarios, such as the device, edge, and cloud.
⚫ Provide unified encoding for dynamic and static graphs.
⚫ Provide unified encoding for standalone and distributed training.

Tensor Low-Level Python API Cell

Layers Optimizer Loss Metric

Functional modules:
⚫ High-level APIs provide management, callback, quantization, mixed
Ops Utils Initializer Other precision, and parallel control APIs for training and inference,
facilitating process control on the entire network.
Executor ⚫ Low-level APIs provide basic tensors, cells, NN-layers, optimizers, and
initialization, helping users flexibly build networks and control
execution processes.
⚫ The executor controls computing execution and interacts with the
MindSpore backend.
19 Huawei Confidential
Subsystem: MindCompiler
⚫ MindCompiler provides the just-in-time compilation capability for MindIR.
 Graph high level optimization (GHLO) is application-oriented and provides frontend optimization and functions, such as
Type derivation, automatic differentiation, second-order optimization, and automatic parallelism.
 Graph low level optimization (GLLO) is hardware-oriented and performs bottom-layer optimization, such as operator
fusion, layout optimization, redundancy elimination, and memory optimization.

Automatic
MindCompiler Mixed precision Type derivation
differentiation

C ++ API Second-order Graph kernel

Auto parallelism
optimization fusion
GHLO Backends: Ascend, GPU, and more

Operator Layout Memory

GLLO selection optimization optimization

MindIR

20 Huawei Confidential
Subsystem: MindRT
The centralized runtime system supports:
• Multiple device types on the device and cloud
MindRT
• Scheduling management of multiple hardware platforms, such
as Ascend, GPU, and CPU
MindRT MindRT lite
• Memory pooling management and efficient memory
runtime ops runtime ops
overcommitment
• Asynchronous operators, heterogeneous execution, and multi-
flow concurrency

host Ascend
Input and output data is
Technology The entire graph is offloaded to transferred through cache Entire graph
avoid extra host-device queues, and the zero copy execution
features interaction overheads. mechanism ensures that
data copies are fully
Ascend hidden.

21 Huawei Confidential
Subsystem: MindData
MindData is responsible for efficiently executing the training data processing pipeline, forming a pipeline with
computing, and promptly importing data for training.
load shuffle map batch repeat

Typical training data processing pipeline

Key functions:
MindData
⚫ Pipeline + parallel execution, improving data processing throughput
⚫ Various data operators
API(Python/C++) ⚫ User-defined Python operators and pipelines (data loading, sampling, and
argumentation)
C++ core ⚫ Heterogeneous hardware acceleration (Ascend/GPU/CPU)
⚫ MindRecord: built-in metadata and aggregated storage
Data graph generation

Data graph execution

Running process:
Data operators (loading/argumentation/sending) 1. Data graph generation: Data graphs are generated based on Python APIs
called by users.
2. Data graph execution: The pipeline executes data operators in a data
graph; this happens in parallel to complete dataset loading, shuffle, data
MindRecord Ascend/GPU/CP argumentation, and batch processing.
/TFRecord/Other U 3. Importing data to device: The processed data is imported to the device for
training.

22 Huawei Confidential
 Subsystem: MindInsight
MindInsight is the debugging and optimization subsystem of MindSpore. It
MindInsight UI provides the training process visualization, model lineage, debugger, and
performance profiling functions.
MindInsight backend
Key functions:
RESTful API • APIs are easy to use, enabling users to easily collect training process
metrics, including computational graphs, scalar data (such as loss and
Training log parsing RPC accuracy), histogram data (such as gradient and weight), and performance
communication data, and display them on the web UI.
Visualization
Hyperparameter Performance
Debugger • Collect training hyperparameters, datasets, and data augmentation
lineage profiling
information to implement model lineage and compare training results.

Running process:
1. Collecting training information: Users collect common training indicators
MindSpore using the callback API, and can decide which information to collect based
on their requirements. For example, use the summary operator to collect
Training information collection APIs Training information exchange
information about the computational graph and the Python API for
Summary operator Python API Summary file RPC information about the Python layer.
2. Generating training logs: Training logs are generated based on the process
Information collection callback Writer pool
information collected during training.
3. Displaying training information: MindInsight opens and parses training logs
FP/BP loop to display the training process information in a graph.

23 Huawei Confidential
Subsystem: MindArmour
MindArmour provides comprehensive, effective, and easy-to-use evaluation tools and enhancement methods for AI
trustworthiness in each domain.
Security Privacy Fairness Transparency Explainability Trustworthy AI

Start We are HERE Key functions: Goal

• Test data generation methods in all scenarios, such as black-and-white
AI Model Trustworthiness Test box adversarial attacks, member and attribute inference attacks, and
data drifts.
Fuzzer Model • Coverage-based fuzzing test process as well as flexible and customizable
Evaluation test strategies and indicators.
• Common methods of adversarial example detection and model
Data/Test Generator Trustworthiness robustness enhancement, including adversarial training and input
Evaluation rebuilding.
• Efficient, adaptive, and differential privacy training and budget statistics
Benign Transform Adversarial Attack algorithms, as well as mathematically proven model for privacy leakage
Coverage constraints.
Membership Attribute Evaluation Running process:
Inference Inference 1. Configuring strategies: Define test strategies based on threat vectors
and trustworthiness requirements, and select the appropriate test data
generation methods.
AI Trustworthiness Enhancement 2. Executing fuzzing test: Generate trustworthiness test data heuristically
based on the model coverage rate and configured strategies.
Robustness Tools Privacy Tools 3. Generating evaluation reports: Generate said reports based on built-in
or user-defined trustworthiness indicators.
Adversarial
Adversarial Adversarial Adaptive Differential Privacy 4. Enhancing trustworthiness: Use preset methods to enhance the
Training
Training Detection Training trustworthiness of AI models.

24 Huawei Confidential
 Contents

1. AI Framework Development Trends and Challenges

2. MindSpore Development Framework

 MindSpore Architecture
◼ MindSpore Key Features

3. MindSpore Development and Application

25 Huawei Confidential
MindSpore Feature: Automatic Parallelism

Challenges MindSpore Key Features

Challenges to efficient distributed training of ultra-large models: Automatically partition an entire graph based on the input and output
data of the operator, and integrate data parallelism and model
NLP models become larger and larger. The memory overhead for parallelism. Cluster topology aware scheduling: The cluster topology is
training ultra-large models such as BERT (340 million)/GPT-2 (1542 aware, and subgraphs are automatically scheduled to minimize
communication costs.
million) exceeds the capacity of a single device. Therefore, the model
NN Graph
needs to be partitioned into multiple devices for execution. Dense MatMul

Currently, manual model parallelism requires model partitioning

Subgraph 1
design and cluster topology awareness, which is difficult to develop, Dense MatMul

and it is hard to ensure high performance and perform tuning. Subgraph 2

Dense MatMul

Network
CPU CPU
Ascend Ascend Ascend Ascend

Effect: The standalone code logic is kept to implement model parallelism,

improving development efficiency by 10 times compared with manual parallelism!

26 Huawei Confidential
MindSpore Feature: Second-order Optimization

Challenges MindSpore Key Features

The second-order matrix is approximated to reduce the computational
Deep learning model training requires a large amount of computing complexity, and then the frequency and dimension of the matrix are
power, and training convergence takes a long time. reduced to accelerate the computation.
The second-order optimization method accelerates model
convergence and reduces the number of training steps. However, it
introduces a large number of complex computation, limiting its
application in deep model training.
The second-order optimizer parameters are updated as follows:
𝜽(𝒕+𝟏) = 𝜽(𝒕) − 𝝐𝑰𝑴−𝟏
𝜽𝒕
𝜵𝒈(𝜽(𝒕) )

Parameter First-order
Learning rate Second-order gradient
information matrix Optimizer Epoch Convergence Time Test Scale
Core problem: The second-order optimizer needs to compute the SGD+MOMENTUM About 60 90 minutes 8-device Ascend 910
inverse matrix of the second-order information matrix. The MindSpore second-order 42 71.5 minutes 8-device Ascend 910
computation workload is heavy, and it can take hours to solve the optimization
second-order matrix directly, creating a technical difficulty.

27 Huawei Confidential
MindSpore Feature: On-Device Execution
Challenges MindSpore Key Features
Challenges to model execution with powerful chip computing power:
The chip-oriented deep graph optimization is used to reduce
Memory wall problems, high interaction overhead, and difficult data supply. Some synchronization waiting time and maximize the parallelism degree of
operations are performed on the host, while others are performed on the device. "data-computing-communication". Data + Entire computational graph
The interaction overhead is much greater than the execution overhead. As a result,
the accelerator usage is low.
to the Ascend chips.

conv CPU
conv
bn
relu6

add

conv GPU
bn Data copy Conditional redirection task
Dependency Notification Task
relu6
kernel1 kernel2 ...
dwconv Effect: Compared with the host-side graph scheduling
bn
High data interaction overhead and difficult
mode, the training performance is improved by 10 times!
relu6 data supply.

28 Huawei Confidential
MindSpore Feature: Deployment and Collaboration in All
Scenarios
Challenges MindSpore Key Features
The diversity of hardware architectures leads to deployment • Unified model IR brings consistent deployment experience.
differences and performance uncertainties in all scenarios, • The graph optimization technology based on software and hardware
collaboration shields scenario differences.
and the separation of training and inference results in model
• Federal meta learning based on device-cloud synergy breaks the boundaries of
isolation. devices and the cloud. The multi-device collaboration model is updated in real
time.

Effect: In the unified architecture, the deployment performance of models in all scenarios is consistent, and the
accuracy of personalized models is improved!
On-demand collaboration in all scenarios and consistent development experience

Device Edge Cloud

29 Huawei Confidential
MindSporeIR
⚫ MindSporeIR is a simple, efficient, and flexible
graph-based functional IR that can represent
functional semantics such as free variables, higher-
order functions, and recursion. It is the program
carrier in the process of auto differentiation and
compilation optimization.

⚫ Each graph represents a function definition graph,

which consists of ParameterNode, ValueNode, and
ComplexNode(CNode).

⚫ The figure shows the def-use relationship.

30 Huawei Confidential
MindSpore Serving: Efficient Deployment of Online Inference Services

MindSpore Serving is a lightweight and high-performance service module that helps MindSpore developers
efficiently deploy online inference services in the production environment.

• Easy-to-use
• One-click release and deployment
• Batching
• High performance and scalability

MindSpore Serving structure

https://gitee.com/mindspore/serving/blob/r1.1/README_CN.md#%E9%85%8D%E7%BD%AE%
E7%8E%AF%E5%A2%83%E5%8F%98%E9%87%8F

31 Huawei Confidential
 Contents

1. AI Framework Development Trends and Challenges

2. MindSpore Development Framework

3. MindSpore Development and Application

◼ Environment Setup
 Application Cases

32 Huawei Confidential
Installing MindSpore

For details about how to install MindSpore, visit https://mindspore.cn/install/en.

MindSpore supports platforms such as Windows, Ubuntu, and CentOS, and hardware such as Ascend 910, CPU,
and GPU.

33 Huawei Confidential
MindSpore Experience
⚫ In MindSpore, the data storage component is Component Description
model_zoo Definition of common network models
a tensor. Common tensor operations are as
communication Data loading module, which provides the dataloader, dataset
follows: definition, and data processing functions such as image and text
 asnumpy() processing

dataset Dataset processing module, such as data reading and

 size() preprocessing
 dim() common Definitions of tensor, parameter, dtype, and initializer
 dtype() context Context class definition, which is used to set parameters for
model running, for example, switching to graph or pynative
 set_dtype() mode
 tensor_add(other: Tensor) akg Automatic differentiation and custom operator libraries
 tensor_mul(ohter: Tensor) nn Definitions of MindSpore cell, loss function, and optimizer
 shape() ops Basic operator definition and backward operator registration
 __str__# Convert into a character string. Training model-related and summary function modules
train
utils Utilities mainly for parameter validation (for internal framework
use)
ME Module Components

34 Huawei Confidential
MindSpore Programming Concept: Operator
Softmax operator Common MindSpore operators are as follows:
- array: Array-related operators
1. Operator - ExpandDims - Squeeze
name and base - Concat - OnesLike
class - Select -StridedSlice
-ScatterNd, etc.
2. Operator
comment - math: Mathematical operators
- AddN - Cos
- Sub - Sin
- Mul - LogicalAnd
- MatMul - LogicalNot
- RealDiv - Less
- ReduceMean - Greater, etc.

- nn: Network operators

3. Operator -Conv2D - MaxPool
initialization. The -Flatten - AvgPool
operator attribute - Softmax - TopK
values are - ReLU - SoftmaxCrossEntropy
initialized. - Sigmoid - SmoothL1Loss
- Pooling - SGD
- BatchNorm - SigmoidCrossEntropy,
4. Shape derivation - etc.
- control: Control operators
- ControlDepend
5. data_type
derivation -random: Random number-related operators

35 Huawei Confidential
MindSpore Programming Concept: Cell
⚫ A cell provides basic modules that define computing execution. Cell objects can be directly
executed.
 __init__: initializes and verifies components such as parameters, cells, and primitives.
 construct: defines the execution process; in graph mode, a graph is compiled for execution,
which is subject to specific syntax restrictions.
 bprop (optional): backward propagation of user-defined modules; if this method is undefined,
the framework automatically generates a backward graph to compute the backward
propagation of the construct part.
⚫ The following cells are predefined in MindSpore: loss functions
(SoftmaxCrossEntropyWithLogits and MSELoss), optimizers (Momentum, SGD, and Adam),
network packaging functions (TrainOneStepCell for network gradient calculation and
update, and WithGradCell for gradient calculation).

36 Huawei Confidential
 Contents

1. AI Framework Development Trends and Challenges

2. MindSpore Development Framework

3. MindSpore Development and Application

 Environment Setup
◼ Application Cases

37 Huawei Confidential
Computer Vision

Image classification Object segmentation Keypoint detection

arXiv:1512.03385 arXiv:1703.06870 arXiv:1611.08050

https://www.mindspore.cn/tutorial/training/zh-CN/r1.1/advanced_use/cv.html#
https://arxiv.org/pdf/1611.08050.pdf
https://arxiv.org/pdf/1706.05587.pdf
38 Huawei Confidential
Natural Language Processing (NLP)

Use MindSpore to train intelligent poetry writing models and deploy prediction services

Model hub

Pre-trained model
Fine- Training +
Pre-training tuning Inference

Poem dataset
Model

Services
Data preprocessing and
postprocessing
MindSpore Serving
Bare metal/Docker deployment

Application flowchart Bert model and training

https://www.mindspore.cn/tutorial/training/en/r1.1/advanced_use/nlp_bert_poetry.html

39 Huawei Confidential
High Performance Computing (HPC)

Computing faces unprecedented challenges due to massive amounts of data and access devices. Therefore, AI
and HPC are slated to converge in the future, with AI transforming tradition HPC.

The 3D ocean model is key for the entire earth system

model. By simulating ocean currents and whirlpools, it
can predict typhoons and tsunamis in real time.

However, the code for conventional ocean models is

complex and they run on CPUs. However, MindSpore
accelerates the GOMO model framework, which runs
on a GPU, significantly improving the model
performance.
GOMO flowchart GOMO prediction

https://gmd.copernicus.org/articles/12/4729/2019/gmd-12-4729-2019.pdf

40 Huawei Confidential
MindSpore Application Case
⚫ We will use the MNIST handwritten digit recognition to demonstrate the MindSpore-based
modeling process.

Prepare Build a Train the Test the

data model model model

• Data preparation • Model architecture • Batch size and epoch • Test sample
• Data visualization • Optimizer setting processing
• Data preprocessing • Loss function • Callback
• Evaluation indicator

41 Huawei Confidential
 Quiz

1. Which of the following operators does nn belong to in MindSpore? ()

A. Math-related operators

B. Network operators

C. Control operators

D. Other operators

42 Huawei Confidential
 Summary

⚫ This chapter introduces the MindSpore framework, design ideas, and features, and
describes the MindSpore environment setup process and development procedure.

43 Huawei Confidential
 More Information
MindSpore official website: https://mindspore.cn/en

Huawei Talent Online website: https://e.huawei.com/en/talent/#/home

WeChat official accounts:

EMUI Huawei Device Open Lab Huawei Developer Contact Huawei Talent
Online
44 Huawei Confidential
Thank you. 把数字世界带入每个人、每个家庭、
每个组织，构建万物互联的智能世界。
Bring digital to every person, home, and
organization for a fully connected,
intelligent world.

Copyright© 2021 Huawei Technologies Co., Ltd.

All Rights Reserved.

The information in this document may contain predictive

statements including, without limitation, statements regarding
the future financial and operating results, future product
portfolio, new technology, etc. There are a number of factors that
could cause actual results and developments to differ materially
from those expressed or implied in the predictive statements.
Therefore, such information is provided for reference purpose
only and constitutes neither an offer nor an acceptance. Huawei
may change the information at any time without notice.
Atlas AI Computing Platform
 Foreword

⚫ This chapter describes Huawei's Ascend AI chips, and hardware and software
architectures of Ascend chips, and full-stack all-scenario solutions of Ascend AI chips.

2 Huawei Confidential
 Objectives

On completion of the course, you will be able to:

 Get an overview of AI chips.
 Understand hardware and software architectures of Huawei Ascend chips.
 Learn about Huawei Atlas AI computing platform.
 Understand industry applications of Atlas.

3 Huawei Confidential
 Contents

1. Overview of AI Chips
◼ Summary

▫ Classification of AI Chips

▫ Current Status of AI Chips

▫ GPU, TPU, and Ascend 310 Design Comparison

▫ Ascend AI Processors

2. Hardware Architecture of Ascend Chips

3. …
4 Huawei Confidential
 Overview and Objectives

⚫ This section is an overview of AI chips, including the introduction, classification, and

status of AI chips, comparison between GPUs and CPUs, and introduction of Ascend
AI processors.

5 Huawei Confidential
Definition
⚫ Four elements of AI: data, algorithm, scenario, and computing power
⚫ AI chips, also known as AI accelerators, are function modules that process massive
computing tasks in AI applications.

6 Huawei Confidential
 Contents

1. Overview of AI Chips
▫ Summary

◼ Classification of AI Chips

▫ Current Status of AI Chips

▫ Design Comparison of GPUs and CPUs

▫ Ascend AI Processors

2. Hardware Architecture of Ascend Chips

3. …
7 Huawei Confidential
Classification of AI Chips
⚫ AI Chips can be divided into four types by technical architecture:
 A central processing unit (CPU): a super-large-scale integrated circuit, which is the computing core and
control unit of a computer. It can interpret computer instructions and process computer software data.
 A graphics processing unit (GPU): a display core, visual processor, and display chip. It is a microprocessor
that processes images on personal computers, workstations, game consoles, and mobile devices, such as
tablet computers and smart phones.
 An application specific integrated circuit (ASIC): an integrated circuit designed for a specific purpose.
 A field programmable gate array (FPGA): designed to implement functions of a semi-customized chip. The
hardware structure can be flexibly configured and changed in real time based on requirements.

8 Huawei Confidential
 Contents

1. Overview of AI Chips
▫ Summary

▫ Classification of AI Chips

◼ Current Status of AI Chips

▫ Design Comparison of GPUs and CPUs

▫ Ascend AI Processors

2. Hardware Architecture of Ascend Chips

3. …
9 Huawei Confidential
Current Status of AI Chips - CPU
⚫ Central processing unit (CPU)
 The computer performance has been steadily improved based on the Moore's Law.
 The CPU cores added for performance enhancement also increase power consumption and cost.
 Extra instructions have been introduced and the architecture has been modified to improve AI performance.
◼ Instructions, such as AVX512, have been introduced into Intel processors (CISC architecture) and vector computing modules, such as
FMA, into the ALU computing module.
◼ Instruction sets including Cortex A have been introduced into ARM (RISC architecture), which will be upgraded continuously.

 Despite that boosting the processor frequency can elevate the performance, the high frequency will cause huge power
consumption and overheating of the chip as the frequency reaches the ceiling.

10 Huawei Confidential
Current Status of AI Chips - GPU
⚫ Graph processing unit (GPU)
 GPU performs remarkably in matrix computing and parallel computing and plays a key role in
heterogeneous computing. It was first introduced to the AI field as an acceleration chip for deep learning.
Currently, the GPU ecosystem has matured.
 Using the GPU architecture, NVIDIA focuses on the following two aspects of deep learning:
◼ Diversifying the ecosystem: It has launched the cuDNN optimization library for neural networks to improve usability
and optimize the GPU underlying architecture.
◼ Improving customization: It supports various data types, including int8 in addition to float32; introduces modules
dedicated for deep learning. For example, the optimized architecture of Tensor cores has been introduced, such as
the TensorCore of V100.

 The existing problems include high costs and latency and low energy efficiency.

11 Huawei Confidential
Current Status of AI Chips - TPU
⚫ Tensor processing unit (TPU)
 Since 2006, Google has sought to apply the design concept of ASICs to the neural network field and
released TPU, a customized AI chip that supports TensorFlow, which is an open-source deep learning
framework.
 Massive systolic arrays and large-capacity on-chip storage are adopted to accelerate the most common
convolution operations in deep neural networks.
◼ Systolic arrays optimize matrix multiplication and convolution operations to elevate computing power and lower
energy consumption.

12 Huawei Confidential
Current Status of AI Chips - FPGA
⚫ Field programmable gate array (FPGA)
 Using the HDL programmable mode, FPGAs are highly flexible, reconfigurable and re-programmable, and
customizable.
 Multiple FPGAs can be used to load the DNN model on the chips to lower computing latency. FPGAs
outperform GPUs in terms of computing performance. However, the optimal performance cannot be
achieved due to continuous erasing and programming. Besides, redundant transistors and cables, logic
circuits with the same functions occupy a larger chip area.
 The reconfigurable structure lowers supply and R&D risks. The cost is relatively flexible depending on the
purchase quantity.
 The design and tapeout processes are decoupled. The development period is long, generally half a year.
The entry barrier is high.

13 Huawei Confidential
 Contents

1. Overview of AI Chips
▫ Summary

▫ Classification of AI Chips

▫ Current Status of AI Chips

◼ Design Comparison of GPUs and CPUs

▫ Ascend AI Processors

2. Hardware Architecture of Ascend Chips

3. Software Architecture of Ascend Chips

4. …
14 Huawei Confidential
Design Comparison of GPUs and CPUs
⚫ GPUs are designed for massive data of the same type independent from each other and pure computing environments that
do not need to be interrupted.
 Each GPU comprises several large-sized parallel computing architectures with thousands of smaller cores designed to handle multiple
tasks simultaneously.

 Throughput-oriented design
◼ With many ALUs and few caches, which improve services for threads, unlike those in CPU. The cache merges access to DRAM, causing latency.

◼ The control unit performs combined access.

◼ A large number of ALUs process numerous threads concurrently to cover up the latency.

 Specialized in computing-intensive and easy-to-parallel programs

15 Huawei Confidential
Design Comparison of GPUs and CPUs
⚫ CPUs need to process different data types in a universal manner, perform logic judgment, and introduce
massive branch jumps and interrupted processing.
 Composed of several cores optimized for sequential serial processing ALU ALU
Control
 Low-latency design ALU ALU
◼ The powerful ALU unit can complete the calculation in a short clock cycle.
Cache
◼ The large cache lowers latency.
◼ High clock frequency
DRAM
◼ Complex logic control unit, multi-branch programs can reduce latency
through branch prediction.
◼ For instructions that depend on the previous instruction result, the logic unit determines the location of the instructions in the
pipeline to speed up data forwarding.

 Specialized in logic control and serial operation

16 Huawei Confidential
 Contents

1. Overview of AI Chips
▫ Summary

▫ Classification of AI Chips

▫ Current Status of AI Chips

▫ Design Comparison of GPUs and CPUs

◼ Ascend AI Processors

2. Hardware Architecture of Ascend Chips

3. Software Architecture of Ascend Chips

4. …
17 Huawei Confidential
Ascend AI Processors
⚫ Neural-network processing unit (NPU): uses a deep learning instruction set to process a large
number of human neurons and synapses simulated at the circuit layer. One instruction is used to
process a group of neurons.
⚫ Typical NPUs: Huawei Ascend AI chips, Cambricon chips, and IBM TrueNorth

⚫ Ascend-Mini
⚫ Architecture: Da Vinci ⚫ Ascend-Max
⚫ Half precision (FP16): 8 Tera-FLOPS ⚫ Architecture: Da Vinci
⚫ Integer precision (INT8): 16 Tera-OPS ⚫ Half precision (FP16): 256 Tera-FLOPS
⚫ 16-channel full-HD video decoder: H.264/H.265 ⚫ Integer precision (INT8): 512 Tera-OPS
⚫ 1-channel full-HD video decoder: H.264/H.265 ⚫ 128-channel full-HD video decoder: H.264/H.265
⚫ Max. power: 8W ⚫ Max. power: 350W
⚫ 12nm FFC ⚫ 7nm

18 Huawei Confidential
 Section Summary

⚫ This section describes AI chips, including classification of AI chips by technologies and

functions, AI chip ecosystem, and comparison between GPUs and CPUs.

19 Huawei Confidential
 Contents

1. Overview of AI Chips

2. Hardware Architecture of Ascend Chips

◼ Logic Architecture of Ascend AI Processors

▫ Da Vinci Architecture

3. Software Architecture of Ascend Chips

4. Huawei Atlas AI Computing Platform

5. Industry Applications of Atlas

20 Huawei Confidential
 Overview and Objectives

⚫ This section describes the hardware architecture of Ascend chips, including the logic
architecture of the Ascend AI processors and Da Vinci architecture.

21 Huawei Confidential
Logic Architecture of Ascend AI Processors
⚫ Ascend AI processor consist of:
 Control CPU
 AI computing engine, including AI core and AI CPU
 Multi-layer system-on-chip (SoC) caches or buffers
 Digital vision pre-processing (DVPP) module

22 Huawei Confidential
 Contents

1. Overview of AI Chips

2. Hardware Architecture of Ascend Chips

▫ Logic Architecture of Ascend AI Processors

◼ Da Vinci Architecture

3. Software Architecture of Ascend Chips

4. Huawei Atlas AI Computing Platform

5. Industry Applications of Atlas

24 Huawei Confidential
Ascend AI Computing Engine - Da Vinci Architecture
⚫ One of the four major architectures of Ascend AI processors is the AI computing engine,
which consists of the AI core (Da Vinci architecture) and AI CPU. The Da Vinci architecture
developed to improve the AI computing power serves as the core of the Ascend AI
computing engine and AI processor.

25 Huawei Confidential
Da Vinci Architecture (AI Core)
⚫ Main components of the Da Vinci architecture:
 Computing unit: It consists of the cube unit, vector unit, and scalar unit.

 Storage system: It consists of the on-chip storage unit of the AI core and data channels.

 Control unit provides instruction control for the entire computing process. It is equivalent to the command center of the AI core and is
responsible for the running of the entire AI core.

26 Huawei Confidential
Da Vinci Architecture (AI Core) - Computing Unit
⚫ Three types of basic computing units: cube, vector, and scalar units, which correspond to matrix, vector and
scalar computing modes respectively.
⚫ Cube computing unit: The matrix computing unit and accumulator are used to perform matrix-related
operations. Completes a matrix (4096) of 16x16 multiplied by 16x16 for FP16, or a matrix (8192) of 16x32
multiplied by 32x16 for the INT8 input in a shot.
⚫ Vector computing unit: Implements computing between vectors and scalars or between vectors. This function
covers various basic computing types and many customized computing types, including computing of data
types such as FP16, FP32, INT32, and INT8.
⚫ Scalar computing unit: Equivalent to a micro CPU, the scalar unit controls the running of the entire AI core. It
implements loop control and branch judgment for the entire program, and provides the computing of data
addresses and related parameters for cubes or vectors as well as basic arithmetic operations.

28 Huawei Confidential
Da Vinci Architecture (AI Core) - Computing Unit

Matrix
ImgCol Calculation Unit Accumulator
Unzip
Input Transpose
Buffer Storage Conversion unit
FP16<->FP32 ReLU
Input Buffer Vector Calculation Unit
Controller Output Buffer
Special Scalar
Computing General
Register
Unit Register

Configure Matrix Operation Queue

Command Event
Port System Scalar
Bus Interface Transmission Vector Operation Queue Synchroniz
Control Instruction Instruction
Unit Module a-tion
Module Cache Processing
Storage Operation Queue Module
Queue

Bus(L2 Buffer/HBM/DDR)

29 Huawei Confidential
Da Vinci Architecture (AI Core) - Storage System (1)
⚫ The storage system of the AI core is composed of the storage unit and corresponding data channel.

⚫ The storage unit consists of the storage control unit, buffer, and registers:

⚫ Storage control unit: The cache at a lower level than the AI core can be directly accessed through the bus interface. The memory can also be
directly accessed through the DDR or HBM. A storage conversion unit is set as a transmission controller of the internal data channel of the AI
core to implement read/write management of internal data of the AI core between different buffers. It also completes a series of format
conversion operations, such as zero padding, Img2Col, transposing, and decompression.

⚫ Input buffer: The buffer temporarily stores the data that needs to be frequently used so the data does not need to be read from the AI core
through the bus interface each time. This mode reduces the frequency of data access on the bus and the risk of bus congestion, thereby
reducing power consumption and improving performance.

⚫ Output buffer: The buffer stores the intermediate results of computing at each layer in the neural network, so that the data can be easily
obtained for next-layer computing. Reading data through the bus involves low bandwidth and long latency, whereas using the output buffer
greatly improves the computing efficiency.

⚫ Register: Various registers in the AI core are mainly used by the scalar unit.

30 Huawei Confidential
Da Vinci Architecture (AI Core) - Storage System (2)
⚫ Data channel: path for data flowing in the AI core during execution of computing tasks
 A data channel of the Da Vinci architecture is characterized by multiple-input single-output. Considering various types and a large
quantity of input data in the computing process on the neural network, parallel inputs can improve data inflow efficiency. On the
contrary, only an output feature matrix is generated after multiple types of input data are processed. The data channel with a single
output of data reduces the use of chip hardware resources.

31 Huawei Confidential
Da Vinci Architecture (AI Core) - Control Unit (1)
⚫ The control unit consists of the system control module, instruction cache, scalar instruction processing queue, instruction
transmitting module, matrix operation queue, vector operation queue, storage conversion queue, and event synchronization
module.
 System control module: Controls the execution process of a task block (minimum task computing granularity for the AI core). After the
task block is executed, the system control module processes the interruption and reports the status. If an error occurs during the
execution, the error status is reported to the task scheduler.

 Instruction cache: Prefetches subsequent instructions in advance during instruction execution and reads multiple instructions into the
cache at a time, improving instruction execution efficiency.
 Scalar instruction procession queue: After being decoded, the instructions are imported into a scalar queue to implement address
decoding and operation control. The instructions include matrix computing instructions, vector calculation instructions, and storage
conversion instructions.
 Instruction transmitting module: Reads the configured instruction addresses and decoded parameters in the scalar instruction queue,
and sends them to the corresponding instruction execution queue according to the instruction type. The scalar instructions reside in the
scalar instruction processing queue for subsequent execution.

33 Huawei Confidential
Da Vinci Architecture (AI Core) - Control Unit (2)
⚫ Instruction execution queue: Includes a matrix operation queue, vector operation queue, and storage conversion queue. Different instructions
enter corresponding operation queues, and instructions in the queues are executed according to the entry sequence.

⚫ Event synchronization module: Controls the execution status of each instruction pipeline in real time, and analyzes dependence relationships
between different pipelines to resolve problems of data dependence and synchronization between instruction pipelines.

34 Huawei Confidential
 Section Summary

⚫ This section describes the hardware architecture of Ascend chips, including the
computing unit, storage unit, and control unit of the core Da Vinci architecture.

36 Huawei Confidential
 Contents

1. Overview of AI Chips

2. Hardware Architecture of Ascend Chips

3. Software Architecture of Ascend Chips

◼ Logic Architecture of Ascend 310

▫ Neural Network Software Flow of Ascend 310

4. Huawei Atlas AI Computing Platform

5. Industry Applications of Atlas

37 Huawei Confidential
 Overview and Objectives

⚫ This section describes the software architecture of Ascend chips, including the logic
architecture and neural network software flow of Ascend AI processors.

38 Huawei Confidential
Logic Architecture of Ascend AI Processor Software Stack (1)
⚫ L3 application enabling layer: It is an application-level encapsulation layer that provides different processing algorithms for specific application
fields. L3 provides various fields with computing and processing engines. It can directly use the framework scheduling capability provided by
L2 to generate corresponding NNs and implement specific engine functions.
 Generic engine: provides the generic neural network inference capability.

 Computer vision engine: encapsulates video or image processing algorithms.

 Language and text engine: encapsulates basic processing algorithms for voice and text data.

39 Huawei Confidential
Logic Architecture of Ascend AI Processor Software Stack (2)
⚫ L2 execution framework layer: encapsulates the framework calling capability and offline model generation
capability. After the application algorithm is developed and encapsulated into an engine at L3, L2 calls the
appropriate deep learning framework, such as Caffe or TensorFlow, based on the features of the algorithm to
obtain the neural network of the corresponding function, and generates an offline model through the
framework manager. After L2 converts the original neural network model into an offline model that can be
executed on Ascend AI chips, the offline model executor (OME) transfers the offline model to Layer 1 for task
allocation.
⚫ L1 chip enabling layer: bridges the offline model to Ascend AI chips. L1 accelerates the offline model for
different computing tasks via libraries. Nearest to the bottom-layer computing resources, L1 outputs operator-
layer tasks to the hardware.
⚫ L0 computing resource layer: provides computing resources and executes specific computing tasks. It is the
hardware computing basis of the Ascend AI chip.

41 Huawei Confidential
 Contents

1. Overview of AI Chips

2. Hardware Architecture of Ascend Chips

3. Software Architecture of Ascend Chips

▫ Logic Architecture of Ascend 310

◼ Neural Network Software Flow of Ascend 310

4. Huawei Atlas AI Computing Platform

5. Industry Applications of Atlas

44 Huawei Confidential
Neural Network Software Flow of Ascend AI Processors
⚫ The neural network software flow of Ascend AI processors is a bridge between the deep learning framework and Ascend AI chips. It realizes
and executes a neural network application and integrates the following functional modules.

⚫ Process orchestrator: implements the neural network on Ascend AI chips, coordinates the whole process of effecting the neural network, and
controls the loading and execution of offline models.

⚫ Digital vision pre-processing (DVPP) module: performs data processing and cleaning before input to meet format requirements for computing.

⚫ Tensor boosting engine (TBE): functions as a neural network operator factory that provides powerful computing operators for neural network
models.

⚫ Framework manager: builds an original neural network model into a form supported by Ascend AI chips, and integrates the new model into
Ascend AI chips to ensure efficient running of the neural network.

⚫ Runtime manager: provides various resource management paths for task delivery and allocation of the neural network.

45 Huawei Confidential
Neural Network Software Flow of Ascend AI Processors
Original Neural
Data
Network Model

Framework TBE

Generate
Offline
Model

DVPP Framework

Load and Execute

Runtime
Process
Choreographer
Coordination (Matrix) TS

Task
Ascend AI
Processor

46 Huawei Confidential
 Contents

1. Overview of AI Chips

2. Hardware Architecture of Ascend Chips

3. Software Architecture of Ascend Chips

4. Huawei Atlas AI Computing Platform

5. Industry Applications of Atlas

47 Huawei Confidential
 Overview and Objectives

⚫ This section describes the main products of Huawei Atlas AI computing platform,
including inference and training.

48 Huawei Confidential
Atlas AI Computing Platform Portfolio
Internet, security, finance, transportation, power, etc.

Application Industry SDK/container engine/basic Cluster management/model

enabling service repository management/data pre-processing

TensorFlow/PyTorch/Caffe/MxNet Public components

AI MindSpore
framework Framework
Framework Adapter
adapter

AscendCL

Unified O&M and configuration management

AXE toolchain (log/profiling/Mind Studio)

Graph optimization engine
Operator/Acceleration/Communication libraries (BLAS, FFT, DNN, Rand, Solver, Sparse, HCCL)
CANN
Runtime

Safety subsystem
Driver

subsystem
Atlas 800 AI inference server
Atlas 200 Atlas 300 inference
512 TOPS INT8
16 TOPS INT8 accelerator card
64 TOPS INT8
Atlas 500 Atlas 900
Chips & 16 TOPS INT8 256–1024 PFLOPS FP16
Hardware Atlas 800 AI training server
Atlas 200 developer Atlas 300 training card
2 PFLOPS FP16
kit 256 TFLOPS FP16

Ascend 310 Da Vinci Ascend 910

Architecture

49 Huawei Confidential
Atlas Accelerates AI Inference
Ascend 310
AI processor

Highest density in the Edge intelligence and

Performance improved 7x for terminal Powerful computing
devices industry (64-channel) cloud-edge
platform for AI inference
for video inference collaboration

Atlas 800 AI server

Atlas 200 Developer Kit (DK) Atlas 300 AI accelerator Atlas 500 AI edge station
Atlas 200 AI accelerator Model: 3000/3010
AI developer kit card Model: 3000
module
Model: 3000 Model: 3000 Model: 3000

50 Huawei Confidential
Atlas 200DK: Strong Computing Power and Ease-of-Use
⚫ 16TOPS INT8 24W

⚫ 1 USB type-C, 2 camera ports, 1 GE port, 1 SD card slot

⚫ 8 GB memory

⚫ Operating temperature: 0°C to 45°C

⚫ Dimensions (H x W x D): 24 mm x 125 mm x 80 mm

Full-Stack AI development on and
off the cloud
Developers Researchers Startups

Set up a dev environment with Local dev + cloud training Code-level demo
one laptop collaboration Implementing the algorithm function
Ultra low cost for local Same protocol stack for Huawei by modifying 10% code based on the
independent environment, with Cloud and the developer kit; reference architecture; interaction
multiple functions and interfaces training on the cloud and with the Developer Community;
to meet basic requirements deployment at local; no seamless migration of commercial
modification required products

51 Huawei Confidential
Atlas 500 AI Edge Station

Powerful
Powerful
computin
computing
g
Easy-to-
Easy-to-deploy
Edge-cloud
collaboration
deploy Edge ⚫ 16 TOPS INT8
intelligence
⚫ 25 w to 40 w Customer traffic
Low statistics
Small Low
Wi-Fi & LTE
Small footprint consumpt
consumption
⚫

footprint
ion ⚫ 64-channel HD video Data
Analysis
analytics in real time

⚫ Fanless design, stable from -

Backend server
40°C to +70°C Camera Atlas 500

52 Huawei Confidential
Atlas 800 AI Server
⚫ An efficient inference platform powered by Kunpeng

⚫ Key functions:
 2 Kunpeng 920 processors in a 2U space
 8 PCIe slots, supporting up to 8 Atlas 300 AI accelerator cards
 Up to 512-channel HD video real-time analytics
 Air-cooled, stable at 5°C to 40°C

Model: 3000
⚫ A flexible inference platform powered by Intel

⚫ Key functions:
 2 Intel® Xeon® SP Skylake or Cascade Lake processors in a 2U
space
 8 PCIe slots, supporting up to 7 Atlas 300/NVIDIA T4 AI
accelerator cards
 Up to 448-channel HD video real-time analytics
Model: 3010
 Air-cooled, stable at 5°C to 35℃

53 Huawei Confidential
Atlas Accelerates AI Training

Ascend 910
AI processor

Training card with ultimate World's most powerful World's fastest AI training
computing power training server cluster

Atlas 300 AI accelerator card Atlas 800 AI server Atlas 900 AI cluster
Model: 9000 Model: 9000/9010

54 Huawei Confidential
Atlas 300 AI Accelerator Card: Highest-Performing Accelerator
Card for AI Training

1802
(images/second)
2x 965 70%
(images/second)

Computing power per Gradient synchronization

card latency

Mainstream Ascend 910 +

training chip + MindSpore
256T FLOPS FP16 TensorFlow Direct 100G RoCE

Atlas 300
Model: 9000 ⚫ Test benchmark:
 ResNet 50 V1.5
 ImageNet 2012
 Optimal batch size respectively

55 Huawei Confidential
Atlas 800 Training Server: Industry's Most Powerful Server for AI
Training

2.5x 25x 1.8x

Density of computing Hardware decoder

power Perf./Watt

Atlas 800
Model: 9000 16384
2P FLOPS/4U
(1080p decoding) images/second 2P FLOPS/5.6kW

56 Huawei Confidential
Atlas 900 AI Cluster: Fastest Cluster for AI Training

Shortest time consumption: 59.8s

76.8s
Time

70.2s

59.8s

Atlas 900 Vendor 1 Vendor 2 Atlas 900

256-1024 PFLOPS FP16 ⚫ Test benchmark:

 Benchmark: ResNet-50 V1.5 model, ImageNet-1k
Industry-leading computing power | Best cluster dataset
network | Ultimate heat dissipation
 Cluster: 1024 Ascend 910 AI processors
 Accuracy: 75.9%
57 Huawei Confidential
Atlas Deep Learning System Accelerates AI Model Training and
Builds Extensive Applications
Video analysis

Algorithms models
Trained models
Parameter Computing Model Tailoring Quantizing AI services
Dataset Gene research
tuning verification

Model training Model deployment

Automated
driving

Weather
forecast

Atlas 300 training Atlas 800 AI training Atlas 900 AI cluster Oil exploration
acceleration card server
Model: 9000 Model: 9000

58 Huawei Confidential
Device-Edge-Cloud Collaboration Enables the Ultimate
Development and User Experience
Device-edge-cloud collaboration for continuous training at data
center and remote model update
Application
Applicatio
system
Video traffic Return unidentified data n system Model file
Model file

Deliver
Deliver inference
inference Remotely
Remotely update models AI
result
result update models
Atlas intelligent edge O&M management Atlas deep learning
platform platform platform
Atlas 200/500/800 FusionDirect Atlas 300/800
Model management Training and inference
Edge inference
and deployment on cloud

Centralized development Centralized O&M Enhanced security

• Centralized development architecture based on • FusionDirector manages up to 50,000 nodes, • Transmission channel encryption
Atlas Da Vinci and CANN, develop once, deploy manages central and edge devices, and remotely • Model encryption, double
everywhere pushes models and upgrades devices assurance

• Edge and data centers use different development

Industry • No O&M management tools; provides only • No encryption/decryption engine;
architectures. Models cannot transfer freely,
APIs, so customers need to develop APIs by models are not encrypted.
solution requiring secondary development.
themselves.

59 Huawei Confidential
 Section Summary

⚫ This section introduces products of Huawei Atlas AI computing platform, mainly

inference products including Atlas 200 DK, Atlas 200 AI accelerator module, Atlas 300
AI accelerator card, Atlas 500 AI edge station, and Atlas 800 AI server. The computing
platforms used for training include Atlas 300 AI accelerator card, Atlas 800 AI server,
and Atlas 900 AI cluster.

60 Huawei Confidential
 Contents

1. Overview of AI Chips

2. Hardware Architecture of Ascend Chips

3. Software Architecture of Ascend Chips

4. Huawei Atlas AI Computing Platform

5. Industry Applications of Atlas

61 Huawei Confidential
 Overview and Objectives

⚫ This section introduces industry application scenarios of the Atlas AI computing

platform.

62 Huawei Confidential
Electric Power: Industry's First Intelligent Unattended Inspection
Solution, with 5x Efficiency
Intelligent analysis at the edge with
ultra-low consumption
Front-end analysis and
return alarms Power 15W 16T Computing
Atlas 300/800
consumption power
Smart AI model training Power consumption 80%↓
AI
cameras Model optimization and
Surveillance and management 3W 2.XT Ideal for edge computing
remote upgrade
platform
Average system Real-time High computing
consumption analytics power
Atlas 200 1W Once per 16TOPS
minute

CHINA SOUTHERN POWER GRID

Manual AI
inspection inspection

Efficiency 5x+ up
System cost 30% down

63 Huawei Confidential
Finance: AI Enables Smart Branches for Banks
Past: Customer experience needs improvement Now: cloud-edge collaboration, smart finance

Branch outlet HQ
Poor experience Video traffic
Return customer ID and Model file

Long queuing time and low Update algorithms and

feature
deploy data
microservices
AI

Atlas 500 AI edge station Applicati FusionDirect Atlas 300/800

efficiency Behavior
Age

Gender Revenue
on Training platform
Interest Occupation Family
system
Credit

Facial Human body Behavior User

recognition recognition analysis profiling

Poor results
Potential VIP customers Customer queuing
Precise VIP Facial recognition-based
duration analysis
missed out identification authentication
Customer
Potential customer Service processing time
complaints 50%
conversion rate 60% 70%

64 Huawei Confidential
Manufacturing: AI Enables Intelligent Production Line with Machine
Vision
Cloud-edge collaboration enables intelligent EL
detection for PV modules

Return unidentified data Model file

Video
traffic
AI
Update algorithms and
Atlas 500 AI edge station Applicatio FusionDirect Atlas 300/800
n system Training platform
deploy microservices

Manual inspection Smart inspection

Unstable Low production Discontinuous High labor Zero omission High production Cloud-edge Reduced labor
results efficiency process costs efficiency synergy costs

Number of
Defective
defective 2 6
battery chip
battery chips
Detection
Detection 1.36s
duration
about 5s duration

Accuracy 33.33% Accuracy 100%

65 Huawei Confidential
Transportation: AI Smooths Highways with 5x Efficiency Boost
Algorithm and Intelligent License
application
Plate Recognition • Low efficiency

Transaction
Networking (LPR) • Long queuing time

Audit
Rate

…
center
Camera
Service aggregation RSU
antenna
Model training and AI
algorithm upgrade Manual charging
Atlas 500
Charging & credit
Lane controller
Big data platform
• Passing efficiency
Smart site 5X Saving energy
Deployed 15,000+ units in China ETC gantry system and reducing
emission

Reliable Easy O&M

Automatic active/standby Unified cloud management
Free-flow charging
switchover
AI-enabled
Lightweight AI inference, real- • Proactive security
Wide temperature range
-40°C to +70°C time computing, and vehicle control
feature library extraction • Road cooperation
management
Ultimate computing AI evolution • Autonomous
power Remote algorithm upgrade, vehicle driving
16 TOPS INT8 continuous evolution for toll Vehicle-route
audit and vehicle-road collaboration
cooperation

66 Huawei Confidential
Supercomputing: Atlas Helps PCL Build CloudBrain Phase II

Peng Cheng Laboratory (PCL)

Innovative basic platform for national
missions

Natural language Peng Cheng CloudBrain Phase II mainly built

Automated driving City brain Smart healthcare Speech recognition
processing Atlas 900,
the world's fastest training cluster
Application layer
Ultimate computing power
Critical infrastructure Level-E AI computing power
Basic platform layer for AI
Top cluster network
HCCL communication supports 100 TB/s non-
Peng Cheng CloudBrain Peng Cheng CloudBrain blocking parameter plane networking
Phase I Phase II
Physical resource layer Ultimate energy efficiency
AI cluster PUE < 1.1

67 Huawei Confidential
Attract More Developers Based on the Ascend Developer
Community

• Held in 10+ cities

Portal
Annual technical • 20+ senior trainers
• Dozens of speeches
salon
Technical
Technical
documents
forums
Ascend
developer • 1,500+ teams
• Annual prize of over

Developers'
community

Community
+ Developer •
RMB1 million
Equal opportunities
rights projects contest for enterprises and
universities
Ascend
Academy • Public cloud
vouchers
Developer • Free certification
• Developer-centric enabling platform
course tickets
• https://ascend.huawei.com/home support • Free Atlas developer
kits

68 Huawei Confidential
 Summary

⚫ This chapter describes the products of Huawei Atlas computing platform and helps
you to understand the working principles of Huawei Ascend chips. It focuses on the
hardware and software architectures of Ascend chips and application scenarios of
the Atlas AI computing platform.

69 Huawei Confidential
 Quiz

1. What are the main applications of Ascend 310? ( )

A. Model inference

B. Model training

C. Model building

70 Huawei Confidential
 Recommendations

⚫ Ascend community:
 https://ascend.huawei.com

71 Huawei Confidential
Thank you. 把数字世界带入每个人、每个家庭、
每个组织，构建万物互联的智能世界。
Bring digital to every person, home, and
organization for a fully connected,
intelligent world.

Copyright©2020 Huawei Technologies Co., Ltd.

All Rights Reserved.

The information in this document may contain predictive

statements including, without limitation, statements regarding
the future financial and operating results, future product
portfolio, new technology, etc. There are a number of factors that
could cause actual results and developments to differ materially
from those expressed or implied in the predictive statements.
Therefore, such information is provided for reference purpose
only and constitutes neither an offer nor an acceptance. Huawei
may change the information at any time without notice.
Huawei Open AI Platform for Smart Devices
 Foreword

⚫ Huawei HiAI is an open artificial intelligence (AI) capability platform for smart
devices, which adopts a "chip-device-cloud" architecture, opening up chip, app, and
service capabilities for a fully intelligent ecosystem. This assists developers in
delivering a better smart app experience for users by fully leveraging Huawei's
powerful AI processing capabilities.

2 Huawei Confidential
 Objectives

After this course, you will be able to:

 Master the usage of Huawei HiAI platform.
 Understand the powerful functions of Huawei HiAI platform.

3 Huawei Confidential
 Contents

1. AI Industry Ecosystem

2. Huawei HiAI Platform

3. Developing Apps Based on Huawei HiAI Platform

4 Huawei Confidential
Huge Opportunities: Foreseeable AI Ubiquity in a $3 Trillion
Market
Estimated AI market scale Involved industries: automobile, finance,
consumer goods and retail, medical care,
(Unit: $1 billion) education, manufacturing,
3500 communications, energy, tourism, culture
3,061 and entertainment, transportation,
3000 logistics, real estate, and environmental
protection
2500
2,147
2000 Computing power
breakthrough
1,507
1500

1,057 Algorithm breakthrough

1000
741
520
500 365 Data breakthrough
256
126 180

0 Data source: Forrester, Transparency Market Research,

Chinese Association for Artificial Intelligence, and
2016 2017 2018 2019 2020 2021 2022 2023 2024 2025
Roland Berger

5 Huawei Confidential
Architecture of the AI Application Platform
AI Finance, healthcare, security, entertainment, education,
Application retail, agriculture, etc.

Technical Computer Speech Natural Language Planning decision Big data

Direction Vision Processing Processing system analysis

AI Elements Computing
Data Algorithm
Power

Basic Cloud
Technology Big Data
Computing

Server
Infrastructure Internet, Sensor, IoT high-performance chip

6 Huawei Confidential
Challenges in AI Capability Development and Application

High thresholds Low efficiency Diverse requests Slow iteration

Demanding requirements Long training cycles Fragmented Limited capability

experience enhancement

6 months: ML & DL Data collection and The entire training cycle Difficulties in model
2 months: Statistics cleansing must be repeated for upgrade
4 months: Linear algebra Model training and each scenario. Difficulties in valid data
3 months: Calculus optimization Lack of inheritability or acquisition
Customized experience transferability
improvement

15 months 3–8 months N-fold workload

7 Huawei Confidential
 Contents

1. AI Industry Ecosystem

2. Huawei HiAI Platform

3. Developing Apps Based on Huawei HiAI Platform

8 Huawei Confidential
HiAI 3.0 - Enabling Ultimate Experience in All-Scenario Smart Life

Single Multiple Distributed

Device Devices

• Ultimate Experience
• Resource Sharing
• Computing Power Sharing

⚫ 4000+ partners ⚫ Over 96 million daily ⚫ Over 600 billion monthly

active users calls

9 Huawei Confidential
Huawei HiAI 3.0: Enabling Distributed All Scenarios by AI

Cloud 1000+ atomized services Device 40+ application Chip 300+ operators
programming interfaces (APIs)

Huawei HiAI Service Huawei HiAI Engine Huawei HiAI Foundation

Chip capability openness for high efficiency

AI capability openness for simplicity
Service capability openness for mutual benefits Quickly convert and migrate existing models
Integrate multiple AI capabilities into apps
Push services based on user requirements to obtain optimal performance based on
simply, making apps smarter and more
in an active manner. heterogeneous scheduling and network process unit
powerful.
(NPU) acceleration.

10 Huawei Confidential
HiAI Foundation
⚫ HiAI Foundation APIs constitute an AI computing library of a
mobile computing platform, enabling developers to efficiently
compile AI apps that can run on mobile devices.
 Leveraging high performance and high precision of Kirin chips,
better device-end AI performance will be delivered by more
powerful computing power.
 Support the largest number of operators (300+) in the industry and
more frameworks, greatly improving flexibility and compatibility.
 The Honghu, Kirin, and AI camera chips enable AI capabilities for S855 Kirin 810 Kirin 990 5G
more devices.
AI running score data

11 Huawei Confidential
HiAI Engine
⚫ HiAI Engine opens app capabilities and
integrates multiple AI capabilities into apps,
making apps smarter and more powerful.
 Provide handwriting recognition and dynamic
gesture recognition capabilities, with 40+
underlying APIs.
 Computer vision and speech recognition will
develop toward a distributed mode, assisting
developers in delivering more all-scenario smart Distributed AI helps sports healthy
life experience. and safe driving

12 Huawei Confidential
HiAI Service
⚫ HiAI Service enables developers to reuse services on multiple devices, such as mobile
phones, tablets, and large screens, with only one service access, efficiently implementing
distribution.

13 Huawei Confidential
Architecture of the HiAI Mobile Computing Platform

Supports diverse mainstream front-end

frameworks.

Provides various upper-layer function

service APIs to ensure efficient running on
mobile devices.

Allows flexible scheduling of heterogeneous

resources, meeting developers' demand to
accelerate neural network model computing and
operator computing

Tool chain Comprehensive Different types Source codes enabling

documents of APIs quick start
14 Huawei Confidential
What can apps benefit from Huawei HiAI?

Ready-to-use Security

Real time Stability Lower cost

15 Huawei Confidential
AI Capability Provider, Accelerating Application Development
Research results of developers' demands for HiAI capabilities:
more than 60% pay attention to CV, ASR, NLU
90.00%

80.00%

70.00%

60.00%

50.00%

40.00%

30.00%

20.00%

10.00%

0.00%
CV capability ASR capability NLP capability Situational Recommended
Search the capability: LBS etc. capability: direct
information in the service
third party
application in one
screen

17 Huawei Confidential
Comprehensive Tools for Developers

18 Huawei Confidential
 Contents

1. AI Industry Ecosystem

2. Huawei HiAI Platform

3. Developing Apps Based on Huawei HiAI Platform

19 Huawei Confidential
The Silent Child

20 Huawei Confidential
 Next Generation Mobile Experience
with Huawei HiAI

Fast Simple Mutual Benefit

22 Huawei Confidential
Connecting Developers and Stimulating Innovation to Achieve
Win-Win ecosystem

Offline connection for in-depth USD1 billion investment, stimulating Innovation competition for
communication innovations in all scenarios continuous development

• Salons in cities • AI application innovation

• HiAI open courses • Openness and innovation of contest
• Technical symposiums device capabilities • Creativity contest for future
• All-scenario digital service application
innovation • AR application innovation
• Cloud service ecosystem co- contest
construction

23 Huawei Confidential
 Welcome to join us to deliver ultimate AI
user experience
24 Huawei Confidential
 Summary

⚫ We believe that AI can make life better by bringing unprecedented convenience for
both back end and devices. However, this requires actual application scenarios that
allow more enterprises and developers to play a part in improving user experience
substantially. Huawei is willing to work with partners to jointly promote intelligent
transformation of industries with more developers and enterprises based on the
HiAI3.0 platform.

25 Huawei Confidential
Thank you. 把数字世界带入每个人、每个家庭、
每个组织，构建万物互联的智能世界。
Bring digital to every person, home, and
organization for a fully connected,
intelligent world.

Copyright©2022 Huawei Technologies Co., Ltd.

All Rights Reserved.

The information in this document may contain predictive

statements including, without limitation, statements regarding
the future financial and operating results, future product
portfolio, new technology, etc. There are a number of factors that
could cause actual results and developments to differ materially
from those expressed or implied in the predictive statements.
Therefore, such information is provided for reference purpose
only and constitutes neither an offer nor an acceptance. Huawei
may change the information at any time without notice.
HUAWEI CLOUD Enterprise Intelligence Application
Platform
 Objectives

On completion of this course, you will be able to:

 Understand the HUAWEI CLOUD Enterprise Intelligence (EI) ecosystem and EI services.
 Understand the Huawei ModelArts platform and know how to use it.
 Understand the application fields of HUAWEI CLOUD EI.

2 Huawei Confidential
 Contents

1. HUAWEI CLOUD EI Overview

2. EI Intelligent Twins

3. AI Services

4. Case Studies of HUAWEI CLOUD EI

3 Huawei Confidential
HUAWEI CLOUD EI

Industry wisdom Algorithms

Industry know-how and deep Extensive algorithm and model
understanding of industry pain points, libraries, general AI services, and
driving AI implementation one-stop development platform

HUAWEI CLOUD
Data Computing power
Conspicuously-defined data EI
sovereignty standards, ensuring Simplified enterprise-grade
clear service and data access AI applications
boundaries

4 Huawei Confidential
HUAWEI CLOUD EI

General APIs Advanced APIs Pre-integrated solutions

ASR TTS Image NLP Home Internet of

CBS ImageSearch VCM VCT City Internet
Vehicles (IoV)

OCR Face Moderation AIS IDS VGS VCC VCR

Logistics Healthcare Campus Manufacturing

ModelArts
Essential platform ExeML GES DLS MLS Batch UPredict RLS
services

Ascend

5 Huawei Confidential
Development of HUAWEI CLOUD EI • 59 cloud services and 159
functions
• PMCs and committers of core • IDC: Huawei • Multi-domain Intelligent
projects ranked No. 1 in Twins
1. Hadoop Core/HBase: 7 China's big data • Dedicated to inclusive AI
2. Spark+CarbonData: 8 market
Cloud EI services
• CarbonData: top Apache • 190+ patents
project
Cloud EI services
Enterprise big data platform
Telecom big data solution (FusionInsight) Cloud EI services
Cloud EI services
(Telco industry)
Big data technology research Reliable and secure self-
management
Traditional BI Performance-oriented and
(Telco industry) equipment-based
Hadoop kernel
optimization and
ETL & analytics community
technology contributions

AI technology research AI practice: Focus on the inside of

Huawei and support intelligent
upgrade.
2002 2007 2011 2013 2015 2017 2019
6 Huawei Confidential
 Contents

1. HUAWEI CLOUD EI Overview

2. EI Intelligent Twins

3. AI Services

4. Case Studies of HUAWEI CLOUD EI

7 Huawei Confidential
TrafficGo
⚫ Traffic Intelligent Twins (TrafficGo) is a comprehensive urban traffic governance solution. Powered by the big data platform, AI algorithms, and expert
experience in the transportation industry, TrafficGo builds a network for urban traffic governance to implement intelligent awareness, diagnosis, and
optimization in all domains. TrafficGo enables 24/7 traffic condition monitoring in all areas, traffic incident detection, real-time regional traffic signal
scheduling, traffic situation display, and key vehicle management. This makes transportation more efficient and safer while yielding new levels of
energy-efficiency.
Solution architecture
Multi-domain collaborative decision-
making technology
Traffic parameter Multi-source data
Computer vision Speech and semantics awareness fusion

Machine Deep Reinforcement Graph Inference Traffic incident

Live traffic situation
learning learning learning engine platform monitoring

Data Lake Traffic status Crowd density

industry insights
Transportation
diagnosis prediction
+
Edge computing

TrafficGo
3G/4G/5G/Ethernet

Congestion cause Traffic signal

Geomagnetic
analysis optimization
Induction
Cameras Checkpoints
sensors coils Traffic control
Floating car Weather optimization
Radars Carrier data
data data

8 Huawei Confidential
Industrial Intelligent Twins
⚫ HUAWEI CLOUD Industrial Intelligent Twins builds an industrial intelligent platform that deeply integrates industrial knowledge and AI to adapt to
frequent changes of working conditions and break the application limit of the industrial mechanism. In addition, Huawei works with industry-leading
know-how customers and partners to build an open and win-win industrial intelligent ecosystem to make enterprises more intelligent and promote
industrial upgrade.

Solution architecture

A simple and efficient platform for industrial AI development

Machine & AI Policy Factory digital Industrial knowledge Knowledge

Industrial intelligent
model configuration twin graph market
data lake
Multimodal Mechanism AI models
data models
Multi-scenario,
Industrial One-stop, Multimodal, Dynamic
data-
Microservice lightweight
knowledge and graph-
low-code update
governance
based
development operating
development environment
mechanism environment
Multimodal data environment
model

Data Mechanism Model

Operating
modeling integration development
platform
Development platform

9 Huawei Confidential
EIHealth
⚫ HUAWEI CLOUD EIHealth covers genome, clinical research, and drug discovery. It focuses on people's health and promotes the combination of AI and
healthcare to improve the service capability and coverage of healthcare. With Huawei's powerful computing, storage, and AI algorithms, EIHealth
enables the genome, disease diagnosis, and pharmaceutical fields.

Solution architecture
1 + 3 + X: 1 EIHealth cloud platform + 3 sub-domain platforms + X enterprise/organization customized platforms
(delivered with ISVs)

Platform Gene Multi-omics AI-assisted patient Medical image Computer-aided drug AI drug
applications database analysis screening modeling discovery discovery

Genome Clinical research Drug discovery

Genome GROMACS AI model for patient Virtual screening of Drug combination

EIHealth assets DeepVariant
AutoML Molecular dynamics screening drugs prediction
EIHealth Data management
Process
Development environment Service management Knowledge graph AI Gallery
components management

AI ModelArts GES OCR KGS

Big data DLI DLF DIS CSS MRS DWS
PaaS CCE SWR AOM RDS DCS APIG
IaaS Atlas Kunpeng x86 OBS SFS LB

10 Huawei Confidential
HeatingGo
Based on the existing auto control facilities and technologies, HeatingGo uses technologies such as AI and big data to implement intelligent heat supply
supervision, operations, and control for heat sources, networks, stations, and users.
It provides a heating system that is inclusive, inexpensive, and controllable.

Cloud
Solution architecture Residential
NB-IoT building
Data training Data
Data inference
preprocessing Indoor temperature
Heating awareness
Data Temperature
aggregation Huawei AI policy 20℃
awareness
Data reporting Policy promotion
Management and
Command and dispatch center of a heating enterprise
control apps of heating
enterprises Data reporting Data reporting
Heat exchanger

X 18℃
Heat
source Heat source
User apps
sensing

Heat source: heat demand Heat exchanger: AI-based Building: AI-based smartUser: temperature control based
forecast and complete smart policies for balancing policies on temperature control panels
closed-loop heat supply and mobile apps

Primary circuit Secondary circuit

11 Huawei Confidential
WaterGo
With technologies including AI and edge computing, Huawei is transitioning cameras into universal sensors capable of sensing all elements related to
water, such as its volume and quality, making them "eyes" for river and lake monitoring. The cameras are widely distributed, intelligent, and responsive.
Leveraging AI technologies, they improve the existing prediction and scheduling system, making water resource management more efficient and
intelligent.

Solution architecture Oriented to digital government

and smart city sectors

Chief
Suitable for drainage and
The universal AI platform Drainage & flood control, water
flood control
provides services such as Video Volume pollution control, and water
Analysis Service (VAS), Pollution resource management
handling
ModelArts, Knowledge Graph, VAS
Quality
Natural Language Processing Management
Geo
(NLP), FusionInsight, and Genius
GeoGenius. Supervision API

Capable of monitoring and

Cabinet
Schedule scheduling water volume
KG
and quality
SDC
Various edge devices and NLP
terminals can access the
network.
HUAWEI
CLOUD

12 Huawei Confidential
GeoGenius
Powered by HUAWEI CLOUD's accumulation in cutting-edge technologies such as AI and big data, GeoGenius builds a one-stop, full-process intelligent
development cloud platform for remote sensing. This cloud platform functions as both a data platform and an intelligent computing platform to help you
focus on mining core values of data and developing application algorithms. It enables rapid application innovation in the industry and provides technical
support for key tasks such as investigation, monitoring, evaluation, supervision, and law enforcement of natural resources, ecology, meteorology,
environmental protection, and oceans.
Ecological environment Agriculture and forestry
Natural resource survey monitoring monitoring

Emergency response and

Weather forecast Marine conservation disaster prevention

13 Huawei Confidential
Smart Logistics Solution
The smart logistics solution provides AI-enabled 3D packing services to improve container loading rate. In addition, it provides the
vehicle route optimization service to reduce customers' transportation costs.

Customers HUAWEI CLOUD Customers

Send a packing request. Work out a 3D packing plan with In the response message of the packing service, obtain
a high loading rate. the packing plan to facilitate packing.
(RESTful API)

Customer API requests

1. Container information Customers
Packing plan
2. Goods information HUAWEI CLOUD

3D packing service

Improve container space Reduce the Improve the overall logistics

utilization, reduce the number operational cost operation efficiency
of boxes

14 Huawei Confidential
Intelligent Stand Allocation Solution
Based on AI algorithms and scenario understanding, the intelligent stand allocation system provides the optimal stand allocation
solution powered by Huawei big data platform and ModelArts, improving the docking rate, passenger experience, and airport
operation efficiency.
Allocation solution
Metric statistics View display
evaluation
Intelligent algorithm Allocation solution
Intelligent stand Rule engines
engine generation
allocation system

Cloud enablement application platform

Big data

Stand information Model information Flight information Stand allocation library …

library library library

Operation and command

Airport Operation resource Flight information Air traffic control
information
system management system management system
platformAOMDP
authorities

15 Huawei Confidential
 Contents

1. HUAWEI CLOUD EI Overview

2. EI Intelligent Twins

3. AI Services

4. Case Studies of HUAWEI CLOUD EI

16 Huawei Confidential
Essential AI Platforms

Huawei HiLens Graph Engine Service (GES)

ModelArts (AI development)
Multimodal AI development application China's first commercial distributed native
One-stop AI development platform platform featuring device-cloud synergy graph engine with independent
intellectual property rights

17 Huawei Confidential
ModelArts
⚫ ModelArts is a one-stop AI development Cloud
platform. It provides data preprocessing,
semi-automated data labeling, distributed AI Gallery

training, automated model building, and

Model
model deployment on the device, edge, training
and cloud to help AI developers build AI model
Batch

models quickly and manage the AI

Device Edge
development lifecycle.
HiLens

Edge Video IoT

Camera Edge station Mobile
server analysis gateway
phone

18 Huawei Confidential
ModelArts Functions
Online learning
Development and management: Notebook + Pycharm SDKs + Immersive development

Data processing Algorithm Training Model Deployment

developmen management management
Data ⚫ Intelligent data management AI apps
Notebookt2.0
filtering
⚫ ⚫ Built-in algorithms (industry
⚫ Model repository
⚫ Intelligent data
⚫ Jupyterlab algorithms)
⚫ Precision tracking ⚫ Cloud real-time
labeling
⚫ ML Studio ⚫ Auto hyperparameter tuning
⚫ Model evaluation service
⚫ Intelligent data
⚫ MindStudio ⚫ Pangu large model
⚫ Model conversion ⚫ Cloud batch service
analysis
⚫ Code samples ⚫ Visualized workflow ⚫ Device-edge-cloud
⚫ Labeling scoring and
⚫ MindSpore samples ⚫ Reinforcement learning synergy
evaluation
⚫ ModelZoo samples ⚫ Federated learning ⚫ Data collection
⚫ Data augmentation
(data refinery) ⚫ Elastic training
⚫ Interactive intelligent
labeling

⚫ Models
⚫ APIs
⚫ Engineering pipelines
⚫ Datasets ⚫ Model sales
⚫ API sales

AI Gallery
AI Gallery

19 Huawei Confidential
ExeML

ExeML is an entry-level service provided by ModelArts for beginners. Powered by the ExeML engine, ExeML enables
even beginners to build, train, verify, and publish models.

Zero coding
Step 1: Step 2: Step 3:
No AI experience Upload data Train the model. Check and publish the
required and label it. model.

20 Huawei Confidential
Data Management
A wide range of data Intelligent data filtering
formats
• Five types of data (image, • Automatic image clustering
audio, video, text, and • Recognition rate of invalid
table) images > 80% Data Team labeling
• Custom data formats management
Automatic pre-labeling Dataset
Team labeling
• Iterative semi-auto data pre-
• Great for ultra-large-scale labeling Team

Intelligent pre-
labeling

Data access
• Labeling efficiency up by 5 times labeling

labeling

Dataset
Iterative intelligent labeling Single
framework Auto feature mining
labeling
• In-depth data optimization
• Adaptive to data and algorithm
suggestions in 30+ dimensions
changes
• General models for various
• Intelligent data filtering and
scenarios
auto pre-labeling

21 Huawei Confidential
Training Platform
Flexible, high-efficiency and cost-effective
Training platform
• Multiple built-in algorithms, import of partner algorithms, custom training
(machine learning/deep learning)
logic and images, and quick switchover between heterogeneous resources
• Linearly improved parallel training capability, auto parameter tuning, and a
wide array of development modes
Notebook/Workflow Model
• Elastic training, economical mode, self-developed chips and software, and Data Model training
ultimate cost-effectiveness management management
Configurable training

Built-in training models, accelerating AI implementation

Algorithm development

Model management
• 100+ algorithms, including image, text, time series, and reinforcement Auto parameter

Model evaluation
learning algorithms tuning

visualization
• One-click training requiring only data source configuration

Dataset

Model
Multiple development modes, meeting different Model training
requirements
• In-cloud development (Notebook+SDK)
• On-premises development (IDE+PyCharm ToolKit)

Elastic training, improving training efficiency

• Turbo mode: Resources are automatically expanded during off-peak hours, accelerating the
training speed by 10 times (from 2 hours to 10 minutes).
• Economic mode: Optimized scheduling reduces the training cost by 30%.

22 Huawei Confidential
Inference Platform
Unified management
• Unified management of models of different vendors, Inference platform
frameworks, and functions
• High-concurrency model deployment, low-latency
access, auto scaling, grayscale release, and rolling Online
upgrade Model Model inference Edge Resource
management deployment inference management
Batch
Flexible deployment inference

• Models can be deployed as real-time and batch inference Deployment control and management
Model import
services on the cloud, edge, and devices. management
Real-time data Resource
Online management
Inference

Result output
Data access
Model deployment
inference
process Model
compression Batch data
and conversion
Batch
Iterative model update inference O&M and
monitoring
Model evaluation
• Hard example mining, automatic identification of and diagnosis
Edge inference
(IEF)
hard examples, and quick adaptation to data
changes

23 Huawei Confidential
AI Gallery
AI Gallery
On-demand subscription and
Dataset Model Industry workflow
Publishing at zero cost quick deployment
Algorithm D chipset operator
• No fees for sellers to • On-demand subscription,
publish a model because deployment, and charging for
Release Subscription
no resources are consumed buyers
• Quick service deployment and low-
Seller ModelArts
Buyer cost O&M because of a full set of
Enterprise-class suite deployment capabilities
customized for AI
Development/I
mport
General AI platform
Optimization
Model finetuning Encrypted deployment
Training Deployment
• Portfolios of cloud models • Confidentiality of container
and algorithms for buyers deployment protects
to optimize models based Cloud Cloud Edge
intellectual property
on differentiated data Built-in algorithm Real-time service Edge site

Parallel training Batch service Device

24 Huawei Confidential
ModelArts Pro
ModelArts Pro is a professional development suite for enterprise-class AI applications. AI development is simplified with the advanced algorithms, quick training, and built-
in workflows and models provided by HUAWEI CLOUD. In addition, customers can quickly develop, share, and launch applications through custom workflow orchestration.
With these, HUAWEI CLOUD aims to create an open AI ecosystem for Inclusive AI. The ModelArts Pro suite consists of NLP Suite, OCR Suite, and Visual Suite, which allows it
to meet AI implementation requirements in different scenarios.
ModelArts Pro
Application asset Application iteration Application
management implementation
Models, services, and
Models, services, and templates Online deployment, Industry
Applications templates HiLens deployment, and
appliances
workflow
package
Step-by-step workflow execution -> Application generation
Creating industry applications using workflows

ModelArts Custom AI components Pre-installed Huawei AI Workflow sharing

AI algorithm, data components Shared with specified users
Workflow processing, and model Data processing, domain and published to AI Gallery Workflow
evaluation model, and deployment

Workflow orchestration and commissioning

Accumulating Huawei AI atom capabilities

Suite NLP Suite OCR Visual HiLens Voice KG Suite Model

Suite Suite Suite Suite algorithms
ModelArts Fundamentals
...

Notebook ML DL GNN RL Search Solver AIBox

25 Huawei Confidential
Built-in Workflows of OBT Suites
OCR Suite NLP Suite HiLens Suite Visual Suite

Single-template
General text Goods Safety helmet
recognition
classification identification detection
workflow

Multi-template
Comment and Steel surface
recognition Mask detection
opinion mining inspection
workflow

Secondary Key information Printing quality

training extraction inspection

Passport Brake pad

recognition inspection

26 Huawei Confidential
Huawei HiLens
⚫ Huawei HiLens, featuring device-cloud synergy, enables you to develop AI applications in multiple modes. It has an easy-to-use
development framework, with an out-of-the-box development environment. It also boasts a skill market with extensive sets of AI
skills and a cloud management platform. Connecting to various computing devices, Huawei HiLens allows you to develop visual and
auditory AI applications, deploy AI applications in real time, and manage a large number of devices.

Huawei HiLens
Industry's first multimodal AI application development platform
featuring device-cloud synergy

Device-cloud Smart Home Smart Store Smart Industry HUAWEI

synergy Solution Solution Solution
CLOUD EI
Multimodal skills HiLens Skill: AI skill application ModelArts
Development Development Management
framework: HiLens framework: HiLens platform: HiLens
Framework Studio Console
Model
Device HiLens Kit, Atlas series, Ascend series...

The HiLens platform has advantages such as device-cloud synergy for inference, rich skill market, convenient development, and powerful
developer ecosystem.
27 Huawei Confidential
Graph Engine Service (GES)
• GES is the first commercial distributed native graph engine with independent intellectual property rights in China. It
facilitates query and analysis on graph structure data based on relationships. It is specifically suited for scenarios
involving social applications, enterprise relationship analysis, knowledge graph, risk control, and recommendations.

Large Innovative graph Compatible Integration

scale engine kernel design with open- Web portal
source APIs
Algorithm Integrated design
development Release Property graphs of analysis and
of higher scales query Visualizer
Eywa high-performance
graph engine
Modeling Service
Abundant graph analysis
algorithm libraries
application
Submit
High-performance graph embedding
High performance computing kernel
Ease of use
Users Mobile client
Distributed high-
performance graph storage
engine

28 Huawei Confidential
ImageSearch
Clothing search and Parts search and boost efficiency
recommendation

Piracy identification and Design search and

copyright protection comparison

29 Huawei Confidential
Speech Interaction Service (SIS)
Intelligent algorithms Self-service optimization
• The speech interaction engine uses innovative • The hot word function allows you to import hot words
algorithms and integrates traditional algorithms and by yourselves to optimize the recognition effect in
deep learning models to achieve high recognition specific domains.
accuracy.

Scenarios
Conference Customer service Human-machine Smart education
Smart call Content analysis
transcription inspection interaction

Real-Time Automatic TTS

Sentence Long Audio Text to Speech
Services Speech Recognition Customization Oral Assessment
Transcription Transcription (TTS)
(RASR) (TTSC)

Endpoint detection Emotion recognition Speaker separation Speech recognition decoder

Voice
technologies
Acoustic model Language model Disfluency detection Waveform splicing

30 Huawei Confidential
Natural Language Processing (NLP)
• NLP provides users with APIs related to natural language processing, including word segmentation, named entity recognition,
sentiment analysis, and machine translation. It can be used in multiple application scenarios, such as intelligent Q&A, semantic
search, public opinion analysis, and intelligent quality inspection.

Public opinion Machine Intelligent quality

Scenarios Intelligent Q&A
analysis
Semantic search
translation inspection

Word Named entity Keyword

NLP Syntax analysis Text similarity Entity linking
segmentation recognition extraction
technologies Sentiment Intent Text Machine
Text classification Text generation
analysis understanding summarization translation

Object Storage Service Relational database Elastic Cloud Server AI development

(OBS) service (ECS) platform
(RDS) (ModelArts)

31 Huawei Confidential
CBS Customization (CBSC)
• Conversational Bot Service (CBS) is a cloud service based on NLP technology and enables dialog AI for different scenarios. With
speech recognition, CBS enables product services to upgrade from graphical user interface (GUI) to voice interaction (VUI) to
improve interaction efficiency and enhance interaction experience.

Scenarios E- Finance and

Government Healthcare Education Catering Tourism Enterprise office Retail
commerce insurance

Intelligent outbound Intelligent inbound Intelligent quality

Solutions Intelligent chatbot Large-screen voice assistant Personal phone assistant
call call inspection

Product
Question Answering Bot (QABot) Intelligent phonebot Speech Analysis IOT assistant
services

Text classification Intention recognition Language Text generation

Language
Emotion recognition generation
understanding
Slot filling Information extraction Text error correction

Word segmentation Word vector Named entity recognition Long Audio

Language Sentence Transcription
NLP Transcription
recognition Automatic Speech
Language model Syntax analysis Keyword extraction
Recognition (ASR)

Model ModelArts platform

training

32 Huawei Confidential
Knowledge Graph
Knowledge Graph is a one-stop knowledge graph building platform that provides functions such as ontology design,
information extraction, knowledge mapping, multi-source convergence, and incremental update.

Knowledge Recommendation
Scenarios Voice assistant Text generation ...
management system

Intelligent Q&A Knowledge search Knowledge Q&A

General application
capabilities Graphic computing Visualization
NLP

Knowledge graph AI computing

Knowledge
compute Metadata Structured data
Graph database Ontology design Model training
platform storage extraction
Knowledge Knowledge Supplementation & Version management Model inference
mapping convergence error correction
Model deployment
Unstructured data Incremental update
Pipeline operation Data labeling
extraction

Infrastructure Infrastructure as a service (IaaS)

33 Huawei Confidential
Optical Character Recognition (OCR)
OCR is an online character recognition service that recognizes characters in images or scanned copies and converts them into editable text. OCR supports
certificate recognition, receipt recognition, custom template recognition, and general table text recognition.

General OCR Receipt OCR Card OCR Domain OCR Custom OCR

General Text OCR VAT Invoice OCR ID Card OCR Electronic Waybill OCR User-defined templates

Dedicated API customization

General Table OCR Vehicle Sales Invoice OCR Driving License OCR Insurance Policy OCR
and development

Web Image OCR Taxi Invoice OCR Vehicle License OCR

Auto Classification OCR Train Ticket OCR Passport OCR

Handwritten Letters & Digits

Quota Invoice OCR Bank Card OCR
OCR

Signature & Seal OCR Toll Invoice OCR Business License OCR

Flight Itinerary OCR Transportation License OCR

Chinese and English Customs

Plate Number OCR
Form OCR

Business Card Recognition

Vehicle Identification Number

OCR Available
Document Check Classification
OCR Coming Soon

34 Huawei Confidential
EI Experience Center

The EI Experience Center allows you to experience AI. It is dedicated to lowering the expertise
requirements for using AI and making AI ubiquitous.

35 Huawei Confidential
EI Training Camps

36 Huawei Confidential
 Contents

1. HUAWEI CLOUD EI Overview

2. EI Intelligent Twins

3. AI Services

4. Case Studies of HUAWEI CLOUD EI

37 Huawei Confidential
Financial Reimbursement
Batch scanning of
paper invoices 24/7 online system running and RPA-
based full-process automation OA/ERP systems

Invoice
Manual review
Classification association
Key information
correction of Ticket OCR
Uploaded images extraction Invoice Repeated
cutting
taken by phones verification detection Image storage
servers

• Multiple access modes: Automatically connects to scanners, high-speed document scanners, and
mobile phones to obtain images in batches. Efficiency
Operation
• Flexible deployment: Supports multiple deployment modes, such as public cloud, HCS, and appliance, improvement and
optimization
cost reduction
and provides unified standard APIs.
• Support for various invoices: VAT invoices/special invoices/electronic invoices/ETC invoices/bills, taxi
invoices/train invoices/travel invoices/fixed-amount invoices/toll invoices
• Multiple invoices in an image: Automatically classifies and recognizes multiple types of invoices. Streamlined Enhanced
• Visualized comparison: The location information is returned and converted into an Excel file for process compliance
statistics and analysis.

38 Huawei Confidential
Medical Insurance Claims
Original Image obtaining Text recognition from Insurance claim
documents drawings by OCR handling

Document scanning

Electronic image

Automatic entry
Result
HUAWEI CLOUD
ID card OCR
High-speed Database
document scanner API calling
Input: image Manual verification
Bank card Output: text Structured
Insurance claim
text data

Scanner Reasonableness check

Medical invoice
Decision archive
Informatization
system
Medical checklist Mobile
phone

• Greatly accelerates data recording and verification, improves user experience, and reduces labor costs.

• Extracts structured information about images, such as the car sales invoice and effectively handles the image rotation, error
line, blur, and deformation issues. The accuracy rate is higher than 98%.

• Automatically detects contract signatures to ensure compliance.

39 Huawei Confidential
Intelligent Customer Service Solution
Machines assisting humans
• The robot preliminarily filters user problems and transfers
the problems that cannot be solved to the customer
service personnel to reduce the workload of. the
Multi-channel
customer service personnel. access All-scenario services Open APIs Service
Robots replace humans to answer questions application
• The robot answers questions more accurately, Web IM Operation management field control Web IM
comprehensively, and quickly, reducing the training system Service
WeChat applet
workload of customer service personnel. system
Intelligent routing API
• The robot identifies the department or business line Assistance

Intelligent
Weibo

service ticket
robot

Intelligent
messaging
corresponding to a specific problem through dialogs with Online

Instant
Website
users, and then transfers the call to the target manual

IM
App JS SDK
service or invokes the target sub-robot to facilitate Training
subsequent customer service data statistics.
Email
Intelligent quality inspection Intelligent Custom
App
• The robot performs offline inspection on the service

Service Go
IVR

Voice robot
Form

center
quality of the manual customer service in batches to

Call
identify user emotions and help seize the blind spots of Web hook
SMS
the customer service to effectively improve the service Third-party
quality. Intelligent App
Phone outbound call apps
Hot issue mining Offline SDK
• Based on the questions raised by users, the robot clusters
the questions, calculates the frequency, and sorts out hot
questions to learn about user feedback in a timely
manner.

40 Huawei Confidential
Large-Screen Voice Assistant in Longzihu, Zhengdong New Area

Challenges
• On the large screen, it is complex to select the content to display.
• It is difficult to quickly extract required information from a large
amount of data.
• Onsite presentation and large-screen interaction cannot be
automated.

Solution
• The large-screen voice assistant solution combines the large-screen
control system with bot data.
• Voice navigation is used to display specific pages on the large screen
through conversations.
• Voice Q&A is available, so you can interact with the big screen to
query metrics and trends through conversations.

Benefits
• Frees your hands and simplifies operations. You do not need to
operate the large screen on your computer;
• Data can be queried in a more timely manner without manual sorting
and searching.

41 Huawei Confidential
Return Visits to the Insurance Industry
1. Proactive learning
3. Semantic understanding 2. Speech recognition investigations through calls

Recognition rate Intention Speech

recognition
2. Real-time
streaming voice
recognition
4. Answering questions, or asking
questions for multiple rounds and
• High recognition accuracy and voice recognition technology that Call center
Intelligent robot filling in slots IVR
has been verified on a large scale. engine 6. Speech synthesis
Service system Repeat steps 2 to 5.
invoking TTS
Hot words 7. Play

5. Answer generation 8. Hang up

• Use hot words to improve the recognition accuracy of professional Architecture and process of
words. inbound call robots

High concurrency and low latency 2. Speech

1. Calling Out
recognition
2. Real-time
3. Semantic
• Background services can be elastically scaled to meet high understanding
ASR streaming voice
concurrency requirements.
CBS smart call
• Real-time stream identification has a low delay and low IVR engine
5. Speech
Call center IVR
synthesis
process customization. Repeat steps 2
6. Play to 5.
• Multiple concurrent calls are supported, and the response time is 4. Answer TTS
7. Hang up
within milliseconds. generation
Architecture and process of
outbound call robots
42 Huawei Confidential
AI-Assisted Diagnosis of COVID-19
Scenario
CT imaging is an important means to evaluate the condition
of COVID-19 patients. Manual screening takes at least 12
minutes, and expert resources are scarce.
HUAWEI CLOUD works with Huazhong University of Science
and Technology and BlueNet to launch the AI-assisted COVID-
19 screening service to deliver AI medical image analysis
capabilities, improving the quality and efficiency of auxiliary
diagnosis.

Solution advantages
• Second-level response: Outputs screening results in
seconds, improving doctors' image reading efficiency;
• 3D reconstruction of lesions: Detects the lesions and
their affected positions, automatically measures the
volume, and outputs anatomy and location through 3D
reconstruction.
• Comparison of follow-up positioning: Helps doctors
effectively evaluate the progress of patients' illnesses
and the efficacy of medication.

43 Huawei Confidential
 Summary

⚫ This course first introduces the HUAWEI CLOUD EI ecosystem, helping you
understand HUAWEI CLOUD EI services. Then, it focuses on ModelArts and AI
services, helping you quickly understand ModelArts. Finally, it introduces success
stories of EI.

44 Huawei Confidential
 Quiz

1. Which of the following scenarios is EI suitable? ( )

A. Smart government

B. Smart city

C. Smart manufacturing

D. Smart finance

45 Huawei Confidential
 More Information

Huawei Talent Online

https://e.huawei.com/en/talent/#/

WeChat official accounts:

EMUI Huawei Device Open Huawei Contact Huawei

Lab Developer Talent Online
46 Huawei Confidential
Thank you. 把数字世界带入每个人、每个家庭、
每个组织，构建万物互联的智能世界。
Bring digital to every person, home, and
organization for a fully connected,
intelligent world.

Copyright©2022 Huawei Technologies Co., Ltd.

All Rights Reserved.

The information in this document may contain predictive

statements including, without limitation, statements regarding
the future financial and operating results, future product
portfolio, new technology, etc. There are a number of factors that
could cause actual results and developments to differ materially
from those expressed or implied in the predictive statements.
Therefore, such information is provided for reference purpose
only and constitutes neither an offer nor an acceptance. Huawei
may change the information at any time without notice.