ISO / IEC AI

Workshop
Series

SC 42 – Artificial Intelligence
Welcome
Thank you for participating!

SC 42 – Artificial Intelligence
Acknowledgments

SC 42 – Artificial Intelligence
Program

SC 42 – Artificial Intelligence
Session 1
Program Talk Title Track Chairs Aprox Start Time (UTC)
Kickoff 13:00
Wael William Diab

Opening Remarks 13:05

Gilles Thonet
Phil Wennblom

Rohit Israni
AI Applications
Catherine Nelson 13:15
Mike Glickman Health informatics and AI, the road to Interoperability
Neil Frost AI potential Applications in Transport
Christophe Preube AI finds awareness in standardization work – the White paper of ISO/TMB SMCC

Peter Deussen
Novel AI Standardization Approaches
Norbert Bensalem 14:25
Kimberly Lucy Creating Trust in AI Through Standards: A Management System Approach
Wo Chang Data Quality for Analytics and Machine Learning
Tim McGarr / Florian Ostmann AI Standards Hub (UK)
Viveka Bonde Novel Standerdization Approach to AI Ethics

SC 42 – Artificial Intelligence
Session 2
Program Talk Title Track Chairs Aprox Start Time (UTC)
Kickoff N/A 5:00
Wael William Diab

Opening Remarks N/A 5:05

Silvio Dulinsky

Emerging AI Requirements Catherine Nelson and Peter Deussen 5:10

Elham Tabassi NIST AI Risk Management Framework
The UNESCO Recommendation on the Ethics of AI - Setting the
Mariagrazia Squicciarini standards for a better and more inclusive future
Liz Coll Can standards deliver consumer trust and confidence in AI

Evaluation Standards for Conformity Assessment of

Daniel Loevenich Trustworthy Cloud-based AI Applications

Emerging AI Technology Trends Norbert Bensalem and Rohit Israni 6:30

Shubhashis Sengupta Emerging AI Trends – as seen by an Industry Practitioner

How specialized AI drives value for Automotive Service

Tilak Kasturi Organizations
William Uppington AI Quality: The Next Big Challenge in AI
Babak Hodjat From Data to Decisions, and Back

SC 42 – Artificial Intelligence
Session 3
Program Talk Title Track Chairs Aprox Start Time (UTC)
Kickoff 22:00
Wael William Diab

AI Applications Rohit Israni and Catherine Nelson 22:05

Charalambos Freed Safety considerations in autonomous products
Development of SRD 63416 “Ethical Considerations of AI
Hajime Yamada when Applied in the AAL Context”
Kenzo Nonami AI Powered UAV and Future Prospects
Use cases and AI application guidelines in international
Fumihiro Maruyama standardization

Novel AI Standardization Approaches Norbert Bensalem and Peter Deussen 23:30

Jochen Friedrich AI Standardisation supporting regulatory needs
Developing a TS on assessment of machine learning
Linzhing Meng / Mike Thieme classification performance
Yonosuke Harada Introduction of Governance Implications of AI Systems
The foundational standards for AI – ISO/IEC 22989 and
Paul Cotton / Milan Patel / Wei Wei ISO/IEC 23053

Closing Remarks Wael William Diab 0:55

Wael William Diab Overview and introduction of SC 42
Program Committee Insights from the workshop

SC 42 – Artificial Intelligence
Thank you

[email protected] Wael’s LinkedIn

Website

LinkedIn Page

SC 42 – Artificial Intelligence
Safety
considerations
in
autonomous
products

SC 42 – Artificial Intelligence
Introduction

SC 42 – Artificial Intelligence
 Nilfisk – Charalambos Freed
• Vice President – Head of Standardization and • IEC Member ACOS TF Collaborative Safety • CLC Convenor TC 61 WG 6 + WG 10
Government Relations
• IEC Secretary SC 61J • CLC Convenor TC 59 X WG 06-03+04
• Chairman Technical Committee EUnited Cleaning
• IEC Convenor TC 31 JMT62784 • CLC Convenor TC 31 WG 24
• Chairman Arbeitskreis Technik VDMA FV
• IEC Secretary TC 61 WG44 • Issue Manager EN 60335-2-67, -68,
Reinigungssysteme
• IEC Secretary TC 61 WG49 -69, -72, -79, EN 62784
• IECEX Management Committee – Head of Greek
• IEC Convenor SC 61J MT 03 • DKE Convenor AK 513.7.10
Delegation
• IEC Convenor SC 59F WG 06 • IEC 1906 Award Winner
• Member WG Testing Bundesverband
Deutsche Industrie • Chairman CSA C234

SC 42 – Artificial Intelligence
SC 42 – Artificial Intelligence
 New trends impacting safety

Intelligent systems: primary drivers of

digital transformation.

Coexisting safety Collaboration vs Coexistence: safety

considerations

Concerns: privacy, data integrity, safety,

security, transparency and trustworthiness
of new technologies.

SC 42 – Artificial Intelligence
Projects related to collaborative safety
ACOS 2nd survey among safety related TCs

SC 42 – Artificial Intelligence
ACOS task force
SMB decision 170/16

SC 42 – Artificial Intelligence
IEC SC 61J: commercial/industrial cleaning equipment

SC 42 – Artificial Intelligence
Transformation of machine: from manual to autonomous mode

„magic addition“

manual autonomous

SC 42 – Artificial Intelligence
 „Magic ingredients“
Parameter
Manual Autonomous
Electrical and mechanical safety Y Y
EMC requirements Y Y
Attended machine Y N
Additional requirements as unattended machine - Y
on
• motor,
• plastics,
• hardware, software,
• environment.
Obstacle and drop-off detection - Y

SC 42 – Artificial Intelligence
 Environment: coexisting safety

Coexisting Safety
machine ensured safety, where humans share the same physical space with automatic / autonomous
machines (robots, mobile robots), but generally act independently or with different tasks.
Collaborative Safety
machine ensured safety, when humans perform intended mutual actions together with
automatic / autonomous machines (robots, mobile robot) to accomplish a common task
SC 42 – Artificial Intelligence
Environment: coexisting safety

IMPORTANT:
Intentional movements of persons towards the appliance in order to harm themselves, is not covered. The
result would be a static appliance…

SC 42 – Artificial Intelligence
Route on getting the safety up and running

Liaison Officer IEC TC 61 → ISO TC 299: Mr. Dejun MA (CN)

Liaison Officer IEC SC 61J to ISO TC 299: Mr. Charalambos Freed (GR)

IEC TC 61 WG 44: Safety of robots for household and similar use

Chair: Mr. Dejun MA (CN); Secretary: Mr. Charalambos FREED (GR)
Standard: Draft based partially on IEC 63327

IEC SC 61J MT 3, IEC 63327: Safety standard for autonomous commercial

floor treatment machines
Chair: Mr. Charalambos FREED (GR)
Secretaries: Mr. Kenneth WILLS (US); Mr. Leonard LETEA (CA)
Standard: IEC 63327; based on CSA/ANSI C22.2 No. 336-17

CSA C234: Safety electrically operated appliances

Chair: Mr. Charalambos FREED (CAN)
Standard: CSA/ANSI C22.2 No. 336-17

SC 42 – Artificial Intelligence
 Commercial environment
Type and purpose of safety-critical function (SCF) Minimum required performance level (PL) as
described in ISO 13849-1

Type 1 machines Type 2 machines

Drop-offs detection PL = d PL = d
Prevent intrusions into stopping/contact zones to PL = b PL = d
prevent crushing + collision
Prevent exceeding autonomous mode speed PL = b PL = d
Provide locked state of drive wheels PL = b PL = b
Provide desired (emergency) switch-off PL = b PL = c
Provide desired stop category 0, 1, or 2 PL = b PL = d
type 1 machine: velocity max. 3 km/h, height max. 50 cm and weight max. 20 kg
type 2 machine: other than a type 1 machine

PL and definition
ISO 13482: Robots and robotic devices — Safety requirements for personal care robots
SC 42 – Artificial Intelligence
 Tests

SC 42 – Artificial Intelligence
Autonomous test – start appliance

Starting of motor-operated appliances

General conditions for the tests: Obstacles detection in direction of motion

SC 42 – Artificial Intelligence
Autonomous test – drop off

SC 42 – Artificial Intelligence
Autonomous test – speed and stopping distance

SC 42 – Artificial Intelligence
Autonomous test – obstacle detection

SC 42 – Artificial Intelligence
Autonomous test – confined space

SC 42 – Artificial Intelligence
Docking Station

SC 42 – Artificial Intelligence
Adding more „intelligence“

SC 42 – Artificial Intelligence
Software and communication

SC 42 – Artificial Intelligence
IEC 60335-1 - excerpt

SC 42 – Artificial Intelligence
IEC 60335-1 - excerpt

SC 42 – Artificial Intelligence
IEC 60335-1 - excerpt

SC 42 – Artificial Intelligence
Classifications
ISO-IEC TR 5469 proposes a classification scheme for AI used in safety related applications

SC 42 – Artificial Intelligence
Thank you

SC 42 – Artificial Intelligence
Development of SRD
63416 “Ethical
Considerations of AI
when Applied in the
AAL Context”

SC 42 – Artificial Intelligence
Hajime Yamada

• Professor Emeritus of Toyo

University, Japan
• HoD for Japan to IEC System
Committee AAL (Active
Assisted Living)
• HoD for Japan to ISO TC 314
(Ageing Societies)

SC 42 – Artificial Intelligence
Population ageing and the value of AAL

• The global population is • Reducing the burden of long-

ageing. term care for older persons
• “United Nations World Population becomes a major policy issue
Prospects 2019” in every country. Active
• In 2018, for the first time in human Assisted Living (AAL)
history, persons aged 65 years or systems help older
over outnumbered children under persons' daily living so that
five years of age worldwide. they can live independently
• In 2050 the 1.5 billion people aged as long as possible. AAL
65 years or over worldwide will can be a solution to
outnumber adolescents and youth population ageing.
aged 15 to 24 years (1.3 billion).
What AAL can do

• For the well, AAL can • For the vulnerable, AAL can
provide an increase in: reduce
• Convenience • Fear - of falls, of adverse
• Time-saving events, of dying and no-
• Self-reliance one knowing
• Ability to monitor their own • Frustration - the loss of
health ability to do ordinary tasks
• Enjoyment of life • Forgetfulness - medication,
appointments, cooking,
location…
• Social isolation
Establishment of IEC SyC AAL in 2015

• The Systems Committee • Enables cross-vendor

shall: interoperability of AAL
• Create a vision of Active systems, services,
Assisted Living that takes into products and components
account the evolution of the • Addresses systems level
market aspects such as safety,
• Foster standardisation which: security and privacy
enables usability and • Communicate the work of the
accessibility of AAL systems SyC appropriately to foster a
and services strong community of
stakeholders
Organization and participation in IEC SyC AAL
Chair: U. Haltrich (DE)/Sec.: M. Siket (CO)

• 14 P members: CA, CN, FR, • WG 1: User Focus

DE, IN, IT, JP, KR, NL, NZ, • WG 2: Architecture and
SE, CH, GB, US Interoperability
• 12 O Members: AT, BE, HR, • WG 3: Quality and Conformity
CZ, DK, FI, HU, MY, NO, RU, assessment
SG, ES • WG 4: Regulatory Affairs
• WG 5: AAL in the connected home
environment
• AG 1: Chair's Advisory Group
• WG 7: Cooperative multiple
• JAG 8: Joint Advisory Group linked systems - Functional safety
to TC 100 and TC 124
• MT 6: IEV - Part 871
AAL use case two-dimensional classification

• Use case category • AAL care recipient’s level of

• Prevention and assistance
management of chronic • Level 0: Independent
long-term conditions • Level 1: Some assistance
• Social interaction • Level 2: Assistance with
• Mobility IADL (transportation,
• Health & wellness housekeeping, medication
management, etc.)
• (Self-)management of daily
life activities at home • Level 3: Assistance with
ADL (walking, bathing,
grooming and getting dressed,
continence and eating, etc.)
AAL use case examples

• Personal Health check • Wandering Detection and

• The care recipient wears, or is Diversion (WDD) for
monitored by, sensors that collect persons with Dementia
vital signs and transmit these to a • In some cases persons with
monitoring centre. The system dementia are exit seeking and
uses AI to monitor the incoming frequently try to leave their
data, assesses any changes in residences. … A WDD system may
ongoing physical or cognitive interact with the person to
health or potential concerns, dissuade them from exiting (e.g.
notifies the user and/or physician encourage a return to bed during
of these concerns. the night). …
Cost effectiveness of AAL system
(IEC SRD 63234-2:2020)

• In case of potential concerns, • Cash Flow Analysis Scenario

“Personal Health check” A – Ramp up to 100,000
system displays an ‘Alert’ to a patients in five years
Clinician/Nurse at the centre. demonstrates Return on
• If deemed appropriate, the Investment of 172%.
Clinician/Nurse will contact • Cash Flow Analysis Scenario
the patient or dispatch a B – Ramp up to 10,000
homecare nurse to visit the patients in five years
patient. demonstrates Return on
Investment of 147%.
Changing conditions of AAL care recipients

• The AAL care recipients’ • Cluster analysis on aging of

conditions change over the Japanese male (H. Akiyama)
years.
LV. 0
• With this change, the degree
of physical activity may 11%
1 19%
decline and simultaneously
70%
their cognition and self-
2
judgment ability may also
decline.
3
Considerations necessary in the context of
AAL: Prior consent
• Since many AAL systems use • For level 0 AAL care recipient,
the user's personal data to the prior consent of the user
provide services to the AAL can be obtained without
care recipient, the consent problem.
should be obtained in • In the case of Level 3 AAL
advance regarding the use of care recipient, it is
AAL system. appropriate to seek the
• No matter what level the consent of not only the AAL
AAL care recipient is, the care recipient but also of
principle of protecting the designated parties such as
dignity of that person shall family members or other
not be violated. substitute decision makers.
Considerations necessary in the context of
AAL: Autonomy or safety
• All individuals have the right • The AAL care recipient
to engage in activities that requests assistance to go for
may put themselves at risk. walk outside on a cold day,
• However, as cognition • the AI system must decide:
declines, users may make a • to allow this request
request of a system that puts without notification to other
their own safety at a higher family members
level of risk and the AI must (respecting autonomy and
determine if the level of risk privacy); or
is still appropriate before • to reject this request for
acting while maximizing the the safety of the AAL care
user’s autonomy. recipient.
Considerations necessary in the context of
AAL: Fail safe
• When the “Personal Health • Multiple AAL systems operate
check” system asks to simultaneously for the AAL
dispatch a homecare nurse to care recipient must not be
visit the patient, independent but
• At the same time if the communicate each other
Wandering Detection and and operate based on the
Diversion system locks the principle of Fail Safe.
door of the person’s house,
• What will happen?
Conclusions: The Needs of developing
guidance document for the use AI in AAL
• The use of AI in AAL must • In addition, the use of AI in
consider ethical AAL must take into account
considerations such as declines of physical, cognitive
privacy, accountability, safety and/or self-judgment abilities
and security, transparency of AAL care recipients.
and explainability, fairness • SyC AAL needs to develop
and non-discrimination etc. an ethical document
that are provided by various treating issues such as who
standards and consortia to ask prior consent,
activities. balance between safety and
autonomy, and fail safe.
Thank you!
 AI Powered
UAV and
Future
Prospects

SC 42 – Artificial Intelligence
翻訳についての詳細を確認するにはソーステキストが必要です
フィードバックを送信
サイドパネル

There are a wide variety of AI applications in the field of unmanned aerial

Introduction vehicles. In particular, the application of AI to the autopilot section, which is the
brain of UAVs, is the most important and most urgent field. Manned aircraft
make various decisions when a skilled pilot experiences an anomaly. On the
other hand, UAVs are now judged by those who are operating on the ground.
This makes it insufficient to deal with abnormalities. It is necessary for the edge
computer mounted on the aircraft to provide guidance, which is the role of the
pilot. This is AI technology. This makes pilotless operation feasible.

SC 42 – Artificial Intelligence
Relationships with the five components of the drone

Drone side ①
Control System Propulsion Bodyframe System
Fixed、Rotary、VTOL
AP System （ Flight performance determined by
（GS,NS,FC, GNSS, Propeller、Motor
airframe structure ）
QZSS, sensors、 ESC、Battery
（Electric、Engine、 Communication
Communication protocol)
Hybrid） ② Systems
③ ④

C2 Link
Grund Control
④ Communication Station
System
⑤
Ground side ACSL 3
 Autopilot consisting of GS, NS and FC
Autopilot(AP）

Current technology：AP=NS＋FC without GS ACSL 4

Revolution of Autonomy & Prospect of Celebrum AP（AI Pilot）

Ideal state of autonomous control in near future (Bio-inspired flight) ACSL 5

5 Levels of Autonomy & Revolution for Celebrum AP
Revolution of Autonomy & Prospect of Celebrum AP

一般社団法人 日本ドローンコンソーシアム（JDC)
 Level 5 Flight as Redundant Celebrum AP and AI Implementation
3rd Stage 2nd Stage

Automatic flight
Positioning Collision avoidance（DAA）
route generation and ＧＮＳＳ, QZSS（CLAS）, Manned and unmanned aerial
search for the Non-GNSS environment vehicles, birds, power lines, trees
（Vision based） (By deep learning by AI Flying
optimal route by AI
smarter every day)

1st Stage ◆Takeoff control, landing control, attitude stabilization control,

◆ Distance control for multiple aircrafts
◆ AI conducts a health check to detect abnormalities during flight Redundant system
as control when an error occurs (One Fail Operative Control)
◆ Implementation of AI for response to local sudden weather events
◆ Control to determine and guide the emergency landing point with AI when an
abnormality occurs
◆ AI implementation and redundant Celebrum AP for the urban flight as Level 5
ACSL 8
 Technical challenges for evolution to cerebral drones
1. Natural environment (wind resistance, low temperature resistance, high temperature resistance, rain
resistance, snow resistance, dust resistance, lightning resistance)
2. Flight in GNSS-resistant lost environment (autonomous flight in non-GPS environment,
Level 5)
3. Airworthiness (redundant AP, reliability, durability, safety and AI implementation,
Level 5)
4. Ground Station GCS Trouble (Completely Autonomous Flight Control, Level 5)

5. Communication resistance trouble (disruption, jamming, cyber security, Level 5)

6. Obstacle autonomous avoidance and real-time routing (SAA & RT-TP, Level 5)

7. Flight performance 1 (long distance, long time flight, high speed flight, 100km / 1 hour / 100km /h) )

8. Flight performance 2 (high-precision landing, safe emergency landing function, within 50

cm))
9. Flight performance 3 (payload increase, transport capacity of about 10 kg )
Collision Avoidance by CAP and AI Implementation with Yaw Control

Cerebrum AP technology and AI ：AP=NS＋FC with GS 10

 AI drone flying in an unknown environment

第１回

一般財団法人 先端ロボティクス財団
Collision Avoidance by CAP and AI Implementation with Edge and Cloud
Edge Computer (CAP)
・ Short-term collision avoidance
・ Physical collision avoidance
・ Return to the original route

Cloud Computer
(UTM)
・ Long-term collision
avoidance
・ Planned collision
avoidance
・ Airspace separation
・ Automatic route
generation
・ By new route flight
13
 Dynamic Map for Autonomous Vehicles VS
Dynamic Map for Autonomous Flight Control of Drone
Dynamic Map for Autonomous Driving Dynamic Map for Autonomous Drone
Type 4: Highly dynamic data
• Airspace: drones, helicopter, manned aircraft,
birds, weather report, UTM info, ATM info,
Airspace info
• Ground: people, cars
Type 3: Transient(Semi) dynamic data
• Airspace: wind, precipitation,
telecommunication info, drone traffic info
• Ground: crowd, drone station info
Type 2: Transient(Semi) static data
• Ground: drone landing spot info. landing spot
traffic info
Type 1: Permanent static data
• Airspace/Ground: high precision 3D map data,
powerlines, building data, trees and vegetation
data, emergency landing area
• Data in dark red can be efficiently processed by AI (Deep Learning) using visual/thermal sensor data
• Note that drones need to consider “Ground” (1) to find a suitable landing spot, or (2) to estimate risk(GRC,
ARC), e.g., risk of falling on people → see SORA

Shimada, H., Yamaguchi, A., Takada, H. and Sato, K. (2015) Implementation and Evaluation of Local Dynamic Map in Safety Driving Systems. Journal of Transportation Technologies, 5, 102-112.
http://dx.doi.org/10.4236/jtts.2015.52010
Industrial Utilization for Drone of AI with Dynamic Map
Objective: Creation of Dynamic Map (DM) by Drone

DM is the result of a mapping technology with

Input: Example 4 : Industrial utilization of AI by Dynamic Map

◆Video feed from camera attached to drone

Output:

◆ Real-time categorization (detection, recognition) and

tracking of each object and object behavior from
video feed

16
Industrial Utilization for Drone of AI with Dynamic Map (cont’d)
These services will use dedicated data analytics algorithms to
predict traffic, movement patterns, etc.
DM-based Traffic/Crowd Surveillance / Disaster
Delivery
Services Management Response

Estimate position
Count & track Estimate traffic Estimate crowd for of fire for safe
vehicles at condition from Track
safe flight path delivery
construction site vehicle movement
trajectories of monkeys
Human action Track direction of fire
recognition

1 17
DM with static and dynamic objects
 Situational Awareness of AI with Dynamic Map
▪ Semantic
segmentation
(pixel-wise labeling)
of video stream

Input video frame

by frame

18
 Situational Awareness of Moving Objects on Superchip
Video Frame Segmentation Confidence

“White” color:
high confidence
in segmentation
NVIDIA Jetson TX1 credit
Hardware results
card sized chip (1TFLOP/s)
Result: 3 frames per sec (non-optimized code) → real-time
 Surveillance by Deep Learning
Multi-Person Detection
▪ Trained on resized images (512x512) for 20K iterations
▪ Single Shot MultiBox Detector (SSD) 72.3% mean Average Precision (mAP)
Animation of Annotated Dataset
Abnormality Detection for Bridge Pier
 インフラ維持管理分野での社会実装の課題

一般社団法人 日本ドローンコンソーシアム（JDC)
Abnormality Detection for Bridge Pier (cont’d)
 Capability of AI Technology for UAV (1)
Manufacturing Process of Drone
1. Parts check, quality inspection and quality assurance in the drone manufacturing process using AI
2. Assembly process check, quality inspection and quality assurance in drone assembly process
using AI

AI based Drone Flight (Flight replacing manned pilots with AI)

1. Health monitoring during AI drone flight, abnormality diagnosis, abnormality detection and cause
investigation
2. AI based Guidance and Path planning using time-environment dynamic map
3. Searching for a crash landing point and determining a crash landing when an abnormality occurs

UTM Pilot License Procedures

1. Airspace management using AI 1. School examiner certification
2. Collision avoidance using AI 2. School operator skill certification
3. Take off and landing management using AI 3. Paper test judgment
4. Weather check during drone flight using AI
5. Dynamic map creation with AI
25
 Capability of AI Technology for UAV (2)
Utilization of AI in precision agriculture
1. Analysis of growth surveys and disease outbreaks in precision agriculture using Deep learning AI
2. Determining the optimal harvest time
3. Others
Utilization of AI by dynamic map
1. These services will use dedicated data analytics algorithms to predict traffic, movement patterns,
disaster responses, etc.
2. Delivery service and optimization
3. Warehouse management
4. Precision 3D mapping and database
5. Security
6. Aerial survey and construction
7. Precise whether report
8. Others
Inspection of infrastructure equipment using AI
1. Abnormality detection for bridge, tunnel, building, tower and chemical plant, etc (See example 5)
2. Advanced maintenance and operation
3. Others 26
EASA AI Roadmap
EASA: European Union Aviation Safety Agency
 Summary Research and development of cerebral drones is being carried out
all over the world in fierce competition through industry-academia-
government collaboration. By the end of the 2020s, drones that
require no human operation may appear. It will be a real “industrial
revolution of the sky". At that time, the world will change games
with AI technology innovation.

SC 42 – Artificial Intelligence
 Thank you
Kenzo Nonami、Ph.D
[email protected]
URL: https://arf.or.jp

SC 42 – Artificial Intelligence
Use cases and
AI application
guidelines in
international
standardization

SC 42 – Artificial Intelligence
Introduction

SC 42 – Artificial Intelligence
Structure of SC 42
WG 1 Foundational standards

WG 2 Big data

WG 3 Trustworthiness

WG 4 Use cases and applications

WG 5 Computational approaches and computational characteristics of AI systems

JWG 1 JWG SC 42 – SC 40: Governance implications of AI

JWG 2 JWG SC 42 – SC 7: Testing of AI-based systems

AG 3 AI standardization roadmapping

AHG 1 Dissemination and outreach

AHG 2 Liaison with JTC 1/SC 38

AHG 4 Liaison with JTC 1/SC 27

SC 42 – Artificial Intelligence
SC 42/WG 4 “Use cases and applications”
• Overview
• Identify AI application domains, context of AI use in those domains and develop guidance
• Collect representative use cases and analyze them for derived requirements

• Officers
• Convenor: Fumi Maruyama
• Secretary: Nobu Hosokawa
• Project Editors:
• Yuchang Cheng (ISO/IEC CD 5338 – AI system life cycle processes, ISO/IEC TR 24030 –
Use cases)
• Shrikant Bhat (ISO/IEC CD 5339 – Guidelines for AI applications)

SC 42 – Artificial Intelligence
ISO/IEC TR 24030 Artificial Intelligence – Use cases

SDO: Standards Developing Organization

SC 42 – Artificial Intelligence
Collaboration through exchange of AI use cases

Vertical
Domain
SDO

Providing relevant AI use cases Collecting novel AI use cases

for their AI applications with new requirements

ISO/IEC JTC 1/SC 42 as systems integration entity

Providing relevant AI use cases Collecting novel AI use cases

for their analysis with new requirements

HorizontalSDO
Horizontal SDO(e.g.
(e.g.ISO/IEC
ISO/IECJTC
JTC1/SC
1/SC27)
27)
Horizontal SDO (e.g. ISO/IEC JTC 1/SC 27)

SC 42 – Artificial Intelligence
Application domain

SC 42 – Artificial Intelligence
AI task

SC 42 – Artificial Intelligence
Use case status

SC 42 – Artificial Intelligence
Example use case “AI solution to quickly identify defects during
quality assurance process on wind turbine blades”

Missing Wrinkle defect

back-wall echo

Foreign object or
dry glass defect

SC 42 – Artificial Intelligence
Example use case in template (1 of 3)
1.ID 4
2. Use case name Detecting defects during quality assurance process on wind turbine blades
3. Application manufacturing
domain
4. Deployment on-premise systems
model
To find an accurate and efficient solution to detect defects without compromising the
5. Objective(s)
detection of in-material damage and risking a loss in the manufacturer’s reputation.
This AI solution can automatically detect defects in produced wind turbine blades through
"imagification", which transforms raw data from the non-destructive ultrasonic testing
(UT) scanner into image data based on RGB where deep learning-based image
recognition can be applied effectively. It achieved high coverage (see 9. KPIs) of more
than 95 % for various defects, e.g. detecting missing back wall echo, foreign objects, dry
glass defects and wrinkle defects. Quality controllers can focus their efforts on suspicious
regions and disregard all clean data; humans are only expected to examine the regions
of blades that are flagged as suspicious by the AI solution. Since it achieved low split
6. Narrative
(see 9. KPIs) of less than 20 %, it reduced evaluation time for UT scanning data by 80 %.
Each blade can be up to 75 meters in length and takes a highly skilled professional
quality controller up to 6 hours to evaluate the UT scanning data in the quality assurance
process. With 5,000 blades produced every year, that adds up to a saving of almost
32,000 person-hours, which translates into significant cost savings, reduced production
lead times, and increased productivity. Today, there is a shortage of ultrasonic engineers
and inspectors. This solution enabled the same inspector to do four to five blades per
day instead of one previously.

SC 42 – Artificial Intelligence
Example use case in template (2 of 3)
The manufacturer is the customer who wants to enhance the efficiency of the quality
assurance process on wind turbine blades while maintaining the quality level of the
products.
7. Stakeholders The actual users of the solution are quality controllers and inspectors of produced
and stakeholder wind turbine blades.
considerations The producer is responsible for deploying an AI solution capable of solving the above
problem of efficiency. The provider is responsible for developing an AI system for the
solution.
Other stakeholders include utility companies and neighborhood communities.

UT Scanning Data
Source: UT scanner
Type: ultrasonic data from scanner vendor
8. Data
Velocity: batch
Characteristics
Variety: single source
Variability: static
Quality: high

Coverage (for accuracy): Proportion of the defects automatically reported without

9. Key manual inspection. Ideal target is 95 %.
performance Split (for efficiency): Proportion of the regions of the blade that are flagged as
indicators (KPIs) suspicious and asking for manual inspection. The less split, the more efficient the total
quality assurance process becomes.

SC 42 – Artificial Intelligence
Example use case in template (3 of 3)
Task(s) recognition
Level of automation partial automation
10. Features of
Method(s) deep learning
use case
Platform deep neural network
Topology convolutional neural network
11. Threats &
Changes in defects of in-material damage over time
vulnerabilities
Challenges:
Achieve the same level as ultrasonic accredited engineers for detecting critical defects.
12. Challenges Issues:
and issues — lack of defect data per defect type;
— how to create good images for deep learning from UT raw data;
— back wall detection.
The solution does not have 100 % reliability. That is where human-machine teaming comes
13. into play. It also lacks robustness and resilience. That is why it needs continuous validation
Trustworthiness (monitoring and retraining).
considerations Although the solution lacks explainability, the human-machine team has explainability, i.e.
the human quality controller and inspector can explain on behalf of the team.
14. Use of
standards; This use case is a good example of human-machine teaming. It provides standardization
standardization opportunities for empowerment and escalation for human-machine teaming.
opportunities
15. Relevant
Affordable and clean energy
SDGs

SC 42 – Artificial Intelligence
Revision of ISO/IEC TR 24030

SC 42 – Artificial Intelligence
ISO/IEC TR 24030 ed.2

SC 42 – Artificial Intelligence
ISO/IEC CD 5339 – Guidelines for AI applications
• Provides a macro-level view of an AI application to facilitate its understanding,
development and use among all stakeholders.
• Includes:
• Approach to identifying an AI application’s stakeholders, context, functional characteristics and non-
functional characteristics;
• AI application framework that can be used to answer the question: “What are the characteristics and
considerations of an AI application?”;
• Guidelines for AI applications based on the make, use and impact perspectives.

• Should be the first document to refer to when developing/using an AI application.

• Provides references to relevant standards SC 42 and other SDOs are developing.

SC 42 – Artificial Intelligence
Developing/using an AI application with ISO/IEC 5339

Establishing AI application context (stakeholder, role and process) <subclauses 5.2 and 5.3>

Confirming AI application characteristics and non-functional properties and considerations < subclauses 5.4 and 5.5>

Establishing AI application framework with stakeholder’s perspective (make, use and impact) < subclauses 6.2 and 6.3>

Answering questions posed in guidelines < subclauses 7.2, 7.3 and 7.4>

SC 42 – Artificial Intelligence
AI application framework

SC 42 – Artificial Intelligence
Example use case in framework (1 of 3)
Perspectives Make

Stakeholders AI producers Data AI developers AI application

providers providers
Ultrasonic
Who Fujitsu Limited scanner Fujitsu Limited Fujitsu Limited
vendor
Provides Develops the AI Takes the
Responsible for the entire data from system developed AI
What system ultrasonic including the system and
scanners trained model provides it as an AI
application

How Built, Applies, Updated Built Built Built

Context Design, Design,

When All Stages Develop, Develop, Verify, Deploy, Operate,
Verify, Validate Monitor
Validate
Where Producer Producer Customer Premises
The producer is expected to
provide a good AI application for
the customer because of a good
Why understanding of the objective
and requirements of the AI
application through dialogues
with the customer.

SC 42 – Artificial Intelligence
Example use case in framework (2 of 3)
Perspectives Use Impact
Stakeholders AI customers AI users Community AI regulators and
policy makers
European
Professional Utility Commission’s
quality companies, Director-General
Who Wind turbine manufacturer controller, users and (DG) for Internal
inspector of neighbours of Market, Industry,
manufacturer wind turbine Entrepreneurship
and SMEs
What
How Applies Applies
When Deploy, Operate, Monitor Deploy, Operate, Deploy,
Operate, Deploy, Operate,
Monitor Monitor Monitor

Where Customer Premises Customer

Premises
Context Any defects when a blade is in operation Professional Neighbours of The regulator
will not only prove catastrophic but also quality the wind makes sure of
inflict major damage to the controllers and turbine sites safety of products
manufacturer’s reputation. The inspectors of the are with the regulations
manufacturer produces over 5,000 wind manufacturer apprehensive for manufacturing
turbine blades every year for use in on are interested in about any process of huge
and off shore wind farms. Each blade can their increased damage products such as
productivity with caused by a wind turbine blades
Why be up to 75 meters in length and takes a
highly skilled professional quality the AI defective wind and liability of
controller up to 6 hours to evaluate the application as turbine. defective products.
scanning data in the quality assurance well as the
process. With the AI system the performance
evaluation time is reduced by 80%, which (accuracy) of the
translates into cost savings, reduced AI application.
production lead times, and increased
productivity.
SC 42 – Artificial Intelligence
Example use case in framework (3 of 3)
AI Characteristics Processes What How
Built with the capabilities of
an AI system that implements
a model to acquire Training phase – Deep learning and Transforms raw data (ultrasonic
information and processes algorithmic imagification testing scanning data) into image
with or without human processes data using RGB
intervention by algorithm or
programming
AI application
(Fujitsu F|AIR Detect defects Inspecting non-destructive ultrasonic
platform) – testing scanning data
execution phase
Applies optimizations or
inferences made with the Decision-making is done through
model to augment decisions, human-machine teaming. The AI
predictions or system can detect defects with high
recommendations in a timely Quality control or coverage (more than 95%) of
manner Decision-making inspection decision- various defects. Humans are only
making expected to examine the regions of
the blade that are flagged by the AI
system. For example, detecting
missing back wall echo, foreign
object of dry glass defect, wrinkle.

Updated and improvements Outcomes are monitored until

made to the model, system or Continuous Deep learning and certain period of time has passed
application by evaluation of validation imagification since the last training or retraining or
interaction outcomes a new production process is
introduced.

SC 42 – Artificial Intelligence
ISO/IEC CD 5338 – AI system life cycle processes
• Defines a set of processes and associated concepts for describing the life cycle
of AI systems based on machine learning and heuristic systems.

• Is based on ISO/IEC/IEEE 15288:2015 and ISO/IEC/IEEE 12207:2017 with

modifications and additions of AI-specific processes from ISO/IEC 22989 and
ISO/IEC 23053.

ISO/IEC/IEEE 15288:2015: Systems and software engineering — System life cycle processes
ISO/IEC/IEEE 12207:2017: Systems and software engineering — Software life cycle processes
ISO/IEC 22989: Information technology — Artificial intelligence — Artificial intelligence concepts and terminology
ISO/IEC 23053: Framework for Artificial Intelligence (AI) Systems Using Machine Learning (ML)

SC 42 – Artificial Intelligence
Life cycle processes for AI systems in a specific domain

AI systems AI system life cycle processes

in a specific domain in a specific domain

Circumstances in a specific domain

AI systems AI system life cycle processes ISO/IEC 5338

AI concepts, terminology and framework

ISO/IEC 22989
ISO/IEC 23053

ISO/IEC/IEEE 15288:2015
Generic systems System/software life cycle processes ISO/IEC/IEEE 12207:2017

SC 42 – Artificial Intelligence
Summary

SC 42 – Artificial Intelligence
Thank you

SC 42 – Artificial Intelligence
Dr. Jochen Friedrich – IBM Technical Relations Europe
[email protected]

© 2022 IBM Corporation

Standardising AI ?
5 Years ago…
People were wondering WHY you would
standardise AI and WHAT you would
possibly want to standardise

2 AI Standardisation supporting regulatory needs | Dr. Jochen Friedrich | [email protected] © 2022 IBM Corporation
Standardising AI – Two Aspects: Compare to Machinery

Example of machinery:
You don’t standardise how the machine works
But you standardise the input, the output and the external factors around the machine

Supply chain Outputs

Inputs Quality
Input quality Post-production
Connectivity Etc.
Etc.

Safety, Environmental factors, Etc.

3 AI Standardisation supporting regulatory needs | Dr. Jochen Friedrich | [email protected] © 2022 IBM Corporation
Standardising AI – Two Aspects: Ethics

AI has brought the topic of ETHICS into standardisation.

How do Humans and AI systems work hand-in-hand

Which are ethical impacts of running and using AI systems
What needs to be addressed for ensuring “ethical AI”
Who takes decisions and holds responsibility

4 AI Standardisation supporting regulatory needs | Dr. Jochen Friedrich | [email protected] © 2022 IBM Corporation
 Good Bad
AI AI
5 AI Standardisation supporting regulatory needs | Dr. Jochen Friedrich | [email protected] © 2020 IBM Corporation
 Transparency Trustworthiness
Risks Governance
Values
Accountability
Processes
Sovereignty
Criteria Control Data
6 AI Standardisation supporting regulatory needs | Dr. Jochen Friedrich | [email protected] © 2020 IBM Corporation
Pioneers in AI Standardisation – IEEE and ISO/IEC JTC 1

THE IEEE GLOBAL INITIATIVE ON The IEEE Global Initiative’s mission is, to ensure every
ETHICS OF AUTONOMOUS AND stakeholder involved in the design and development of
INTELLIGENT SYSTEMS autonomous and intelligent systems is educated, trained, and
empowered to prioritize ethical considerations so that these
technologies are advanced for the benefit of humanity.
[https://standards.ieee.org/industry-connections/ec/autonomous-systems/]

ISO/IEC JTC 1 SC 42 Serve as the focus and proponent for JTC 1's standardization
program on Artificial Intelligence
ARTIFICIAL INTELLIGENCE
Provide guidance to JTC 1, IEC, and ISO committees
developing Artificial Intelligence applications
[https://www.iso.org/committee/6794475.html]

7 AI Standardisation supporting regulatory needs | Dr. Jochen Friedrich | [email protected] © 2022 IBM Corporation
 IBM Policy Lab: Precision Regulation for Artificial Intelligence
5 policy imperatives for companies, based on whether they are a provider or owner (or both) of an AI system

Designate a lead AI ethics official Different rules for different risks

Responsible for trustworthy AI Initial high-level assessment of the technology’s potential for
Accountable for internal guidance and compliance harm
mechanisms In-depth and detailed assessment for higher-risk applications
Oversee risk assessments and harm mitigation strategies Document assessment processes in detail to be auditable,
and retain them for a minimum period of time

Don’t hide your AI Explain your AI Test your AI for bias.

Promote transparency is through Maintain audit trails surrounding Responsibility that AI systems are fair
disclosure input and training data and secure
Make purpose of an AI system Make available documentation Take remedial actions as needed, both
clear to consumers and informing about confidence before sale or deployment and after an
businesses measures, levels of procedural AI system is operationalised
regularity, error analysis

See https://www.ibm.com/blogs/policy/ai-precision-regulation/

8 AI Standardisation supporting regulatory needs | Dr. Jochen Friedrich | [email protected] © 2020 IBM Corporation
 US Policy Initiatives Around AI

NATIONAL ARTIFICIAL INTELLIGENCE INITIATIVE ACT OF 2020 (DIVISION E, SEC. 5001)

AI IN GOVERNMENT ACT OF 2020 (DIVISION U, TITLE I)
EO 13859, MAINTAINING AMERICAN LEADERSHIP IN AI
EO 13960, PROMOTING USE OF TRUSTWORTHY AI IN FEDERAL GOVERNMENT
OMB (EO Office of Management and Budget) GUIDANCE FOR THE REGULATION OF AI
NATIONAL ARTIFICIAL INTELLIGENCE INITIATIVE ACT OF 2020 (DIVISION E, SEC. 5001)

➔ ensure continued U.S. leadership in AI R&D

➔ lead the world in the development and use of trustworthy AI systems in public and private
sectors
➔ prepare the present and future US workforce for the integration of artificial intelligence systems
across all sectors of the economy and society
➔ support opportunities for international cooperation with strategic allies on R&D, assessment,
and resources for trustworthy AI systems
[https://www.ai.gov/legislation-and-executive-orders/]
9 AI Standardisation supporting regulatory needs | Dr. Jochen Friedrich | [email protected] © 2022 IBM Corporation
 European Union High Level Expert Group on AI

Ethics Guidelines on Artificial Intelligence:

The Guidelines put forward a human-centric approach on AI and list 7
key requirements that AI systems should meet in order to be
trustworthy.

Human agency and oversight Diversity, non-discrimination and

fairness
Technical robustness and safety
Societal and environmental well-
Privacy and Data governance being
Transparency Accountability
[https://ec.europa.eu/futurium/en/european-ai-alliance/ai-hleg-steering-group-european-ai-alliance.html ]

10 AI Standardisation supporting regulatory needs | Dr. Jochen Friedrich | [email protected] © 2020 IBM Corporation
OECD.AI – Focus on AI with a Broad Scope

[https://oecd.ai/en/]

11 AI Standardisation supporting regulatory needs | Dr. Jochen Friedrich | [email protected] © 2022 IBM Corporation
Good governance and ethical standards are of key interest for the
provision and adoption of AI technologies.
Governments around the world look at promoting the uptake of AI
technologies and ensuring leadership and societal benefits of AI via
appropriate regulatory and policy measures.
Open and voluntary standardisation has come a good way in providing
relevant standards for ensuring high-quality, trustworthy and ethical AI
systems.
Open and voluntary standards play a key role for delivering “good AI”.
 AI Regulation and Standardisation
European Commission issued a proposal for an AI regulation
It includes a risk-based approach and is modelled according to the New Legislative Framework, i.e. with
harmonised European Standards as a basis for demonstrating compliance.
EU New Legislative Framework (NLF)

Well established
and well working
processes for
technical
regulation

Link to
international
standardisation

13 AI Standardisation supporting regulatory needs | Dr. Jochen Friedrich | [email protected] © 2020 IBM Corporation
New regulated areas: Artificial Intelligence

AI
Regulation
(draft)

https://ec.europa.eu/newsroo
m/dae/items/709090

Article 40 – Harmonised standards

High-risk AI systems which are in conformity with harmonised standards or parts thereof the
references of which have been published in the Official Journal of the European Union shall be
presumed to be in conformity with the requirements set out in Chapter 2 of this Title, to the extent
those standards cover those requirements.

14 AI Standardisation supporting regulatory needs | Dr. Jochen Friedrich | [email protected] © 2022 IBM Corporation
A “Back Stop” in the Draft Regulation on AI

Article 41 – Common specifications

1. Where harmonised standards referred to in Article 40 do not exist or where the Commission
considers that the relevant harmonised standards are insufficient or that there is a need to
address specific safety or fundamental right concerns, the Commission may, by means of
implementing acts, adopt common specifications in respect of the requirements set out in
Chapter 2 of this Title. Those implementing acts shall be adopted in accordance with the
examination procedure referred to in Article 74(2).

15 AI Standardisation supporting regulatory needs | Dr. Jochen Friedrich | [email protected] © 2022 IBM Corporation
EU Draft Regulation on AI – Essential Requirements

Essential requirements as laid down in EU Draft Regulation, Chapter 2, Articles 9-15:

Risk management system

Data and data governance ISO/IEC JTC 1 SC 42 has work

ongoing on all of these topics and
Technical documentation issues
Record-keeping
Objective must be to have
Transparency and provision of information international standards supporting
to users regulatory and policy needs globally
Human oversight Ideally international standards will be
accepted as European standards
Accuracy, robustness and cybersecurity

16 AI Standardisation supporting regulatory needs | Dr. Jochen Friedrich | [email protected] © 2022 IBM Corporation
EU Rolling Plan for ICT Standardisation – Chapter on AI

Developed by the European Commission in collaboration with

the EU Multi-Stakeholder Platform for ICT Standardisation, an
expert advisory group of the European Commission

Covering 35+ policy areas and identifying standardisation

needs for supporting policy objectives

Invitation to ICT standardisation community to get active on

standardisation: “Actions” in part B of each chapter

Does not cover standards for EU regulation

PDF version: https://ec.europa.eu/docsroom/documents/49834

Web version: https://joinup.ec.europa.eu/collection/rolling-plan-ict-standardisation/rolling-plan-2022

17 AI Standardisation supporting regulatory needs | Dr. Jochen Friedrich | [email protected] © 2022 IBM Corporation
Actions on AI in the EU Rolling Plan for ICT Standardisation (1/2)

Action 1 SDOs should establish coordinated linkages with, and adequately consider European requirements or
expectations from initiatives, including policy initiatives, and organisations contributing to the discourse on AI
standardisation. This in particular includes the contents of the EU proposal for an AI Regulation and of the standardisation
request on AI issued by the European Commission in 2022 as well as the orientations set in the 2021 review of the
Coordinated Plan

Action 2 SDOs should further increase their coordination efforts around AI standardisation both in Europe and
internationally in order to avoid overlap or unnecessary duplication of efforts and aim to the highest quality to ensure a
trustworthy and safe deployment of this technology.

Action 3 ESOs should coordinate with the Commission and appropriately direct their activities to ensure that the
objectives set in the standardisation request on AI issued in 2022 are adequately and timely fulfilled

Action 4 Taking into account the cross-sectorial aspects of the proposed AI Regulation and the interactions between the
AI Regulation and existing or future sectorial safety legislation (for example the proposed new EU Regulation on
machinery products), ESOs shall devote specific attention to the elaboration of standards on the methodology of risk
assessment of cyber-physical products powered by AI and on the testing framework.

18 AI Standardisation supporting regulatory needs | Dr. Jochen Friedrich | [email protected] © 2022 IBM Corporation
Actions on AI in the EU Rolling Plan for ICT Standardisation (2/2)

Action 5 SDOs should appropriately consider cybersecurity and related aspects of artificial intelligence, to identify gaps
and develop the necessary standards on safety, privacy and security of artificial intelligence, to protect against malicious
artificial intelligence and to use artificial intelligence to protect against cyber-attacks

Action 6 EC/JRC to coordinate with SDOs and other initiatives on developing a standardisation landscape and gap
analysis for AI. This work should include recommendations for an action plan

Action 7 Stakeholders in open source to identify relevant open source projects in the field of AI, e.g. providing tools for
testing, benchmarking etc.

19 AI Standardisation supporting regulatory needs | Dr. Jochen Friedrich | [email protected] © 2022 IBM Corporation
Challenges
There are challenges ahead for regulation making use
of standardisation in the area of AI
We should address the challenges in a constructive
way
We should look at best practices and good examples
for solving issues that are seen

20 AI Standardisation supporting regulatory needs | Dr. Jochen Friedrich | [email protected] © 2022 IBM Corporation
Challenges – Ethics, Values and AI Standardisation

§ STAN
DARD

Values are set by law Standards provide technical

and reflect ethics ways to fulfill requirements set
by law

Standards don’t set values per se but will have to provide ways to enable operation
according to values
Standards must comply with values and law and must not undermine law
Standards don’t set ethics but define paths for operating in an ethical way

21 AI Standardisation supporting regulatory needs | Dr. Jochen Friedrich | [email protected] © 2022 IBM Corporation
Challenges – Updating and Self-Learning

Updates,
Compliance at
self-learning,
point in time
…

The traditional use case: The new situation:

Placing a product on the market Operating a system

Standards for AI are process oriented, inter alia defining requirements for ensuring
compliance of an AI system throughout the entire life-cyle
New methods for conformity assessment may have to be developed over time taking into
account specifics of software systems
Difference between stand-alone AI systems and embedded AI systems

22 AI Standardisation supporting regulatory needs | Dr. Jochen Friedrich | [email protected] © 2022 IBM Corporation
Challenges – Market Surveillance

Market surveillance is critical for

the functioning of the system.

What is required so that market surveillance authorities can properly assess compliance of
AI systems?
Do conformity assessment procedures need to evolve?
Which information do market authorities require (source code?; algorithms?; …)

23 AI Standardisation supporting regulatory needs | Dr. Jochen Friedrich | [email protected] © 2022 IBM Corporation
Where do we need to get to?

Regulators around the world should rely on international standards for

addressing and supporting regulatory needs and policy objectives

Standards provide a proven method for defining the way on how

technology must be built for fulfilling regulatory requirements

Standards provide a proven basis for self-assessment of conformity

for bringing trusted, secure and safe technology to the market

24 AI Standardisation supporting regulatory needs | Dr. Jochen Friedrich | [email protected] © 2022 IBM Corporation
Open Source – Complementing Standardisation

Tools and technologies developed in Open Source are available and

should be considered to support and complement standardisation

Standardisation organisations Community creation around

and open source standards provides for fast
communities should partner feedback cycles and
on identifying areas for innovative technology
synergy improvements

25 AI Standardisation supporting regulatory needs | Dr. Jochen Friedrich | [email protected] © 2022 IBM Corporation
Collaboration is key

Experts
Standards bodies need to collaborate
to maximise synergy and avoid
duplication of efforts

are rare ISO/IEC JTC 1 SC 42 has a lead role

internationally but should also take an
obligation for promoting collaboration
New methods, e.g. open source
technologies, need to be considered
and partnering needs to be done

26 AI Standardisation supporting regulatory needs | Dr. Jochen Friedrich | [email protected] © 2022 IBM Corporation
 Thanks very much for your attention

Jochen Friedrich
[email protected]
https://www.linkedin.com/in/jochenfriedrich/
https://twitter.com/jfopen @jfopen
@[email protected]
27 AI Standardisation supporting regulatory needs | Dr. Jochen Friedrich | [email protected] © 2022 IBM Corporation
Developing a TS on
assessment of ML
classification
performance

SC 42 – Artificial Intelligence
Why does “Machine Learning Classification
Performance Assessment” matter?

SC 42 – Artificial Intelligence
ML classification types addressed in TS 4213
Binary Multi-Class Multi-Label
• Each sample labelled as one • Each sample labelled as one • Each sample can be labelled as one or more
of two mutually exclusive of three or more mutually classes – labels not mutually exclusive
classes exclusive classes • Sample can have multiple labels
• Often positive or negative with • ML model can predict some labels correctly,
reference to a categorization Example – ML classification but fail to predict other correct labels
software learns to categorize • ML model can predict some labels correctly,
Example – ML classification images as “dog”, “cat” or “other” but predict others incorrectly
software learns to mark email as based on labels assigned by a
“spam” or “not spam” based on human reviewer Example - ML classification software learns to
feedback from email recipient categorize text as one or more of opinion, news,
hostile, misinformation or disinformation based
on labels assigned by a human reviewer

SC 42 – Artificial Intelligence
How can you use TS 4213?

SC 42 – Artificial Intelligence
Informative Annexes

Annex A Annex B
Multi-class classification performance illustration Illustration of ROC curve derived from
classification results

Annex C
Summary information on ML classification Annex D
benchmark tests Chance-corrected cause-specific mortality fraction

SC 42 – Artificial Intelligence
Thank you
[email protected]

[email protected]

SC 42 – Artificial Intelligence
Introduction of
Governance
Implications of
AI Systems

SC 42 – Artificial Intelligence
 Yonosuke Harada,
SC42 JWG1 Convenor
Hohnor Prof. Institute of Information Security

Introduction

SC 42 – Artificial Intelligence
Topics

SC 42 – Artificial Intelligence
Background
AI is the opportunity and risk for the organization

SC 42 – Artificial Intelligence
Background
Governance is common issues

SC 42 – Artificial Intelligence
Background
Governance is common issues

SC 42 – Artificial Intelligence
Why Governance
definition from ISO31000 Risk Management

SC 42 – Artificial Intelligence
Why Governance
use of AI brings new source of risk ?

SC 42 – Artificial Intelligence
ISO/IEC 38507:2022
just published by joint effort by ISO/IEC JTC1 SC42 and SC40

SC 42 – Artificial Intelligence
Overview of AI and AI systems (clause 5)
technical background (base line and not for technicians/engineers)

SC 42 – Artificial Intelligence
Use of AI
concept introduced in ISO/IEC38507

SC 42 – Artificial Intelligence
when to apply

SC 42 – Artificial Intelligence
Where to apply
AI is used every where (embedded in software)

AI life cycle
development

retirement
Operation
inception

Design

Governance gating process

Figure is modified from original Figure 2 in ISO/IEC38507

SC 42 – Artificial Intelligence
potential user/reader
not only for Governing Bodies

SC 42 – Artificial Intelligence
Where to start AI governance ?
Set a goal

SC 42 – Artificial Intelligence
Where to start AI governance ?
Responsibility and accountability of AI

- -

SC 42 – Artificial Intelligence
ISO/IEC 38507:2022
helps governing body and the organization

SC 42 – Artificial Intelligence
Governing body and Management

SC 42 – Artificial Intelligence
Model of Use of AI implication for the organization
Use of AI will
Oversight of AI enhance customer
satisfaction and
Market communication
needs

Use of AI for
AI
Stakeholder Use of AI will
differences
Organization
expectation
decrease cost
by increasing Governing Bodies
completive
advantage Use of AI
Use of AI impact both
Governing body and
management
Change of business style,
Managers
responsibility etc

Figure is modified from original Figure 3 in ISO/IEC38507

SC 42 – Artificial Intelligence
Model of Use of AI implication for the organization

Oversight of AI

Use of AI introduces new issues

Governing Bodies - Decision making
- Data use
- Culture and values
set and oversee a set of principles for its use of AI - Compliance
the achievement - Risk
Decision Culture
of outcomes making
Data use
and values
Compliance Risk

Managers

Figure is modified from original Figure 3 in ISO/IEC38507

SC 42 – Artificial Intelligence
Policies to address use of AI

SC 42 – Artificial Intelligence
Risk management for use of AI

SC 42 – Artificial Intelligence
Summary

SC 42 – Artificial Intelligence
Thank you

SC 42 – Artificial Intelligence
The foundational
standards for AI
ISO/IEC 22989 and
ISO/IEC 23053

SC 42 – Artificial Intelligence
Introduction

SC 42 – Artificial Intelligence
ISO/IEC 22989 AI concept & terminology

SC 42 – Artificial Intelligence
22989: AI system definition
ü The most challenging task that SC42 WG1 experts spent most time and effort in 22898
ü Hard to reach consensus, because different people with different background has
different view
ü Many contributions from experts and NBs including definitions from internal/external
sources such as OBP, EU AI HLEG, EU AI Act, OECD, Oxford etc
ü Current approach contains two parts: main body describes what AI system does + note
describes how AI system works

AI system
AI systemsystem that generates outputs such as content, forecasts, recommendations or
engineered
decisions for a given set of human-defined objectives
engineered system that generates outputs such as content, forecasts, recommendations or decisions
for a1given
Note set ofThe
to entry: human-defined
engineeredobjectives
system can use various techniques and approaches related to
artificial intelligence (3.1.3) to develop a model (3.1.23) to represent data, knowledge (3.1.21),
Note 1 to entry:
processes, The engineered
etc. which can be usedsystem can use tasks
to conduct various(3.1.35).
techniques and approaches related to
artificial intelligence (3.1.3) to develop a model (3.1.23) to represent data, knowledge (3.1.21), processes,
etc. 2
Note which can be
to entry: AIused to conduct
systems tasks (3.1.35).
are designed to operate with varying levels of automation (3.1.7).
Note 2 to entry: AI systems are designed to operate with varying levels of automation (3.1.7).

SC 42 – Artificial Intelligence
22989: Level of automations

SC 42 – Artificial Intelligence
22989: AI functional overview
Human design choices, engineering and oversight

Machine Learning
Training data

Continuous learning
Model

Outputs:
- Predictions
Inputs: - Actions
Processing
- Production data - Recommendations
- Information - Decisions

SC 42 – Artificial Intelligence
22989: AI system life cycle model

Verification and

Operation and
development

Deployment

Re-evaluate

Retirement
monitoring
Design and

validation
Inception

DevOps

Transparency and explainability

Security and privacy

Risk management

Governance

SC 42 – Artificial Intelligence
22989: AI system life cycle processes
Inception
Objectives
Requirements
Risk assessment
Risk treatment plan
Polices and compliance
Design and development
Approach
Architecture
Code
Training and validation data
Risk treatment

Re-evaluate Verification and validation

Evaluate operating results Test data

Refine objectives System verification
Refine requirements Risk monitoring and review
Risk monitoring and review

Operation and monitoring Deployment

Operating data input
Model execution Runtime deployment
Model update Model deployment
Risk management Risk treatment

Retirement Continuous validation

Test data If using continuous
Data disposal System validation
Risk management learning
Model disposal
Decommission and discard Continuous improvement

SC 42 – Artificial Intelligence
22989: AI stakeholder roles
AI Stakeholder

AI provider AI producer AI customer AI partner AI subject Relevant authorities

AI platform provider AI developer AI user AI system integrator Data subject Policy makers

AI product or service
Data provider Other subject Regulators
provider

AI evaluator

AI auditor

SC 42 – Artificial Intelligence
ISO/IEC 23053 Machine learning framework

SC 42 – Artificial Intelligence
ISO/IEC 23053 Machine learning framework
Software
Data tools and
techniques

ML model

This framework is used for the structure of the document, but

also to illustrate the machine learning process Perform a task
to solve a
problem

SC 42 – Artificial Intelligence
Machine learning pipeline

SC 42 – Artificial Intelligence
Supervised machine learning process based on the
ML pipeline
Model Retraining

Make
Setup the task Run trained prediction
Algorithm Model Model System
Define Model training and deployed
selection selection evaluation validation
problem model Make
decision

Software tools and techniques

Data ML algorithms Optimisation Evaluation

preparation methods metrics

Data

Labelled
Acquired data Prepared Data Validation data Test data Production data
Training data

Model creation Model evaluation Model use

SC 42 – Artificial Intelligence
ISO/IEC 42001 AI management system

SC 42 – Artificial Intelligence
ISO/IEC 42001 AI management system

SC 42 – Artificial Intelligence
Use of management system and related governance and risk
standards

SC 42 – Artificial Intelligence
Summary

SC 42 – Artificial Intelligence
Thank you

SC 42 – Artificial Intelligence
22989: Compare AI definition with EU AI Act
22989 AI System AI system in EU AI Act
engineered system that generates outputs such as artificial intelligence system’ (AI system) means software
content, forecasts, recommendations or decisions for a that is developed with one or more of the techniques and
given set of human-defined objectives approaches listed in Annex I and can, for a given set of
human-defined objectives, generate outputs such as
Note 1 to entry: The engineered system can use various content, predictions, recommendations, or decisions
techniques and approaches related to artificial intelligence influencing the environments they interact with
(3.1.3) to develop a model (3.1.23) to represent data,
knowledge (3.1.21), processes, etc. which can be used to ANNEX I List
conduct tasks (3.1.35). (a) Machine learning approaches, including supervised,
unsupervised and reinforcement learning, using a wide
Note 2 to entry: AI systems are designed to operate with variety of methods including deep learning;
varying levels of automation (3.1.7). (b) Logic- and knowledge-based approaches, including
knowledge representation, inductive (logic) programming,
knowledge bases, inference and deductive engines,
(symbolic) reasoning and expert systems;
(c) Statistical approaches, Bayesian estimation, search
and optimization methods.

SC 42 – Artificial Intelligence
22989: Compare AI definition with OECD

22989 AI System AI system in OECD

engineered system that generates outputs such as An AI system is a machine-based system that can, for a
content, forecasts, recommendations or decisions for a given set of human-defined objectives, make predictions,
given set of human-defined objectives recommendations or decisions influencing real or virtual
environments. It does so by using machine and/or human-
Note 1 to entry: The engineered system can use various based inputs to:
techniques and approaches related to artificial intelligence i. perceive real and/or virtual environments;
(3.1.3) to develop a model (3.1.23) to represent data, ii. abstract such perceptions into models through
knowledge (3.1.21), processes, etc. which can be used to analysis in an automated manner (e.g. with ML, or
conduct tasks (3.1.35). manually); and
iii. use model inference to formulate options for
Note 2 to entry: AI systems are designed to operate with information or action.
varying levels of automation (3.1.7). AI systems are designed to operate with varying levels of
autonomy.

SC 42 – Artificial Intelligence
22989: AI eco system Vertical sectors

AI systems

AI functions
Decision

Reasoning

Machine Learning Engineering

(e.g. evolutionary computation, swarm intelligence)

Model Procedural knowledge

Task Declarative knowledge

Other technologies
Data for machine learning Software tools and Software tools and
techniques techniques
Training data Data pre-processing Logic programming

Validation data Categories of

ML Algorithms Expert systems
Test data
Optimization methods
Production data Evaluation metrics

Big data and data sources — Cloud and edge computing

Resource pools

Compute Storage Network

CPU FPGA GPU ASIC

Cluster management
Resource scaling

Resource management

Resource provisioning

SC 42 – Artificial Intelligence
22989: Mapping with OECD AI life cycle
Traceability

Risk management includes privacy, digital security, safety, bias

AI system life cycle

Design, data and models Deployment Operation + defined by OECD

Verification + validation
• Planning monitoring
• Design
• Data collection
• Processing
• Model building

Retirement

Verification and

Operation and
development

Deployment

Re-evaluate

Retirement
Design and

monitoring
validation
Inception

AI system life cycle in

DevOps this document
Transparency and explainability

Security and privacy

Risk management
Governance

SC 42 – Artificial Intelligence
Artificial Intelligence
Overview of the AI Standards Program and Novel Ecosystem Approach
ISO/IEC Workshop Series, Inaugural Event, May 2022

Wael William Diab, Chair SC 42 (Artificial Intelligence)

SC 42 – Artificial Intelligence
Acknowledgement SC 42 – Artificial Intelligence

Heather Benko (SC 42 Committee Manager)

Andy Dryden, Jose Alcorta (ISO TPMs)
ISO Communications Team (Clare, Vivienne, Liz, Catherine, Barnaby)
IEC Communications Team (Mike, Gabriella, Antoinette)
SC 42 Officers
 Enabling the Digital Transformation SC 42 – Artificial Intelligence

Data
Scientist

Resource Management/Monitoring, Analytics Libraries, etc.

BDRA Interface
BDRA Ecosystem Components
Data Sources Analytics Data Consumers
- Sensors - End users
Computing Resources
- Simulations - Repositories
Resources
- Modeling - Systems
- Etc. - Etc.

Support Infrastructure Analytics Application

Data Sources Services Analytics Services
Database Services Visualization & BI Services
Distributed File System Services Value-added Content Services
Infrastructure Services Security and Privacy Services

Artificial Intelligence Data Science and the Data Ecosystem

- not just one technology - Big Data ecosystem, AI related data and analytics
- variety of SW / HW technologies applied in applications - includes BD reference architecture, data quality, AI data aspects

Changing how we live, work and paly…

AI Use Cases, Applicability and Growth SC 42 – Artificial Intelligence

Traditionally, AI had been focused on large scale problems that were either too hard and complex
to solve with traditional compute methods or were in specialized emerging areas
This is no longer the case. Machine learning has widened the applicability of AI. Focus on the
digital transformation has created a demand for services and more intelligent analytics.
Examples:
§ AI expert systems are helping healthcare professionals make better decisions for patients with proper
trustworthiness measures designed into the system,
§ AI deployment in the smart manufacturing sector where it is driving higher efficiencies by allowing robots to
work alongside human workers with the proper safety measures designed into the system,
§ AI deployment in the financial ecosystem where it is enabling applications that range from asset management
that takes into account factors such as the clients risk to fraud detection that reduces false-positives
Emerging applications are numerous and diverse e.g. consumer, retail, digital assistants,
expert systems such as smart grid, marketing intelligence tools, enterprise etc.
Thus, it is not surprising that IDC estimates that 75% of enterprise applications will use AI.
The market is forecast to accelerate further in 2022 with 18.8% growth and remain on track to
break the $500 billion mark by 2024.
Ecosystem is ripe for standardization
The Need for an Ecosystem Perspective SC 42 – Artificial Intelligence

Historically, IT systems and their governing standards were based on well understood environments
§ Early approaches focused on performance for a specific problem definition
§ As IT became ubiquitous, considerations such as cost, sustainability, security and privacy played a
an increasingly important role in defining requirements
Digital transformation of industries has changed the landscape for IT standardization. For instance:
§ Emerging non-technical requirements such as ethical and societal considerations and the ability to
design trustworthy systems are key aspects
§ Stakeholder diversity has increased considerably (eg. regulatory, social scientists etc.)
§ Early engagement by the various stakeholders has become the norm
§ The application domains and associated use cases have increased dramatically
§ Understanding uses, proving business cases and developing standards are now concurrent
§ The “data ecosystem” is as important as hardware, software and operational technologies
§ Enabling certification, 3rd party audit and increasing end-user confidence increasingly important
SC 42 and the Holistic AI Ecosystem SC 42 – Artificial Intelligence

A new approach to standardization is needed that

§ Takes into account the context of use of the technology by looking at both technology capability
and non-technical requirements such as business requirements, regulatory and policy
requirements, application domain needs, and ethical and societal concerns
§ Translating the above into technical requirements
§ Building foundational standards that allow communities to build upon such as terminology, use
cases, application guidance and reference architectures
§ Linking technology innovation communities such as proprietary implementations, research,
SDOs and open-source communities
The result not only accelerates technology adoption but it also takes into account its context of
use and builds an ecosystem
AI, Big Data and associated analytics are examples of this technology, it’s use and IT’s evolution
SC 42 has adopted this holistic ecosystem approach providing the glue between requirements
and technical requirements through the horizontal deliverables the committee develops
 Bridging the Gap – An Ecosystem Approach SC 42 – Artificial Intelligence

Example of
Non-Technical Application Regulatory and Business
Emerging Societal
Trends and Domain Policy Requirements /
Requirements
Requirements Requirements Requirements KPIs

SC 42
Ecosystem Assimilating Requirements for Context of Use of AI, Big Data and Analytics
Perspective

Horizontal and Foundational Projects

SC 42
Enabling and Accelerating Wide Adoption while Addressing Concerns

Deliverables Foundational Data Reference Use Application

Frameworks Guidance Interop MSS
Concepts Stds Architectures Cases Guidelines

Application Standards Open Source Projects Proprietary Solutions

Ecosystem Domain specific standards OS projects including AI Proprietary solutions leveraging
Deliverables that build on SC 42 algorithms leveraging SC SC 42 deliverables for
horizontal deliverables 42 ecosystem deliverables requirements and gudance
&
Implementations Across Diverse Application Domains
Implementations E.g. AI-enabled Smart Manufacturing, Health Care, Financial Systems, Consumer Devices, ITS etc.
Introduction and Overview of SC 42 SC 42 – Artificial Intelligence

Part of the ISO and IEC Family

§ Committee on AI standardization under the joint initiative on Information Technology (JTC 1)
§ IEC and ISO produce international standards and with participation by country – one country one vote
§ Collaboration with long established sister ISO and IEC committees covering broad rage of app domains
§ Works with JTC 1 committees producing horizontal stds in key areas such as security, cloud, IoT, governance
Scope
§ Standardization in the area of Artificial Intelligence
§ Serve as the focus and proponent for JTC 1's standardization program on Artificial Intelligence
§ Provide guidance to JTC 1, IEC, and ISO committees developing Artificial Intelligence applications
Growing Program of Work and Stakeholders
§ Projects: 28 active (10 added within last year). 11 published. 2 NP ballots. New study items being discussed
§ Participation: 50 nations (35P/15O) with annual growth. New regions represented. 250+ delegates at plenaries
§ SC 42 “customers” consuming our standards and participants are increasingly diverse e.g. from data scientists
to regulators to application domain experts to social scientists
Extensive collaborations (internal and external) – over 45 liaisons including Cat A e.g. recently EUROCAE, IIOC
8
 SC 42 – Artificial Intelligence

Structure of SC 42
WG 1 Foundational standards JWG 2 (SC 42 – SC 7) Testing of AI-
based system
WG 2 Data
AG 3 AI standardization
WG 3 Trustworthiness roadmapping
WG 4 Use cases and applications AHG 4 Liaison with SC 27
WG 5 Computational approaches
and computational
characteristics of AI systems
Key Topics: Foundational Standards SC 42 – Artificial Intelligence

Overview and motivation

§ AI has generated interest across a very diverse and growing set of stakeholders
§ Introduce a common language that can be used across stakeholder groups
§ Building blocks for other SC 42 projects and ISO/IEC TC application standards
§ Addresses unique AI requirements to enable certification and audit via MSS
Current focus areas
§ AI Concepts and Terminology*
§ Framework for AI Systems Using Machine learning*
§ Management System Standard (ISO/IEC 42001)
§ NP ballot on new project related to AI system impact assessment
§ Discussion on the concept of risk for input to ISO TMB/IEC SMB joint task force
* AI Standards Lifecycle contributions included in these projects
Key Topics: Data – AI, Big Data, Analytics SC 42 – Artificial Intelligence

Overview
§ Data ecosystem, characteristics, properties and quality essential to AI, Big Data and analytics work
§ Big Data deals with extensive datasets by considering characteristics, such as volume, variety, velocity, variability, which allows
scalable technology to efficiently store, manipulate, manage and analyze these datasets
§ AI systems acquire, process and apply knowledge, which relies on data, its properties and quality
§ Analytics derives insights from data, which requires data acquisition, collection, validation and processing

History, motivation and current focus areas

§ History
§ BD work initiated as JTC 1/WG 9 in 2015 and moved to SC 42 in 2018 creating SC 42/WG 2 (Big Data)
§ WG 2 expanded to data in context of AI, BD and analytics April 2020 focusing SC 42’s data work in one WG
§ Foundational Big Data work completed and published: ISO/IEC 20546 and ISO/IEC BDRA 20547 series
§ Areas of current work
§ BD: Process management framework for Big Data analytics to leverage across the organization irrespective of industry
§ AI/Analytics:
– Multipart series AI – data quality for analytics and ML – 5 parts including newly added part 5 on data governance
– New AI – Data lifecycle framework
§ Areas of current study and project proposals under development
§ Data ecosystem study areas include AI Data AHG, Data Quality AHG and road-mapping
Key Topics: Trustworthiness SC 42 – Artificial Intelligence

Overview
§ Looking at a wide range of issues related to trustworthiness, security and privacy within the context of AI
Motivation and current focus areas
§ Hot topic due to regulatory landscape (e.g. European privacy laws; discussions about social media engines)
§ Stakeholders view this as a necessary area for the success and broad market adoption of AI. Essential to AI application areas
§ Published work: AI trustworthiness overview, overview of robustness of neural networks and bias in AI systems
§ Areas of current work
§ Formal methodologies for the assessment of neural networks
§ Risk management framework for AI based on ISO 31000
§ Systems and SW Quality Requirements and Evaluation (SQuaRE) -- Quality Model for AI systems
– Companion on quality evaluation guidelines
§ Functional Safety and AI Systems being developed in close collaboration with IEC TC 65/SC 65A
§ Objectives and approaches for explainability of ML models and AI systems
§ Controllability of automated artificial intelligence systems
§ Treatment of unwanted bias in classification and regression machine learning tasks
§ Transparency taxonomy of AI systems
§ Comprehensive areas of study and road-mapping
 COMMON AI-RELATED ETHICAL AND
SOCIETAL ISSUES* SC 42 – Artificial Intelligence

THE SINGULARITY

JOB DISPLACEMENT

INADVERTENT
ALGORITHMIC
CONSEQUENCES
ALGORITHMIC
BIASES
PRIVACY

*Source: Accenture (Liongosari) 13

Key Topics: Societal Concerns and Ethics SC 42 – Artificial Intelligence

Overview
§ Adoption of transformative technologies like AI have impacts that go beyond the technology
§ AI-specific trustworthiness issues e.g. reliability, privacy, security, explainability, controllability
§ Emerging issues related to the context of use of the technology
– e.g. algorithmic bias, safety directives in industrial AI, eavesdropping
§ AI ethical considerations not limited to SC 42 but extend to ISO/IEC TCs in their applications
Motivation and current focus areas
§ Standards can mitigate ethical issues allowing for broad responsible adoption. Stakeholders include
§ industry, regulatory, technologists, interest groups, app domain, society at large
§ Ethical considerations and societal concerns considered across entire SC 42 program
§ Dedicated projects
– Overview of AI ethical and societal concerns with tie-in to trustworthiness and exemplary use cases
– PWI on AI best practice guidance for mitigating ethical and societal concerns
§ Integrated into and enabled by entire SC 42 deliverables portfolio. For instance,
– governance, MSS, use cases, application guidelines, (with IEC SC 65A) safety etc.
§ SC 42 collaborating with other work programs e.g. PAI, OECD, UNESCO, EC, ISO SMCC, IEC ACOS
Key Topics: Use Cases and Applications SC 42 – Artificial Intelligence

Overview
§ Identify AI application domains, context of AI use in those domains and develop guidance
§ Collect representative use cases and analyze for derived requirements
Motivation and current focus areas
§ Interest in AI continues to grow across application domains and use cases
§ Use cases are the “currency” between SDO committees
§ By looking at different domains, ensures SC 42 deliverables are “broad enough to be horizontal”
§ Novel approach that includes trustworthiness, ethics and societal concerns
§ Published areas of work
§ ISO/IEC 24030 with over 130 use cases including contributions from liaisons
§ Current and new areas of work
§ Guidelines for AI applications
– Application guidance enables SDOs and OS communities developing AI app projects to leverage SC 42’s work
– Macro-level view of an application to facilitate its understanding, development and use amongst all stakeholders
– “What are the characteristics and considerations of an AI application?
§ AI lifecycle for software developers – AI system life cycle processes
§ Revision of ISO/IEC 24030:2021 for additional use cases
§ AI Service Ecosystem assigned to WG 4 and will be incorporated into the existing two projects above
Key Topics: Computational Methods SC 42 – Artificial Intelligence

Overview
§ Heart of AI looking at computational approaches and characteristics of AI systems
Motivation and current focus areas
§ Published work
§ Overview project: Overview of the state of the art of computational approaches for AI
systems, describing: a) main computational characteristics of AI systems; b) main algorithms
and approaches used in AI systems, referencing use cases contained in ISO/IEC TR 24030
§ Current work
§ Desire to have some key industry agreed tenants for classification performance of algorithms –
currently no such internationally agreed upon tenants / norms
– Assessment of classification performance for machine learning models and algorithms
§ Reference architecture of knowledge engineering
§ Overview of machine learning computing devices
§ Areas of current study and road-mapping
§ NP ballot on new project related to verification and validation analysis of AI systems
Key Topics: Governance Implications of AI SC 42 – Artificial Intelligence

Overview
§ Fueled by the digital transformation, AI technologies are being rapidly adopted
across industries, cities, homes and infrastructures
§ Thus, the need to address governance implications for the use of AI in
organizations has become of paramount importance
Motivation and current focus areas
§ The motivation is to help organization boards and executives ask and
answer key questions about AI technologies
§ By combining the expertise of SC 42, which is looking at the entire AI
ecosystem, with that of SC 40, which is looking at IT governance, a joint
working group has been established to develop an ISO/IEC standard on the
governance implications of AI
§ Project published April 2022!
Key Topics: Testing of AI-Based Systems SC 42 – Artificial Intelligence

Overview
§ As adoption and deployment across industry sectors of AI continues, a need to
address testing of AI-based systems is key
Motivation and current focus areas
§ AI testing is similar to conventional software testing, but it also faces AI-specific
challenges. These challenges are discussed and approaches to mitigate are
introduced.
§ Guidelines for the testing of AI systems across the AI system life cycle are provided.
Moreover, guidelines related to AI testing useful for AI systems development in
accordance with their specified functional and non-functional requirements.
§ By combining the expertise of SC 42, which is looking at the entire AI
ecosystem, with that of SC 7, which is looking at software systems, a joint
working group has been established to develop the ISO/IEC standard
§ Project in Working Draft
Key Topics: AI Standardization Roadmapping SC 42 – Artificial Intelligence

Overview, motivation and current focus areas

Driven by the
§ rapid technology advancement
§ broad adoption
§ evolution of AI landscape
§ context of use of AI and increasing diversity of stakeholders
§ growing program of work
SC 42 has setup an advisory group to
§ recommend new areas of work
§ recommend further development of existing areas
§ maintain a relationships document between the various projects
Examples discussion areas include conformity assessment, sustainable AI etc.
Key Topic: Management Systems Standard
ISO/IEC 42001 SC 42 – Artificial Intelligence

Motivation
AI Management
§ AI technologies bring AI-specific concerns beyond those of traditional IT systems. For example
System Standard
§ ML based AI system may provide different results depending on the training data used
§ The choice of training data when using an AI system is an additional process that an organization needs to
perform to ensure the intended overall system performance
§ Consumers of AI products and services may lack trust in the AI supplier organization
§ Assurance that the organization considered for fairness, inclusiveness, accountability etc. of AI system
MSS Enables Assessment of Conformance and Auditability of the Process
§ MSS designed with AI-specific process requirements
§ Allows organizations to check how well it meets their objectives in the use of an AI system
§ For trusted 3rd party performing a check or audit, a certificate of conformance can be issued
MSS Brings Benefits to the Application Domains Deploying AI Applications
§ MSS extendable to application domains. Sector-specific implementations expand the applicability
of the MSS, which in turn enable conformance/audit and broadens AI adoption
ISO/IEC 42001 pulls together all the SC 42 AI platform work from a management system perspective
Concluding Remarks SC 42 – Artificial Intelligence

SC 42 is the first of its kind international standards committee looking at the full AI ecosystem
§ AI, Big Data and related analytics are key technologies enabling the digital transformation

SC 42 has a rapidly growing work program

§ Strong growth and execution. 11 published projects and 28 active projects in 7 working groups. 2 NP ballots
§ Robust study program for anticipated new work addressing AI ecosystem and system level concerns with AI

SC 42 engaging in extensive outreach and global collaboration

§ Tremendous outreach via ISO, IEC and national bodies. Extensive and diverse liaison network

Part of the ISO, IEC and JTC 1 families

§ Access to broad, diverse and numerous committees that range from horizontal to vertical areas
§ System integration committee providing guidance to ISO, IEC and JTC 1 committees looking at AI applications

Opportunity for international standards to fuel AI market growth and accelerate adoption
Excellent opportunity to engage – if you are interested, please contact your national body
SC 42 – Artificial Intelligence
SC 42 – Artificial Intelligence
SC 42 – Artificial Intelligence
SC 42 – Artificial Intelligence
SC 42 – Artificial Intelligence
SC 42 – Artificial Intelligence
 SC 42 – Artificial Intelligence

Annex A
Additional Links and Information
Additional Links and Information SC 42 – Artificial Intelligence
§ SC 42 Committee website
§ Press Coverage Related to SC 42 Overview and Program of Work
§ IEC news A governance framework for organizations deploying AI systems (Apr 29th 2022)
§ IEC news New report focuses on convergence of AI and Industrial IoT (Mar 10th 2022)
§ ISO innovation article on Information technology growth and the role of AI and associated AI standards (Jan 2022)
§ IEC e-tech Computational approaches for AI systems (Jan 25th 2022)
§ ISO news Enabling An AI-Ready Culture - SC 42’s Novel MSS Approach (Nov 2021)
§ IEC e-tech Standards help address bias in artificial intelligence technologies (Nov 8th 2021)
§ IEC news Growing AI standards committee concludes plenary (Oct 22nd 2021)
§ IEC news IEC and ISO artificial intelligence plenary begins (Oct 22nd 2021)
§ ISO publication White Paper on Smart Manufacturing (Oct 2021)
§ RAPS Article Enabling the digital transformation of industry: The roles of AI, big data, analytics, and related data ecosystem (June 1st 2021)
§ IEC news International standards instill confidence in artificial intelligence technologies (July 22nd 2021)
§ IEC news IEC and ISO artificial intelligence committee broadens standards work programme (May 17th 2021)
§ IEC e-tech IEC and ISO publish over 130 emerging AI use cases (May 17th 2021)
§ IEC news IEC/ISO standards committee for artificial intelligence begins spring plenary (Apr 30th 2021)
§ IEC e-tech New standard to enhance trustworthiness of artificial intelligence systems (March 15th 2021)
§ ISO news article on Getting Big on Data (Nov 5th 2020)
§ IEC e-tech article on International standards committee for AI ecosystem expands into new areas (Sep 15th 2020)
Additional Links and Information SC 42 – Artificial Intelligence
§ Press Coverage Related to SC 42 Overview and Program of Work
§ IEC e-tech article on IEC and ISO publish TR which provides overview of big data framework and reference architecture (Aug 24th 2020)
§ IEC e-tech article on Achieving trustworthy AI with standards (June 8th 2020)
§ ISO news SC 42 virtual plenary as an example of standards innovation during COVID-19 (May 15th, 2020)
§ IEC news announcing the key outcomes of the 5th plenary and added focus on data ecosystem (May 7th, 2020)
§ IEC e-tech article on SC 42’s holistic ecosystem approach to AI standardization (Feb 2020)
§ IEC e-tech article on New IEC and ISO Standard will enable big data adoption across industry sectors (Feb 15th 2020)
§ IEC e-tech article on IEC and ISO AI committee (SC 42) expands programme of work (Jan 2020)
§ ISO focus Nov/Dec 2019 magazine on AI and the SC 42 program of work (November 2019)
§ ISO focus landing page for edition including links to download PDF (above in English) in various languages and individual articles
§ IEC news AI standards help accelerate digitalization of smart manufacturing (Dec 2019)
§ IEC news announcing the key outcomes of the 4th plenary (Nov 11th, 2019)
§ IEC e-tech article on Establishing trustworthiness is vital in our human-machine world (July 15th 2019)
§ IEC e-tech article on Artificial intelligence and big data: a paradigm shift in healthcare (May 15th 2019)
§ IEC news announcing the key outcomes of the 3rd plenary (April 23rd, 2019)
§ IEC news announcing the start of the 3rd plenary (April 9th 2019)
§ ISO news article (18th October 2018)
§ JTC 1 press committee article (30th May 2018)
Additional Links and Information SC 42 – Artificial Intelligence
§ Press Coverage Related to SC 42 Formation
§ IEC e-tech article (17th May 2018). Additional circulations
§ ISO retweeted the article (September 2018)
§ Published on ANSI (US National Body) website
§ Published on UNE (Spain National Body) website (September 2018)
§ Published on ILNAS (Luxemburg National Body) website (27th April 2018)
§ Note: not a direct reprint but used the photo
§ Published on Robotics Automation and News Magazine
§ ANSI news article on the formation of SC 42 (16th January 2018)
§ Introduction of SC 42 in the IEC MSB White Paper on Artificial Intelligence
Additional Links and Information SC 42 – Artificial Intelligence
§ Press Coverage Related to SC 42 Participation at Key Industry and International Events
§ IEC Medium Publications
§ IEC news on How standardization can contribute to an international framework for AI (Oct 20th 2021)
§ IEC news Young Professionals learn about international standards for artificial intelligence (Oct 15th 2021)
§ IEC blog on Webinar on regulations and artificial intelligence technologies (Dec 10th 2020)
§ IEC blog on AI standards on the agenda at IOT Solutions World Congress (Dec 8th 2020)
§ IEC blog on IEC and ISO present AI standardization work during event by European Commission (Oct 28th 2020)
§ IEC blog on Trustworthiness is key to services and products using AI and IoT technologies (Mar 3rd, 2020)
§ IEC blog on AI standards on the agenda at IOT Solutions World Congress (Nov 21st 2019)
§ IEC blog on AI and IoT industry leaders to consider a digital trust framework at Berlin forum (May 15th 2019)
§ Global Standards Collaboration (GSC-22) 2019 Session on Artificial Intelligence
§ ISO news on Standards cooperation is key to making AI and smart cities a reality (April 4th 2019)
§ IEC blog on 22nd Global Standards Collaboration meeting discusses need for standards to accelerate AI technology innovation and
adoption (April 3rd 2019)
§ JTC 1 Info
§ JTC 1 info article on IEC and ISO present on the AI Ecosystem Standardization Program at the European Commission Workshop (Oct
16th 2020)
§ Industrial Internet Consortium (IIC)
§ IIC blog on from IEC on Standards for AI on the Agenda at IoT Solutions World Congress
§ IoT Solutions World Congress (IoTSWC)
§ IoTSWC promotion of the IEC blog on AI standards on the agenda at IOT Solutions World Congress
Additional Links and Information SC 42 – Artificial Intelligence

§ Other media coverage

§ Twitter
§ ISO (@isostandards)
§ Tweet Chat on standards on Artificial Intelligence with Chair of SC 42 (25th October). Hashtags: #ISOchat #Standards4AI
§ IEC (@IECStandards)
§ Article on New international standard will help organization boards and executive managers ask and answer key questions about AI
technologies (12th February 2019)
§ Article on International standards play a key role in addressing the ethical, technical, safety and security aspects (6th February 2019)
§ Article and video on Standardization can help eliminate data bias in AI (4th February 2019)
§ Article and video on Chair of SC 42 explains the growing influence of AI in Smart Manufacturing (4th February 2019)
§ Article on Chair of SC 42 will lead a session at the CEN/CENELEC workshop on Trustworthy Artificial Intelligence (10th Aug 2018)

§ IEC Medium Publications

§ IEC blog on New IEC and ISO Standard will enable big data adoption across industry sectors (Mar 30th, 2020)
§ IEC blog on Important questions around AI technologies in smart manufacturing (Jan 8th, 2020)
§ IEC blog on New IEC and ISO Standard will enable big data adoption across industry sectors (Mar 30th, 2020)
§ IEC blog on Trustworthiness is key to services and products using AI and IoT technologies (Mar 3rd, 2020)
§ IEC blog on How standards help people trust AI (Jan 15th, 2020)
§ IEC blog on Important questions around AI technologies in smart manufacturing (Jan 8th, 2020)
Additional Links and Information SC 42 – Artificial Intelligence

§ Other media coverage

§ IEC Medium Publications
§ IEC blog on Establishing trustworthiness is vital in our human-machine world (Sep 9th 2019)
§ IEC blog on The need for Big Data Standards (April 24th 2019)
§ IEC blog on New international standard will offer risk management framework for AI (March 18th 2019)
§ IEC blog on Helping organization boards and executives ask and answer key questions about AI technologies (Feb 12th 2019)
§ IEC e-tech article on AI in healthcare: keeping data safe and building trust (January 25th 2019)
§ IEC blog on Making AI safe (January 23rd 2019)
§ IEC e-tech article on Healthcare needs doctors and machines (December 10th, 2018)
§ IEC e-tech article on Eliminating data bias from machine learning systems (November 13th 2018)
§ IEC e-tech article on Smart homes are getting smarter (November 6th 2018)
§ IEC e-tech article on Machine learning is not a synonym for AI (October 17th 2018)
§ IEC e-tech article on Rethinking the healthcare ecosystem (reference to SC 42)
§ IEC e-tech article on Standards development organizations play key role in enabling remote daily life
§ Publications referencing SC 42 work
§ IEC e-tech on Luxembourg highlights role of technical standardization in adoption of artificial intelligence (Sep 21st 2021)
§ ILNAS white paper on AI and technical standardization
§ IEC e-tech article on How Standards Australia contributes to the global artificial intelligence ecosystem (Jan 20th 2021)
Additional Links and Information SC 42 – Artificial Intelligence

§ Other media coverage

§ ISO Multimedia
§ ISO video interview with Chair of SC 42 on Standards and Artificial Intelligence (November 14th 2018)
§ Artificial Intelligence and the role of International Standards in the implementation of this technology
§ ISO video interview with Chair of SC 42 on Standards and Artificial Intelligence Continued (November 14th 2018)
§ Artificial Intelligence and easing the mind of end-users including AI trustworthiness, ethics and societal concerns

§ IEC Multimedia
§ IEC video interview with SC 42 Chair, SC 42/WG 3 Connvenor and ISO/IEC 24368 Editor on AI Ethics (Nov 2019)
§ IEC video interview with Chair of SC 42 on How can we ensure AI is safe for Healthcare? (April 6th 2019)
§ IEC video interview with Chair of SC 42 on Is it too early to use machine learning for cybersecurity? (April 5th 2019)
§ IEC video interview with Chair of SC 42 on To what extent is AI ready for standardization? (April 5th 2019)
§ IEC video interview with Chair of SC 42 on What are some of the challenges you see with AI? (March 26th 2019)
§ IEC video interview with Chair of SC 42 on How to Define Artificial Intelligence (March 26th 2019)
§ IEC video interview with Chair of SC 42 on Why do we need standards for AI? (March 26th 2019)
§ IEC video interview with Chair of SC 42 on Artificial Intelligence (February 4th 2019)
§ The growing influence of AI in Smart Manufacturing and the important role of standards
§ IEC video interview with Chair of SC 42 on Artificial Intelligence (February 4th 2019)
§ Standardization can help eliminate data bias in AI
 SC 42 – Artificial Intelligence

Annex B
Details of Publications, SC 42 WGs, Liaisons and Program of Work
Published Standards – AI – 2022 SC 42 – Artificial Intelligence

ISO/IEC 38507 -- Information technology -- Governance of IT -- Governance implications

of the use of artificial intelligence by organizations
§ Publication date: 2022-04
This document provides guidance for members of the governing body of an organization to enable and govern
the use of Artificial Intelligence (AI), in order to ensure its effective, efficient and acceptable use within the
organization.
This document also provides guidance to a wider community, including:
— executive managers;
— external businesses or technical specialists, such as legal or accounting specialists, retail or industrial
associations, or professional bodies;
— public authorities and policymakers;
— internal and external service providers (including consultants);
— assessors and auditors.
This document is applicable to the governance of current and future uses of AI as well as the implications of
such use for the organization itself.
This document is applicable to any organization, including public and private companies, government entities
and not-for-profit organizations. This document is applicable to an organization of any size irrespective of their
dependence on data or information technologies.
Published Standards – AI – 2021 SC 42 – Artificial Intelligence

ISO/IEC TR 24029-1:2021 Artificial Intelligence -- Assessment of the robustness of neural networks --

Part 1: Overview
§ Publication date: 2021-03
This document provides background about existing methods to assess the robustness of neural networks.

ISO/IEC TR 24030:2021 Information technology -- Artificial Intelligence -- Use cases

§ Publication date: 2021-05
This document provides a collection of representative use cases of AI applications in a variety of domains.

ISO/IEC TR 24027:2021 Information technology -- Artificial Intelligence -- Bias in AI systems and AI

aided decision making
§ Publication date: 2021-11
This document addresses bias in relation to AI systems, especially with regards to AI-aided decision-making. Measurement techniques
and methods for assessing bias are described, with the aim to address and treat bias-related vulnerabilities. All AI system lifecycle
phases are in scope, including but not limited to data collection, training, continual learning, design, testing, evaluation and use.

ISO/IEC TR 24372:2021 Information technology -- Artificial Intelligence -- Overview of computational

approaches for AI systems
§ Publication date: 2021-12
This document provides an overview of the state of the art of computational approaches for AI systems, by describing: a) main
computational characteristics of AI systems; b) main algorithms and approaches used in AI systems, referencing use cases contained in
ISO/IEC TR 24030.
Published Standards – AI – 2020 SC 42 – Artificial Intelligence

ISO/IEC TR 24028:2020 Information technology -- Artificial Intelligence --

Overview of trustworthiness in artificial intelligence
§ Publication date: 2020-05
This document surveys topics related to trustworthiness in AI systems, including the following:
— approaches to establish trust in AI systems through transparency, explainability, controllability, etc.;
— engineering pitfalls and typical associated threats and risks to AI systems, along with possible
mitigation techniques and methods; and
— approaches to assess and achieve availability, resiliency, reliability, accuracy, safety, security and
privacy of AI systems.
The specification of levels of trustworthiness for AI systems is out of the scope of this document.
Published Standards – Big Data – 2020 SC 42 – Artificial Intelligence

ISO/IEC 20547-1:2020 Information technology -- Big data reference architecture

-- Part 3: Framework and application process
§ Publication date: 2020-08
The ISO/IEC 20547 series is intended to provide users with a standardized approach to developing and
implementing big data architectures and provide references for approaches.
This document describes the framework of the big data reference architecture and the process for how a
user of the document can apply it to their particular problem domain.

ISO/IEC 20547-3:2020 Information technology -- Big data reference architecture

-- Part 3: Reference architecture
§ Publication date: 2020-03
The ISO/IEC 20547 series is intended to provide users with a standardized approach to developing and
implementing big data architectures and provide references for approaches.
This document describes the reference architecture in terms of User and Functional views.
The reference architecture presented in this document provides an architecture framework for describing
the big data components, processes, and systems to establish a common language for the various
stakeholders named as big data reference architecture (BDRA).
Published Standards – Big Data – 2018, 2019 SC 42 – Artificial Intelligence

ISO/IEC TR 20547-2:2018 Information technology -- Big data reference architecture --

Part 2: Use cases and derived requirements
§ Publication date: 2018-01
ISO/IEC TR 20547-2:2018 provides examples of big data use cases with application domains and technical
considerations derived from the contributed use cases.

ISO/IEC TR 20547-5:2018 Information technology -- Big data reference architecture --

Part 5: Standards roadmap
§ Publication date: 2018-02
ISO/IEC TR 20547-5:2018 describes big data relevant standards, both in existence and under development,
along with priorities for future big data standards development based on gap analysis.

ISO/IEC 20546:2019 Information technology -- Big Data -- Overview and Vocabulary

§ Publication date: 2019-02
This document provides a set of terms and definitions needed to promote improved communication and
understanding of this area. It provides a terminological foundation for big data-related standards.
This document provides a conceptual overview of the field of big data, its relationship to other technical areas
and standards efforts, and the concepts ascribed to big data that are not new to big data.
SC 42 Projects, Status and Leadership SC 42 – Artificial Intelligence

SC 42/WG 1 Foundational standards

§ Terms of reference: Development of foundational standards for Artificial Intelligence
§ Convenor: Paul Cotton (Canada)
§ ISO/IEC 22989: Artificial Intelligence Concepts and Terminology
§ Editor: Wei Wei (Germany)
§ Status: Publication
§ ISO/IEC 23053: Framework for Artificial Intelligence Systems Using Machine Learning
§ Editor: Milan Patel (United Kingdom)
§ Status: Publication
§ ISO/IEC 42001: Information technology -- Artificial Intelligence -- Management System
§ Editor: Marta Janczarski (Canada)
§ Status: CD Ballot
SC 42 Projects, Status and Leadership SC 42 – Artificial Intelligence

SC 42/WG 2 Data
§ Terms of reference: Standardization in relation to data in the context of artificial intelligence, big data, and data
analytics
§ Convenor: Wo Chang (United States)
§ ISO/IEC 5259-1: Data quality for analytics and ML -- Part 1: Overview, terminology, and examples
§ Editor: Suwook Ha (Korea)
§ Status: Working draft
§ ISO/IEC 5259-2: Data quality for analytics and ML -- Part 2: Data quality measures
§ Editor: Kyoung-Sook Kim (Japan)
§ Status: Working draft
§ ISO/IEC 5259-3: Data quality for analytics and ML -- Part 3: Data quality management requirements and guidelines
§ Editor: Matthis Eicher (Germany)
§ Status: Working draft
§ ISO/IEC 5259-4: Data quality for analytics and ML -- Part 4: Data quality process framework
§ Editor: Wanzhong Ma (China)
§ Status: Working draft
SC 42 Projects, Status and Leadership SC 42 – Artificial Intelligence

SC 42/WG 2 Data
§ ISO/IEC 5259-5: Data quality for analytics and ML -- Part 5: Data quality governance
§ Editor: Gyeung-Min Kim (Korea)
§ Status: Working draft
§ ISO/IEC 8183 Information technology -- Artificial intelligence -- Data life cycle framework
§ Editor: Colin Crone (United Kingdom)
§ Status: CD Ballot Comment Resolution
§ ISO/IEC 24668: Information technology -- Artificial Intelligence -- Process management framework for Big data
analytics
§ Editor: Gautam Banerjee (India)
§ Status: FDIS
SC 42 Projects, Status and Leadership SC 42 – Artificial Intelligence

SC 42/WG 3 Trustworthiness
§ Terms of reference: Standardization in the area of AI Trustworthiness
§ Convenor: David Filip (Ireland)
§ Secretariat: Aditya Mohan (Ireland)
§ ISO/IEC 24029-2: Artificial Intelligence (AI) -- Assessment of the robustness of neural networks -- Part 2: Formal
methods methodology
§ Editor: Arnault Ioualalen (France)
§ Status: CD Ballot
§ ISO/IEC 23894 -- Information technology -- Artificial intelligence -- Risk management
§ Editor: Peter Deussen (Germany)
§ Status: DIS Ballot
§ ISO/IEC TR 24368: Information technology -- Artificial Intelligence (AI) -- Overview of Ethical and Societal Concerns
§ Editor: Viveka Bonde (Sweden)
§ Status: Publication
SC 42 Projects, Status and Leadership SC 42 – Artificial Intelligence

SC 42/WG 3 Trustworthiness
§ ISO/IEC TR 5469: Artificial Intelligence (AI) -- Functional Safety
§ Editor: Takashi Egawa (Japan), Riccardo Mariani (Italy)
§ Status: CIB for comments from SC 42 and IEC TC 65/SC 65A
§ ISO/IEC 25059 -- Software engineering -- Systems and software Quality Requirements and Evaluation (SQuaRE) --
Quality Model for AI-based systems
§ Editor: Adam Leon Smith (United Kingdom)
§ Status: CD Ballot
§ ISO/IEC TS 6254 -- Information technology -- Artificial intelligence -- Objectives and approaches for explainability of
ML models and AI systems
§ Editor: Jaeho Lee (Korea)
§ Status: Working Draft
§ ISO/IEC TS 5471 -- Artificial intelligence -- Quality evaluation guidelines for AI systems
§ Editor: Olivier Blais (France)
§ Status: Working Draft
SC 42 Projects, Status and Leadership SC 42 – Artificial Intelligence

SC 42/WG 3 Trustworthiness
§ ISO/IEC TS 8200 -- Artificial intelligence -- Controllability of automated artificial intelligence systems
§ Editor: Xiaoqi Cao (China)
§ Status: Working Draft
§ ISO/IEC 12791: Information technology -- Artificial Intelligence (AI) -- Treatment of unwanted bias in classification
and regression machine learning tasks
§ Editor: Adam Leon Smith (United Kingdom)
§ Status: Working Draft
§ ISO/IEC TS 12792: Information technology -- Artificial Intelligence (AI) -- Transparency taxonomy of AI systems
§ Editor: Rania Wazir (Austria)
§ Status: Working Draft

§ Information technology -- Artificial Intelligence (AI) -- Best practice guidance for mitigating ethical and societal
concerns
§ Editor: Viveka Bonde (Sweden)
§ Status: PWI
SC 42 Projects, Status and Leadership SC 42 – Artificial Intelligence

SC 42/WG 4 Use cases and applications

§ Terms of reference: Use cases and applications for AI standardization
§ Convenor: Fumihiro Maruyama (Japan)
§ Secretariat: Nobuhiro Hosokawa (Japan)
§ ISO/IEC 5338: Information technology -- Artificial Intelligence (AI) -- AI system lifecycle processes
§ Editor: Yuchang Cheng (Japan)
§ Status: CD Ballot
§ ISO/IEC 5339: Information technology -- Artificial Intelligence (AI) -- Guidelines for AI applications
§ Editor: Shrikant Bhat (India)
§ Status: Working draft
§ ISO/IEC TR 24030:2021 Revision: Information technology -- Artificial Intelligence (AI) – Use cases
§ Editor: Yuchang Cheng (Japan)
§ Status: Working Draft
SC 42 Projects, Status and Leadership SC 42 – Artificial Intelligence

SC 42/WG 5 Computational approaches and computational characteristics of AI systems

§ Terms of reference: Standardization in the area of computational approaches and computational characteristics of AI
systems
§ Convenor: Ning Sun (China)
§ Past Convenor: Tangli Liu (China) until April 29th 2022
§ ISO/IEC TS 4213: Assessment of classification performance for machine learning models
§ Editor: Michael Theime (United States) and Lingzhong Meng (China)
§ Status: Publication
§ ISO/IEC 5392: Information technology -- Artificial Intelligence (AI) -- Reference architecture of knowledge
engineering
§ Editor: Ruiqi Li (China)
§ Status: CD Ballot
§ ISO/IEC TR : Information technology -- Artificial intelligence -- Overview of machine learning computing devices
§ Editor: Xiaoqi Cao (China)
§ Status: Working Draft
SC 42 Projects, Status and Leadership SC 42 – Artificial Intelligence

SC 42/JWG 1 Governance implications of AI (Completed)

[Joint WG with SC 40. Administered by SC 42]
§ Convenor: Yonosuke Harada (Japan)
§ Co-Convenor: Gyeung-Min Kim (Republic of Korea)
§ ISO/IEC 38507 -- Information technology -- Governance of IT -- Governance implications of the use of
artificial intelligence by organizations
§ Editor: Peter Brown (United Kingdom)
§ Status: Published!
SC 42/JWG 2 Testing of AI Systems
[Joint WG with SC 7. Administered by SC 42]
§ Convenor: Adam Leon Smith (United Kingdom)
§ Co-Convenor: Stuart Reid (United Kingdom)
§ ISO/IEC 29119-11 -- Information technology -- Artificial intelligence -- Testing for AI systems -- Part 11
§ Editor: Jonghong Jeon (Korea), Stuart Reid (United Kingdom)
§ Status: Working Draft
SC 42 Projects, Status and Leadership SC 42 – Artificial Intelligence

SC 42 Active AGs/AHGs on Specific Topics

§ AHG 4 on Liaison with JTC 1/SC 27 – Convenor: Peter Duessen (Germany)
§ AG 3 on AI standardization roadmap – Convenor: Patrick Bezombes (France)
SC 42 Completed AGs on Specific Topics
§ AG 1 on AI Management Systems Standard – Convenor: Jim McFie (Canada)
§ AG 2 on AI Systems engineering – Convenor: Luigi Troiano (Italy)
SC 42 Completed AHGs
§ AHG 1 on Dissemination and outreach – Convenor: Wael William Diab (SC 42 Chair), Secretariat: Heather
Benko (SC 42 Committee Manager)
§ AHG 2 on Liaison with JTC 1/SC 38 – Convenor: Peter Duessen (Germany)
§ AHG 3 on Intelligent systems engineering – Convenor: Luigi Troiano (Italy)
§ AHG 5 on AI standardization landscape and roadmap – Convenor: Patrick Bezombes (France)
§ Societal concerns
§ Study groups terms of reference
§ Business plan review 2018, 2019, 2020 and 2021
§ Ethical and sustainable AI proposal
SC 42 Projects, Status and Leadership SC 42 – Artificial Intelligence

SC 42 Completed Study Groups

§ SC 42/SG 1 Computational approaches and characteristics of artificial intelligence systems
§ Convenor: Tangli Liu (China)
§ Secretariat: Qun Zhang (China)
§ Status
– SG report to be submitted by the SG leadership team to SC 42 by May 31st for consideration
– NWIPs under discussion in the SG have been assigned to WG 5 to continue discussion and consideration
§ SC 42/SG 2: Trustworthiness
§ Convenor: David Filip (Ireland)
§ Secretariat: Barry Smith (Ireland)
§ Status
– Study group report on robustness completed and accepted by SC 42
– Remaining items of study from terms of reference assigned as tasks to SC 42/WG 3
§ SC 42/SG 3: Use cases and applications
§ Convener: Fumihiro Maruyama (Japan)
§ Secretariat: Nobuhiro Hosokawa (Japan)
§ Status: Remaining items of study from terms of reference assigned as tasks to SC 42/WG 4
SC 42 Liaisons SC 42 – Artificial Intelligence

SC 42 has established an extensive and comprehensive set of liaisons for collaboration

§ SC 42 provides guidance to ISO, IEC and JTC 1 committees on AI applications
§ Reflects strong internal and external interest in the AI standardization program of work
Approved Category A External Liaisons Approved Category A External Liaisons
§ EC – European Commission § OECD – Organisation for Economic Co-operation and Development
§ SC 42 liaison officer: Ray Walshe § OECD liaison officers: Karine Perset, Luis Aranda

§ ETUC – European Trade Union Confederation § SC 42 liaison officer: Rohit Israni

§ EUROCAE – The European Organization for Civil Aviation Equipment § Consumers International

§ euRobotics AISBL § Small Business Standards (SBS)

§ Big Data Value Association (BDVA) § Open Geospatial Consortium (OGC)

§ SC 42 liaison officers: Abdellatif Touimi, Ray Walshe § OGC liaison officers: George Percivall, Ingo Simonis

§ BDVA liaison officers: Ana Garcia Robles, Abdellatif Touimi § Independent International Organization for Certification (IIOC)

§ Partnership on AI (PAI) § IEEE

§ SC 42 liaison officer: Tarek Besold § ITU
§ PAI liaison officers: Terah Lyons, Peter Eckersley, Steven Adler § SC 42 liaison officer: Yoav Evenstein
§ ITU liaison officers: Reinhard Scholl, Bilel Jamoussi
 SC 42 Liaisons SC 42 – Artificial Intelligence
Approved Internal Liaisons to SC 42 Approved Internal Liaisons to SC 42
§ JTC 1/SC 7 – Software and systems engineering § JTC 1 (WG 11) – Smart Cities
§ Officers: Stuart Reid and Shuji Kinoshita § Officer: Howard Choe

§ JTC 1/SC 24 – Computer graphics, image processing and environmental data § ISO/CASCO
representation
§ ISO/PC 317 – Consumer protection: privacy by design for consumer goods and
§ JTC 1/SC 27 – Information security, cybersecurity and privacy protection services

§ JTC 1/SC 29 – Coding of audio, picture, multimedia and hypermedia information § ISO/TC 20 – Aircraft and space vehicles

§ JTC 1/SC 32 – Data management and interchange § ISO/TC 20/SC 16 – Unmanned aircraft

§ JTC 1/SC 34 – Document description and processing languages § ISO/TC 37 – Language and terminology

§ JTC 1/SC 36 – Information technology for learning, education and training § ISO/TC 37/SC 3 – Management of terminology resources
§ Officer: Jon Mason § ISO/TC 42 – Photography
§ JTC 1/SC 37 – Biometrics § Officer: Scott Foshee (United States)
§ Officer: Markku Metsämäki (Finland) § ISO/TC 69 – Applications of statistical methods
§ JTC 1/SC 38 – Cloud computing and distributed platforms § Officer: Radouane Oudrhiri (United Kingdom)
§ Officer: Toshiro Suzuki (Japan) § ISO/TC 211 – Geographic information/Geomatics
§ JTC 1/SC 40 – IT Service Management and IT Governance § ISO/TC 215 – Health infomatics
§ Officer: Terry Landers (Ireland)
§ ISO/TC 262 – Risk management
§ JTC 1/SC 41 – Internet of things and related technologies
§ ISO/TC 268 – Sustainable cities and communities
§ Officers: Osten Franberg (Sweden) Luke Fay (United States)
§ ISO/TC 269 – Railway applications
 SC 42 Liaisons SC 42 – Artificial Intelligence
Approved Internal Liaisons to SC 42
§ ISO/TC 307 – Blockchain and distributed ledger technologies
§ Officer: Janna Lingenfelder (Germany)

§ ISO/TC 309 – Governance of organizations

§ Officer: Michael Kayser

§ IEC/SyC AAL
§ Officer: Ulrike Haltrich

§ IEC/SyC SM
§ Officer: Alexander McMillan

§ IEC/TC 62 – Electrical equipment in medical practice

§ IEC/SC 62C – Equipment for radiotherapy, nuclear medicine and radiation

dosimetry

§ IEC/TC 65 – Industrial – Process measurement, control and automation

§ Officers: Rudy Belliardi (TC 65 Secretary) and Wael William Diab (SC 42
Chair)

§ IEC/SC 65A – System aspects

§ IEC/TC 100 – Audio, video and multimedia systems and equipment

SC 42 Liaisons SC 42 – Artificial Intelligence
Approved Internal Liaisons from SC 42 Approved Internal Liaisons from SC 42
§ IEC/SyC AAL – Active Assisted Living § JTC 1/SC 32 – Data management and interchange
§ SC 42 Officer: David Martin (United States) § SC 42 Officers: Wo Chang (US) and Guang Liang (China)

§ IEC/SyC Smart Cities § JTC 1/SC 34 – Document description and processing languages
§ SC 42 Officer: Tangli Liu (China) § SC 42 Officers: Jaeho Lee (Korea)

§ IEC/SyC SM – Smart Manufacturing § JTC 1/SC 36 – Information technology for learning, education and training
§ SC 42 Officer: Wael William Diab (SC 42 Chair) § SC 42 Officer: Bruce Peoples (United States)

§ IEC/TC 62 – Electrical equipment in medical practice § JTC 1/SC 37 – Biometrics

§ IEC/TC 65 – Industrial – Process measurement, control and automation § SC 42 Officers: Brianna Brownell (Canada), Frank Rudzicz (Canada)

§ SC 42 Officers: Wei Wei (Germany), Rudy Belliardi (TC 65 § JTC 1/SC 38 – Cloud computing and distributed platforms
Secretary) and Wael William Diab (SC 42 Chair)
§ SC 42 Officers: Peter Deussen (Germany), David Filip (Ireland)
§ IEC/TC 65/SC 65A – System Aspects
§ JTC 1/SC 39 – Sustainability for and by Information Technology
§ SC 42 Officer: Takashi Egawa (Japan)
§ SC 42 Officer: Yoav Evenstein (Israel)
§ JTC 1/SC 7 – Software and systems engineering
§ JTC 1/SC 40 – IT Service Management and IT Governance
§ SC 42 Officers: Yuchang Cheng (Japan) and Adam Leon Smith (UK)
§ SC 42 Officer: Geoff Clarke (Australia)

§ JTC 1/SC 27 – IT security techniques § JTC 1/SC 41 – Internet of things and related technologies

§ SC 42 Officers: Peter Deussen (Germany), Sun Yan (China) § SC 42 Officer: Wei Wei (Germany)

§ JTC 1/SC 29 – Coding of audio, picture, multimedia and hypermedia § JTC 1 (WG 11) – Smart cities
information § SC 42 Officer: Tangli Liu (China)
§ SC 42 Officers: Wo Chang (United States) and Abdellatif Benjelloun § ISO CASCO
Touimi (UK)
SC 42 Liaisons SC 42 – Artificial Intelligence
Approved Internal Liaisons from SC 42 Approved Internal Liaisons from SC 42
§ ISO/TC 22/SC 32 – Electrical and electronic components and general § ISO/TC 309 – Governance of organizations
system aspects
§ SC 42 Officer: Victoria Hailey (Canada)
§ ISO/TC 37 – Language and terminology
§ IEC/TC 100 – Audio, video and multimedia systems and equipment
§ SC 42 Officer: David Filip (Ireland)
§ ISO/TC 268 – Sustainable cities and communities
§ ISO/TC 37/SC 3 – Management of terminology resources
§ SC 42 Officer: David Filip (Ireland)

§ ISO/TC 69 – Applications of statistical methods

§ SC 42 Officer: Radouane Oudrhiri (UK)

§ ISO/TC 204 – Intelligent Transport Systems

§ SC 42 Officer: Wael William Diab (Chair)

§ ISO/TC 215 – Health informatics

§ SC 42 Officer: Paolo Alcini (Italy)

§ ISO/TC 262 – Risk management

§ SC 42 Officer: Pat Baird (United States)

§ ISO/TC 299 – Robotics

§ SC 42 Officer: David Dubois (Canada)

§ ISO/TC 307 – Blockchain and distributed ledger technologies

§ SC 42 Officers: Li Bin (China) and Dapeng Zhang (China)
 SC 42 – Artificial Intelligence

Annex C
Additional Information on SC 42 Meeting Schedule
Upcoming Meetings SC 42 – Artificial Intelligence

10th Plenary meeting 13th Plenary meeting 16th Plenary meeting

§ October, 2022 § April, 2024 § October, 2025

§ Sydney, Australia § Seoul, South Korea § Berlin, Germany

§ Confirmed § Confirmed § Confirmed

11th Plenary meeting 14th Plenary meeting 17th Plenary meeting

§ April, 2023 § October, 2024 § April, 2026

§ Tel Aviv, Israel § France § Salerno, Italy

§ Confirmed § Confirmed § Tentative

12th Plenary meeting 15th Plenary meeting

§ October, 2023
§ Vienna, Austria
§ Confirmed
Past Meetings SC 42 – Artificial Intelligence

9th Plenary meeting 6th Plenary meeting 3rd Plenary meeting

§ April 18th to 29th, 2022 § October 19th – 30th, 2020 § April 8th – 12th, 2019
§ Virtual § Virtual § Dublin, Ireland

8th Plenary meeting 5th Plenary meeting 2nd Plenary meeting

§ October 18th to 29th, 2021 § April 6th – 20th, 2020 § October 18th – 20th, 2018
§ Virtual § Virtual § Sunnyvale, CA, USA

7th Plenary meeting 4th Plenary meeting 1st Plenary meeting

§ April 26th – May 7th, 2021 § October 7th – 11th, 2019 § April 18th – 20th, 2018
§ Virtual § Tokyo, Japan § Beijing, China
 SC 42 – Artificial Intelligence

Annex D
SC 42 Member National Bodies
SC 42 Member National Bodies SC 42 – Artificial Intelligence

The following national bodies are participating members of SC 42 (P-members)

§ Australia (SA), Austria (ASI), Belgium (NBN), Canada (SCC), China (SAC),
Congo, The Democratic Republic of the (OCC), Cyprus (CYS), Denmark
(DS), Finland (SFS), France (AFNOR), Germany (DIN), India (BIS), Ireland
(NSAI), Israel (SII), Italy (UNI), Japan (JISC), Kazakhstan (KAZMEMST),
Kenya (KEBS), Korea, Republic of (KATS), Luxembourg (ILNAS), Malta
(MCCAA), Netherlands (NEN), Norway (SN), Portugal (IPQ), Russian
Federation (GOST R), Saudi Arabia (SASO), Singapore (SSC), Spain (UNE),
Sweden (SIS), Switzerland (SNV), Uganda (UNBS), United Arab Emirates
(MoIAT-STR), United Kingdom (BSI), United States (ANSI)
The following national bodies are observing members of SC 42 (O-members)
§ Argentina (IRAM), Benin (ANM), Brazil (ABNT), Côte d'Ivoire (CODINORM),
Hong Kong (ITCHKSAR), Hungary (MSZT), Indonesia (BSN), Lithuania
(LST), Mexico (DGN), New Zealand (NZSO), North Macedonia (ISRSM),
Philippines (BPS), Poland (PKN), Romania (ASRO), South Africa (SABS),
Ukraine (DSTU)
 SC 42 – Artificial Intelligence

Annex E
Overview of Guidelines for AI Applications (ISO/IEC 5339)
ISO/IEC 5339 Guidelines for AI Applications SC 42 – Artificial Intelligence

§ Macro-level view
§ The document provides a macro-level view of an AI application to facilitate its understanding,
development and use amongst all stakeholders

§ Document includes
§ motivation and objectives;
§ an approach to identifying an AI application’s stakeholders, context, functional characteristics and non-
functional characteristics;
§ an AI application framework that can be used to answer the question: “What are the characteristics
and considerations of an AI application?”;
§ guidelines for AI applications based on the make, use and impact perspectives

§ Progress
§ submitted for an eight-week CD ballot starting May 2nd, 2022
 SC 42 – Artificial Intelligence

Annex F
Big Data Trends and Emerging Projects
Key Topics: Big Data Background SC 42 – Artificial Intelligence

Overview
§ Big Data is extensive datasets — primarily in the data characteristics of volume, variety,
velocity, and/or variability — that require a scalable technology for efficient storage,
manipulation, management, and analysis. Note: Big data is commonly used in many different
ways, for example as the name of the scalable used to handle big data extensive datasets.
§ Big Data deals with characteristics that for an application domain cannot be efficiently
processed using traditional technologies and techniques in order to extract value
Key Drivers of Big Data
§ Key drivers in understanding the Big Data paradigm – how this is different from traditional data
storage and compute / processing applications
§ Volume: too big data set(s) characteristics
§ Velocity: arrives too fast
§ Variability: changes too fast
§ Variety: too diverse
§ The applications generating this data or requiring its analysis may have one more of the
above aspects present
Emerging applications are creating a paradigm shift and enabling predictive analytics
Big Data Analysis and Predictions
SC 42 – Artificial Intelligence

Tremendous market growth and job creation

Strong support that Big Data adoption is happening, cross-cutting and has significant
implications and potential
Paradigm shift and predictive analytics
IDC – Predictions from the IDC FutureScape for Big Data and Analytics
1. Visual data discovery tools will be growing 2.5 times faster than rest of the business intelligence (BI)
market. By 2018, investing in this enabler of end-user self service will become a requirement for all
enterprises.
2. Over the next five years spending on cloud-based Big Data and analytics (BDA) solutions will grow three
times faster than spending for on-premise solutions. Hybrid on/off premise deployments will become a
requirement.
3. Shortage of skilled staff will persist. In the U.S. alone there will be 181,000 deep analytics roles in 2018 and
five times that many positions requiring related skills in data management and interpretation.
4. By 2017 unified data platform architecture will become the foundation of BDA strategy. The unification will
occur across information management, analysis, and search technology.
5. Growth in applications incorporating advanced and predictive analytics, including machine learning, will
accelerate in 2015. These apps will grow 65% faster than apps without predictive functionality.
6. 70% of large organizations already purchase external data and 100% will do so by 2019. In parallel more
organizations will begin to monetize their data by selling them or providing value-added content.
Big Data Analysis and Predictions
Emerging Applications and Trends for BD SC 42 – Artificial Intelligence

IDC – Predictions from the IDC FutureScape for Big Data and Analytics
7. Adoption of technology to continuously analyze streams of events will accelerate in 2015 as it is applied to Internet of
Things (IoT) analytics, which is expected to grow at a five-year compound annual growth rate (CAGR) of 30%.
8. Decision management platforms will expand at a CAGR of 60% through 2019 in response to the need for greater
consistency in decision making and decision making process knowledge retention.
9. Rich media (video, audio, image) analytics will at least triple in 2015 and emerge as the key driver for BDA technology
investment.
10. By 2018 half of all consumers will interact with services based on cognitive computing on a regular basis.

Gartner Bid Data Motivators within ICT Space

§ On weather or not “Big data hype or substance?”
§ Beyond all the discussions, adoption of big data is simply inevitable
§ Within key IT trends
§ Identifies Big Data expertise as essential
§ Identifies Big Data expertise as needed within Web-scale I, IoT and others
§ Benefits of big data are not limited solely to better decision making
§ fewer than half of big data projects focus on direct decision making
§ most big data projects are geared to generating deeper business insights and optimizing, automating or even
designing new processes
Process Management Framework – SC 42 – Artificial Intelligence

Big Data Analytics

The proposed standard provides a framework for developing processes to effectively leverage big data analytics across the
organization irrespective of the industries/sectors. It specifies process management for big data analytics with the five process
categories as listed below with help of Process Reference Model, Measurement Framework and Assessment Model:

Process Reference Process Measurement

Model Framework
Context Attributes
Purpose Attribute Outcomes
Outcomes Rating Scale

Process Assessment Model

Process Capability by processes from Process groups

 SC 42 – Artificial Intelligence

Annex G
Overview of JTC 1
Part of the ISO, IEC and JTC 1 Family SC 42 – Artificial Intelligence

JTC 1
§ Jointly established under ISO and IEC covering the field of Information Technology
§ ICT building blocks for global markets
§ Standards for business and consumer applications
§ 33 P-members and 62 O-members
§ About 5000 active participants developing 580+ standards; over 3000 published
§ Technical areas within JTC 1 include
§ Coded character sets – Telecommunications and information exchange between systems – Software and systems engineering – Cards
and security devices for personal identification – Programming languages – Digitally recorded media – Computer graphics, image
processing – IT security techniques – Office equipment (printing) – Coding of audio, picture, multimedia (JPEG, MPEG) – Automatic ID
and data capture (RFID) – Data management – Document description, processing – User interfaces, IT for learning, education, training
– Biometrics – Cloud computing – IT Sustainability – IT governance – Internet of Things – Artificial Intelligence – Smart cities – 3D
printing and scanning – Quantum Computing
§ Strategic topics covered within JTC 1 include
§ Digital transformation
– Increased cooperation with other ISO and IEC TCs
– Working with policy makers: standards and regulations
§ Systems integration
§ Cooperation with consortia
§ Trustworthiness
§ Emerging technologies
§ Open source
 SC 42 – Artificial Intelligence

Annex H
ISO/IEC AI Workshop Series
ISO/IEC AI Workshop Series
Inaugural Workshop on 24th and 25th May 2022 (Online)
SC 42 – Artificial Intelligence

4 Content Tracks Benefits

AI Applications

This track will look at emerging AI applications, public projects, use cases and
case studies with the goal of identifying insights relating to AI application
requirements, providing an overview of supporting horizontal standardization
work and discuss roadmaps of application domain verticals

Emerging AI Technology Trends

AI technology and capability is evolving at a rapid pace. This track will look at Innovation Market Intelligence Thought Leadership
emerging AI technology areas and trends with the goal of discussing new areas workshop will focus on identify emerging trends, participating in this
and looking at the interplay of research, industry activity and standardization state of the art, priorities requirements, insights and interactive event will help to
and requirements opportunities position your organization
Novel AI Standardization Approaches as an industry leader

In addition to building on well established information and operational

technologies, standards and open-source efforts, AI introduces technology-
specific challenges through its learning nature. This track will look at such
challenges and discuss the innovative standards approaches to address them

Emerging AI Requirements

Viewed as a digital transformation enabler or through its potential to deliver

transformative insights for the betterment of society, AI is expected to become a Enhanced Stakeholders Information
ubiquitous technology. Thus, requirements to ensure responsible adoption Attract new stakeholders Gain insight into alternative
continue to emerge from a diverse set of perspectives. This track will look at the Enhanced networking
opportunities to join standardization POVs and needs through
context of use (e.g. ethics) and emerging requirements highlighting the role
standards can play to complement other efforts such as industry, regulatory etc. activities Q&A and panel discussion
ISO/IEC AI Workshop Sessions SC 42 – Artificial Intelligence

24th 1300 – 1600 UTC 25th 2200 – 26th 0100 UTC

Opening Remarks AI Applications B
AI Applications A Novel AI Standardization B
Novel AI Standardization A Closing Remarks
Session registration Session registration

25th 0500 – 0800 UTC Workshop Website

Emerging AI Requirements
Emerging Tech Trends
Session registration
Contacts:
Wael William Diab, Chair SC 42 (Artificial Intelligence): [email protected]
Heather Benko, Committee Manager SC 42 (Artificial Intelligence): [email protected]

SC 42 – Artificial Intelligence