Machine Learning And Probabilistic Graphical

Models For Decision Support Systems Kim Phuc

Tran download

https://ebookbell.com/product/machine-learning-and-probabilistic-
graphical-models-for-decision-support-systems-kim-phuc-
tran-43881298

Explore and download more ebooks at ebookbell.com

Here are some recommended products that we believe you will be
interested in. You can click the link to download.

Applied Stochastic Processes Chaos Modeling And Probabilistic

Properties Of Numeration Systems Vincent Granville Phd

https://ebookbell.com/product/applied-stochastic-processes-chaos-
modeling-and-probabilistic-properties-of-numeration-systems-vincent-
granville-phd-44871872

Applied Stochastic Processes Chaos Modeling And Probabilistic

Properties Of Numeration Systems Vincent Granville

https://ebookbell.com/product/applied-stochastic-processes-chaos-
modeling-and-probabilistic-properties-of-numeration-systems-vincent-
granville-44887996

Machine Learning A Probabilistic Perspective Adaptive Computation And

Machine Learning Series Illustrated Kevin P Murphy

https://ebookbell.com/product/machine-learning-a-probabilistic-
perspective-adaptive-computation-and-machine-learning-series-
illustrated-kevin-p-murphy-34631540

Probabilistic Machine Learning For Finance And Investing Early Release

1st Edition Deepak Kanungo

https://ebookbell.com/product/probabilistic-machine-learning-for-
finance-and-investing-early-release-1st-edition-deepak-
kanungo-48522166
Probabilistic Machine Learning For Finance And Investing Deepak
Kanungo

https://ebookbell.com/product/probabilistic-machine-learning-for-
finance-and-investing-deepak-kanungo-46122252

Probabilistic Machine Learning For Finance And Investing 1 Converted

Deepak K Kanungo

https://ebookbell.com/product/probabilistic-machine-learning-for-
finance-and-investing-1-converted-deepak-k-kanungo-56074418

Hardwareaware Probabilistic Machine Learning Models Learning Inference

And Use Cases 1st Ed 2021 Laura Isabel Galindez Olascoaga

https://ebookbell.com/product/hardwareaware-probabilistic-machine-
learning-models-learning-inference-and-use-cases-1st-ed-2021-laura-
isabel-galindez-olascoaga-36497074

Machine Learning And Internet Of Things For Societal Issues Ch

Satyanarayana

https://ebookbell.com/product/machine-learning-and-internet-of-things-
for-societal-issues-ch-satyanarayana-46750136

Machine Learning And Deep Learning Techniques For Medical Science K

Gayathri Devi

https://ebookbell.com/product/machine-learning-and-deep-learning-
techniques-for-medical-science-k-gayathri-devi-47103764
Machine Learning and Probabilistic
Graphical Models for Decision
Support Systems
Editor
Kim Phuc Tran
Univ. Lille, ENSAIT, ULR 2461 - GEMTEX - Génie et Matériaux Textiles
F-59000 Lille, France

p,
p,
A SCIENCE PUBLISHERS BOOK
A SCIENCE PUBLISHERS BOOK
First edition published 2022
by CRC Press
6000 Broken Sound Parkway NW, Suite 300, Boca Raton, FL 33487-2742
and by CRC Press
4 Park Square, Milton Park, Abingdon, Oxon, OX14 4RN

© 2022 Taylor & Francis Group, LLC

CRC Press is an imprint of Taylor & Francis Group, LLC

Reasonable efforts have been made to publish reliable data and information, but the author and publisher cannot assume
responsibility for the validity of all materials or the consequences of their use. The authors and publishers have attempted to
trace the copyright holders of all material reproduced in this publication and apologize to copyright holders if permission to
publish in this form has not been obtained. If any copyright material has not been acknowledged please write and let us know
so we may rectify in any future reprint.
Except as permitted under U.S. Copyright Law, no part of this book may be reprinted, reproduced, transmitted, or utilized
in any form by any electronic, mechanical, or other means, now known or hereafter invented, including photocopying,
microfilming, and recording, or in any information storage or retrieval system, without written permission from the publishers.
For permission to photocopy or use material electronically from this work, access www.copyright.com or contact the
Copyright Clearance Center, Inc. (CCC), 222 Rosewood Drive, Danvers, MA 01923, 978-750-8400. For works that are not
available on CCC please contact [email protected]
Trademark notice: Product or corporate names may be trademarks or registered trademarks and are used only for identification
and explanation without intent to infringe.
Library of Congress Cataloging‑in‑Publication Data (applied for)

ISBN: 978-1-032-03948-0 (hbk)

ISBN: 978-1-032-03950-3 (pbk)
ISBN: 978-1-003-18988-6 (ebk)
DOI: 10.1201/9781003189886

Typeset in Times New Roman

by Radiant Productions
 Preface

The last decades have witnessed the rapid growth of advanced technologies and their application that is
leading to the fourth industrial revolution. The recent development of information and communication
technologies has engendered to add intelligence into the industrial process to drive continuous
improvement, knowledge transfer, and data driven-based decision-making. The Internet of Things (IoT)
is one of the main technologies used to enable enabling industrial organizations to rapidly automate
and digitize traditional business processes. A huge volume of data collected can feed real-time analytic
solutions provided by Artificial Intelligence (AI), Big Data Analytics, and Decision Support Systems
(DSS), which can lead to optimal industrial operations. Based on modern technologies of the IoT, the
process of collecting, transforming, and storing data from all stages of the industrial process becomes
easier and more efficient, promoting the era of Big Data. AI algorithms provide powerful tools for
exploiting the wealth of data generated in the IoT. By extracting useful information and features from
Big Data, the AI algorithms allow complex tasks such as monitoring, and optimizing the production
process to be performed smartly and efficiently. To combine human knowledge with these above results,
DSS is integrated to help managers to make better decisions in their work.
In the era of Big Data, DSS has become vital for organizations. Machine Learning, a subfield
of AI, is a useful technology to process and analyze Big Data are a useful methodology for DSS
with a combination of data dictated and human-driven analytics. DSS applications can be used in a
vast array of diverse fields, such as making operational decisions, medical diagnosis, and predictive
maintenance. There is a lot of research in the literature regarding the development and application
of DSS. In this book, the chapters are proposed in such a way as to explore every important aspect
of Machine Learning and Probabilistic Graphical Models for Decision Support Systems. This book
presents recent advancements in research, new methods and techniques, and applications of DSS with
Machine Learning and Probabilistic Graphical Models, which are very powerful techniques to extract
knowledge from Big Data effectively and interpret decisions. The book undertakes to stimulate scientific
exchange, ideas, and experiences in the field of DSS applications. Researchers and practitioners alike
will benefit from this book to enhance the understanding of machine learning, Probabilistic Graphical
Models, and, especially, their use in DSS in the context of decision making with uncertainty. The real-
world case studies in various fields with guidance and recommendations for the practical applications of
these studies are introduced in each chapter. Current researches, trends, future directions, opportunities,
etc. will be discussed, making it friendly for beginners and young researchers.
20 November 2021 Kim Phuc Tran
Univ. Lille, ENSAIT, ULR 2461 - GEMTEX - Génie et Matériaux Textiles
F-59000 Lille, France.
[email protected]
 Contents

Preface iii
Acronyms xi

1. Introduction to Machine Learning and Probabilistic Graphical Models for Decision 1

Support Systems
Kim Phuc Tran
1 Scope of the Research Domain 1
2 Structure of the Book 2
3 Conclusion 3
References 4

2. Decision Support Systems for Healthcare based on Probabilistic Graphical 5

Models: A Survey and Perspective
Ali Raza, Kim Phuc Tran, Ludovic Koehl and Shujun Li
1 Introduction 5
1.1 Probabilistic Modeling 6
1.2 Applications of PGMs 7
2 Decision Support Systems in Healthcare 8
2.1 Probabilistic Graphical Models 9
2.2 Bayesian Networks: Directed Graphical Models 10
2.3 Markov Random Fields 12
2.4 Deep Neural Networks 14
2.5 Neural Networks with Probabilistic Graphical Models 16
3 Artificial Intelligence in Healthcare Applications 17
4 Healthcare Decision Support Systems based on Probabilistic Graphical Models 19
5 Perspectives for Healthcare Decision Support Systems based on Probabilistic 20
Graphical Models
6 Case Studies 21
6.1 Logistic Regression for ECG Classification 21
6.2 Variational Autoencoder for ECG Anomaly Detection 22
7 Conclusions 26
References 27
 Contents

3. Decision Support Systems for Anomaly Detection with the Applications in Smart 34
Manufacturing: A Survey and Perspective
Quoc‑Thông Nguyen, Tung Nhi Tran, Cédric Heuchenne and Kim Phuc Tran
1 Introduction 34
2 Decision Support Systems for Smart Manufacturing 35
3 Anomaly Detection in Smart Manufacturing 37
3.1 Smart Predictive Maintenance 37
3.2 Integrated Wearable Technology 38
3.3 Production Monitoring 40
3.4 Real-time Cybersecurity 41
4 Difficulties and Challenges of Anomaly Detection Applications in Smart 42
Manufacturing
5 Perspectives for Anomaly Detection in Smart Manufacturing 43
6 Case Studies 47
6.1 Anomaly Detection in Production Monitoring 47
6.2 Anomaly Detection in Predictive Maintenance 49
7 Concluding Remarks 52
References 53

4. Decision Support System for Complex Systems Risk Assessment with Bayesian 62
Networks
Ayeley Tchangani
1 Introduction 62
2 Bayesian Technology 64
3 BN Model for Event Oriented Risk Management 64
3.1 Variables Identification 64
3.2 Relationships Identification 66
3.3 Usage of the model 67
3.4 Illustrative Case Study in Natural Risk Management 67
4 BN for Risk Management in Industrial Systems 71
5 DBN for Risk Management of Industrial Systems 75
5.1 Brief Presentation of DBN 75
5.2 Illustrative Case Study 76
6 EOOBN for Risk Management 78
6.1 Extended Object Oriented Bayesian Network 80
6.1.1 Construction of an EOOBN 80
6.1.2 Case Study 83
7 Conclusion 85
References 86

v
 Machine Learning and Probabilistic Graphical Models for Decision Support Systems

5. Decision Support System using LSTM with Bayesian Optimization for Predictive 88
Maintenance: Remaining Useful Life Prediction
Huu Du Nguyen and Kim Phuc Tran
1 Introduction 88
2 Predictive Maintenance and Remaining Useful Life Prediction 89
3 Machine Learning based Decision Support System for Predictive Maintenance 92
4 Long Short Term Memory Networks using Bayesian Optimization 93
4.1 Long Short Term Memory Networks 93
4.2 Bayesian Optimization 95
5 Decision Support System for Remaining Useful Life Prediction using LSTM with 96
Bayesian Optimization
6 A Case Study 98
7 Conclusion and Perspectives 99
References 101

6. Decision Support Systems for Textile Manufacturing Process with Machine 107
Learning
Zaohao Lu, Zhenglei He, Kim Phuc Tran, Sebastien Thomassey, Xianyi Zeng and
Mengna Hong
1 Introduction 107
2 Relevant Literatures 108
2.1 Intelligent Techniques used for Textile Process Modeling 109
2.1.1 Artificial Neural Networks 109
2.1.2 Fuzzy Logic 110
2.1.3 Fuzzy Inference System 111
2.1.4 Support Vector Machine 111
2.1.5 Gene Expression Programming 111
2.2 Decision-making of Textile Manufacturing Process 111
2.2.1 Classic Methods 112
2.2.2 Meta-heuristic Methods 112
2.2.3 Multi-criteria Meta-heuristic Methods 112
3 Case Study: Decision-making of Denim Ozonation 114
3.1 Problem Formulation 114
3.2 Methodology 115
3.2.1 ANN Model 115
3.2.2 Determining the Criteria Weights using the AHP 116
3.2.3 The Markov Decision Process 117
3.2.4 The RL Algorithm: Q-learning 118
3.3 Case Study 118
3.3.1 Results and Discussion 119
4 Conclusion 120
References 121

vi
 Contents

7. Anomaly Detection Enables Cybersecurity with Machine Learning Techniques 124

Truong Thu Huong, Nguyen Minh Dan, Le Anh Quang, Nguyen Xuan Hoang,
Le Thanh Cong, Kieu‑Ha Phung and Kim Phuc Tran
1 Introduction 124
2 Cybersecurity of Industrial Systems 125
2.1 Cyberattack Detection for Industrial Control Systems 126
2.2 Anomaly Detection for Time-series Data 127
3 Machine Learning-based Anomaly Detection for Cybersecurity Applications 128
3.1 Data Driven Hyperparameter Optimization of One-Class Support Vector 129
Machines for Anomaly Detection in Wireless Sensor Networks
3.1.1 Anomaly Detection Scheme 129
3.1.2 Illustrative Example in WSN Anomaly Detection 132
3.2 Real Time Data-Driven Approaches for Credit Card Fraud Detection 132
3.2.1 Anomaly Detection Scheme 132
3.2.2 Illustrative Example in Credit Card Fraud Detection 133
3.3 Nested One-Class Support Vector Machines for Network Anomaly Detection 134
3.3.1 Nested OCSVMs and Anomaly Detection Scheme 134
3.3.2 Illustrative Example in Network Anomaly Detection 136
3.4 A Data-Driven Approach for Network Anomaly Detection and Monitoring 137
Based on Kernel Null Space
3.4.1 Anomaly Detection Scheme 138
3.4.2 Illustrative Example in Network Anomaly Detection 141
4 Federated Learning-based Anomaly Detection for Cybersecurity Applications 142
4.1 Security System Architecture for IoT Systems 142
4.1.1 Design of Edge-Cloud System Architecture 142
4.1.2 Data Pre-processing at the Edge 144
4.1.3 Detection Mechanism 144
4.1.4 Performance Evaluation 149
4.1.5 Summary 157
4.2 Anomaly Detection in Industrial Control System—Smart Manufacturing 157
4.2.1 Federated Learning-based Architecture for Smart Manufacturing 158
4.2.2 Anomaly Detection Algorithm using Hybrid VAE-LSTM Model at 161
Edge Devices
4.2.3 Data Pre-processing 166
4.2.4 Detection Performance Evaluation 167
4.2.5 Evaluation on Edge Computing Efficiency 169
4.2.6 Summary 174
5 Difficulties, Challenges, and Perspectives for Machine Learning-based Anomaly 175
Detection for Cybersecurity Applications
6 Conclusion 177
References 178

vii
 Machine Learning and Probabilistic Graphical Models for Decision Support Systems

8. Machine Learning for Compositional Data Analysis in Support of the Decision 184
Making Process
Thi Thuy Van Nguyen, Cédric Heuchenne and Kim Phuc Tran
1 Introduction 184
2 Modeling of Compositional Data 185
3 Machine Learning for Multivariate Compositional Data 187
3.1 Principal Component Analysis 188
3.2 Clustering 191
3.3 Classification 193
3.3.1 Support Vector Machine Classification using Ilr—Transformation 194
3.3.2 Support Vector Machine Classification using Dirichlet Feature 196
Embedding Transformation
3.4 Regression 198
4 Anomaly Detection using Support Vector Data Description 203
4.1 Support Vector Data Description 203
4.2 Anomaly Detection using SVDD with Dirichlet Density Estimation 204
4.2.1 Transform CoDa using Dirichlet Density Estimation 204
4.2.2 Anomaly Detection using SVDD with Dirichlet Density-transformed Data 206
4.2.3 An Example of Anomaly Detection using SVDD 207
5 Conclusion 209
References 210

9. Decision Support System with Genetic Algorithm for Economic Statistical Design 216
of Nonparametric Control Chart
Alejandro Marcos Alvarez, Cédric Heuchenne, Phuong Hanh Tran and Alireza Faraz
1 Introduction 216
2 Background 218
2.1 Statistical Process Monitoring with Control Chart 218
2.2 Parametric and Nonparametric Control Charts 219
2.2.1 The x– Chart 219
2.2.2 The SN Chart 219
2.2.3 The SR Chart 220
2.3 Related Works 221
3 Economic Statistical Design of SN & SR Control Charts 222
4 Experiments 225
5 Results Discussion 228
6 Conclusions 231
References 232
Appendix 234

viii
 Contents

10. Jamming Detection in Electromagnetic Communication with Machine Learning: 252

A Survey and Perspective
Jonathan Villain, Virginie Deniau and Christophe Gransart
1 Introduction 252
2 Electromagnetic Waves Communication Jamming 253
2.1 Susceptibility of the Physical Layer in Presence of a Jamming Signal 253
2.2 Smart Jamming 255
3 Difficulties and Challenges of Electromagnetic Waves Communication Anomaly 256
Detection
3.1 Detection on Physical Layers 256
3.2 Smart Jamming Detection 256
3.3 Transmission and Mobility 257
3.4 Transmitter Location 257
4 Machine Learning Techniques for Electromagnetic Waves Communication Anomaly 258
Detection
4.1 Classification Algorithms Specificities 258
4.2 ML for Jamming Detection Algorithm for a TETRA Base Station Receiver 260
4.3 ML for Jamming Detection in 5G Radio Communication 260
4.4 ML for Jamming Detection in IoT Network 261
4.5 More Applications of ML for Jamming 261
5 A Case Study 262
5.1 Preliminary Description of the Measurement Test Site 262
5.2 Jamming Signals 263
5.3 Device Setting 263
5.4 Spectrum Analysis 265
5.5 Learning and Result 266
6 Conclusion 267
References 269

11. Intellectual Support with Machine Learning for Decision-making in Garment 272
Manufacturing Industry: A Review
Yanni Xu and Xiaofen Ji
1 Introduction 272
2 Problems in Garment Manufacturing 273
3 Garment Manufacturing using Machine Learning 276
4 Popular Machine Learning Algorithms 277
5 Potential Machine Learning Applications in Garment Manufacturing 280
6 Case Study 284
7 Conclusion 287
References 288

ix
 Machine Learning and Probabilistic Graphical Models for Decision Support Systems

12. Enabling Smart Supply Chain Management with Artificial Intelligence 294
Thi Hien Nguyen, Huu Du Nguyen, Kim Duc Tran, Dinh Duy Kha Nguyen and
Kim Phuc Tran
1 Introduction 294
2 AI for Demand Forecasting 296
3 AI for Logistics 297
4 AI for Production 298
5 AI for Decision Support Systems in SCM 299
6 Blockchain Technique for SCM 301
7 Case Study 303
8 Conclusion 304
References 305
Index 311
About the Editor 317

x
 Acronyms

5G 5 Generation DSSS Direct-Sequence Spread

AD Anomaly Detection Spectrum
AE Autoencoder DT Decision Tree
AHC Agglomerative Hierarchical ECG Electrocardiograph
Clustering ED Economic Design
AI Artificial Intelligence EKNS Enhanced Kernel Null Space
ANN Artificial Neural Network ELBO Evidence Lower Bound
ARL Average Run Length EM ElectroMagnetic
ARP Address Resolution Protocol EML Edge-Machine-Learning
ASC Agriculture Supply Chain ERP Enterprise Resource Planning
AUC Area Under the ROC Curve ESD Economic Statistical Design
BC Blockchain EWMA Exponentially Moving Average
BN Bayesian Network FL Federated Learning
BO Bayesian Optimization FN False Negative
BS BackSpace FP False Positive
BT Blockchain Technique FPR False Positive Rate
CE Cross Entropy GA Genetic Algorithm
CL Centralized Learning GAN Generative Adversarial Networks
CLSC Closed-Loop Supply Chain HTM Hierarchical Temporal Memory
CNN Convolutional Neural Network IC In-control
CoDa Compositional Data ICS Industrial Control System
CPD Conditional Probability IDS Intrusion Detection System
Distribution IF Isolation Forest
CPS Cyber-Physical System IIoT Industrial Internet of Thing
CS Computer Science IoT Internet of Thing
DBN Deep Belief Network IPS Intrusion Prevention System
DDoS Distributed DoS IT Information Technology
DDS Decision Support System KDE Kernel Density Estimation
DL Deep Learning KL Kullback-Leibler
DNN Deep Neural Network KNN K-Nearest Neighbors
DOE Design of Experiment KPCA Kernel Principle Component
DoS Denial of Service Analysis
DR Detection Rate KQE Kernel Quantile Estimator
DSS Decision Support System LCL Lower Control Limit
 Machine Learning and Probabilistic Graphical Models for Decision Support Systems

LOCF Last Observation Carried RL Reinforcement Learning

Forward RMSE Root Mean Squared Error
LoS Line of Sight RNN Recurrent Neural Network
LR Linear Regression ROC Receiver Operating
LSTM Long-Short Term Memory Characteristic
M2M Machine-to-Machine RSME Root Mean Square Error
MAD Mean Absolute Deviation RSSI Received Signal Strength
MAPE Mean Absolute Percentage Error Indicator
MAR Missing at Random RUL Remaining Useful Lifetime
MCAR Missing Completely at Random SCA Side Channel Attack
ML Machine Learning SCADA Supervisory Control and Data
MLP Multi Layer Perceptron Acquisition
MNAR Missing not at Random SCM Supply Chain Management
MQTT Message Queuing Telemetry SDN Software Defined Network
Transport SM Smart Manufacturing
MRF Markov Random Field SN charts Shewhart charts using the Sign
MSE Mean Square Error Statistics
MU-MIMO Multi User-Multiple-Input SPC Statistical Process Control
Multiple-Output SPM Statistical Process Monitoring
NN Neural Network SR charts Shewhart charts using the
NR New Radio Wilcoxon Signed-rank Statistics
OC Out-of-control SSH Secure Shell
OCSVM One Class Support Vector SVc Support Vector classification
Machine SVDD Support Vector Data Description
OFDM Orthogonal Frequency Division SVM Support Vector Machine
Multiplexing TDA Topological Data Analysis
OoS Out of Sight TETRA TErrestrial Trunked RAdio
PCA Principal Component Analysis UCL Upper Control Limit
PdM Predictive Maintenance URL Uniform Resource Locator
PDR Packet Delivery Ratio VAE Variational Autoencoder
PGM Probabilistic Graphical Models VR Virtual Reality
PLM Product Lifecycle Management Wi-Fi Wireless Fidelity
QoS Quality of Service WSN Wireless Sensor Network
RBF NN Radial Basis Function Neural XAI Explainable Artificial
Network Intelligence
RF Random Forest XGBoos eXtreme Gradient Boosting
RFID Radio Frequency IDentification

xii
 Chapter 1

Introduction to Machine Learning and

Probabilistic Graphical Models for
Decision Support Systems
Kim Phuc Tran

1 Scope of the Research Domain

We are now witnessing the rapid development and powerful application of advanced technologies,
leading to the 4th industrial revolution or Industry 4.0 1 . Digitization is changing every aspect of
society and industry. Sensors with integrated Internet of Things (IoT) technology are being used
more and more widely in the digital transformation process of businesses. The wide use of cyber-
physical systems and the IoT lead to the era of Big Data. The massiveness, complexity, and
heterogeneity of data streams require advanced computing technologies which are now performed
efficiently thanks to the availability of Artificial Intelligence (AI) and Cloud Computing 2 .
Industry 4.0 can provide automatic solutions to different sectors such as manufacturing,
healthcare, automation, supply chain management. However, there are many challenges in
Industry 4.0 such as shorter product life cycles, the need for resources to design, manufac-
ture, and quality control that decision-making processes in companies are becoming extremely
complex and require more and more knowledge 3 . In this context, decision-making based on
the data gathered from the process of data-driven decision-making is essential. Data-driven
decision-making is a technology that brings a lot of benefits to the decision-making process of
enterprises. As an essential tool, Decision Support System (DSS) is designed to assist compa-
nies to support the decision-making process and making more effective decisions. A DSS is an
information system that analyses data from organizations and presents it so that managers can
make decisions more easily 4 .
In the era of Big Data, DSS has become vital for organizations. DSS applications can be
used in a vast array of diverse fields, such as making operational decisions, medical diagnosis,
and predictive maintenance. In industry 4.0, the emergence of increasingly complex Big Data
brings more challenges to the current DSS technology. As a powerful solution, in this case,
Machine learning is a useful technology for decision support systems to solve complex decision

Univ. Lille, ENSAIT, ULR 2461 - GEMTEX - Génie et Matériaux Textiles, F-59000 Lille, France.
Email: [email protected]
 Machine Learning and Probabilistic Graphical Models for Decision Support Systems

problems 5 . In addition, in the context of decision-making under uncertainty, Probabilistic

Graphical Modeling (PGM) is a rich framework that provides a powerful formal framework to
model complex data 6 . PGM is widely used throughout machine learning and in many real-time
applications in various fields of engineering. These are the primary reasons for using PGMs.
PGMs are acknowledged as a powerful framework for complex domains. Bringing together graph
theory and probability theory, PGMs can be used to represent relations compactly and permit
efficient inference in the presence of uncertainty 7 . This book is devoted to the development
of a decision support system based on machine learning and PGMs with applications in many
fields such as manufacturing, healthcare, supply chain management, predictive maintenance,
cybersecurity, etc.

2 Structure of the Book

This is the first book that presents recent advancements of research, new methods, and tech-
niques with applications in DSS using Machine Learning and PGMs which are very powerful
techniques to extract knowledge from Big Data effectively and interpret decisions. It explores
Bayesian Network 6 , Long Short-Term Memory (LSTM) networks 1 , Reinforcement Learning 8;5 ,
Anomaly Detection 9 , Intrusion Detection 10 , etc. The book contains 12 chapters.
In the Introductory chapter “Introduction to Machine Learning and Probabilistic Graphical
Models for Decision Support Systems”, the book editor Kim Phuc Tran elaborates on the
peculiarities of Decision Support Systems using Machine Learning and Probabilistic Graphical
Models. He determines recent research streams and summarizes the structure and contributions
of the book.
Ali Raza, Kim Phuc Tran, Ludovic Koehl, and Shujun Li investigate in their chapter, “De-
cision Support Systems for Healthcare based on Probabilistic Graphical Models: a survey and
perspective”, a survey and perspective about the development of Probabilistic Graphical Mod-
els based Decision Support Systems for Healthcare. This chapter fills the gap in the literature
by identifying and analyzing research on the application of DSSs for Healthcare. The authors
review and discuss open research issues that are important for this research stream.
Quoc-Thong Nguyen, Tung Nhi Tran, Cédric Heuchenne, and Kim Phuc Tran develop in
their chapter, “Decision Support Systems for Anomaly Detection with the Applications in Smart
Manufacturing: a survey and perspective”, a survey of the key techniques that enable SM,
including IIoT, Big Data, DSS and AI with several important perspectives for the decentralised
techniques in Smart Manufacturing.
Ayeley Tchangani develop in his chapter, ‘Decision Support System for Complex Systems
Risk Assessment with Bayesian Networks, an overview of how to use different Bayesian tech-
nology tools to model and analyze risk management problems.
Huu Du Nguyen and Kim Phuc Tran develop in their chapter, “Decision Support System
using LSTM with Bayesian optimization for Predictive Maintenance: Remaining Useful Life
Prediction” an ML based method to build a DSS using the LSTM with a Bayesian optimization
algorithm for predicting Remaining Useful Life Prediction.
Zaohao Lu, Zhenglei He, Kim Phuc Tran, Sebastien Thomassey, Xianyi Zeng, and Mengna
Hong develop in their chapter, “Decision Support Systems for Textile Manufacturing Process
with Machine Learning” a decision support system combining the Artificial Neural Network
2
 Introduction to Decision Support Systems

model, Analytic Hierarchy Process and Q-learning for supporting the decision-making of textile
manufacturing process.
Truong Thu Huong, Nguyen Minh Dan, Le Anh Quang, Nguyen Xuan Hoang, Le Thanh
Cong, Kieu Ha Phung, and Kim Phuc Tran develop in their chapter, “Anomaly Detection
enables Cybersecurity with Machine Learning techniques” an overview of Cybersecurity issues
for Industrial systems, IoT-based Industrial systems, and the cyberattack detection issues for
Industrial Control Systems.
Thi Thuy Van Nguyen, Cédric Heuchenne and Kim Phuc Tran develop in their chapter,
“Machine learning for compositional data analysis in Support of the Decision Making Process”
a review of several researches related to applying ML to compositional data, including principal
component analysis, clustering, classification, and regression. They introduced a transformation
method based on Dirichlet density estimation to transform CoDa into real data and apply this
transformed data in anomaly detection using Support Vector Data Description.
Alejandro Marcos Alvarez, Cédric Heuchenne, Phuong Hanh Tran and Alireza Faraz develop
in their chapter, “Decision support system with Genetic Algorithm for economic statistical de-
sign of Nonparametric control chart” an economic statistical design (ESD) for two nonparametric
control charts based on the sign and the Wilcoxon signed-rank tests. Genetic Algorithm based
DSS is used to find the optimal parameters of the designed charts.
Jonathan Villain, Virginie Deniau and Christophe Gransart develop in their chapter, “Jam-
ming Detection in Electromagnetic communication with Machine Learning: a survey and per-
spective” an overview on the threat of the jamming on the use of wireless communication and
they shown the interest of ML to help to counter this threat.
Yanni Xu and Xiaofen Ji develop in their chapter, “Intellectual Support with Machine Learn-
ing for Decision-making in Garment Manufacturing Industry: A Review” insights and relevant
references for both researchers and practitioners on the machine learning-based decision support
for smart manufacturing in garment industry 4.0.
Thi Hien Nguyen, Huu Du Nguyen, Kim Duc Tran, Dinh Duy Kha Nguyen, and Kim Phuc
Tran develops in their chapter, “Enabling Smart Supply Chain Management with Artificial
Intelligence” a comprehensive overview of the applications of the AI Technique in Supply Chain
Management.

3 Conclusion
The book undertakes to stimulate scientific exchanges, ideas, and experiences in the field of DSS
applications. Researchers, postgraduate students, and practitioners alike will benefit from this
book to enhance the understanding of Machine Learning, Probabilistic Graphical Models, and
their use in DSS in practice, and especially in the context of decision making with uncertainty.
The real-world case studies in various fields with guidance and recommendations for the practical
applications of these studies are introduced in each chapter.

3
 Machine Learning and Probabilistic Graphical Models for Decision Support Systems

References
[1] H.D. Nguyen, K.P. Tran, S. Thomassey and M. Hamad. Forecasting and anomaly detection
approaches using LSTM and LSTM Autoencoder techniques with the applications in supply
chain management. International Journal of Information Management, 57: 102282, 2021.

[2] K.P. Tran. Artificial intelligence for smart manufacturing: Methods and applications. Sensors,
21(16): 5584, 2021.

[3] K.P. Tran, S. Thomassey, X. Zeng, C. Yi, Z. He and J. Xu. Modeling of textile manufacturing
processes using intelligent techniques: A review. The International Journal of Advanced
Manufacturing Technology, pp. 1–29, 2021.

[4] C.W. Holsapple, R.H. Bonczek and A.B. Whinston. Foundations of Decision Support Systems.
Academic Press, 2014.

[5] J. Xu, S. Thomassey, X. Zeng, C. Yi, Z. He and K.P. Tran. Multi-objective optimization of
the textile manufacturing process using deep-q-network based multi-agent reinforcement
learning. Journal of Manufacturing Systems, 2021.

[6] K. Topuz and D. Delen. A probabilistic bayesian inference model to investigate injury severity
in automobile crashes. Decision Support Systems, p. 113557, 2021.

[7] D. Koller and N. Friedman. Probabilistic Graphical Models: Principles and Techniques. MIT
Press, 2009.

[8] J. Xu, S. Thomassey, X. Zeng, C. Yi, Z. He and K.P. Tran. A deep reinforcement learning based
multi-criteria decision support system for optimizing textile chemical process. Computers in
Industry, 125: 103373, 2021.

[9] T.H. Nguyen, K.D. Tran, P.H. Tran, A. Ahmadi Nadi and K.P. Tran. Application of Machine
Learning in Statistical Process Control Charts: A Survey and Perspective, pp. 7–42. Springer
International Publishing, Cham, 2022.

[10] M.L. Dao, D.L. Tran, M.D. Nguyen, A.Q. Le, D.T. Bui, K.P. Tran, T.H. Truong and P.B. Ta.
Detecting cyberattacks using anomaly detection in industrial control systems: A federated
learning approach. Computers in Industry, 132: 103509, 2021.

4
 Chapter 2

Decision Support Systems for Healthcare based

on Probabilistic Graphical Models: A Survey
and Perspective
Ali Raza,1,2 Kim Phuc Tran,1,* Ludovic Koehl1 and Shujun Li2

1 Introduction
Probabilistic graphical modeling (PGM) deals with the branch of machine learning which studies
the use of probability distributions to describe a given event to make useful predictions about
it. PGM is widely used throughout machine learning and in many real-world applications. Such
techniques can be used to address problems in fields such as medicine, language processing, and
computer vision. This combination of theory and powerful applications makes PGMs one of the
most interesting topics in the modern era of artificial intelligence (AI). One major advantage
of probabilistic models is that they provide an idea about the uncertainty associated with
predictions. Such ideas related to uncertainty and confidence are of extreme utility when it
comes to sensitive and critical machine learning applications, such as clinical healthcare. To
understand probabilistic models at the abstract level, let us consider a classification problem
with N classes. If the model is probabilistic it will provide a probability for each of the N classes
for a given input, i.e., the model which provides a probability distribution over the N classes.
Usually, we consider the class with the highest probability as the output class. Typical examples
of probabilistic models in machine learning are logistic regression, hidden Markov models and
Bayesian classifiers, and neural networks with the softmax function (we will discuss in detail in
later sections). Note that logistic regression based on the sigmoid function can be considered
as an exception, as it provides the probability in relation to one class only.
Another way to understand the difference between probabilistic and non-probabilistic models
is their respective objective functions. For example, in linear regression, the objective function
is based on the squared, where the objective is to minimize the Mean Squared Error (MSE) or
Root Mean Squared Error (RMSE), the later is defined by Eq. 1.
n
1
RMSE = (y − y )2 , (yi − yi )2 (1)
n i=1

1Univ. Lille, ENSAIT, ULR 2461 - GEMTEX - Génie et Matériaux Textiles, F-59000 Lille, France.
2School of Computing & Institute of Cyber Security for Society (iCSS), University of Kent, UK.
* Corresponding author: [email protected]
 Machine Learning and Probabilistic Graphical Models for Decision Support Systems

Here, n is the total number of data samples, yi is the true label, and yi is the predicted label.
�

The intuition behind this is to calculate the RSME by predicting a given data point based on
the difference between the actual value and the predicted value. As the objective function
here is not based on probabilities, but on the absolute difference between the actual value and
the predicted value they can be considered as non-probabilistic models. Typical examples of
non-probabilistic models are Support Vectors Machines (SVMs) and @Shujun: sigmoid predicts
the probability of occurrence of a binary outcome. It is a special case of linear regression
as it predicts the probabilities of outcome using log function. We use the activation function
(sigmoid) to convert the outcome into categorical value. In regards to probabilistic models, such
as neural networks with softmax output function, the objective function is usually cross-entropy
(binary cross-entropy in case of a binary classifier), given by Eq. 2. Here, p(yi ) and yi is the
predicted label and true label of data sample i respectively. The intuition behind cross-entropy
is; if the probabilistic model predicts the true class of a data point with high confidence, the
loss will be less.
n
1n
CS = − (log (p(yi))) (2)
n i=1

As we notice that cross-entropy is based on probabilities, such models can be regarded as

probabilistic models. Therefore, to differentiate between probabilistic and non-probabilistic
models, one of the easiest ways is to analyze the loss function of the model.

1.1 Probabilistic Modeling

To understand probabilistic modeling, the simplest way would be to define a real-world model
in the form of the mathematical equation.
y = αx, (3)

where y is the dependent variable which we want to predict, and x is the independent variable,
upon which y is dependent, paramaterized by α. For example, y may be the price of a car, and x
are features that affect price, e.g., color, the number of seats, the engine size, etc. We assume
that y is a linear function of x. However, real-world events are very complicated to model
because they involve a certain amount of uncertainty. Therefore, we model such events in the
form of probability distributions, represented as p(x, y). The probabilistic aspect of modeling
has significant importance, because we cannot perfectly predict the future as the world is often
stochastic. Moreover, we need to assess the confidence o f o ur p redictions. I t i s o ften t he case
that predicting a single value is not enough, we need the system to output its beliefs about what
is going on in the event. To overcome this, we can write the probability model as a product of
conditional probabilities.
n
(4)
n
P (y, x1 , x2 , . . . , xn ) = p(y) p(xi |y)
i=1

A small number of parameters (e.g., weights in case of machine learning) can be used to describe
each factor p(xi |y).
6
 Decision Support Systems for Healthcare based on Probabilistic Graphical Models

1.2 Applications of PGMs

PGMs have a number of diverse real-world applications. Typical examples of such applications
include image generation, inpainting, denoising, language translation, speech recognition, and
diagnosis in clinical healthcare and medicine. In this subsection, we provide an overview of
applications in healthcare and medicine.
PGMs can assist clinical practitioners in diagnosing diseases and prognoses. For example, in 1
Bayesian networks (which we will discuss later) based model has been developed for diagnosing
pneumonia. Their model was able to distinguish patients with pneumonia from patients with
other diseases with high sensitivity (0.95) and specificity (0.965), and was used for many years
in the clinical practice. Figure 1 outlines the network proposed in 1 .

Figure 1: Structure of Bayesian network. Reprinted from Aronsky and Haug 1 .

Regarding the application of PGMs in healthcare, probabilistic methods lie primarily in

the realm of AI. The AI community first encountered these methods in the search of building
computerized systems designed to perform complex tasks, such as medical diagnosis, at an
expert level. Researchers in this domain quickly realized the need for methods that allow the
integration of evidence and information to provide support for decision making under certain
uncertainty. Furthermore, academia has recognized that Decision Support Systems (DSS) have
the utmost important role in computer-based information systems and play a crucial role in
supporting managers in their semi-structured or unstructured decision-making activities. Using
a predefined set of rules, DSS extracts knowledge from complex data and presents it in an
appropriate way. For instance, Gorry and Scott Morton 2 claimed that information systems
should exist only to support decisions. Thereafter, there has been an exponentially growing
amount of research in the area of DSS. Medical diagnosis is one of the most important research
subjects in medical informatics. Hence, a lot of research is being carried out in the application of
DSS in healthcare. By adopting proper DSS, healthcare can be made easily accessible to remote
and large populations. Furthermore, physicians can have easy access to medical records, medical
7
 Machine Learning and Probabilistic Graphical Models for Decision Support Systems

test results, medical images, and information about medication remotely anytime 3 . Moreover,
healthcare requires responsibility in managing a large amount of healthcare-related information.
It can be done by proper modeling of information that field experts can continuously build a
strong policy of welfare. The main goal of DSS is to provide experts with information when
it is needed. Such systems provide knowledge, models, and data processing tools to help the
experts in making efficient and better decisions in many situations. The goal of such systems
is to resolve several problems in healthcare to help patients and their families, and the clinical
practitioners manage their healthcare by providing better access to these services 4 .
A lot of research has been carried in the application of DSS in healthcare. Feinleib 5 suggest
that data mining methods are promising in the application of DSS in healthcare. A prototype
of a system for self-management in healthcare to assist patients with diabetes and to track their
blood glucose levels has been developed in 6 . Goldberg et al. 7 used web-based DSS for depart-
ments related to emergency to assess the performance features source of the recommendations
generated by experts. The results show that a remote clinical decision support system decreases
time-to-trial in the decision support to clinical interventions.
In regards to the importance of DSS in healthcare, this chapter reviews some of the re-
search work on healthcare DSS based on Probabilistic Graphical Models (PGMs) 8 and machine
learning. The rest of the chapter is organized as follows: Section 2 discusses decision support
systems in Healthcare. Section 3 presents a review about the application of artificial intelli-
gence in healthcare. Section 4 discusses healthcare DSSs based on PGMs. Section 5 provides
perspectives for Healthcare DSSs based on PGMs. Section 6 provides case studies of DSS in
healthcare. Section 7 concludes the chapter.

2 Decision Support Systems in Healthcare

DSS in healthcare are intended to assist physicians and other health practitioners in decision-
making tasks. It can be also defined as a computerized algorithm that uses data from a number
of patients to generate case-specific or encounter-specific advice 9 . Decision support systems in
healthcare have been studied and explored extensively in the healthcare industry. These systems
links observations with health knowledge to determine health options by health practitioners
for improved healthcare. The main idea of healthcare DSS is a set of rules derived from medical
professionals applied to dynamic knowledge. Data mining is well suited to give decision support
for healthcare. There are several probabilistic classification techniques available that can be
used for healthcare decision support systems. Various techniques are being used for differential
diagnoses. Health decision support systems provide a number of soft computing techniques to
derive useful information from data repositories, human knowledge, and literature to support
decision-making across operational and clinical healthcare processes.
DSSs are an important part of modern healthcare organizations. They help felicitate pa-
tients, practitioners, and healthcare stakeholders by providing patient-centered information and
expert health knowledge 10 . To improve the efficiency and quality of healthcare, healthcare
decision-making uses the knowledge obtained from the smart decisions systems. For example,
automated DSSs for ECG is available in primary health care units and hospitals to fulfill the
increasing healthcare requirements of prognosis in the domain of heart diseases. A lot of studies
have used healthcare DSSs to promote individualized cardiovascular prevention 11;12;13 . DSSs
8
 Decision Support Systems for Healthcare based on Probabilistic Graphical Models

provide timely information at the point of care to inform patient care decisions. The use cases
of decision support systems can be summarized as follow;
1. Clinical Management: DSS can alert healthcare practitioners to reach out to patients who
have not followed management schedules, or are due for a follow-up, and help identify
patients eligible for research based on specific criteria 14 .
2. Diagnosis Support: DSSs for healthcare diagnosis, known as diagnostic decision support
systems (DDSs) have traditionally provided computerized support, whereby they might be
provided an input (data/user selections), and then the output of possible diagnoses 15;16;17 .
Moreover, the healthcare industry generates a large amount of data. Consequently, DSSs
are used extensively to capture and transfer information. Therefore, in this section, we will
briefly overview various classification techniques for healthcare decision support systems. In
other words, this section summarizes some of the historical and state of the art decision support
systems in healthcare, and analyzes the success factors needed for widespread deployment, and
postulates the future trends of the field in the context of a new decision management paradigm.

2.1 Probabilistic Graphical Models

PGMs were developed in the early 1980s by researchers working in mathematics, AI, and econ-
omy with the purpose of solving complex problems which were proven not to be solvable by
methods existing so far. PGMs are a rich framework for encoding probability distributions over
complex domains: joint (multivariate) distributions over large numbers of random variables
that interact with each other. These representations exist in the intersection of statistics and
computer science, depending on concepts from probability theory, graph algorithms, machine
learning, and more. They are the basis for the state-of-the-art methods in a number of applica-
tions, such as healthcare, image processing, speech, and natural language processing, etc. They
are also the building blocks in the formulation of many machine learning problems. PGMs allow
dealing with problems that were not solvable with traditional probabilistic methods or other
artificial intelligence techniques.
Depending on whether the graph is directed or undirected, we can classify graphical models
into Bayesian networks (BN) and Markov networks (MN), respectively, as shown in Figure 2.

Figure 2: Types of PGMs.

9
 Machine Learning and Probabilistic Graphical Models for Decision Support Systems

Both types contain the properties of conditional probabilities and independence. However,
they can encode the different sets of independence and the conditional probabilities of the
distribution. Each type is discussed as follows.

2.2 Bayesian Networks: Directed Graphical Models

BN is a knowledge-based graphical representation 18;19 that depicts a set of variables and their
probabilistic relationships among diseases and the corresponding symptoms. In other words,
BN represent probability distributions that can be calculated by-product of the local conditional
probability distribution. To understand, let us use the notation I(p) to denote the set of all in-
dependencies for a joint distribution p. If p(x|y) = p(x)p(y), we say x⊥y ∈ I(p). The Bayesian
network can describe many independencies in I(p); such independencies can be retrieved from
the directed graphs. For example, a Bayes net G with three nodes A, B and C could have es-
sentially three different possible structures with different independence assumptions, as shown
in Figure 3.

Figure 3: Bayesian networks over three variables, encoding different types of dependencies.

These structures describe the independencies in a three-variable BN. We can extend it to

general networks by recursively applying them over any larger graph. This leads to a notion
called d-separation (where d stands for directed).
BN are used to find the probability of possible diseases to occur, given their symptoms.
These networks take advantage of their property to require the knowledge and conclusions of
domain experts in the form of probabilities. However, it is not viable for large complex systems
given multiple symptoms. To understand the BN, let us consider a canonical example of a
researchers network. The setting of the graph, as shown in Figure 4, consists of four variables
which are given as follows.

1. Difficulty: Takes values 0 and 1 for minimum and maximum difficulty, respectively.

2. Intelligence: Takes values 0 and 1 for not intelligent and intelligent, respectively.

3. Research output: Takes values 1, 2 and 3 for good, average and bad research, respectively.

4. Research articles: shows the number of research articles published.

10
 Decision Support Systems for Healthcare based on Probabilistic Graphical Models

Figure 4: Bayesian networks: Directed graphical models.

The edges in the graph show the dependencies in the graph. The Research Output of a re-
searcher depends on the Difficulty of the research area and the Intelligence of the researcher.
The Research output, in turn, determines the number of publications. Note that the direction
of arrows shows the cause-effect relationships. Difficulty affects the Research Output score, but
the Research Output does not influence the Difficulty. Finally, let us look at the tables asso-
ciated with each of the nodes. Formally, these are called conditional probability distributions
(CPDs), as shown in Figure 5. The CPDs for Difficulty and Intelligence are easy to compute,
because these variables are independent. The tables basically encode the probabilities of these
variables, taking values from 0 to 1. You might have noticed, the values in each of the rows must
sum to 1.

Figure 5: Bayesian network with conditional probability distributions (CPDs).

11
 Machine Learning and Probabilistic Graphical Models for Decision Support Systems

Next, let us look at the CPD for Research Output. Each row corresponds to the values that
its parent (Difficulty and Intelligence) can take, and each column corresponds to the values
that Research Output can take. Each cell has the conditional probability p(RsearchOutput =
RS|Intelligence = I, Dif f iculty = D), that is, given that the value of Intelligence and
Difficulty , what is the probability of the value of Research Output being RS. For example,
as P (ResearchOutput = RS 1 |Dif f iculty = D1 , Intelligence = I 1 ) is 0.5, that is, if the
intelligence of the researcher and the difficulty of the research area is high, then the probability
of the research output to be good is 0.5. The CPD for Research Articles is easy to understand
with the above knowledge. Because it has one parent, the conditional probabilities will be of the
form P (ResearchArticles = RA1 |ResearchOutput = RS 1 ), that is, what is the probability
of Research Articles being RA, given that the value of Research Output is RS. Each row now
corresponds to a pair of values of Research Output. Again, the row values add up to 1. An
essential requirement for BN is that the graph must be a directed acyclic graph (DAG).
Thanathornwong 20 designed a clinical decision support system to help general practitioners
assess the need for orthodontic treatment in patients with permanent dentition. Particularly, a
BN is used as the underlying model for assessing the need for orthodontic treatment. Around
one thousand permanent dentition patient’s datasets chosen from a hospital record system were
prepared in which one data element represented one participant with information for all variables
and their stated need for orthodontic treatment. The proposed system in this work provided
promising results; it showed a high classification accuracy for classifying groups into needing
and not needing orthodontic treatment.

2.3 Markov Random Fields

Although BN can compactly represent interesting probability distributions some distributions
may have independent assumptions that cannot be well represented by the structure of a BN. To
address such challenges there exists another technique for compactly representing and visualizing
a probability distribution that is based on the language of undirected graphs called Markov
random fields (MRF) 21;22 . Let us take a motivating example to understand the MRF. Suppose
that we are modeling voting preferences among persons W, X, Y, and Z. Let us say that (W,X),
(X,Y), (Y,Z), and (Z,W), are relatives. Moreover, suppose that relatives have similar voting
preferences. These relationships can be naturally depicted by undirected graphs, as shown in
Figure 6.
A MRF is a probability distribution p over variables defined by an undirected graph G in
which nodes represent variables xi i = 1, . . . , N . The probability p is given by.
1 �
p(x1 , ..., xN ) = βc (Xc )
Z c∈C

Where C denotes the set of cliques of G, and each factor βc is a non-negative function over
cliques. The partition function � �
Z= βc (Xc ),
x1 ,...,XN c∈C

12
 Decision Support Systems for Healthcare based on Probabilistic Graphical Models

Figure 6: Undirected graphical representation of a joint probability of voting preferences over four individ-
uals. Colors illustrates the pairwise preference present in the model.

is a normalizing constant which sums the distribution to one. Hence, given a graph G, there
might be factors in the probability distribution whose scope is any clique in G, it can be a single
node, an edge, etc. It is important to note that there is no need to specify a factor for each
clique. In the example above, a factor is defined over each edge (which is a clique of two nodes).
Nevertheless, cliques over single nodes have been specified.
In regards to the application of BN and MRF Rajinikanth et al. 23 present a Firefly Algo-
rithm and Shannon Entropy (FA+SE) based multi-threshold to increase the pneumonia lesion
and implements MRF segmentation to identify the lesions with better accuracy. 24 developed a
system based on BN which uses Bayesian reasoning to compute posterior probabilities of pos-
sible diagnoses depending on the given symptoms. This system was developed for diagnosis in
Internal Medicine and now covers about 1500 diagnoses in this domain, based on thousands of
findings. Barnett et al. 25 proposed a system called DXplain which uses a modified form of the
BN. It generates a list of ranked diagnoses associated with the given symptoms. It finds its
use particularly for healthcare practitioners who lack computer expertise. It is also used as a
reference with a searchable database of diseases and clinical manifestations. SimulConsult 26 ,
utilizes BN to input data in a scalable fashion and compute probabilities, accomplishing it by
focusing specialty by specialty. It uses a statistical pattern-matching method which consists of
the onset and offset of the findings in each disease. Table 1 presents a summary of a few past
developments and applications of BN and MRF in the healthcare sector.

Table 1: Applications of MRF and BN in healthcare.

Scheme Method Use Case

27 MRF Vertebral Tumor Prediction
28 MRF Tumor segmentation and gene-expression based classification
29 MRF Segmented MRI-based partial volume correction in PET
30 MRF Unsupervised 4D myocardium segmentation
31 MRF EMR-based medical knowledge representation and inference
32 BN Identifying Risk Factors of Depression in Middle-Aged Persons
33 BN Electrocardiogram (ECG) or Heart rate monitoring
34 BN Human activity recognition (HAR)

13
 Machine Learning and Probabilistic Graphical Models for Decision Support Systems

2.4 Deep Neural Networks

Deep Neural Networks (DNN) 35;36 are non-knowledge-based decision support systems which
are adaptive in nature. They learn from existing knowledge and experiences (data). A typical
workflow of neural networks in healthcare is shown in Figure 7.

Figure 7: The illustration of major phases for development of DNN based healthcare systems. Figure
adopted from 37 .

The architecture of DNN mainly consists of three layers: Input, Output, and Hidden layer(s).
These networks are made of nodes called neurons. Weights and biases are the connection between
nodes of different layers, which are used to propagate the input between the nodes. DNN are
able to work with incomplete data which gives educated guesses about missing data and gets
improved with adaptive system learning. A method for training an unsupervised fashion is
autoencoders 38 . An autoencoder learns features of a dataset, typically of lower dimensions.
Autoencoder is a type of DNN that learns to reconstruct its input in the output. It has an
internal representation layer that describes a code used to represent the input, and it is made
up of an encoder that translates the input into the latent space, and a decoder that maps
the latent space to reconstructed input. A lot of improvement has been made so far in the
architecture and algorithm of DNN to make them learn without any supervised pretraining.
Such as, the use of RELU activation f (z) = max(z, 0), which learns more efficiently in a
multi-layer model.
A typical DNN is depicted in Figure 8. In this DNN the convolutional layer along with
the max-pooling layers are used for feature extraction and the dense layer is used for classifica-
tion. The output dense layer often uses a sigmoid function in the case of binary classification
and a softmax function in the case of multiclass classification. DNN have been used in several
14
 Decision Support Systems for Healthcare based on Probabilistic Graphical Models

applications, such as image classification, computer vision, activity recognition, and deep rein-
forcement learning. For example, Kharat and Dudul 39 proposed a healthcare decision support
system based on Jordan/Elman neural network for the diagnosis of epilepsy. The proposed
system obtained comparatively a high overall training accuracy 99.83% and testing accuracy of
99.92%. A decision support system based on a DNN for the classification of heart-related dis-
eases into 5 categories of heart disease with 97.5% accuracy by using multilayer perceptron with
backpropagation training algorithm is proposed in 40 . Janghel et al. 41 proposed a decision sup-
port system using an artificial neural network to classify the fetal delivery method into normal
or surgical. They primarily used three different algorithms to train the neural network: radial
basis function, back propagation algorithm, and learning vector quantization network with an
accuracy 99%, 93.75%, and 87.5% respectively. Researchers have proposed a large number of
methods to apply DSS in healthcare. For example, 42 explains the role played by the DSS.
Luque Gallego 43 describes the medical decision-support system for the mediastinal staging of
non-small cell lung cancer, which is also known as called Mediastinet. Table 2 presents summary
of selected works in the applications of DNN for healthcare.

Figure 8: A typical deep Neural Network (DNN) architecture.

Table 2: Applications of DNN in healthcare.

Scheme Method Use Case

44 MLP Diagnosing diabetes
45 RNN Clinical intervention prediction and understanding
46 NN Emotion recognition for healthcare surveillance
47 CNN ECG biometric recognition
48 RNN ECG signal denoising
49 DNN ECG-based cardiac arrest pulse detection
50 CNN ECG classification
51 CNN ECG arrhythmias detection

15
 Machine Learning and Probabilistic Graphical Models for Decision Support Systems

2.5 Neural Networks with Probabilistic Graphical Models

There are certain limitations and challenges for neural networks NN.

1. Explanations: DNN are often complex and difficult to explain the reasons behind their
predictions and decisions. Explanations about the model’s decision making important for
some applications such as medical prognosis and diagnosis 52 .

2. Uncertainty measure: DNN cannot provide a quantification of the uncertainty of their

decisions and outputs.

3. Robustness: similar to the first point, it is hard to know about the aspect of input DNN
are using to take decisions.

PGM can help to solve these shortcomings, thereafter there is an opportunity to use these ap-
proaches to take advantage of their complementary strengths. For example, PGM provides a
practical way to represent dependence relationships between variables and spatial relations 53;54 .
DNN outperforms other approaches in classification. Thus to integrate PGM and DNN is by
representing the structure of a complex problem through PGM, followed by the use of DNNs
as classifiers for different elements of the underlying problem. The DNN, trained on labeled
data, provide an initial estimation; then these initial estimates can be combined and improved
through belief propagation in the graphical model. This approach can be used for efficient
training of the model, as each one only considers a particular dataset. Spatial analysis problems
are hybrid systems in which the above-mentioned systems can be useful. For instance, human
activity estimation, in which body parts have a certain spatial structure; this structure pro-
vides constraints that can be used by a graphical model. The spatial constraints between the
distinct elements in the model can be represented in terms of a Markov network, showing the
constraints as the local joint probabilities of neighboring elements. These elements are detected
and classified using a DNN. Another type of such problem is temporal modeling. In temporal
modeling, the outputs change and evolve over the time usually depending on the previous state,
for example, time-series. Markov chain and hidden Markov are often used to represent such
problems. In the hybrid system, DNN can be used to classify the state-based observation, and
the Markov model can be used for encoding the temporal relations. The application of such
systems is human activity recognition and speech recognition. A toy example of such hybrid
systems has been shown in Figure 9. Variational Autoencoders (VAE) are another such example.
Due to the increasing popularity of VAEs in anomaly detection, they have been used in various
fields, such as, Healthcare 55 , cybersecurity 56 and various other applications being discovered
with time. One of the important use of VAE in healthcare is anomaly detection. The idea is to
train the VAE using normal data and note down the corresponding reconstruction error. When
the VAE is subjected to anomaly data the reconstruction error is usually high. Hence, data
with reconstruction error more than that of the normal data is considered anomalous. In order
to give more in-depth details owing to the growing utility of VAEs in anomaly detection, we will
discuss a case study about VAE in healthcare anomaly detection in Section 6.2. Researchers
have proposed such hybrid architectures for human activity recognition. For example, in 57 an
architecture for the recognition of the human posture in video sequences was developed. The
proposed model consists of a convolutional neural network-based detector and a hidden markov
16
 Decision Support Systems for Healthcare based on Probabilistic Graphical Models

Figure 9: A hybrid DNN classifier and a hidden Markov model architecture.

model (CoHMM). The integration of both models allows learning spatial and temporal depen-
dencies. The detector recognizes the different joints based on a convolutional neural network
(CNN), and uses the spatial correlations between neighboring regions through a conditional
random field (CRF) 58 . Whereas, the CoHMMs computes the best possible movement sequence
among interacting processes.
More interesting research that combines deep neural networks and graphical models include:
conditional random fields as recurrent neural networks have been summarized in 59 .

3 Artificial Intelligence in Healthcare Applications

In the last decade, Artificial intelligence (AI) has revolutionized the healthcare system 60;61 . AI
is bringing a paradigm shift to healthcare, powered by the increasing availability of healthcare
data and rapid progress of analytic techniques. AI can be applied to various types of healthcare
data (structured and unstructured) to help practitioners diagnose the underlying health issue
in the early stages with more accuracy and efficiency. In this section, we review selected current
AI applications in healthcare and also discuss their future applications. The usefulness and
the advantages of AI have been extensively discussed in 62;63;64 . AI constructs data analysis
algorithms to extract features from data. AI algorithm’s inputs include patient ‘traits’ and
sometimes medical outcomes of interest. A patient’s traits commonly include baseline data,
such as age, gender, disease history, Xray-images, ECG. This may also include disease-specific
data, gene expressions, physical examination results, clinical symptoms, medication. Depending
on the outcomes and the input data, machine learning (ML) (a subclass of AI, which has been
extensively used in healthcare) can be classified into two major categories: unsupervised learning
and supervised learning. Unsupervised learning (UL) is a type of algorithm that learns patterns
from untagged data. In unsupervised learning, through mimicry, the machine is forced to build
a compact internal representation of the underlying traits of data. In supervised learning (SL)
the data is tagged or labeled by a human, e.g., as “car” or “fish”, etc. Unsupervised learning is
well known for feature extraction, while supervised learning is suitable for predictive modeling
17
 Machine Learning and Probabilistic Graphical Models for Decision Support Systems

by building some relationships between the input and the outcome of interest. Recently, a
hybrid of unsupervised learning and supervised learning, known as semisupervised learning
has been proposed which is suitable for scenarios where the outcome is missing for certain
subjects. Clustering and principal component analysis (PCA) are two famous and extensively
used unsupervised learning techniques. Clustering groups data points with similar features
together into clusters, without using the labeled outcome information. Clustering algorithms
predict the cluster labels as output for the given input data point by maximizing and minimizing
the similarity of the data within and between the clusters. Most popular clustering algorithms
include k-means, hierarchical, and Gaussian mixture clustering. On the other hand, PCA plays
a key role in the dimension reduction of complex data. Especially, when the recorded data is
multi-dimensional. For example, the number of genes in a genome-wide association study. PCA
works by projecting data into a few principal component (PC) directions, without losing too
much information about the underlying data. It is sometimes recommended to use PCA for
multi-dimensional data and then use clustering for better, and efficient clustering of data.
In regards to supervised learning, it considers the subject’s outcomes together with their
features, and goes through a certain training process to determine the best outputs associated
with the inputs that are closest to the mean outcomes. Usually, the output formulations vary
and are dependent on the underlying method and problem being solved. For example, the
outcome can be the probability of getting a particular clinical event, the expected value of a
disease level, or the expected survival duration.
The application of AI especially in healthcare is well studied in literature 65 . For example,
the internet of medical things (IoMT) integrates healthcare devices, sensors and machine learn-
ing algorithms to provide new applications in healthcare 66 . Machines based on AI can add
support in healthcare by providing continuous automatic monitoring and alerting the health-
care provider or clinical practitioners through an alert system. Moreover, these devices can also
help in decision-making through DSS. One of the major advantages of this transformation is the
transition of tasks from a manual, hectic and time-consuming methodology to smart, automatic,
and time-efficient systems in healthcare. Additionally, these systems help clinical practitioners
to attend to patients in emergency cases by providing timely information. DNN has always
outperformed in healthcare by providing hybrid architectures and blended concepts like CNN,
to enable new healthcare solutions. Due to the variety of healthcare data including clinical data,
HAR data, it is difficult for humans to infer the data for decision making. Accordingly, ML
has been used in healthcare for better understanding of data and for better decision-making
process 67 . For example, Azimi et al. 68 proposed a CNN-based classifier architecture for a health
case study on an ECG classification. Tahmassebi et al. 69 proposed ML-based algorithms for
early prediction of pathological complete response (PCR) to neoadjuvant chemotherapy and
survival outcome of breast cancer patients using multiparametric magnetic resonance imag-
ing (mpMRI) data and eight different ML-based classifiers. In this regard, decision-making is
incorporated at the edge thereby sending notifications to the user in the case of disease detec-
tion. This gives timely information for decision-making at the initial stage of the healthcare
monitoring and improving the healthcare system.
In conclusion, we believe that AI has an important role to play in healthcare in the future. In
the form of machine learning, it plays a primary role in the development of precision medicine,
and healthcare solutions. Although early efforts at providing prognosis, diagnosis, and treat-

18
 Decision Support Systems for Healthcare based on Probabilistic Graphical Models

ment recommendations are challenging, it can be seen that AI will ultimately cope with these
challenges as well. Given the fast research advances in AI for imaging analysis, it can be seen
that most radiology and pathology images will be examined at some point by machines using
AI. As for now, automatic speech and text recognition systems are already employed for tasks
like patient communication and for clinical notes, and usage of such systems is continuously
increasing.
One of the greatest challenges to AI application in healthcare is to ensure their adoption in
daily clinical practice. For adoption to take place, AI systems must be approved by regulators,
and standardized to the extent that similar products work in a similar way. Such challenges
will be overcome ultimately, but they will take time for the technologies themselves to be
practical enough. As a result, we see limited use of AI in clinical practice for the coming
decade, but with the rapidly improving research can make use of such systems in real life soon.
For more interesting works about the application of artificial intelligence in DSS, we recommend
to read 70;71 .

4 Healthcare Decision Support Systems based on

Probabilistic Graphical Models
DSS 72;73;74 have been widely used in the field of healthcare for assisting physicians and other
healthcare professionals with decision-making tasks, for example, for analyzing patient data
75;76;77;78
. DSS are mainly based on two mainstream approaches: knowledge-based and non-
knowledge based. The knowledge-based DSS consists of two principal components: the knowl-
edge database and the inference engine. The knowledge database contains the rules and as-
sociations of compiled data which often take the form of if–then rules, whereas the inference
engine combines the rules from the knowledge database with the real patients’ data in order
to generate new knowledge and to propose a set of suitable actions. Different methodologies
have been proposed for designing healthcare knowledge databases and inference engines, such
as the ontological representation of information 79 . The nonknowledge-based DSS have no di-
rect clinical knowledge about a particular healthcare process, however, they learn clinical rules
from past experiences and by finding patterns in clinical data. For example, various machine
learning algorithms such as decision trees represent methodologies for learning healthcare and
clinical knowledge. Both of these approaches could be used in conjunction with Ambient Intel-
ligence (AmI) technologies. Indeed, the sensitive, adaptive, and unobtrusive nature of AmI is
particularly suitable for designing decision support systems capable of supporting medical staff
in critical decisions. In particular, AmI technology enables the design of the third generation
of telecare systems. The first generation was the panic-alarms gadgets, often worn as pendants
or around the wrist to allow a person to summon help in the case of a fall or other kinds of
health emergencies. The second generation of telecare systems uses sensors to automatically
detect situations where assistance or medical decisions are needed. Finally, the third generation
represents AmI-based systems that move away from the simple reactive approach and adopt
a proactive strategy capable of anticipating emergency situations. As a result, DSS could be
used with multimodal sensing and wearable computing technologies for constantly monitoring
all important signs of a patient and for analyzing such data in order to take real-time decisions

19
 Machine Learning and Probabilistic Graphical Models for Decision Support Systems

and opportunely support people. Finally, DSS are jointly used with the AmI paradigm for
enhancing communications among health personnel such as doctors and nurses. For example, 80
introduced a DSS based on context-aware knowledge modeling aimed at facilitating the com-
munication and improving the capability to take decisions among clinical practitioners located
in different locations.

5 Perspectives for Healthcare Decision Support Systems

based on Probabilistic Graphical Models
Probabilistic models such as deep neural networks have become popular in medical applications,
especially as healthcare support for computerized-aided diagnosis and prognosis. Although, such
probabilistic models provide promising results and attract attention in healthcare research, real-
world implementation of such models would not be that easy. Firstly, there are no clear regu-
lations. Current regulations lack standards to measure the safety and efficacy of probabilistic
models. In order to overcome such issues, the United-States food and drug administration pro-
vided guidance for assessing probabilistic models systems 81 . It classifies probabilistic models to
be general wellness products, which are loosely regulated as long as the models are intended for
general wellness and have low risk to the users. They also provide guidance for adaptive design
in healthcare trials. Secondly, since healthcare data is highly sensitive, exchanging it among
geographically distributed parties governs privacy and security challenges. Moreover, the data
should be protected under general data protection regulations. Techniques like encryption, dif-
ferential privacy, and federated learning can be applied to provide security and privacy to the
data. However, such techniques come with a trade-off between privacy, security, and efficiency.
Another hurdle in the implementation of probabilistic systems in healthcare is data ownership
and incentives. Currently, there are no clearly defined regulations for the ownership of data.
Moreover, most of the current healthcare environment does not provide incentives to the data
owners for sharing data on the system. Nevertheless, research is underway to stimulate data
sharing. The research is oriented toward changing the health service payment systems. Many
payers, such as insurance companies, have shifted from rewarding the physicians by shifting the
treatment volume to the treatment outcome. Additionally, the payers are also reimbursed for a
medication or a treatment procedure by considering its efficiency. Under such an environment,
all the parties in the healthcare system, the clinical physicians, pharmaceutical companies, and
patients, have more incentives to compile and exchange information.
Other than the regulations, the key challenges and perspectives for the implementation of
probabilistic systems in healthcare include those intrinsic to the science of ML, logistical dif-
ficulties in implementation, and consideration of the barriers to adoption as well as of the
necessary socio-cultural or pathway changes. Robust peer-reviewed clinical evaluation as part
of randomized controlled trials should be developed as a standard for evidence generation, but
in practice, it may not always be appropriate or feasible. Performance evaluations should focus
to capture real clinical applicability and be interpretable and understandable to the intended
users. Research for regulation to access the innovation with the potential for harm, alongside
thoughtful post-market surveillance is needed to ensure that patients are not exposed to dan-
gerous health and finical risks. Methodologies should be developed to enable and make direct
20
 Decision Support Systems for Healthcare based on Probabilistic Graphical Models

comparisons of probabilistic models, including the use of independent, local, and baseline test
datasets. Research and development of probabilistic algorithms must be vigilant to potential
dangers, including dataset shift, accidental fitting of confounders, unintended bias, the issues
of generalization to new datasets, and the unintended bad consequences of new algorithms on
health outcomes.
In summary, the key future perspectives about the implementation of probabilistic models
in healthcare are as follow.
1. The data should be regulated properly by clear regulatory policies. The proper mecha-
nism should be developed to ensure the security and privacy of data under general data
protection regulations. For example, techniques like federated learning 82 can address such
issues, but issues in federated learning like data heterogeneity, privacy leakages needed to
be addressed 83;84 .

2. Proper metrics should be developed to measure the risk and unintended harm to users by
probabilistic models for healthcare and clinical practice.

3. Proper interpretable guidance and mechanisms should be developed to understand the re-
sults of probabilistic models for healthcare and clinical practice. For example, explainable
artificial intelligence can be used to interpret the result of deep neural networks 51 .

4. Proper incentive mechanisms should be developed to reward the data owners 85 .

6 Case Studies
Discriminative and generative models are widely used machine learning models for ECG clas-
sification in healthcare. For example, logistic regression, support vector machines, are popular
discriminative models and VAE and autoencoder are examples of generative models. In this
section, we provide a case study to explore the discriminative model’s graphical structure as
PGM, using logistic regression as an example for ECG classification. We also provide a case
study to explore the generative model’s graphical structure as PGM, using VAEs as an example
for ECG anomaly detection.

6.1 Logistic Regression for ECG Classification

Suppose that we are solving a classification problem to decide if an ECG signal is benign or not.
We have a joint model over labels Y = y , and features X = x1 , ...xn . The joint distribution of
the model is represented as p(Y, X) = P (y, x1 , ...xn ). Our aim is to estimate the probability of
benign ECG signal: P (Y = 1|X). To get the conditional probability P (Y |X), discriminative
models assume the functional form for P (Y |X) and estimate parameters of P (Y |X) directly
from training data.
In Figure 10, the circles represent variable(s) and the arrow indicates what probabilities can
be inferred. In our example, X is the ECG signal and Y is the unknown class of the ECG
signal. We see that the arrow is pointing from X to Y , indicating that we can infer P (Y |X)
directly from the given X .
21
 Machine Learning and Probabilistic Graphical Models for Decision Support Systems

Figure 10: Directed graphical model.

Figure 11: Graphical representation of input and output relationship.

Figure 11 represents the probability distribution of the model when feature X is expanded.
We can see that each feature xi depends on all the previous features. This will have no effect as
the model simply treats X as given facts and it estimates P (Y |X). As mentioned earlier that
the model estimates the probability from the training data:

P (Y |X) = p(y|x1 . . . xn ) (5)

In logistic regression, we parameterize the probability as

1
P (Y = 1|X; β) = n (6)
1 + exp(β0 + i=1 βi xi )
Here, maximum likelihood estimation is used to estimate the parameters, followed by classifi-
cation into benign and not benign ECG.

6.2 Variational Autoencoder for ECG Anomaly Detection

In this section, we will present VAEs as a case study. VAE are a type of generative deep learning
method which learns latent representations. Figure 12 shows a typical structure of a VAE. VAE
have also been used to draw images.
VAE consist of two main parts: encoder and decoder, where the encoder part models
E(Z|X), Z is the latent representation, and X is the data. E(Z|X) is the function that
maps the data to the latent variables. The decoder function D(X|Z) learns to generate
22
 Decision Support Systems for Healthcare based on Probabilistic Graphical Models

Figure 12: The structure of the variational autoencoder.

new data using the latent variables. It should be noted that in VAE, unlike the autoen-
coders, the distribution of Z is forced to be close to normal distribution as possible. With
VAE parametric distribution can be achieved. Hence, during the run time, we can construct
new samples from the normal distribution and feed them to the encoder function to generate
samples, as depicted in Figure 14. The main difference between traditional autoencoder and
VAE is that the former has no continuous latent space, while the latter has continuous latent
space (a sample is mapped to a probability distribution with a certain mean and variance).
Figure 13 depicts a comparison between the mapping of input data to latent space by an au-
toencoder and a VAE.

Figure 13: (a) mapping of an input to latent space by autoencoder, (b) mapping of an input to latent space
by VAE.

The main objective of this case study is to use the VAE to learn the latent representations
of the data. We will use the VAE to map the data to latent representation. Thereafter, we
will visualize features to see the model has generalized enough to learn the data clustering or
to differentiate the data as normal or not normal. Note that we do not use labels because VAE
are unsupervised machine learning approaches. To show the applicability, we will use the ECG
healthcare dataset for anomaly detection and visualize the features that the model has learned.
The reason for using VAE is to get rid of labeling the data, as labeling data can be a hectic
task. In this case study, we trained a convolutional VAE for ECG anomaly detection. We
trained the VAE on normal (a particular distribution) ECG signals so that, when not normal
(different distribution) ECG signals are fed into the VAE, the reconstruction loss is expected
23
 Machine Learning and Probabilistic Graphical Models for Decision Support Systems

to be higher than a certain threshold. The threshold is usually the reconstruction error of VAE
for normal data. If for certain data, the reconstruction error crosses the threshold, we see that
data point as not normal. In other words, via the reconstruction loss, we can keep track if an
ECG signal belongs to a particular distribution or not. The VAE has optimized over 2 losses,
the Kullback–Leibler (kl)-loss and reconstruction loss (the difference between the input ECG
and the reconstructed ECG). The kl-loss is the difference between the distribution of the latent
space and a standard Gaussian with mean zero and standard deviation one. In other words,
kl-loss is used the minimize the distance between the distribution between distinct classes yet
keep them separable. The kl loss between two distributions A and B can be calculated as the
negative sum of probability of each event in A, multiplied by the log probability of B over the
probability of the event in A, as given by Eq. 7.

B(x)
(7)
�
KL(A||B) = − A(x) log( )
x∈X
A(x)

Where, || is divergence. This is to compress the distribution of the latent space to the
standard distribution. This helps the decoder to map from every area of the latent space when
decoding the input ECG signal. Figure 14 shows a graphical representation of the VAE for new
sample generation.

Figure 14: VAE as a graphical model and its use to generate new samples.

We used the public baseline ECG dataset 86 to train and test our VAE for anomaly detection.
Figure 15 shows a scatter plot of the latent space generated by the encoder for the test dataset,
after training for 50 epochs, with stochastic gradient descent optimization. The color of each
point reflects its associated reconstruction error. In other words, it shows the marking of each
data point that has crossed the error threshold as an anomaly in dark violet color, and the
normal data point as yellow. We can clearly see one large cluster of points that seem quite on
the normal side (yellow dots), with a dark-colored cluster with a relatively high error term on
the sides. It should be noted that this VAE was just a toy example without any hyper-parameter
tuning. The performance can be enhanced by optimizing the hyper-parameters and adjusting
the layer structure of the VAE.
24
 Decision Support Systems for Healthcare based on Probabilistic Graphical Models

Figure 15: Anomaly detection using VAE.

VAE are widely used for a variety of machine learning tasks. This case study was an example
to show VAE’s application. The applicability of VAE can be enhanced by using XAI. While
reconstructing a sample in anomaly detection methods such as class activation maps can be
utilized to tap the neurons which fire for a given input and by applying max-polling over
the activation maps we can generate a spatial saliency map which shows the regions of input
signal that contribute more tho the output signal. For a reconstruction which is marked as an
anomaly, the regions with high vales of gradient mapping will be the contributor. Hence, the
region which cause the anomalous behaviour can be trace out. The example shown in Figure 16

Figure 16: Explanation of the regions responsible for a particular class.

25
 Machine Learning and Probabilistic Graphical Models for Decision Support Systems

was introduced in 51 to trace back the regions which are responsible for predicting a particular
class. The regions in red are responsible for classification of the ECG signal in to the output class
(color weighted scale shows contribution of each region). Similar approaches can be adopted in
anomaly detection to trace and explain the regions which cause the anomalous behaviour.

7 Conclusions
In this chapter, we discussed different DSS based on probabilistic graphical methods and machine
learning. DSS were proven to be a useful tool. They help in the reduction of prescription
errors, and help in prognosis, with a higher capacity than the previously used methods. DSS
has been shown to help healthcare practitioners and providers and in a variety of decisions
making and diagnosis tasks, and as of now, they actively and efficiently support in providing
quality healthcare service. Moreover, they were proved to be useful in the standardization of
protocols, adjustments with a target, and warning systems. We noticed that both DSS based on
classical PGM and DSSs based on advanced machine learning methods are extensively used in
healthcare. However, PGM has fallen out a little due to the ubiquity of probabilistic methods
like neural networks. Nevertheless, we believe they still have the potential to be relevant in the
future, because of their explanatory and intuitive nature. They can be used for modeling casual
relationships and can be useful in learning the representation of abstract or high-level concepts.
As we saw in the chapter that combining neural networks with graphical models could be very
useful in the domain of machine learning, especially in healthcare DSS.
Meanwhile, we must take extra measures and precautions and careful analysis when creating,
implementing, and maintaining DSS. In this regard complete solutions will be required in prac-
tice, especially as DSS continue to evolve in complexity through advances in AI, interoperability,
explanations, and new sources of data.

26
 Decision Support Systems for Healthcare based on Probabilistic Graphical Models

References
[1] D. Aronsky and P.J. Haug. Diagnosing community-acquired pneumonia with a Bayesian
network. In: Proceedings of the AMIA Symposium, American Medical Informatics
Association, p. 632, 1998.

[2] G.A. Gorry and M.S. Scott Morton. A framework for management information systems, 1971.

[3] J. Horsky, J. Aarts, L. Verheul, D.L. Seger, H. van der Sijs and D.W. Bates. Clinical
reasoning in the context of active decision support during medication prescribing.
International Journal of Medical Informatics, 97: 1–11, 2017.

[4] M.W.L. Moreira, J.J.P.C. Rodrigues, V. Korotaev, J. Al-Muhtadi and N. Kumar. A

comprehensive review on smart decision support systems for health care. IEEE Systems
Journal, 13: 3536–3545, 2019.

[5] D. Feinleib. Big Data Bootcamp: What Managers Need to know to Profit from the Big Data
Revolution. A Press, 2014.

[6] A. Sunyaev and D. Chornyi. Supporting chronic disease care quality: Design and
implementation of a health service and its integration with electronic health records. Journal
of Data and Information Quality (JDIQ), 3: 1–21, 2012.

[7] Howard S. Goldberg, Marilyn D. Paterno, Robert W. Grundmeier, Beatriz H. Rochaab, Jeffrey
M. Hoffman, Eric Tham, Marguerite Swietlik, Molly H. Schaeffer, Deepika Pabbathi, Sara J.
Deakyne, Nathan Kuppermann and Peter S. Dayan. Use of a remote clinical decision support
service for a multicenter trial to implement prediction rules for children with minor blunt
head trauma. International Journal of Medical Informatics, 87: 101–110, 2016.

[8] D. Koller and N. Friedman. Probabilistic Graphical Models: Principles and Techniques. MIT
Press, 2009.

[9] J. Wyatt and J. Liu. Basic concepts in medical informatics. Journal of Epidemiology &
Community Health, 56: 808–812, 2002.

[10] S.B. Clauser, E.H. Wagner, E.J.A. Bowles, L. Tuzzio and S.M. Greene. Improving modern
cancer care through information technology. American Journal of Preventive Medicine,
40: S198–S207, 2011.

[11] Jason Chiang, John Furler, Douglas Boyle, Malcolm Clark and Jo-Anne Manski-Nankervis.
Electronic clinical decision support tool for the evaluation of cardiovascular risk in general
practice: A pilot study. Australian Family Physician, 46: 764, 2017.

[12] P.A. Williams, R.D. Furberg, J.E. Bagwell and K.A. LaBresh. Usability testing and adaptation
of the pediatric cardiovascular risk reduction clinical decision support tool. JMIR Human
Factors, 3: e17, 2016.

27
 Machine Learning and Probabilistic Graphical Models for Decision Support Systems

[13] A.A. Montgomery, T. Fahey, T.J. Peters, C. MacIntosh and D.J. Sharp. Evaluation of computer
based clinical decision support system and risk chart for management of hypertension in
primary care: Randomised controlled trial. Bmj, 320: 686–690, 2000.

[14] R.T. Sutton, D. Pincock, D.C. Baumgart, D.C. Sadowski, R.N. Fedorak and K.I. Kroeker. An
overview of clinical decision support systems: Benefits, risks, and strategies for success.
NPJ Digital Medicine, 3: 1–10, 2020.

[15] E.S. Berner and T.J. La Lande. Overview of clinical decision support systems. In: Clinical
Decision Support Systems, Springer, pp. 3–22, 2007.

[16] S. Belciug and F. Gorunescu. Intelligent Decision Support Systems—A Journey to Smarter
Healthcare, Springer, 2020.

[17] C. Schaarup, L.B. Pape-Haugaard and O.K. Hejlesen. Models used in clinical decision support
systems supporting healthcare professionals treating chronic wounds: Systematic literature
review. JMIR Diabetes, 3: e8316, 2018.

[18] J. Pearl. Bayesian networks, 2011.

[19] D. Heckerman. A tutorial on learning with Bayesian networks. Innovations in Bayesian

Networks, 33–82, 2008.

[20] B. Thanathornwong. Bayesian-based decision support system for assessing the needs for
orthodontic treatment. Healthcare Informatics Research, 24: 22, 2018.

[21] G.R. Cross and A.K. Jain. Markov random field texture models. IEEE Transactions on Pattern
Analysis and Machine Intelligence, 25–39, 1983.

[22] C. Wang, N. Komodakis and N. Paragios. Markov random field modeling, inference &
learning in computer vision & image understanding: A survey. Computer Vision and Image
Understanding, 117: 1610–1627, 2013.

[23] V. Rajinikanth, S. Kadry, K.P. Thanaraj, K. Kamalanand and S. Seo. Firefly-algorithm

supported scheme to detect covid-19 lesion in lung ct scan images using shannon entropy
and markov-random-field. arXiv preprint arXiv:2004.09239, 2020.

[24] Depeursinge Adrien, Vargas Alejandro, Gaillard Frédéric, Platon Alexandra, Geissbuhler
Antoine, Poletti Pierre-Alexandre and Müller Henning. Case-based lung image categorization
and retrieval for interstitial lung diseases: Clinical workflows. International Journal of
Computer Assisted Radiology and Surgery, 7(1): 97–110, 2012.

[25] G.O. Barnett, J.J. Cimino, J.A. Hupp and E.P. Hoffer. DXplain: An evolving diagnostic
decision-support system. Jama, 258: 67–74, 1987.

[26] SimulConsult Inc. A simultaneous consult on your patient’s diagnosis, 2022.

28
 Decision Support Systems for Healthcare based on Probabilistic Graphical Models

[27] A. Alsiddiky, W. Awwad, K. Bakarman, H. Fouad and N.M. Mahmoud. Magnetic resonance
imaging evaluation of vertebral tumor prediction using hierarchical hidden markov random
field model on internet of medical things (iomt) platform. Measurement, 159: 107772, 2020.

[28] A.B. Ashraf, S.C. Gavenonis, D. Daye, C. Mies, M.A. Rosen and D. Kontos. A multichannel
markov random field framework for tumor segmentation with an application to classification
of gene expression-based breast cancer recurrence risk. IEEE Transactions on Medical
Imaging, 32: 637–648, 2012.

[29] A. Bousse, S. Pedemonte, B.A. Thomas, K. Erlandsson, S. Ourselin, S. Arridge and B.F.
Hutton. Markov random field and gaussian mixture for segmented mri-based partial volume
correction in pet. Physics in Medicine & Biology, 57: 6681, 2012.

[30] L. Cordero-Grande, G. Vegas-Sánchez-Ferrero, P. Casaseca-de-la Higuera, J.A. San-Román-

Calvar, A. Revilla-Orodea, M. Martín-Fernández and C. Alberola-López. Unsupervised 4d
myocardium segmentation with a markov random field based deformable model. Medical
Image Analysis, 15: 283–301, 2011.

[31] C. Zhao, J. Jiang, Y. Guan, X. Guo and B. He. Emr-based medical knowledge representation
and inference via markov random fields and distributed representation learning. Artificial
Intelligence in Medicine, 87: 49–59, 2018.

[32] F.J. Costello, C. Kim, C.M. Kang and K.C. Lee. Identifying high-risk factors of depression in
middle-aged persons with a novel sons and spouses bayesian network model 8: 562, 2020.

[33] P. Gupta, B. Bhowmick and A. Pal. Mombat: Heart rate monitoring from face video using
pulse modeling and bayesian tracking. Computers in Biology and Medicine, 121: 103813,
2020.

[34] Z. Zhou, H. Yu and H. Shi. Human activity recognition based on improved bayesian
convolution network to analyze health care data using wearable IOT device. IEEE Access,
8: 86411–86418, 2020.

[35] O.I. Abiodun, A. Jantan, A.E. Omolara, K.V. Dada, N.A. Mohamed and H. Arshad. State-of
the-art in artificial neural network applications: A survey, Heliyon, 4: e00938, 2018.

[36] W. Liu, Z. Wang, X. Liu, N. Zeng, Y. Liu and F.E. Alsaadi. A survey of deep neural network
architectures and their applications. Neurocomputing, 234: 11–26, 2017.

[37] A. Qayyum, J. Qadir, M. Bilal and A. Al-Fuqaha. Secure and robust machine learning for
healthcare: A survey. IEEE Reviews in Biomedical Engineering, 14: 156–180, 2021.

[38] I. Goodfellow, Y. Bengio, A. Courville and Y. Bengio. Deep Learning, volume 1. MIT Press
Cambridge, 2016.

29
 Machine Learning and Probabilistic Graphical Models for Decision Support Systems

[39] P. Kharat and S. Dudul. Clinical decision support system based on jordan/elman neural
networks. In: 2011 IEEE Recent Advances in Intelligent Computational Systems. IEEE,
pp. 255–259, 2011.

[40] M. Gudadhe, K. Wankhade and S. Dongre. Decision support system for heart disease based
on support vector machine and artificial neural network. In: 2010 International Conference
on Computer and Communication Technology (ICCCT). IEEE, pp. 741–745.

[41] R. Janghel, A. Shukla, R. Tiwari and P. Tiwari. Clinical decision support system for fetal
delivery using artificial neural network. In: 2009 International Conference on New Trends
in Information and Service Science. IEEE, pp. 1070–1075, 2009.

[42] K. Rajalakshmi, S.C. Mohan and S.D. Babu. Decision support system in healthcare industry.
International Journal of Computer Applications, 26: 42–44, 2011.

[43] M. Luque Gallego. Probabilistic graphical models for decision making in medicine, 2009.

[44] O. Karan, C. Bayraktar, H. Gümüşkaya and B. Karlık. Diagnosing diabetes using neural
networks on small mobile devices. Expert Systems with Applications, 39: 54–60, 2012.

[45] H. Suresh, N. Hunt, A. Johnson, L.A. Celi, P. Szolovits and M. Ghassemi. Clinical intervention
prediction and understanding with deep neural networks, 322–337, 2017.

[46] M. Dhuheir, A. Albaseer, E. Baccour, A. Erbad, M. Abdallah and M. Hamdi. Emotion

recognition for healthcare surveillance systems using neural networks: A survey. In:
2021 International Wireless Communications and Mobile Computing (IWCMC). IEEE,
pp. 681–687, 2021.

[47] R.D. Labati, E. Muñoz, V. Piuri, R. Sassi and F. Scotti. Deep-ecg: Convolutional neural
networks for ecg biometric recognition. Pattern Recognition Letters, 126: 78–85, 2019.

[48] K. Antczak. Deep recurrent neural networks for ecg signal denoising. arXiv preprint
arXiv:1807.11551, 2018.

[49] A. Elola, E. Aramendi, U. Irusta, A. Picón, E. Alonso, P. Owens and A. Idris. Deep neural
networks for ecg-based pulse detection during out-of-hospital cardiac arrest. Entropy,
21: 305, 2019.

[50] Y. Wu, F. Yang, Y. Liu, X. Zha and S. Yuan. A comparison of 1-d and 2-d deep convolutional
neural networks in ecg classification. arXiv preprint arXiv:1810.07088, 2018.

[51] A. Raza, K.P. Tran, L. Koehl and S. Li. Designing ecg monitoring healthcare system with
federated transfer learning and explainable AI. arXiv preprint arXiv:2105.12497, 2021.

[52] F.K. Došilović, M. Brčić and N. Hlupić. Explainable artificial intelligence: A survey. In: 2018
41st International Convention on Information and Communication Technology, Electronics
and Microelectronics (MIPRO), IEEE, pp. 0210–0215, 2018.

30
 Decision Support Systems for Healthcare based on Probabilistic Graphical Models

[53] M.J. Johnson, D. Duvenaud, A.B. Wiltschko, S.R. Datta and R.P. Adams. Structured vaes:
Composing probabilistic graphical models and variational autoencoders. arXiv preprint
arXiv:1603.06277 2 (2016), 2016.

[54] F. Le, M. Srivatsa, K.K. Reddy and K. Roy. Using graphical models as explanations in deep
neural networks, 283–289, 2019.

[55] T. Fernando, H. Gammulle, S. Denman, S. Sridharan and C. Fookes. Deep learning for
medical anomaly detection—A survey. arXiv preprint arXiv:2012.02364, 2020.

[56] Truong Thu Huong, Ta Phuong Bac, Dao Minh Long, Tran Duc Luong, Nguyen Minh Dan,
Le Anh Quang, Le Thanh Cong, Bui Doan Thang and Kim Phuc Tran. Detecting cyberattacks
using anomaly detection in industrial control systems: A federated learning approach.
Computers in Industry, 132: 103509, 2021.

[57] R.B. Zebadúa. Human body pose tracking based on spatio-temporal joints dependency
learning, 2018.

[58] X. Chu, W. Yang, W. Ouyang, C. Ma, A.L. Yuille and X. Wang. Multi-context attention for
human pose estimation. In: Proceedings of the IEEE Conference on Computer Vision and
Pattern Recognition, pp. 1831–1840, 2017.

[59] S. Zheng, S. Jayasumana, B. Romera-Paredes, V. Vineet, Z. Su, D. Du, C. Huang and P.H.
Torr. Conditional random fields as recurrent neural networks. In: Proceedings of the IEEE
International Conference on Computer Vision, pp. 1529–1537, 2017.

[60] A. Panesar. Machine Learning and AI for Healthcare. Springer, 2019.

[61] R. Shah and A. Chircu. Iot and AI in healthcare: A systematic literature review. Issues in
Information Systems, 19, 2018.

[62] S.E. Dilsizian and E.L. Siegel. Artificial intelligence in medicine and cardiac imaging:
Harnessing big data and advanced computing to provide personalized medical diagnosis and
treatment. Current Cardiology Reports, 16: 441, 2014.

[63] V.L. Patel, E.H. Shortliffe, M. Stefanelli, P. Szolovits, M.R. Berthold, R. Bellazzi and A.
Abu-Hanna. The coming of age of artificial intelligence in medicine. Artificial Intelligence
in Medicine, 46: 5–17, 2009.

[64] S. Jha and E.J. Topol. Adapting to artificial intelligence: Radiologists and pathologists as
information specialists. Jama, 316: 2353–2354, 2016.

[65] A. Qayyum, J. Qadir, M. Bilal and A. Al-Fuqaha. Secure and robust machine learning for
healthcare: A survey. arXiv preprint arXiv:2001.08103, 2020.

[66] S. Durga, R. Nag and E. Daniel Survey on machine learning and deep learning algorithms used
in internet of things (iot) healthcare. In: 2019 3rd International Conference on Computing
Methodologies and Communication (ICCMC), pp. 1018–1022, 2019.
31
 Machine Learning and Probabilistic Graphical Models for Decision Support Systems

[67] K.R. Bisaso, G.T. Anguzu, S.A. Karungi, A. Kiragga and B. Castelnuovo. A survey of machine
learning applications in hiv clinical research and care. Computers in Biology and Medicine,
91: 366–371, 2017.

[68] I. Azimi, J. Takalo-Mattila, A. Anzanpour, A.M. Rahmani, J.-P. Soininen and P. Liljeberg.
Empowering healthcare iot systems with hierarchical edge-based deep learning. In:
Proceedings of the 2018 IEEE/ACM International Conference on Connected Health:
Applications, Systems and Engineering Technologies, pp. 63–68, 2018.

[69] Amirhessam Tahmassebi, Georg J. Wengert, Thomas H. Helbich, Zsuzsanna Bago-Horvath,

Sousan Alaei, Rupert Bartsch, Peter Dubsky, Pascal Baltzer, Paola Clauser, Panagiotis
Kapetas, Elizabeth A. Morris, Anke Meyer-Baese and Katja Pinker. Impact of machine
learning with multiparametric magnetic resonance imaging of the breast for early prediction
of response to neoadjuvant chemotherapy and survival outcomes in breast cancer patients.
Investigative Radiology, 54: 110, 2019.

[70] S. Montani and M. Striani. Artificial intelligence in clinical decision support: A focused
literature survey. Yearbook of Medical Informatics, 28: 120–127, 2019.

[71] M. Fernandes, S.M. Vieira, F. Leite, C. Palos, S. Finkelstein and J.M. Sousa. Clinical decision
support systems for triage in the emergency department using intelligent systems: A review.
Artificial Intelligence in Medicine, 102: 101762, 2020.

[72] E. Turban. Implementing decision support systems: A survey. In: 1996 IEEE International
Conference on Systems, Man and Cybernetics. Information Intelligence and Systems (Cat.
No. 96CH35929), volume 4, IEEE, pp. 2540–2545, 1996.

[73] S.B. Eom, S.M. Lee, E. Kim and C. Somarajan. A survey of decision support system
applications (1988–1994). Journal of the Operational Research Society, 49: 109–120, 1998.

[74] S. Eom and E. Kim. A survey of decision support system applications (1995–2001). Journal
of the Operational Research Society, 57: 1264–1278, 2006.

[75] M. Omichi, Y. Maki, T. Ohta, Y. Sekita and S. Fujisaku. A decision support system for
regional health care planning in a metropolitan area. Japan-hospitals: The Journal of the
Japan Hospital Association, 3: 19–23, 1984.

[76] G. Acampora, D.J. Cook, P. Rashidi and A.V. Vasilakos. A survey on ambient intelligence in
healthcare. Proceedings of the IEEE, 101: 2470–2494, 2013.

[77] R. Snyder-Halpern. Assessing health care setting readiness for point of care computerized
clinical decision support system innovations. Outcomes Management for Nursing Practice,
3: 118–127, 1999.

[78] Mohammad Raza Perwez, Naveed Ahmad, Muhammad Sajid Javaid and Muhammad Ehsan
ul Haq. A critical analysis on efficacy of clinical decision support systems in health care

32
 Decision Support Systems for Healthcare based on Probabilistic Graphical Models

domain. In: Advanced Materials Research, volume 383, Trans. Tech. Publ., pp. 4043–4050,
2012.

[79] M.C. Kaptein, P. Markopoulos, B. De Ruyter and E. Aarts. Persuasion in ambient intelligence.
Journal of Ambient Intelligence and Humanized Computing, 1: 43–56, 2010.

[80] O. Anya, H. Tawfik, S. Amin, A. Nagar and K. Shaalan. Context-aware knowledge modelling
for decision support in e-health. In: The 2010 International Joint Conference on Neural
Networks (IJCNN), pp. 1–7, 2010.

[81] J. Graham. Artificial intelligence, machine learning, and the fda, 2016.

[82] M. Aledhari, R. Razzak, R.M. Parizi and F. Saeed. Federated learning: A survey on enabling
technologies, protocols, and applications. IEEE Access, 8: 140699–140725, 2020.

[83] J.H. Yoo, H. Jeong, J. Lee and T.-M. Chung. Federated learning: Issues in medical application,
2021.

[84] N. Papernot, P. McDaniel, A. Sinha and M. Wellman. Towards the science of security and
privacy in machine learning. arXiv preprint arXiv:1611.03814, 2016.

[85] Y. Zhan, J. Zhang, Z. Hong, L. Wu, P. Li and S. Guo. A survey of incentive mechanism
design for federated learning. IEEE Transactions on Emerging Topics in Computing, 2021.

[86] G.B. Moody and R.G. Mark. The impact of the mit-bih arrhythmia database. IEEE Engineering
in Medicine and Biology Magazine, 20: 45–50, 2001.

33
 Chapter 3

Decision Support Systems for Anomaly

Detection with the Applications in Smart
Manufacturing: A Survey and Perspective
Quoc-Thông Nguyen,1,2,# Tung Nhi Tran,1,# Cédric Heuchenne3,* and
Kim Phuc Tran2

1 Introduction
With the growth of the volume of data collected in manufacturing, Big Data offers a tremendous
opportunity in the transformation of today’s manufacturing paradigm to smart manufacturing
(SM) and helps the scientists and engineers have Artificial Intelligence (AI)-driven Industrial
Internet of Things (IIoT) solutions working in real-time and being more accurate and efficient 1 .
The development of AI algorithms and Big Data analytic helps to transfer a human experience
to technological developments, providing ways for IIoT solutions to maximize value creation
across asset and operations life-cycles while improving profitability. Machine Learning (ML)
and Deep Learning (DL) are AI subsets that are used to assess produced data and provide useful
information about the manufacturing industry 2 . Therefore, ML and DL play important roles in
the development of SM with various applications especially anomaly detection in 3 , machine fault
diagnosis 4 , intrusion detection 5 , production monitoring 6 . Furthermore, to react and adapt to
the constantly changing industrial environment, manufacturers nowadays need a support system
that allows them analyze information to come up with reasonable decisions. Decision support
systems (DSS) in this scenario integrate human talents with computer capabilities to offer
effective data administration, reporting, analytics, modeling, and planning. By integrating these
advanced technologies in manufacturing, the factories enable it to optimize the performance,
quality, control, and transparency of the manufacturing process. While there is a large amount of
research related to these technologies in the context of SM, there are still challenges in applying
them in a practical setting 7;8;9 .

1 International Research Institute for Artificial Intelligence and Data Science, Dong A University, Danang, Vietnam.
2
Univ. Lille, ENSAIT, ULR 2461 - GEMTEX - Génie et Matériaux Textiles, F-59000 Lille, France.
3 HEC Management School, University of Liège, Liège, Belgium.

* Corresponding author: [email protected]

# These authors contributed equally
 Decision Support Systems for Anomaly Detection with the Applications in Smart Manufacturing

The first problem is that, to have a flexible and scalable IIoT platform, the central computing
of all data collected in manufacturing is the way for growth and innovation. Thus, cloud
computing is important to empower the workloads and applications, reduce costs, and increase
release velocity, and agility. With the recent advancements in cloud computing, enterprises can
have large storage and enough computing capacity to process collected data from IIoT devices.
Traditionally, IIoT is only used for data collection and offloads computationally intensive tasks
to cloud servers. Yet, offloading computationally intensive tasks to a cloud centre may result
in a delay, due to the time needed to transmit, process, and receive a large amount of data.
This is a serious drawback in a smart factory that must perform massive analysis in real-time.
To overcome this limitation, the concept of edge computing can be used in a smart factory 10 .
Edge computing is the technology that makes it possible to quickly perform the necessary
computational task in the network edge, i.e., between data producers and the cloud centre.
The workload that is concentrated in the central cloud can be reduced. Moreover, in some
situations, traditional ML approaches require combining data at one location which not only
prevents systems from dealing with vastly distributed data and training models on the edge
nodes but also presses communication systems in factories. In that case, Federated Learning
(FL) approach can represent a good solution. In 11 , the authors mentioned that operating an
FL system can decrease the amount of bandwidth used in transferring data between the edge
and the cloud by 35%. Integrating FL with other learning frameworks is a potential method
that can solve existing problems in SM. Therefore, this chapter is the first to show a survey on
ML and DL techniques used for anomaly detection in some aspects of SM. Then, difficulties
and challenges in applying them in a real context are discussed. Finally, our perspectives for
solving these problems are recommended.
The chapter is organised as follows: In Section 2, we briefly describe the concept of DSS.
The IIoT-based background and techniques for anomaly detection in SM are presented in
Section 3. Some difficulties and challenges also mentioned in Section 4. In Section 5, we
propose an alternative approach for the decentralised system. The case studies are given in
Section 6. Section 7 provides some concluding remarks.

2 Decision Support Systems for Smart Manufacturing

In SM, the future factory will be more conscious and intelligent to independently perform
complex tasks, i.e., “smart factory” 12 . It leads to increased demand in making precise decisions
as soon as possible. In addition, with the wide use of electronic sensing devices, wireless sensor
networks and other advanced technologies in the IIoT, there would be a tremendous amount
of information that needs to be processed to come up with rational decisions. Thus, decision
makers will encounter complex situations in the decision-making. This process becomes more
challenging due to insufficient and complex data, it is difficult to do it in the traditional way.
The formulation process is due to failed and weak decisions. Thanks to the development of
technology such as cloud computing, the Internet of Things (IoT), and Big Data which are key
elements in digital transformation, decision support and decision support system (DSS) turn into
accessible and reliable. However, it seems difficult to have an exact definition of it. In 13 and 14 ,
DSS is mentioned as human and management systems that depend on management science

35
 Machine Learning and Probabilistic Graphical Models for Decision Support Systems

and adapt technologies tools through simulation and information technology. In some recent
researches, a DSS includes all of the people who are in the organisation and the hardware,
software, models and data 15 . According to 16 , DSS is described as an interactive computer-
based system or subsystem designed to help decision-makers use communication technology,
data, documents, knowledge, and/or models to identify and solve problems, complete tasks in
the decision-making process, and make decisions. DSS are divided into five categories depending
on their major sources of information 17 :
1. Communication-Driven DSS is a type that uses communication technologies and net-
works to facilitate the communication and collaboration of decision-relevant. It empha-
sizes communication, collaboration, and shared decision-making support using technol-
ogy. The most fundamental level of functionality is a simple bulletin or threaded e-mail.
Communications-driven DSS allows two or more individuals to interact, share and coordi-
nate their activities.
2. Data-Driven DSS emphasize access to and manipulation of a time series of internal
company data and in some systems real-time and external data. Business intelligence
systems for operational or strategic use are most often data-driven.
3. Document-Driven DSS integrate a variety of storage and processing technologies to
provide complete document retrieval and analysis. A search engine is a powerful decision-
aiding tool associated with a document-driven DSS.
4. Model-Driven DSS emphasize access to and manipulation of a quantitative model (e.g.,
an algebraic, financial, optimization, or simulation model). Model-driven DSS use data
and parameters provided by decision makers to aid them in analyzing a situation, but they
are not usually data-intensive.
5. Knowledge-Driven DSS suggest or recommend actions to managers. These DSS are
person–computer systems with specialized problem-solving expertise. A knowledge-driven
DSS uses AI and statistical technologies. Knowledge storage and processing technologies
are the dominant component in the architecture of these systems.
As mentioned above, DSS plays an important role in SM. Promoting effective strategies to
support decision-making can not only be through a comprehensive understanding of the dynam-
ics involved, but also the possibility of enriching these processes with the valuable information
collected at different levels. The last point represents a key aspect of effective decision-making,
especially when we consider decentralization. In addition, it is necessary to analyze the amount
of information collected to avoid the overloading of decision variables and the increasing com-
plexity of the decision-making process. Another problem is how to integrate different tasks (that
is, decisions at different levels) that need to be made so that the existing system works more
efficiently. There are 2 common methods to overcome this challenge: centralized or decentral-
ized.
1. In the centralized decision-making process, a single, decision center is acquainted with all
the system information. The central node is responsible for system planning and has the
ability to manage the operations performed by all nodes on the network. The central node
makes decisions with the aim to optimize the objectives of the entire network 18 .
36
 Decision Support Systems for Anomaly Detection with the Applications in Smart Manufacturing

2. In the decentralized decision-making models each individual independent network entity

makes its own decisions in order to optimize its own objectives. In this process, there is
more than one decision-maker so the collaboration of these nodes is an essential compo-
nent. They need to be connected and to exchange their information as well as decisions.
Moreover, depending on the degree of collaboration, the nodes’ decisions are impacted by
the other ones in the network 19 .
In this chapter, we will focus on the Anomaly Detection approaches used as DSS in SM that
we will discuss in the next section.

3 Anomaly Detection in Smart Manufacturing

With the emerged industry in the last decade under the name SM, the maintenance, protection,
and cybersecurity systems have been evolved to face the novel challenges and conditions. In
the chapter, we will focus on 4 most important axes of anomaly detection in SM: Smart Pre-
dictive Maintenance, Integrated Wearable Technology, Production Monitoring, and Real-time
Cybersecurity.

3.1 Smart Predictive Maintenance

Equipment maintenance is a vital factor in SM and directly affects the service life of the equip-
ment and its production efficiency. By using IIoT and wireless sensors devices to monitor
equipment status, advanced and pervasive Predictive Maintenance (PdM) applications can be
developed, thus reducing maintenance costs, and avoiding dangerous situations. PdM is gener-
ally indicated based on the assessment of the health status of key components, regardless of the
maintenance status, and its fundamental purpose is to make predictability achievable. That is,
the incipient problems that may evolve into catastrophic failures can be predicted accurately,
and effective measures can be applied to avoid these failures on the basis of the prediction
results. This approach can minimise maintenance costs and also extend the useful life of the
equipment. With the rise of AI, data-driven methods for fault diagnosis and remaining useful life
prediction have been popular issues in PdM system research. The process of data-driven PdM
can be divided into four stages: (1) operational assessment; (2) data acquisition; (3) feature
engineering, and (4) modelling.
Currently, with the massively available data used to solve numerous problems, ML has been
widely used in computer science and other fields, e.g., the PdM of industrial equipment, such as
Linear Regression (LR), Support Vector Machine (SVM), Decision Tree (DT), Random Forest
(RF). Although high-performance algorithms are continually developed, generally, employing
efficiently and straightforward methods were only first considered by 20;21 . An LR model can be
provided to industry experts with interpretability and that its predictive performance outper-
formed in terms of predicting whether mechanical equipment or components run correctly 22 .
The SVM model is used to tackle the tasks for binary classification. Specifically, the SVM model
and its extension can be utilised to solve multi-class tasks since the diversity of fault types and
the ability of mapping low dimension features to hyperplanes 23 . A Digital Twin model can de-
compose a complex decision-making process into a collection of more straightforward decisions
37
 Another Random Document on
Scribd Without Any Related Topics
 CHAPTER X

DOWN THE WHITE NILE

We lingered lovingly around Hippo camp for two more days,
moving to other lagoons and overflows of the river with the launch,
and striking out inland in search of the great herd of elephants. But
although their recent presence was on all sides proclaimed by
snapped-off trees and trampled ground, and broad lanes cut through
the grass, we saw none of them; and a tribe of natives who helped
to carry home a variety of buck one afternoon, informed us upon
expert authority that the whole herd had been alarmed by the arrival
of strangers and the sound of firing, and had retired three days'
journey from the river's bank. These natives—of the Lado Enclave—
were gentleman-like folk, and I parleyed long with them upon their
affairs. They were stark naked and very dignified, with graceful
athletic bodies, long tapering well-bred hands, and bright keen eyes.
The local chief exhibited all these characteristics in a superior
degree, and his natural preeminence was recognized with
instantaneous obedience by his followers. We loaded them with
gifts. First, quantities of meat and hides; then chocolate all round—
they love sweet things—three pieces of sugar for each, at least one
empty bottle per man, and tin pots and card-board boxes almost
without limit. The chief showed a fine taste in all these things, and
annexed at once in the Imperial style whatever took his fancy, to
whomsoever it belonged. I cast about for some means of doing him
especial honour, and luckily remembered that I had bought a
Japanese kimono for a dressing-gown in passing through Port Said
on the journey out. With this he was forthwith enrobed, and I must
say he assumed the flowing garment with that easy grace and
natural self-possession which are the gifts of a wilderness life. Thus
the fabrics of Cathay were by the enterprise of Europe introduced
into the heart of Africa.
When, finally, with much reluctance we left this attractive place
and pushed off determinedly into the stream, we lost no time in
making Nimule. Steaming throughout the night and all next day
along a broad flood contained by high and healthy slopes—now
clothed with forest, now with waving grass—we approached, at
about four in the afternoon, the mountains beneath which is the
administrative station of Nimule. Hitherto the course of the Nile since
it left the Albert Lake had been smooth and open—a broad, steady-
flowing river everywhere navigable to vessels of not more than four
feet draught. But at Nimule, after a reach of more than a hundred
and seventy miles of unobstructed waterway, the river turns a sharp
right angle and enters a long succession of granite gorges, through
which it plunges in ceaseless cataract for a hundred and twenty
miles. It is here at the head of these rapids that one of the great
reservoirs of the Upper Nile must some day be constructed. "I spent
hours," said Sir William Willcocks, the "practical mystic" of hydraulic
engineering, "looking at the site, and seeing in a vision a great
regulating work of the future." And indeed the exact scientific
control of the whole vast system of Central African waters, of the
levels of every lake, of the flow of every channel, from month to
month and from day to day throughout the year, is a need so
obvious and undisputed as to leave argument unemployed.
The change in the character of the river separated us finally from
our flotilla. From Nimule to Gondokoro we must again proceed by
land, and the swift and easy progress of the last few days must be
exchanged for the steady grind of marches. It was this stage which
had always been painted to me as the most dangerous and
unhealthy in our whole journey, and I had pictured to myself eight
days of toil through swamp and forest amid miasma and
mosquitoes. These anticipations were not sustained. Of the
disadvantages of the track along the river bank I cannot speak; but
the upper road over the hills is certainly excellent and healthy, and
runs throughout over firm dry undulations of a bright, breezy, scrub-
covered country.
At Nimule we touched the telegraph wire again, and from the
Reuter's accumulations which I studied, I learned that Parliament
would not meet till the 19th of January. This gave another ten days'
more rope, and I began to realize how much the spirit of these
wonderful lands had taken possession of me, for it was only with the
greatest reluctance and difficulty, that I forced myself to continue my
homeward journey without first turning back with the launch and
circumnavigating Lake Albert. No exertion or inconvenience seemed
too great to win a few more glimpses of these enchanted seas and
gardens, on which I may perhaps not look again, but from whose
spell I can never be free. Porters to be fed from day to day, the
Sirdar's steamer waiting at the Soudan frontier, public meetings
looming heavily in the far-off distance, drove me onward; and with
feelings of keen and genuine regret we addressed ourselves to the
march to Gondokoro.

Fording the Asua.

 This was accomplished uneventfully in six stages, three of which
were double marches. The country was pleasant and healthy, the
scenery imposing, and, under a fierce sun, the air was cool. Each
morning we started before dawn, and by noon had camped by the
side of one of the tributary rivers or streams which flow into the
Nile. Of these the Asua was the most important, and the picture of
the long safari fording it and coming into camp among the palm-
trees of the southern bank is one which lingers pleasantly in my
memory. But this I must say—somehow after Nimule the charm was
broken, and none of the regions through which the traveller passes
in the long-drawn descent of the Nile revive in any degree those
delicious sensations of wonder and novelty which are associated
with the great lakes and the kingdoms of Uganda, Usoga, and
Unyoro, to say nothing of what I have not been fortunate enough to
see—Toro, Ankole, the Semliki, and the Mountains of the Moon.
At the end of the sixth day we arrived at Gondokoro. The last
march had been long and scorching. The moisture seemed to have
gone from the air, and the vegetation, abundant though it was,
seemed parched and stunted. The approaches to Gondokoro are
beset by a herd of three hundred elephants of peculiar ill-fame.
Nearly all the eligible tuskers have been killed. The females and
young bulls are fierce and wary, and, taught by frequent contact
with the white man, and protected by the sacred game laws,
exercise a lawless and tyrannical power over the whole region. On
every side their depredations are to be seen. Great trees pushed
over in careless sport, native plantations trampled into ruin, the
roads rendered precarious for the traveller, the mails often
interrupted for days at a time, and occasional loss of life, are the
features of this domination. And it seems likely to last a long time,
for I was informed that the young bulls would not be sufficiently
grown for about forty years, and even then, as the two white officers
in the station are not allowed to shoot more than one elephant
apiece each year, the nuisance will only gradually be abated.
Rogue elephants are of course fair game at any time, and the day
before we arrived at Gondokoro, the young civil officer of the station
had encountered one in a manner which he was scarcely likely to
forget. For, having pursued this evil-doer for some time, he at last
got into an excellent position, and was about to fire at a distance of
thirty yards when suddenly the elephant, without even trumpeting
rushed furiously upon him, and, paying no attention to the two
heavy bullets which struck him in the head, chased the officer twice
round an uncommonly small bush; and then, distracted by the
spectacle of the native gun-bearer in flight, turned off after this new
prey, and, overtaking the poor wretch, smashed him to pieces with
one blow of his terrible trunk. "Cet animal est très méchant; quand
on l'attaque, il se défend." We reached the bungalow, which serves
as the seat of government, in time to see the tusks of this man-
slayer, who had died of his wounds, brought in by the tribe whose
plantations he had so often ravaged.
Gondokoro, like most of the names which figure so imposingly
upon the African map, is not a numerously populated town. There
are about six houses and a number of native huts. There is,
however, a telegraph station, a prison, a court-house, and the lines
of a company of native police and King's African Rifles. Here the Nile
again becomes navigable, and offers an unbroken waterway open to
large vessels until the Shabluka cataract is reached, a hundred miles
below Khartoum and fifteen hundred miles from Gondokoro. And
here at the river's bank, seen through a tracery of palms, were the
white funnel and superstructure of the Sirdar's steamer with all the
letters and newspapers; and which, instead of pursuing us across
Uganda, had "come through the other way."
"Had come through the other way"—it is an easy phrase to write:
but how much it signifies in the modern history of Africa! Ten or
eleven years ago this journey which I was now able to make so
easily, so prosperously, so comfortably, would have been utterly
impossible. The Dervish empire, stretching from Wady Halfa or Abu
Hamed to Wadelai, interposed a harsh barrier which nothing but a
stricken field could sweep away; and these long reaches of the Nile
which now bore a fleet of fifty steamers were silent in the embrace
of a devastating barbarism. A grim slaughter which had strewn the
sands of Kerreri, twelve hundred miles to the North, with jibba-clad
corpses "like snow-drifts" had blasted a passage, and the Nile was
free.
Embarked at Gondokoro we passed out of the sphere of the
Colonial Office into the domain of that undefined joint authority
which regulates the Soudan, which flies two flags side by side on
every public building, and which you can only correspond with
through the British Foreign Office.

The Belgian Officials at Lado.

 Gondokoro.

Henceforward our journey was comfortable, and regular. Yet

though I had no official work to do and was merely coming home
the shortest way, I could not traverse the Soudan without the
keenest interest. When one has started from Cairo and padded up
the Nile to Wady Halfa, crossed the desert railway to the Atbara,
marched thence two hundred miles to the battle of Omdurman, one
feels one has seen something of the Nile. Yet now we had followed it
the other way from its source for nearly five hundred miles, and yet
twelve hundred more intervened before even Omdurman was
reached; and as the mighty and peerless river unrolled its length and
immemorial history, the feelings of reverence, without which no
traveller can drink its sweet waters, grew in intensity.
I yield to no one in recognition of the constructive and
reconstructive work which Sir Reginald Wingate and his able officers
have, with scanty means and in spite of grave military dangers,
wrought in the Soudan. Yet it is not possible to descend the Nile
continuously from its source at Ripon Falls without realizing that the
best lies behind one. Uganda is the pearl. The Nile province and the
Lado Enclave present splendid and alluring panoramas. Even the
march from Nimule to Gondokoro is through a fertile and inspiring
region. But thereafter the beauty dies out of the landscape and the
richness from the land. We leave the regions of abundant rainfall, of
Equatorial luxuriance, of docile peoples, of gorgeous birds and
butterflies and flowers. We enter stern realms of sinister and
forbidding aspect, where nature is cruel and sterile, where man is
fanatical and often rifle-armed. Cultivation—nay, vegetation, is but a
strip along the river bank: and even there thorn-bushes and prickly
aloes are its chief constituents. We enter two successive deserts as
contrasted in their character, as redoubtable in their inhospitality, as
Dante's Circles of the Inferno: the Desert of Sudd and the Desert of
Sand.

Review at Khartoum.
 Soudan Government Steamer "Dal."

About a hundred miles from Gondokoro the White Nile enters and
spills itself in a vast and appalling swamp. Of the action of this
tremendous sponge, whether beneficial in regulating the flow, or
harmful in wasting the water through evaporation, nothing need
here be said. But its aspect is at once so dismal and so terrifying
that to travel through it is a weird experience. Our steamer, with the
favouring current, made at least seven miles an hour, and, as the
moon was full, we travelled night and day. For three days and three
nights we were continuously in this horrible swamp into which the
whole of the United Kingdom could be easily packed. By day from
the roof of the high pilot-house a commanding view revealed hour
after hour, in every direction, one uninterrupted ocean of floating
vegetation spreading to far horizons. The papyrus-plant is in itself a
beautiful, graceful, and venerable thing. To travel through the sudd,
is to hate it for evermore. Rising fifteen feet above the level of the
water, stretching its roots twenty or even thirty feet below, and so
matted and tangled together that elephants can walk safely upon its
springy surface, papyrus is the beginning and end of this melancholy
world. For hundreds of miles nothing else is to be perceived—not a
mountain-ridge blue on the horizon, scarcely a tree, no habitation of
man, no sign of beast. The silence is broken only by the croaking of
innumerable frog armies, and the cry of dreary birds.
The vigorous operations of the sudd-cutters have opened, and the
constant traffic of steamers has preserved and improved, a channel
about a hundred yards wide, winding by loops and corkscrews
through the swamp. The river presents a depth of thirty feet along
this course, and greater vessels could thread its length for nearly a
thousand miles. The navigation is intricate and peculiar. Indeed, it
would seem to be an art by itself. No effort is made by the Arab
pilots, who alone are employed, to avoid collisions with the banks.
On the contrary, they rely upon them as an essential feature of their
management of the steamer. The vessel bumps regularly at almost
every corner from one cushion of sudd to the other, or plunges its
nose into the reeds and waits for the currents to carry its stern
round, bumps again and recovers its direction. Sometimes where the
twists were very sharp we would turn completely round, not once
but two or three times, and our movements round an S-curve were
even more complicated. The bumps occasionally swept us out of our
chairs and sent us sprawling on the deck. In this strange fashion we
waltzed along at full speed for about seventy or eighty hours.
Meanwhile the Nile was accomplishing its destiny. Its vast tributary
rivers, the Sobat and the Bahr-el-Ghazal, came to reinforce its flow.
The miles spread out behind us in a long succession of hundreds. At
length the sudd expanses begin to contract. Distant mountains rise
against the steel-blue sky in serrated silhouette, and gradually draw
in upon the river. Islands of earth and trees, peaks of sharp rock
break here and there the awful monotony of waving reeds. At last
the banks become firm and clear-cut walls of yellow sand, fringed in
places with palms and shady trees, and everywhere bristling with
undergrowth of thorns. We leave the wilderness of moisture, we
approach the wilderness of drought. But first, in a middle region,
vast areas of dusty scrub-covered plains, not wholly incapable of
cultivation in the rainy season, supporting always flocks and herds,
now flank both sides of the river. The camel caravans pad slowly
across them under the blaze and glitter of the heat. The mirage
begins to twist and blur the landscape with deceptive waters. At
intervals of forty or fifty miles are the stations of the Soudan
Government, each trim and regular with its public buildings, its
storehouses, the lines of beehive huts of its garrison, a tangle of
native sailing-craft, and always, or nearly always, one or two white
gunboats of war-time days now turned policemen of the river.
Thus we reach in time Fashoda—now called Kodok for old sake's
sake; and here are clusters of Shillooks who (by request) stand
pensively on one leg in their natural attitude, and smart companies
of Soudanese troops and British officers, civil and military—the
whole clear-cut under sun-blaze dry light, veiled only in dancing
dust-devils piteously whipped by strong hot winds. All this was like a
piece of the Omdurman campaign to me—the old familiar Soudan,
so often made known to British minds by pen, pencil, and
photograph during nearly twenty years of war, unfolded itself feature
by feature. Yet we were still five hundred miles south of Khartoum!
 A SHELUK AT KODOK (FASHODA).
At Meshra-er-Zeraf we stopped for two days to shoot, by the
Sirdar's invitation, in the extensive game reserve, and were fortunate
in securing a buffalo and various antelope. We wandered through a
harsh country, of white sand and tussocks of coarse grass, more
grey than green, with leafless black thorn-trees densely tangled; yet
it seemed full of game. In three hours' walk on the second morning I
shot a fine waterbuck, two reed-bucks, and two of a beautiful herd
of roan antelope, who walked slowly down to water past our
ambuscade. And, be it remembered, that the pleasure and
excitement of such sport are in these lands always heightened by
the possibility that at any moment the hunters may come upon
game of much more serious quality—lion or buffalo; so that no one
cares to be more than a few yards from his heavy rifle or give his
mind wholly to the buck he stalks. Surely they are perverse,
unenterprising folk who spend fortunes each year in preserving with
so much artificial care, and to the inconvenience of other dwellers in
a small island, well-counted herds of more or less tame deer, when
in a month, and for less expense than the year's rent of their forests,
they could pursue wild animals of every kind in their natural haunts
and gain experiences that would last them all their lives.
I was so much elated by this jolly morning's sport and the near
approach of civilized conditions—for after all, contrast is an element
in pleasure—that I permitted myself to rejoice at the safe and happy
outcome of this long journey, and to exult in our complete immunity
from serious accident or illness or even fever. How extravagant were
the accounts of the dangers of African travel! How easy to avoid the
evil chances of the road! Reasonable precautions, steady exercise,
regular quinine—were these not in themselves the guarantees of
safety? Thus I reckoned, and with specious reasons, but in a bad
hour. We were not yet at our journey's end.
Twenty-four hours' steaming from Meshra-er-Zeraf brought us
near Khartoum. The character of the country was unchanged. Yellow
sand-slopes drank at the Nile brim; thorn-scrub fringed the river on
either side; but date-palms mingled even more frequently and
numerously with the vegetation, and brown mud-built villages with
brown mud-coloured populations multiplied as the miles slipped
swiftly by. At length a solitary majestic tree, beneath whose spacious
branches and luxuriant foliage a hundred persons might have found
shelter from the relentless sun—Gordon's tree—advertised us of the
proximity of Khartoum. Soon on the one bank came into view the
vast mud labyrinth of Omdurman, with forests of masts rising along
the shore, and on the other, among palm-groves ever clustering
thicker, sprang the blue and pink and crimson minarets of new
Khartoum. Khartoum—the new Khartoum, risen from its ruins in
wealth and beauty—a smiling city sitting like a queen throned at the
confluence of the Niles, the heart and centre of a far-reaching and
formidable authority, disclosed herself to the traveller's eye. Sharp to
the right turns the steamer, leaving the dull placid waters of the
sovereign river we have so long followed, and shouldering a more
turbulent current of clearer water, swings up-stream along its noble
feudatory, the Blue Nile. And passing by the side of high stone
embankments crowned by palms, the steamer enters into a modern
Oriental port and city, and is soon surrounded by its palaces, its
mosques, its warehouses and its quays.

The Palace, Khartoum.

 Nearly ten years have passed since the Dervish domination was
irretrievably shattered on the field of Omdurman, and every year has
been attended by steady and remarkable progress in every sphere of
governmental activity in every province of the Soudan. Order has
been established, and is successfully, though precariously,
maintained even in the remotest parts of Kordofan. The railway has
reached the Southern bank of the Blue Nile, connects Khartoum with
Cairo and with the Red Sea, waits only for the construction of a
bridge to cross the river and enter the fertile regions of the
Ghezireh. A numerous fleet of steamers maintains swift and regular
communication along the great waterways. The revenue has risen
from a few thousands a year in 1899 to considerably over a million
pounds in 1907. Improved methods of agriculture have increased the
wealth of the country; the prevention of massacre and famine has
begun to restore its population. Slavery has been abolished, and
without affronting the religion or seriously disturbing the customs of
the people, a measure of education and craftsmanship has been
introduced.
These great changes which are apparent throughout the whole
Soudan are nowhere presented in so striking and impressive form as
in the capital. A spacious palace, standing in a beautiful garden, has
risen from the ruins where Gordon perished. Broad thoroughfares
lighted by electricity, and lined with excellent European shops, lead
with geometrical precision through the city. A system of steam
tramways in connection with ferry boats, patronized chiefly by the
natives, renders communication easy throughout Khartoum, and
between Khartoum, Omdurman, and Halfyah. A semi-circle of
substantial barracks, arranged upon a defensive scheme, protects
the landward approaches. The Gordon College hums with scholarly
activity—Moslem and Christian, letters or crafts; and seven thousand
soldiers of all dress march past the British and Egyptian flags on
occasions of ceremony.
 George Scrivings.

Yet neither these inspiring facts—the more impressive by contrast

with my memories of ten years before—nor the gracious hospitality
of the Sirdar—more responsible than any other man for the whole of
this tremendous task of reconstruction and revival—were to prevent
me from taking away a sombre impression of Khartoum. As our
steamer approached the landing-stage I learned that my English
servant, George Scrivings, had been taken suddenly ill, and found
him in a condition of prostration with a strange blue colour under his
skin. Good doctors were summoned. The hospital of Khartoum, with
all its resources, was at hand. There appeared no reason to
apprehend a fatal termination. But he had been seized by a violent
internal inflammation, the result of eating some poisonous thing
which we apparently had escaped, and died early next morning after
fifteen hours' illness, with almost every symptom of Asiatic cholera.
Too soon, indeed, had I ventured to rejoice. Africa always claims
its forfeits; and so the four white men who had started together
from Mombasa returned but three to Cairo. A military interment
involves the union of the two most impressive rituals in the world.
The day after the Battle of Omdurman it fell to my lot to bury those
soldiers of the 21st Lancers, who had died of their wounds during
the night. Now after nine years, in very different circumstances,
from the other end of Africa, I had come back to this grim place
where so much blood has been shed, and again I found myself
standing at an open grave, while the yellow glare of the departed
sun still lingered over the desert, and the sound of funeral volleys
broke its silence.

The remainder of our journey lay in tourist lands, and the

comfortable sleeping-cars of the Desert Railway, and the pleasant
passenger steamers of the Wady Haifa and Assouan reach soon
carried us prosperously and uneventfully to Upper Egypt; and so to
Cairo, London, and the rest.
Philae.
 CHAPTER XI

THE VICTORIA AND ALBERT RAILWAY

My journey is at an end, the tale is told, and the reader who has
followed so faithfully and so far has a right to ask what message I
bring back. It can be stated in three words. Concentrate upon
Uganda!
Over the greater part of the north-east quarter of Africa, British
influence or authority in one form or another is supreme. But when I
turn my mind over all those vast expanses, excluding only Egypt,
there is no region which offers prospects to compare in hopefulness
with those of the Protectorate of Uganda. The Soudan is far greater
in extent and importance, and Great Britain is at no charge in
respect to it. But the Soudan is clearly inferior in fertility. The East
African Protectorate possesses not only enormous coast-lands of
great value, but noble plateaux where the air is as cool as an English
spring. But we already spend on East Africa—and upon the needs of
its expensive white settlers—more than the whole revenue of
Uganda; and yet the promise is not so bright. Northern Somaliland is
a desert of rocks and thorn bushes peopled by rifle-armed fanatics,
on which we spend nearly half as much as the whole annual grant-
in-aid of Uganda. And between Somaliland and Uganda there is this
contrast presented in its crudest form—a barren land with dangerous
inhabitants; and a fruitful land with a docile people. What is least
worth having, is most difficult to hold: what is most worth having, is
easiest.
 The union under scientific direction in Uganda (and I include in
this popular name Usoga, Unyoro, Toro and Ankole, etc.) of
unequalled fertility with a population of high intelligence and social
quality, in a region of extraordinary waterways, must, unless some
grievous error or neglect should intervene, result in remarkable
economic developments. Already more than half the traffic which
passes down the railway to Mombasa comes from beyond the lake.
Yet scarcely any money has ever been spent on Uganda. No
European roads exist, no railways have been built, no waterfalls are
harnessed, no public works of any serious description have been
undertaken. A poor little grant-in-aid has barely supported the day-
to-day cost of European administration, and practically nothing in
cash or credit has been available for the development of the country.
But it is alive by itself. It is vital; and in my view, in spite of its
insects and its diseases, it ought in the course of time to become the
most prosperous of all our East and Central African possessions, and
perhaps the financial driving wheel of all this part of the world. It is
far from my desire to disparage the East African Protectorate, or to
suggest diminution of activity or support. Both Protectorates are
necessary to each other and should advance together; but in view of
their relative positions, and looking at the situation as it is to-day,
my counsel plainly is—"Concentrate upon Uganda!" Nowhere else in
Africa will a little money go so far. Nowhere else will the results be
more brilliant, more substantial or more rapidly realized.
Cotton alone should make the fortune of Uganda. All the best
qualities of cotton can be grown in the highest perfection, a hundred
thousand intelligent landowners occupying twenty thousand square
miles of suitable soil are eager to engage in the cultivation. An
industrious and organized population offers the necessary labour.
Merely at the request of the Government cotton has been planted
experimentally on a considerable scale throughout Uganda. The
figures of production—though of course they are only the first
beginnings—show a surprising expansion. Great care is required,
and steps have already been taken to secure that the quality of
cotton exported from Uganda is not deteriorated or its reputation
prejudiced by hasty or untutored action, that only the seeds which
yield the best results should be distributed, and that no
indiscriminate mixture should be permitted. The Government must
control the culture. Experts must watch the ginneries and educate
the native cultivator. Roads must be made to enable the crop to be
marketed. The scientific organization of the cotton-growing
resources of Uganda has now been definitely undertaken. A special
grant of £10,000 a year will in future be devoted to this purpose,
and the whole process will be supervised by European officers in
close touch through the Colonial Office with the highest Manchester
authorities and the British Cotton-Growing Association. In the
opinion of the ablest observers the next five years will see a very
remarkable development in cotton production, even though the
means available to foster it continue to be slender.
But cotton is only one of those tropical products for which the
demand of civilized industry is almost insatiable, and which can
nowhere in the world be grown more cheaply, more easily, more
perfectly than between the waters of the two great lakes. Rubber,
fibre, cinnamon, cocoa, coffee, sugar may all be cultivated upon the
greatest scale; virgin forests of rare and valuable timber await the
axe; and even though mineral wealth may perhaps never lend its
hectic glory to Uganda, the economic foundations of its prosperity
will stand securely upon a rich and varied agriculture. A settler's
country it can never be. Whatever may be the destinies of the East
African Highlands, the shores of the great lakes will never be the
permanent residence of a white race. It is a planter's land, where
the labours of the native population may be organized and directed
by superior intelligence and external capital. For my own part I
rejoice that the physical conditions of the country are such as to
prevent the growth in the heart of happy Uganda of a petty white
community, with the harsh and selfish ideas which mark the jealous
contact of races and the exploitation of the weaker. Let it remain a
"planter's land." Let the planters, instead of being the agents of
excited syndicates with minds absorbed in the profits of shareholders
thousands of miles away, be either Europeans of substance and
character who have given proofs of their knowledge of natives and
their ability to deal skilfully and justly with them, or better still—say I
—let them be the disinterested officers of the Government, directing
the development of the country neither in their own, nor any other
pecuniary interest, but for the general good of its people and of the
Empire of which it forms a part.
But if the immediate inflexion of British policy in Eastern Africa
should be, without prejudice, but with precedence of other
provinces, to accelerate the economic and social development of
Uganda, what are the first steps to take? I might have much to say
of Forestry and Agriculture; of an extended system of technical
education similar to that given at the Gordon College at Khartoum,
here perhaps in part to be achieved through grants in aid of the
existing missionary schools; of road-making, indispensable to
progress, of motor-transport, and of water-power. But let me make
my message brief and unclouded, and as before expressed in three
words, "Build a Railway."
The clusters of colonial possessions which have been acquired on
the east and west coasts of Africa, so rapidly and with so little cost
or bloodshed, will unquestionably prove an invaluable, if not indeed
a necessary feature of the British Empire. From these vast
plantations will be drawn the raw materials of many of our most
important industries; to them will flow a continuous and broadening
train of British products; and in them the peculiar gifts for
administration and high civic virtues of our race may find a healthy
and an honourable scope. Some of these great estates, like
Southern Nigeria on the west coast, are already so prosperous as
not only to be self-supporting, but able to assist with credit and
subvention the progress of neighbours less far advanced. Others are
still a charge upon our estimates. We are annually put to the
expense of grants-in-aid more or less considerable for Northern
Somaliland, the East African Protectorate, Nyassaland and Uganda.
Heavy upon the finance of all the East Coast hangs the capital
charge of the Uganda Railway. In no way will these charges be
eased or removed except by the rise of one or more of the territories
concerned to economic buoyancy, or by the growth of railway traffic
down the Uganda trunk consequent upon development. Under
present conditions the progress made from year to year is steady
and encouraging. The charges upon the Colonial Estimates diminish
regularly every year. Every year the administration of the different
Governments increases in elaboration, in efficiency and consequently
in cost. The extra charge is met ever more fully by the returning
yield of a grateful soil. Except for the chances of war, rebellion,
pestilence, and famine which brood over the infancy of tropical
protectorates, but which may be averted or controlled, it would be
easy to calculate a date—not too remote—by which all contribution
from the British tax-payer would be unnecessary. The movement of
events is encouraging; but there is one method by which it can be
made far more sure and far more swift, by which all adverse
chances are minimized, and all existing resources stimulated and
multiplied—railways.
I would go so far as to say that it is only wasting time and money
to try to govern, or still more develop, a great African possession
without a railway. There can be no security, progress, or prosperity
without at least one central line of rapid communication driven
through the heart of the country. Where, as in Northern Somaliland,
the land itself is utterly valueless, a mere desert of rocks and scrub,
or where the military dangers are excessive and utterly
disproportioned to any results that can ever be reaped—withdrawal
and concentration are the true policy. But if for any reason it be
decided to remain and to administer, a railway becomes the prime of
absolute necessities. Till then all civilized government is extravagant
and precarious, and all profitable commerce practically impossible.
These considerations have lately led a British Government to
sanction the extensive railways, nearly 600 miles long, now being
rapidly constructed in Northern and Southern Nigeria; and the same
arguments apply, though in my view with increased force, to the
Uganda Protectorate.
It is not usually realized that the Uganda railway does not pass
through Uganda. It is the railway to Uganda and not of Uganda. It
stops short of the land from which it takes its name, and falls
exhausted by its exertions and vicissitudes, content feverishly to lap
the waters of the Victoria Nyanza. Uganda is reached, but not
traversed by steam communication in any form. Yet the extension of
the railway from the western shores of the Victoria to the Albert
Nyanza would not only carry it through much of the most valuable
and fertile country within its radius, but as I shall show could far
more than double its effective scope.
It may be accepted as an axiom that in the present state of
development in these African protectorates, it is scarcely ever, and
indeed I think never, worth while to build railways in competition
with waterways. Railways should in new countries be in supplement
of, and not in substitution for, lakes and navigable rivers. No doubt
direct through-routes of railway, where bulk is not broken and all
delays and changings are avoided, show an imposing advantage in
comparison with a mere alternation of water stages and railway
links. There could be no doubt which was the better if only you leave
out the question of cost. But it is just this question of cost which
cannot be left out, which clamorously dominates the proposition
from the beginning. For first-class countries may afford first-class
railways and trains de luxe, but second-class countries must be less
ambitious, and young new jungle-born countries are satisfied, or
ought to be, if they get any railway at all. The differences between
the best railway in the world and the worst, are no doubt
impressive; but they become utterly insignificant when contrasted
with the difference between the worst railway in the world and no
railway at all. For observe, the comparison is not with perfect lines of
European communication, nor with anything like them, nor even
with a waggon on a turnpike road. It is with a jogging, grunting,
panting, failing line of tottering coolies, men reduced to beasts of
burden, that the new pioneer line must be compared—that is to say,
with the most painful, most degrading, slowest and feeblest method
of transportation which has ever disgraced the world. And compared
with that, any line of steam-communication, however primitive,
however light, however interrupted, is heaven.
 I am endeavouring to guide the reader to a positive proposal of a
modest and practical character, I mean the construction of a new
railway which might be called "The Victoria and Albert Railway,"
although it would virtually be an extension of the existing Uganda
line. This railway should traverse the country between the great
lakes, and join together these two noble reservoirs with all their
respective river connections. The distance is not great. Two hundred
and fifty miles would exceed the largest computation; and perhaps a
line of one hundred and fifty miles would suffice. If the cost of this
railway were estimated, as I am informed is reasonable, at a
maximum figure of £5,000 a mile, the total sum involved would be
between £1,250,000 and £750,000.
The supreme advantage of making a railway debouch upon a
great lake, is that every point on the lake shore is instantly put in
almost equal communication with railhead. Steamers coast round on
circular tours, and whatsoever trade or traffic may offer along the
whole circumference, is carried swiftly to the railway. Lakes are in
fact the catchment areas of trade, and it is by tapping and uniting
them that the economic life of Central Africa can be most easily and
swiftly stimulated.
Two routes present themselves with various competing
advantages by which the Victoria and Albert Railway may proceed.
The first, the most obvious, most desirable and most expensive, is
straight across the Highlands of Toro, through the best of the cotton
country, from a point on the Victoria Lake in the neighbourhood of
Entebbe, to where the Semliki river runs into the southern end of
Lake Albert. The second would practically follow the footsteps
recorded in these pages. It does not offer a direct line. It does not
pass during the whole of its length through cultivated and inhabited
country. It does not reach the Albert Lake at the most convenient
end. But it is far cheaper than the other. It is only 135 miles long
instead of nearly 250. It connects not only the two great lakes, but
also Lake Chioga with all its channels and tributaries, in one system
of unbroken steam communication.
 Briefly this latter project would consist of two links of railway: the
first about sixty miles long from Jinja (or Ripon Falls) to Kakindu, the
first point where the Victoria Nile becomes navigable: the second
about seventy-five miles long from the neighbourhood of Mruli to the
Nile below the Murchison Falls and near its mouth on the Albert. By
these two sections of railway, together only 135 miles in length, a
wonderful extent of waterways would be commanded; to wit: 1.
Thirty miles of the Victoria Nile navigable from Kakindu to Lake
Chioga. 2. Lake Chioga itself, with its long arms and gulfs stretching
deeply into the whole of the fertile regions to the south-west of
Mount Elgon, and affording a perimeter of navigable coastline
accessible by steamers, of certainly not less than 250 miles. 3. All
that reach of the Victoria Nile navigable from Lake Chioga to Foweira
when the rapids ending in the Murchison Falls begin again—70 miles.
4. Thirty miles from below the falls to the Albert Lake. 5. The whole
of the Albert Lake shores—250 miles. 6. The Semliki river navigable
(once a sandbar has been passed) for sixty miles. 7. The glorious
open reach of the White Nile from the Albert Lake to Nimule—120
miles. Thus by the construction of only 135 miles of railroad, swift
modern communication would be established over a total range of
800 miles: or for an addition of one-fifth to its length and one-eighth
to its cost the effective radius of the Uganda railway would be more
than doubled. Such railway propositions are few and far between.
I do not prejudge the choice of these two routes. Both are now
being carefully surveyed. The advantages of the longer and more
ambitious line across Toro are perhaps superior. But the cost is also
nearly twice as great; and cost is a vital factor—not merely to the
government called upon to find money, but still more to the
commercial soundness of an enterprise which is permanently
crippled, if its original capital charges are allowed notably to exceed
what the estimated earnings would sustain. The question is one
which will require severe and patient examination, the nicest
balancings between competitive advantages, the smoothest
compromises between the practical and the ideal.
 But let us now look forward to a time—not, I trust, remote—when
by one route or the other the distance between the Victoria and
Albert Lakes has been spanned by a railway, and when the
Mountains of the Moon are scarcely four days' journey from
Mombasa. The British Government will then be possessed of the
shortest route to the Eastern Congo. The Uganda railway will be able
to offer rates for merchandise and railway material with which no
other line that can ever be constructed will ever be able to compete.
The whole of that already considerable, though as yet stifled trade,
which feebly trickles back half across Africa by Boma to the Atlantic,
which is looking desperately for an outlet to the northward, which
percolates in driblets through Uganda to-day, will flow swiftly and
abundantly to the benefit of all parties concerned down the Uganda
trunk, raising that line with steady impulse from the status of a
political railway towards the level of a sound commercial enterprise.
In no other way will the British tax-payer recover his capital. The
advantages are great and the expense moderate. Larger
considerations may postpone, and the imperative need of the fullest
surveys will in any case delay construction; but I cannot doubt that
the Victoria and Albert railway is now the most important project
awaiting action in the whole of that group of Protectorates which Sir
Frederick Lugard used proudly to call "our East African Empire."
But let us proceed one step further in the development of the
communications of north-east Africa. When an extension of the
Uganda railway has reached the Albert Nyanza, only one link will be
missing to connect the whole of the rail and waterway system of
East Africa and Uganda with the enormous system of railways and
riverways of Egypt and the Soudan, to connect the Uganda with the
Desert railway, to join the navigation of the great lakes to the
navigation of the Blue and White Niles. Only one link will be missing,
and that a very short one; the distance of 110 miles from Nimule to
Gondokoro, where the Nile is interrupted by cataracts. Of the
commercial utility of such a link in itself I have nothing to say; but as
a means of marrying two gigantic systems of steam communication,
it will some day possess a high importance; and thereafter over the
whole of the north-east quarter of the African continent under the
influence or authority of the British Crown, comprising a total
mileage by rail and river of perhaps 20,000 miles, uninterrupted
steam communication will prevail.
The adventurous and the imaginative may peer out beyond these
compact and practicable steps into a more remote and speculative
region. Perhaps by the time that the junction between the Uganda
and Soudan rail and water systems has been effected, the Rhodes
Cape to Cairo railway will have reached the southern end of Lake
Tanganyika: and then only one comparatively short hiatus will bar a
complete transcontinental line, if not wholly of railroad, at least of
steam traffic and of comfortable and speedy travel.
Then, perhaps it will be time to make another journey; but as the
reader, who will no doubt take care to secure a first-class tourist
ticket, will no longer require my services as guide, I shall take this
opportunity of making him my bow.
 FOOTNOTE
[1] "I am informed by the courtesy of Mr. Lydekker of the
British Natural History Museum, that the true name of the white
rhinoceros found in Uganda is Rhinoceros Simus Cottoni.
'Burchell's white rhinoceros' is the designation of the southern
race; but I have preserved in the text the name commonly used
in Uganda."

Transcriber's Note:
Obvious typographical errors have been corrected.
Inconsistent spelling and hyphenation in the original
document have been preserved.
Welcome to our website – the perfect destination for book lovers and
knowledge seekers. We believe that every book holds a new world,
offering opportunities for learning, discovery, and personal growth.
That’s why we are dedicated to bringing you a diverse collection of
books, ranging from classic literature and specialized publications to
self-development guides and children's books.

More than just a book-buying platform, we strive to be a bridge

connecting you with timeless cultural and intellectual values. With an
elegant, user-friendly interface and a smart search system, you can
quickly find the books that best suit your interests. Additionally,
our special promotions and home delivery services help you save time
and fully enjoy the joy of reading.

Join us on a journey of knowledge exploration, passion nurturing, and

personal growth every day!

ebookbell.com