Department of Computer Science and Engineering

Lesson Plan

Course Title: Neural Network & Fuzzy Logic Course Code: CSE 451
Credit: 3.0 Contact Hours: 39 (3 Hour/Week)
Level/Term: 7th Type - Core/Major: Core
Section: A/B/C

Prerequisite: CSE 111 (Structured Programming) or instructor's consent.

Session: Spring 2020

Instructor: Md. Ataur Rahman (AR) | Faisal Ahmed (FA)

Class schedule: Tuesday (1.00 – 2.30, Room: 609) | Wednesday (01.00 – 2.30, Room: 609) – Sec: A
Saturday (08.30 – 10.00, Room: 408) | Tuesday (11.30 – 01.00, Room: 409) – Sec: B
Monday (9.30 – 11.00, Room: 404) | Wednesday (11.30 – 1.00, Room: 808) – Sec: C

Counseling Time: Saturday (12.00 – 1.00) Room No: 607 | 503

Email address: [email protected] Phone No: 01717453828 (Sec-A/C)
[email protected] 01521315943 (Sec - B)
Rationale:
This course tends to help the learners to break into cutting-edge AI by gaining knowledge
about design and working principle of Artificial Neural Networks (ANN) and Deep Learning. In
essence, students will also see how these concepts relates to the biological neuron of human brain. This
course in general also teaches you how Neural Networks (NN) Learning actually works, rather than
presenting only a cursory or surface-level description. It also helps to gain the basics of different
application areas and evaluation of fuzzy logic and fuzzy set theories. So, after completing, you will be
able to understand the key parameters in a neural network's architecture and apply them to your own
applications.

Course Objectives: The objective of this course is to introduce -

• The concept of Artificial Neural Network (NN) related to biological neuron model.
• Different types of NN Architecture.
• Application of NN into classification and clustering problem.
• Concepts of Artificial Intelligence (AI), Pattern Recognition, Computer Vision and Natural
Language Processing (NLP).
• Basic approaches of Fuzzy Logic, Fuzzy Set Theories.

Course Outcomes (COs): Upon completion of this course, students should be able to:

1. Have a deeper understanding about Neural Network (NN) as a subdomain of modern Machine
Learning (ML) algorithms and their Applications.
 2. Realize how different components are combined to form the structural building block of a NN
that can mimic Human Brains in terms of decision making.
3. Understand the notion of a Single-layer (Perceptron) and Multi-layer NN and their basic working
mechanism.
4. Grasps the idea behind Training of a NN through Loss/Cost and Forward and Back-propagation.
5. Gain in-depth knowledge about the mathematics behind the Training of a simple NN (Gradient
Descent, Computation Graph, Vectorization etc.).
6. Present and critically assess current research on neural networks and their Relationship with
other research areas such as Pattern Recognition, Natural Language Processing etc.
7. Work with uncertainty of a problem using fuzzy logic rules.

Course Description (Catalog Description): This course deals with the theory and applications of
neural networks and fuzzy logic. Topics discussed are single-layer and multi-layer perception,
supervised and unsupervised neural network, Hebbian network, Hopfield network, recurrent network,
fuzzy set theory, fuzzy logic control, adaptive fuzzy neural network, genetic algorithm, and evolution
computing. Applications to control, pattern recognition, nonlinear system modeling, speech and image
processing are also introduced.

Reading Lists and Books:

Textbooks:
1. Make Your Own Neural Network: A Gentle Journey Through the Mathematics of Neural
Networks, and Making Your Own Using the Python Computer Language – By Tariq Rashid
2. Deep learning – By Ian Goodfellow and Yoshua Bengio and Aaron Courville - MIT press
3. Neural Networks, Fuzzy Logics and Genetic Algorithms: Synthesis and Application,
International Edition – By S. RAJASEKARAN, G. A. VIJAYALAKSHMI PAI
4. Neural Network Designs, 2nd edition, PWS Pub. Co – Martin T Hagan, Howard B Demuth,
Mark Hadson Beale.
5. Fuzzy and Neural Approach in Engineering- By Lefteri H. Tsoukalas, Robert E. Uhrig

Reading Lists and References:

1. Machine Learning Glossary.
2. Neural Networks and Deep Learning – By Michael Nielsen.
3. The Ultimate Guide to Artificial Neural Networks (ANN).
4. Deep Learning Neural Networks Explained in Plain English.
5. Deep Learning - List of Readings and Survey Papers.
6. Mathematics for Machine Learning.
7. Jain, A. K., Mao, J., & Mohiuddin, K. M. (1996). Artificial neural networks: A tutorial.
Computer, 29(3), 31-44.
8. Simon Haykin: Neural Networks- A Comprehensive Foundation, 2nd edition, Prentice Hall,
1998.

Teaching Strategy:
Typical methodologies are Class lectures, web-access, self-study, critics writing, problem
formulation, scientific paper analysis and student presentation.
Assessment Strategy:
Class Tests, Quizzes/Assignments/Homework’s, Class Participation, Midterm and Final Exam.

Marks distribution:

Description Marks
Class Attendance/ Participation 10
Class Test 10
Quizzes/Assignments/Homework’s 10
Midterm 20
Final Exam 50

Weekly Lecture Schedule:

Teaching Course Assessment

Week Topic
Strategy Outcome Strategy
Introduction:
• Difference Between AI/ML/DL,
• Overview of different subcategories of Machine
Learning and their working principle.
Class
Slides/Oral 01, 02
A Gentle Introduction to NN: Participation
• Decision Boundary.
01 • Classification Example: AND, OR, XOR.
• Biological Vs. Artificial NN.
Applications of Neural Network:
• An abstract example of NN.
• Supervised Learning with NN. Slides 01, 06 Homework
• Popular NN (ANN, CNN, RNN).
• Why Deep Learning is taking off?
Structural Building Blocks of NN:
• Input/Output Layers.
• Weights, Bias. Slides/Paper 02 CT-1
• Activation Function (non-linearity).
• Summation Vs. Matrix.
02 Single layer Perceptron (A Simple NN):
• Linear regression and Decision Boundaries w.r.t
Weights and Biases.
• Training set, Input/Output. Hand Notes 02, 03 CT - 1
• Forward Propagation.
• Why single layer Perceptron doesn’t work?
 Multi-Layer NN:
• Hidden Layer. Assignment
• XOR Calculation Using Summation Formula. (Paper
• ReLU Activation Function. Study)
• Vectorized Representation of Multi-Layer NN. Hand Notes 03
-
• Adding Bias as a weight. Mid Term
Examination
03
How NN's Learns: Assignment
• Abstract intuition behind: (Paper
o Weight Initialization Slides
Study)
(Random vs. Zero) and 02, 03,04
o Expected Vs Predicted Output.
-
Notes
o Loss and Cost Function. Mid Term
• Back-propagation Basics. Examination
Logistic Regression (LR):
• The pitfalls of Linear Regression.
• Binary Classification.
• NN standard Notations. Mid Term
Slides 05, 06
Examination
Logistic Regression Function:
• From Linear to Logistic Regression.
• Sigmoid Activation Function.
Loss and Cost Function:
04
• Training/Dev/Test set.
• Loss function:
o M.S.E
o Convex Vs. Non-convex. Slides
Mid Term
o Local Vs. Global Optima. and 05
Examination
o Log/Cross-Entropy Loss. Notes
o L1 and L2 M.A.E
• Cost function.
• SoftMax Activation Function.
Gradient Descent (G.D):
• Finding the Global Optima.
• Derivatives Assignment
• Weights and Bias Update. (Paper
• Learning Rate (intuition). Slides 05 Study)
-
Computation Graph: Final Exam
05 • More Derivatives.
• A simple example of Computation Graph.
Backward Propagation: Assignment
• Back-prop using Computation Graph. Slides (Paper
• Completing the G.D with Back and 04 Study)
propagation for LR. Hand Notes -
Final Exam
 Backward Propagation:
• G.D with ‘M’ training examples.
• G.D Algorithm using Summation. Assignment
• Vectorized G.D. Slides (Paper
• Updating Weights and Bias. and 04 Study)
• Iteration Vs. Batch Vs. Epochs Hand Notes -
06 Final Exam
A Step by Step Backpropagation Example
for Multi-Layer NN.
Introduction to Fuzzy Logic- Fuzzy set, fuzzy
set operations- union, intersection, complement, Lecture
07 Final Exam
properties of fuzzy set. notes, slides
Introduction to Fuzzy Logic- extension
principles, alfa-cuts. Lecture Assignment/
Fuzzy relations- Properties, Basic operations, 07
notes, slides Final Exam
compositions of fuzzy relations.
07 Fuzzy number-Representation, properties,
addition, subtraction of discrete and continuous
fuzzy number. Lecture CT-2, Final
07
Fuzzy number-addition and subtraction of notes, slides Exam
discrete fuzzy number through extension principle
Fuzzy number: multiplication and division of
fuzzy number Lecture CT-2, Final
07
Fuzzy linguistic description- Linguistic notes, slides Exam
08 variables and values, implication relations.
Fuzzy linguistic description- fuzzy inference, Lecture
07 Final Exam
and composition notes, slides

NB: No student will be allowed to attend mid-term and final exam if his\her attendance is less than 50%.