Artificial

Intelligence
Model Building Using Keras

Copyright Intellipaat. All rights reserved.

 Agenda
01 Why Keras? 02 What Is Keras?

03 Models in Keras 04 Layers in Keras

05 Regularization Techniques 06 Batch Normalization

07 Keras Workflow 08 Use Case 1: Sequential Model

09 Use Case 2: Functional Model

Copyright Intellipaat. All rights reserved.
There are countless Deep Learning
frameworks available today. Why do
we prefer Keras the most?

Copyright Intellipaat. All rights reserved.

Why Keras?

Keras prioritizes
developer experience

Copyright Intellipaat. All rights reserved.

Why Keras?

Keras is broadly
adopted in the
industry and among
the research
community

Copyright Intellipaat. All rights reserved.

Why Keras?

Keras makes it easy to

turn models into
products

Copyright Intellipaat. All rights reserved.

Why Keras?

Keras supports
multiple backend
engines and does not
lock you into one
ecosystem

Copyright Intellipaat. All rights reserved.

Why Keras?

Keras has strong multi-

GPU support and
distributed training
support

Copyright Intellipaat. All rights reserved.

Why Keras?

Keras development is
backed by key
companies in the Deep
Learning ecosystem

Copyright Intellipaat. All rights reserved.

What Is Keras?

Keras is a high-level neural networks API. It is written in Python and can run on top of Theano, TensorFlow, or CNTK. It is designed
to be modular, fast, and easy to use

• It was developed by François Chollet, a Google engineer

• It was developed with the concept of:

‘Being able to go from idea to result with the least possible delay is key to

doing good research’

• Keras doesn't handle low-level computation. Instead, it relies on a

specialized, well optimized tensor manipulation library to do so, serving as

the "backend engine" of Keras

• So, Keras is the high-level API wrapper for the low-level API

Copyright Intellipaat. All rights reserved.

Let us understand the different
types of models in Keras

Copyright Intellipaat. All rights reserved.

Composing Models in Keras

You can create two types of models available in Keras, i.e., the Sequential model and the Functional model

Copyright Intellipaat. All rights reserved.

Sequential Models

▪ The Sequential model is a linear stack of layers

▪ You can create a Sequential model by passing a list of layer instances to the constructor

▪ Stacking convolutional layers one above the other can be an example of a sequential model

Copyright Intellipaat. All rights reserved.

Sequential Models

from keras.models import Sequential

from keras.layers import Dense, Activation
model = Sequential([
model = Sequential()
Dense(32, input_shape=(784,)), You can also simply add
layers via the .add() model.add(Dense(32, input_dim=784))
Activation('relu’), method:
model.add(Activation('relu'))
Dense(10),
Activation('softmax’),
])

Copyright Intellipaat. All rights reserved.

Functional Models

The Keras functional API is used for defining complex models, such as multi-output models, directed acyclic graphs, or models with
shared layers

Three unique aspects of the Keras Functional API

Models are defined by creating instances of layers and connecting are as follows:

them directly to each other in pairs, then specifying the layers to act as • Defining the Input

the input and output to the model • Connecting Layers

• Creating the Model

Copyright Intellipaat. All rights reserved.

Defining the Input

• Unlike in the Sequential model, you must create and define a

standalone Input Layer that specifies the shape of the input data

• The Input Layer takes a shape argument which is a tuple that

indicates the dimensionality of the input data

• In the case of one-dimensional input data, such as for a from keras.layers import Input
visible = Input(shape=(2,))
multilayer perceptron, the shape must explicitly leave room for

the shape of the mini-batch size used when splitting the data

while training the network

• Therefore, the shape tuple is always defined with a hanging last

dimension when the input is one-dimensional

Copyright Intellipaat. All rights reserved.

Connecting Layers

• Layers in the model are connected pairwise

• This is achieved by specifying where the input comes

from while defining each new layer

• A bracket notation is used to specify the layer from from keras.layers import Input
from keras.layers import Dense
which the input is received to the current layer, after
visible = Input(shape=(2,))
hidden = Dense(2)(visible)
the layer is created

• Example: We can create the Input Layer as above,

and then create a Hidden Layer as a Dense Layer

that receives input only from the Input Layer

Copyright Intellipaat. All rights reserved.

Creating the Model

• After creating all of your model layers and

connecting them together, you must

define the model from keras.models import Model

from keras.layers import Input
• Keras provides a model class that you can from keras.layers import Dense
visible = Input(shape=(2,))
use to create a model from your created
hidden = Dense(2)(visible)

layers. It requires you to only specify the model = Model(inputs=visible, outputs=hidden)

Input and Output Layers

• Example:

Copyright Intellipaat. All rights reserved.

Predefined Neural Network Layers

• Keras has a number of predefined layers, such as:

1. Core Layers 6. Normalization Layers

2. Convolutional Layers 7. Noise Layers

3. Pooling Layers 8. Embedding Layers

4. Locally-connected Layers 9. Merge Layers

5. Recurrent Layers 10. Advanced Activation Layers

Copyright Intellipaat. All rights reserved.

Performing Regularization Using Keras

Take a look at this graph:

• As we move toward the right in this graph, our model tries to learn too well the details and the noise from the training data, which results in poor

performance on the unseen data

• In other words, while going toward the right, the complexity of the model increases such that the training error reduces but the testing error doesn’t. This

is shown in the graph on the next slide

Copyright Intellipaat. All rights reserved.

Performing Regularization Using Keras

• Have you come across a situation where your model performed exceptionally well on train data but was not able to predict test data?

• Or, were you ever on the top of a competition in public leaderboard only to fall hundreds of places in the final ranking?

• Do you know how complex neural networks are and how it makes them prone to overfitting? This is one of the most common problems Data Science

professionals face these days

Copyright Intellipaat. All rights reserved.

Performing Regularization Using Keras

Regularization is a technique which makes slight modifications to the learning algorithm such that the model generalizes better. This in turn improves the

model’s performance on the unseen data as well

Copyright Intellipaat. All rights reserved.

Performing Regularization Using Keras

How does Regularization help reduce overfitting?

Copyright Intellipaat. All rights reserved.

Performing Regularization Using Keras

How does Regularization help reduce overfitting?

Let’s consider a neural network which is overfitting on the training data as shown in the above image

Copyright Intellipaat. All rights reserved.

Performing Regularization Using Keras

How does Regularization help reduce overfitting?

Regularization penalizes the weight matrices of the nodes

Copyright Intellipaat. All rights reserved.

Performing Regularization Using Keras

Assume that our regularization coefficient is so high that some of the weight matrices are nearly equal to zero

This will result in a much simpler linear network and slight underfitting of the training data

Copyright Intellipaat. All rights reserved.

Performing Regularization Using Keras

• Such a large value of the regularization coefficient is not that useful

• We need to optimize the value of the regularization coefficient in order to obtain a well-fitted model as shown in the image below

Copyright Intellipaat. All rights reserved.

Performing Regularization Using Keras

• Dropout produces very good results and is consequently the most frequently used Regularization technique in the field of Deep Learning

• Let’s say our neural network structure is akin to the one shown below:

So, what does dropout do?

Copyright Intellipaat. All rights reserved.

Dropout

• At every iteration, it randomly selects some nodes and removes them, along with all of their incoming and outgoing connections as shown below:

• So, each iteration has a different set of nodes, and this results in a different set of outputs. It can also be thought of as an ensemble technique in

Machine Learning

Copyright Intellipaat. All rights reserved.

Dropout

• Ensemble models usually perform better than a single model as they capture more randomness. Similarly, dropout also performs better than a

normal neural network model

• The probability of choosing how many nodes should be dropped out is the hyperparameter of the dropout function. As seen in the image below,

dropout can be applied to both the Hidden Layers as well as the Input Layers

• Due to these reasons, dropout is usually preferred when we have a large neural network structure in order to introduce more randomness

Copyright Intellipaat. All rights reserved.

Dropout

In Keras, we can implement dropout using the Keras core layer

from keras.layers.core import Dropout

model = Sequential([
Dense(output_dim=hidden1_num_units, input_dim=input_num_units, activation='relu'),
Dropout(0.25),

Dense(output_dim=output_num_units, input_dim=hidden5_num_units, activation='softmax'),

])

Copyright Intellipaat. All rights reserved.

Data Augmentation

• The simplest way to reduce overfitting is to increase the size of the training data

• Let’s consider, we are dealing with images

• There are a few ways of increasing the size of the training data—rotating the image, flipping, scaling, shifting, etc.

• In the below image, some transformation has been done on the handwritten digits dataset

• This technique is known as Data Augmentation

• This usually provides a big leap in improving the accuracy of the model

• It can be considered as a mandatory trick to improve our predictions

Copyright Intellipaat. All rights reserved.

Data Augmentation

• In Keras, we can perform all these transformations using ImageDataGenerator

• It has a big list of arguments which you can use to pre-process your training data

• Example:

from keras.preprocessing.image import ImageDataGenerator

datagen = ImageDataGenerator(horizontal flip=True)
datagen.fit(train)

Copyright Intellipaat. All rights reserved.

Batch Normalization

• When the data is fed through a deep neural network and weights and parameters adjust those values, sometimes making the data too big or too small, it

becomes a problem. By normalizing the data in each mini-batch, this problem is largely avoided

• Batch Normalization normalizes each batch by both mean and variance reference

• It is just another layer, you can use to create your desired network architecture

• It is generally used between the linear and the non-linear layers in your network, because it normalizes the input to your activation function so that you're

centered in the linear section of the activation function (such as Sigmoid)

Copyright Intellipaat. All rights reserved.

Batch Normalization

• When the data is fed through a deep neural network and weights
A normal Dense fully connected layer looks like this:
and parameters adjust those values, sometimes making the data

too big or too small, it becomes a problem. By normalizing the

model.add(layers.Dense(64, activation='relu'))
data in each mini-batch, this problem is largely avoided

• Batch Normalization normalizes each batch by both mean and

variance reference

• It is just another layer, you can use to create your desired network

architecture

• It is generally used between the linear and the non-linear layers in

your network, because it normalizes the input to your activation

function so that you're centered in the linear section of the

activation function (such as Sigmoid)

Copyright Intellipaat. All rights reserved.

Batch Normalization

• When the data is fed through a deep neural network and weights
A normal Dense fully connected layer looks like this:
and parameters adjust those values, sometimes making the data

too big or too small, it becomes a problem. By normalizing the

model.add(layers.Dense(64, activation='relu'))
data in each mini-batch, this problem is largely avoided

• Batch Normalization normalizes each batch by both mean and

variance reference
To make it Batch normalization enabled, we have to tell the
• It is just another layer, you can use to create your desired network Dense Layer not to use bias, since it is not needed, and thus
it can save some calculation. Also, put the Activation Layer
architecture after the BatchNormalization() layer

• It is generally used between the linear and the non-linear layers in

your network, because it normalizes the input to your activation model.add(layers.Dense(64, use_bias=False))
model.add(layers.BatchNormalization())
function so that you're centered in the linear section of the model.add(Activation("relu"))

activation function (such as Sigmoid)

Copyright Intellipaat. All rights reserved.

Let us see how to build models in
Keras!

Copyright Intellipaat. All rights reserved.

Building Models in Keras

Let us see the 4-step workflow in developing neural networks with Keras

Define the Training Data

Copyright Intellipaat. All rights reserved.

Building Models in Keras

Let us see the 4-step workflow in developing neural networks with Keras

Define a Neural Network

Model

Copyright Intellipaat. All rights reserved.

Building Models in Keras

Let us see the 4-step workflow in developing neural networks with Keras

Configure the Learning

Process

Copyright Intellipaat. All rights reserved.

Building Models in Keras

Let us see the 4-step workflow in developing neural networks with Keras

Train the Model

Copyright Intellipaat. All rights reserved.

Building a Simple Sequential Model

Copyright Intellipaat. All rights reserved.

Use Case 1

Here, we will try to build a Sequential Network of Dense Layers, and the dataset used is MNIST. MNIST is a classic dataset of
handwritten images, released in 1999, and has served as the basis for benchmarking classification algorithms

Copyright Intellipaat. All rights reserved.

Building a Keras Model Using Functional
Model API

Copyright Intellipaat. All rights reserved.

Use Case 2

Here, we will be using Keras to create a simple neural network to predict, as accurately as we can, digits from handwritten images. In
particular, we will be calling the Functional Model API of Keras and creating a 4-layered and 5-layered neural network.
Also, we will be experimenting with various optimizers: the plain Vanilla Stochastic Gradient Descent optimizer and the Adam’s optimizer.
We will also introduce dropout, a form of regularization technique, in our neural networks to prevent overfitting

Copyright Intellipaat. All rights reserved.

Quiz

Copyright Intellipaat. All rights reserved.

Quiz 1

Keras uses TensorFlow in the backend?

A True

B False

Copyright Intellipaat. All rights reserved.

Answer 1

Keras uses TensorFlow in the backend?

A True

B False

Copyright Intellipaat. All rights reserved.

Quiz 2

Models is Keras are of type..

A Sequential and Functional API

B Linear and Functional API

C Sequential and Batch

D None of these

Copyright Intellipaat. All rights reserved.

Answer 2

Models is Keras are of type..

A Sequential and Functional API

B Linear and Functional API

C Sequential and Batch

D None of these

Copyright Intellipaat. All rights reserved.

Quiz 3

Dropout is the other name for Data

Augmentation?

A Yes

B No

Copyright Intellipaat. All rights reserved.

Answer 3

Dropout is the other name for Data

Augmentation?

A Yes

B No

Copyright Intellipaat. All rights reserved.

Thank you!

Copyright Intellipaat. All rights reserved.

India: +91-7847955955

US: 1-800-216-8930 (TOLL FREE)

[email protected]

24/7 Chat with Our Course Advisor

Copyright Intellipaat. All rights reserved.