Scribd
Upload
17 views

Lab

The document discusses various techniques for building and training neural network models for classification tasks including image and time series data, including: - Initializer techniques like Xavier and Kaiming initialization to prevent exploding or vanishing gradients - Regularization techniques like dropout layers and batch normalization to prevent overfitting - Using transfer learning with a pre-trained VGG16 model for image classification - Early stopping to select the best model weights during training to avoid overfitting

Uploaded by

Atul Draws
Copyright
© © All Rights Reserved
Available Formats
Download as PDF, TXT or read online on Scribd
17 views

Lab

The document discusses various techniques for building and training neural network models for classification tasks including image and time series data, including: - Initializer techniques like Xavier and Kaiming initialization to prevent exploding or vanishing gradients - Regularization techniques like dropout layers and batch normalization to prevent overfitting - Using transfer learning with a pre-trained VGG16 model for image classification - Early stopping to select the best model weights during training to avoid overfitting

Uploaded by

Atul Draws
Copyright
© © All Rights Reserved
Available Formats
Download as PDF, TXT or read online on Scribd
You are on page 1/ 12

history = modelMLP.

fit(X_train, y_train, epochs=50, batch_size=64, validation_data=(X_test,


y_test), verbose=2)

i) Xavier Initialization:
Xavier initialization helps in initializing the weights in a way that prevents vanishing or
exploding gradients. You can use the glorot_uniform initializer for Xavier initialization.

from keras.initializers import glorot_uniform

model = Sequential()

model.add(Dense(256, activation='relu', input_dim=(28 * 28),


kernel_initializer=glorot_uniform()))
model.add(Dense(256, activation='relu',
kernel_initializer=glorot_uniform()))
model.add(Dense(10, activation='softmax',
kernel_initializer=glorot_uniform()))

ii) Kaiming Initialization:


Kaiming Initialization is typically used with activation functions like ReLU. You can use the
he_uniform initializer for Kaiming Initialization.

from keras.initializers import he_uniform

model = Sequential()

model.add(Dense(256, activation='relu', input_dim=(28 * 28),


kernel_initializer=he_uniform()))
model.add(Dense(256, activation='relu',
kernel_initializer=he_uniform()))
model.add(Dense(10, activation='softmax',
kernel_initializer=he_uniform()))

iii) Dropout Layers:


Dropout layers help prevent overfitting by randomly setting a fraction of input units to 0 at
each update during training. You can add Dropout layers like this:

from keras.layers import Dropout

model = Sequential()

model.add(Dense(256, activation='relu', input_dim=(28 * 28),


kernel_initializer=he_uniform()))
model.add(Dropout(0.5)) # Add dropout with a dropout rate of
0.5
model.add(Dense(256, activation='relu',
kernel_initializer=he_uniform()))
model.add(Dropout(0.5)) # Add dropout with a dropout rate of
0.5
model.add(Dense(10, activation='softmax',
kernel_initializer=he_uniform()))

iv) Batch Normalization:


Batch Normalization normalizes the input of each layer, which helps in training deep neural
networks. You can add BatchNormalization layers like this:

from keras.layers import BatchNormalization

model = Sequential()

model.add(Dense(256, input_dim=(28 * 28),


kernel_initializer=he_uniform()))
model.add(BatchNormalization())
model.add(Activation('relu'))
model.add(Dense(256, kernel_initializer=he_uniform()))
model.add(BatchNormalization())
model.add(Activation('relu'))
model.add(Dense(10, activation='softmax',
kernel_initializer=he_uniform()))

modelMLP = Sequential()
modelMLP.add(Dense(256, activation='relu',input_dim=3072))
modelMLP.add(Dense(256, activation='relu'))
modelMLP.add(Dense(10, activation='softmax'))

for CNN:
#preprocessing data

xtrain = np.reshape(xtrain, (60000, 28,28,1))


xtest = np.reshape(xtest, (10000, 28,28,1))

xtrain = xtrain.astype('float32')
xtest = xtest.astype('float32')

xtrain = xtrain/255
xtest = xtest/255

#building model

model = Sequential()
model.add(Conv2D(filters = 25, kernel_size=(3,3),
activation='relu', input_shape = (28,28,1)))
model.add(MaxPooling2D(2,2))
model.add(Conv2D(filters = 64, kernel_size=(3,3),
activation='relu'))
model.add(MaxPooling2D(2,2))
model.add(Conv2D(filters = 64, kernel_size=(3,3),
activation='relu'))
model.add(MaxPooling2D(2,2))
model.add(Flatten())
model.add(Dense(128, activation = 'relu'))
model.add(Dense(10, activation = 'softmax'))

TRANSFER LEARNING:

from keras.datasets import mnist


import tensorflow_datasets as tfds

(xtrain, ytrain), (xtest, ytest) = tfds.load("mnist",


split=["train[:70%]", "train[:30%]"], batch_size=-1,
as_supervised=True)

import tensorflow as tf
from keras.applications.vgg16 import VGG16
from keras.applications.vgg16 import preprocess_input

# Loading VGG16 model


base_model = VGG16(weights="imagenet", include_top=False,
input_shape=(32, 32, 3))

base_model.trainable = False # Not trainable weights

# Preprocessing input (assuming MNIST images are converted to 3


channels and resized to 32x32)
xtrain = tf.image.grayscale_to_rgb(xtrain) # Convert to RGB
xtrain = tf.image.resize(xtrain, (32, 32)) # Resize to 32x32
xtrain = preprocess_input(xtrain)

xtest = tf.image.grayscale_to_rgb(xtest) # Convert to RGB


xtest = tf.image.resize(xtest, (32, 32)) # Resize to 32x32
xtest = preprocess_input(xtest)

from keras.models import Sequential


from keras.layers import Dense, Flatten

model = Sequential()
model.add(base_model)
model.add(Flatten())
model.add(Dense(128, activation = 'relu'))
model.add(Dense(10, activation = 'softmax'))

EARLY STOPPING

from tensorflow.keras.callbacks import EarlyStopping

model.compile(
optimizer='adam',
loss='categorical_crossentropy',
metrics=['accuracy'],
)

es = EarlyStopping(monitor='val_accuracy', mode='max',
patience=5, restore_best_weights=True)

model.fit(train_ds, train_labels, epochs=50,


validation_split=0.2, batch_size=32, callbacks=[es])
# Define parameters
n_input_nifty = 3 # Adjust as needed
n_features_nifty = nifty_data.shape[1] # Assuming all columns are used
as features

# Create time series generators


generator_nifty = TimeseriesGenerator(train_data, train_data,
length=n_input_nifty, batch_size=1)
generator_test_nifty = TimeseriesGenerator(test_data, test_data,
length=n_input_nifty, batch_size=1)
# Build and compile the model
model_nifty = Sequential()
model_nifty.add(SimpleRNN(units=100, input_shape=(n_input_nifty,
n_features_nifty), activation='tanh'))
model_nifty.add(Dense(units=1, activation='tanh'))
model_nifty.compile(loss='mean_squared_error', optimizer='adam')

# Train the model


model_nifty.fit(generator_nifty, epochs=20)

# Evaluate the model on the test data


evaluation_result = model_nifty.evaluate(generator_test_nifty)

# Print the evaluation result


print(f'Mean Squared Error on Test Data: {evaluation_result}')

# Predictions on test data


test_predictions = model_nifty.predict(generator_test_nifty)

# Inverse transform the scaled predictions to get original values


test_predictions_original =
scaler_nifty.inverse_transform(test_predictions)

# Optionally, you can compare the predicted values with the actual
values
actual_values = nifty_data.values[1608 + n_input_nifty:]

# Create a DataFrame to compare actual and predicted values


comparison_df = pd.DataFrame({
'Actual': actual_values.flatten(),
'Predicted': test_predictions_original.flatten()
})

# Display the comparison DataFrame


print(comparison_df)

You might also like