Scribd
Upload
36 views

Taller - 02 - Taller - Ipynb - DarwinMerchan

The document discusses training a convolutional neural network model on the MNIST dataset. It imports necessary libraries, initializes GPU parameters, downloads and preprocesses the MNIST dataset, defines the model architecture with convolutional and linear layers, and trains the model in a loop over 5 epochs. For each epoch, it runs the training process to update weights and outputs loss, and evaluates the model on the test set to output loss and accuracy. The trained model achieves an initial accuracy of 97.76% and a final accuracy of 98.82% after 5 epochs. Finally, it tasks training the same model on the Fashion-MNIST dataset.

Uploaded by

jerryman1
Copyright
© © All Rights Reserved
Available Formats
Download as PDF, TXT or read online on Scribd
36 views

Taller - 02 - Taller - Ipynb - DarwinMerchan

The document discusses training a convolutional neural network model on the MNIST dataset. It imports necessary libraries, initializes GPU parameters, downloads and preprocesses the MNIST dataset, defines the model architecture with convolutional and linear layers, and trains the model in a loop over 5 epochs. For each epoch, it runs the training process to update weights and outputs loss, and evaluates the model on the test set to output loss and accuracy. The trained model achieves an initial accuracy of 97.76% and a final accuracy of 98.82% after 5 epochs. Finally, it tasks training the same model on the Fashion-MNIST dataset.

Uploaded by

jerryman1
Copyright
© © All Rights Reserved
Available Formats
Download as PDF, TXT or read online on Scribd
You are on page 1/ 8

keyboard_arrow_down CNN

Darwin Merchan Delgado


Taller 2

Importando bibliotecas

import torch
import torch.nn as nn
import torch.nn.functional as F
from torchvision import datasets, transforms

Inicializando parámetros para el uso de la GPU

use_cuda = torch.cuda.is_available()
device = torch.device("cuda" if use_cuda else "cpu")
train_kwargs = {'batch_size': 500}
test_kwargs = {'batch_size': 500}
if use_cuda:
cuda_kwargs = {'num_workers': 1, 'pin_memory': True, 'shuffle': True}
train_kwargs.update(cuda_kwargs)
test_kwargs.update(cuda_kwargs)

use_cuda

False

Descargando conjunto de datos (aplica transformaciones)

transform = transforms.Compose([ transforms.ToTensor(), transforms.Normalize((0.5,), (0.3,)) ])

dataset1 = datasets.MNIST('../data', train=True, download=True, transform=transform)


dataset2 = datasets.MNIST('../data', train=False, transform=transform)
train_loader = torch.utils.data.DataLoader(dataset1, **train_kwargs)
test_loader = torch.utils.data.DataLoader(dataset2, **test_kwargs)

Downloading http://yann.lecun.com/exdb/mnist/train-images-idx3-ubyte.gz
Downloading http://yann.lecun.com/exdb/mnist/train-images-idx3-ubyte.gz to ../data/MNIST/raw/train-images-idx3-ubyte.gz
100%|██████████| 9912422/9912422 [00:00<00:00, 107951287.72it/s]
Extracting ../data/MNIST/raw/train-images-idx3-ubyte.gz to ../data/MNIST/raw

Downloading http://yann.lecun.com/exdb/mnist/train-labels-idx1-ubyte.gz
Downloading http://yann.lecun.com/exdb/mnist/train-labels-idx1-ubyte.gz to ../data/MNIST/raw/train-labels-idx1-ubyte.gz
100%|██████████| 28881/28881 [00:00<00:00, 31044514.05it/s]
Extracting ../data/MNIST/raw/train-labels-idx1-ubyte.gz to ../data/MNIST/raw

Downloading http://yann.lecun.com/exdb/mnist/t10k-images-idx3-ubyte.gz
Downloading http://yann.lecun.com/exdb/mnist/t10k-images-idx3-ubyte.gz to ../data/MNIST/raw/t10k-images-idx3-ubyte.gz
100%|██████████| 1648877/1648877 [00:00<00:00, 39908199.29it/s]Extracting ../data/MNIST/raw/t10k-images-idx3-ubyte.gz to ../data/MNIST/raw

Downloading http://yann.lecun.com/exdb/mnist/t10k-labels-idx1-ubyte.gz
Downloading http://yann.lecun.com/exdb/mnist/t10k-labels-idx1-ubyte.gz to ../data/MNIST/raw/t10k-labels-idx1-ubyte.gz
100%|██████████| 4542/4542 [00:00<00:00, 14688148.63it/s]
Extracting ../data/MNIST/raw/t10k-labels-idx1-ubyte.gz to ../data/MNIST/raw

keyboard_arrow_down Red convolucional


Definición del modelo

class Net(nn.Module):
def __init__(self):
super(Net, self).__init__()
self.conv1 = nn.Conv2d(1, 32, 3, 1)
self.conv2 = nn.Conv2d(32, 64, 3, 1)
self.dropout1 = nn.Dropout(0.25)
self.dropout2 = nn.Dropout(0.5)
self.fc1 = nn.Linear(9216, 128)
self.fc2 = nn.Linear(128, 10)

def forward(self, x):


x = F.relu(self.conv1(x))
x = F.relu(self.conv2(x))
x = F.max_pool2d(x, 2)
x = self.dropout1(x)
x = torch.flatten(x, 1)
x = F.relu(self.fc1(x))
x = self.dropout2(x)
x = self.fc2(x)
return F.log_softmax(x, dim=1)

def train(model, device, train_loader, optimizer, epoch):


model.train()
for batch_idx, (data, target) in enumerate(train_loader):
data, target = data.to(device), target.to(device)
optimizer.zero_grad()
output = model(data)
loss = F.nll_loss(output, target)
loss.backward()
optimizer.step()
if batch_idx % 20 == 0:
print('Train Epoch: {} [{}/{} ({:.0f}%)]\tLoss: {:.6f}'.format(epoch,
batch_idx * len(data), len(train_loader.dataset),
100. * batch_idx / len(train_loader), loss.item()))

def test(model, device, test_loader):


model.eval()
test_loss = 0
correct = 0
with torch.no_grad():
for data, target in test_loader:
data, target = data.to(device), target.to(device)
output = model(data)
test_loss += F.nll_loss(output, target, reduction='sum').item()
pred = output.argmax(dim=1, keepdim=True)
correct += pred.eq(target.view_as(pred)).sum().item()
test_loss /= len(test_loader.dataset)
print('\nTest set: Average loss: {:.4f}, Accuracy: {}/{} ({:.2f}%)\n'.format(test_loss,
correct, len(test_loader.dataset), 100. * correct / len(test_loader.dataset)))

Main execution

model = Net().to(device)
optimizer = torch.optim.Adadelta(model.parameters(), lr=1.0)
scheduler = torch.optim.lr_scheduler.StepLR(optimizer, step_size=1, gamma=0.7)
for epoch in range(5):
train(model, device, train_loader, optimizer, epoch)
test(model, device, test_loader)
scheduler.step()

Train Epoch: 0 [0/60000 (0%)] Loss: 2.302887


Train Epoch: 0 [10000/60000 (17%)] Loss: 0.472961
Train Epoch: 0 [20000/60000 (33%)] Loss: 0.283980
Train Epoch: 0 [30000/60000 (50%)] Loss: 0.240869
Train Epoch: 0 [40000/60000 (67%)] Loss: 0.120151
Train Epoch: 0 [50000/60000 (83%)] Loss: 0.211381

Test set: Average loss: 0.0721, Accuracy: 9776/10000 (97.76%)

Train Epoch: 1 [0/60000 (0%)] Loss: 0.120275


Train Epoch: 1 [10000/60000 (17%)] Loss: 0.105161
Train Epoch: 1 [20000/60000 (33%)] Loss: 0.101851
Train Epoch: 1 [30000/60000 (50%)] Loss: 0.089838
Train Epoch: 1 [40000/60000 (67%)] Loss: 0.095962
Train Epoch: 1 [50000/60000 (83%)] Loss: 0.101294

Test set: Average loss: 0.0473, Accuracy: 9845/10000 (98.45%)

Train Epoch: 2 [0/60000 (0%)] Loss: 0.087204


Train Epoch: 2 [10000/60000 (17%)] Loss: 0.105048
Train Epoch: 2 [20000/60000 (33%)] Loss: 0.060012
Train Epoch: 2 [30000/60000 (50%)] Loss: 0.052956
Train Epoch: 2 [40000/60000 (67%)] Loss: 0.059987
Train Epoch: 2 [50000/60000 (83%)] Loss: 0.117764

Test set: Average loss: 0.0404, Accuracy: 9866/10000 (98.66%)

Train Epoch: 3 [0/60000 (0%)] Loss: 0.098848


Train Epoch: 3 [10000/60000 (17%)] Loss: 0.068928
Train Epoch: 3 [20000/60000 (33%)] Loss: 0.060862
Train Epoch: 3 [30000/60000 (50%)] Loss: 0.049525
Train Epoch: 3 [40000/60000 (67%)] Loss: 0.058006
Train Epoch: 3 [50000/60000 (83%)] Loss: 0.060907

Test set: Average loss: 0.0388, Accuracy: 9862/10000 (98.62%)

Train Epoch: 4 [0/60000 (0%)] Loss: 0.061178


Train Epoch: 4 [10000/60000 (17%)] Loss: 0.062550
Train Epoch: 4 [20000/60000 (33%)] Loss: 0.058064
Train Epoch: 4 [30000/60000 (50%)] Loss: 0.020286
Train Epoch: 4 [40000/60000 (67%)] Loss: 0.046676
Train Epoch: 4 [50000/60000 (83%)] Loss: 0.081748

Test set: Average loss: 0.0363, Accuracy: 9882/10000 (98.82%)

Entrenamiento 1 con 5 epocas

Inicia con 97.43% y termina en 98.79%

Entrenamiento 2 con 20 epocas


Inicia con 97.75% y termina en 99.00%

Entrenamiento 3 con 50 epocas


Inicia con 97.59% y termina en 99.00%, desde la epoca 17 se mantiene en ese valor

keyboard_arrow_down Tarea 1
Entrenar la red convolucional pero esta vez con el conjunto de datos Fashion-MNIST

transform = transforms.Compose([ transforms.ToTensor(), transforms.Normalize((0.5,), (0.3,)) ])

dataset1 = datasets.FashionMNIST('../data', train=True, download=True, transform=transform)


dataset2 = datasets.FashionMNIST('../data', train=False, transform=transform)
train_loader = torch.utils.data.DataLoader(dataset1, **train_kwargs)
test_loader = torch.utils.data.DataLoader(dataset2, **test_kwargs)
# TODO
class Net(nn.Module):
def __init__(self):
super(Net, self).__init__()
self.conv1 = nn.Conv2d(1, 32, 3, 1)
self.conv2 = nn.Conv2d(32, 64, 3, 1)
self.dropout1 = nn.Dropout(0.25)
self.dropout2 = nn.Dropout(0.5)
self.fc1 = nn.Linear(9216, 128)
self.fc2 = nn.Linear(128, 10)

def forward(self, x):


x = F.relu(self.conv1(x))
x = F.relu(self.conv2(x))
x = F.max_pool2d(x, 2)
x = self.dropout1(x)
x = torch.flatten(x, 1)
x = F.relu(self.fc1(x))
x = self.dropout2(x)
x = self.fc2(x)
return F.log_softmax(x, dim=1)

def train(model, device, train_loader, optimizer, epoch):


model.train()
for batch_idx, (data, target) in enumerate(train_loader):
data, target = data.to(device), target.to(device)
optimizer.zero_grad()
output = model(data)
loss = F.nll_loss(output, target)
loss.backward()
optimizer.step()
if batch_idx % 20 == 0:
print('Train Epoch: {} [{}/{} ({:.0f}%)]\tLoss: {:.6f}'.format(epoch,
batch_idx * len(data), len(train_loader.dataset),
100. * batch_idx / len(train_loader), loss.item()))

def test(model, device, test_loader):


model.eval()
test_loss = 0
correct = 0
with torch.no_grad():
for data, target in test_loader:
data, target = data.to(device), target.to(device)
output = model(data)
test_loss += F.nll_loss(output, target, reduction='sum').item()
pred = output.argmax(dim=1, keepdim=True)
correct += pred.eq(target.view_as(pred)).sum().item()
test_loss /= len(test_loader.dataset)
print('\nTest set: Average loss: {:.4f}, Accuracy: {}/{} ({:.2f}%)\n'.format(test_loss,
correct, len(test_loader.dataset), 100. * correct / len(test_loader.dataset)))

keyboard_arrow_down Main Execution


model = Net().to(device)
optimizer = torch.optim.Adadelta(model.parameters(), lr=1.0)
scheduler = torch.optim.lr_scheduler.StepLR(optimizer, step_size=1, gamma=0.7)
for epoch in range(50):
train(model, device, train_loader, optimizer, epoch)
test(model, device, test_loader)
scheduler.step()
Train Epoch: 44 [30000/60000 (50%)] Loss: 0.274733
Train Epoch: 44 [40000/60000 (67%)] Loss: 0.252805
Train Epoch: 44 [50000/60000 (83%)] Loss: 0.238308

Test set: Average loss: 0.2514, Accuracy: 9098/10000 (90.98%)

Train Epoch: 45 [0/60000 (0%)] Loss: 0.247925


Train Epoch: 45 [10000/60000 (17%)] Loss: 0.204291
Train Epoch: 45 [20000/60000 (33%)] Loss: 0.294529
Train Epoch: 45 [30000/60000 (50%)] Loss: 0.259717
Train Epoch: 45 [40000/60000 (67%)] Loss: 0.198837
Train Epoch: 45 [50000/60000 (83%)] Loss: 0.265916

Test set: Average loss: 0.2514, Accuracy: 9098/10000 (90.98%)

Train Epoch: 46 [0/60000 (0%)] Loss: 0.297817


Train Epoch: 46 [10000/60000 (17%)] Loss: 0.304898
Train Epoch: 46 [20000/60000 (33%)] Loss: 0.223439
Train Epoch: 46 [30000/60000 (50%)] Loss: 0.240747
Train Epoch: 46 [40000/60000 (67%)] Loss: 0.250715
Train Epoch: 46 [50000/60000 (83%)] Loss: 0.258945

Test set: Average loss: 0.2514, Accuracy: 9098/10000 (90.98%)

Train Epoch: 47 [0/60000 (0%)] Loss: 0.282230


Train Epoch: 47 [10000/60000 (17%)] Loss: 0.204891
Train Epoch: 47 [20000/60000 (33%)] Loss: 0.288569
Train Epoch: 47 [30000/60000 (50%)] Loss: 0.229663
Train Epoch: 47 [40000/60000 (67%)] Loss: 0.282118
Train Epoch: 47 [50000/60000 (83%)] Loss: 0.233623

Test set: Average loss: 0.2514, Accuracy: 9098/10000 (90.98%)

Train Epoch: 48 [0/60000 (0%)] Loss: 0.229652


Train Epoch: 48 [10000/60000 (17%)] Loss: 0.255917
Train Epoch: 48 [20000/60000 (33%)] Loss: 0.260929
Train Epoch: 48 [30000/60000 (50%)] Loss: 0.270742
Train Epoch: 48 [40000/60000 (67%)] Loss: 0.243654
Train Epoch: 48 [50000/60000 (83%)] Loss: 0.248164

Test set: Average loss: 0.2514, Accuracy: 9098/10000 (90.98%)

Train Epoch: 49 [0/60000 (0%)] Loss: 0.268348


Train Epoch: 49 [10000/60000 (17%)] Loss: 0.291086
Train Epoch: 49 [20000/60000 (33%)] Loss: 0.227961
Train Epoch: 49 [30000/60000 (50%)] Loss: 0.242620
Train Epoch: 49 [40000/60000 (67%)] Loss: 0.261256
Train Epoch: 49 [50000/60000 (83%)] Loss: 0.240179

Test set: Average loss: 0.2514, Accuracy: 9098/10000 (90.98%)

Entrenamiento 1 con 5 epocas Inicia con 82.79% y termina en 90.00%

Entrenamiento 2 con 20 epocas Inicia con 85.14% y termina en 90.97%

Entrenamiento 3 con 50 epocas Inicia con 83.91% y termina en 99.98%, desde la epoca 22 se mantiene en ese valor

Reportar el desempeño alcanzado con la red convolucional en el conjunto de datos Fashion-MNIST

Conclusión : El entrenamiento de esta red neuronal, puedo evidenciar que a mayor entrenamiento se desempeña mejor llegando a valores altos
como 99.98%

keyboard_arrow_down Tarea 2
Entrenar la red convolucional pero esta vez con el conjunto de datos CIFAR10

transform = transforms.Compose([ transforms.ToTensor(), transforms.Normalize((0.5,0.5,0.5), (0.3,0.3,0.3)) ])


dataset1 = datasets.CIFAR10('../data', train=True, download=True, transform=transform)
dataset2 = datasets.CIFAR10('../data', train=False, transform=transform)
train_loader = torch.utils.data.DataLoader(dataset1, **train_kwargs)
test_loader = torch.utils.data.DataLoader(dataset2, **test_kwargs)

from torchvision.utils import make_grid


import matplotlib.pyplot as plt
for images, labels in train_loader:
fig, ax = plt.subplots(figsize=(12, 12))
ax.set_xticks([]); ax.set_yticks([])
ax.imshow(make_grid(images[:64], nrow=8).permute(1, 2, 0))
break

output Downloading https://www.cs.toronto.edu/~kriz/cifar-10-python.tar.gz to ../data/cifar-10-python.tar.gz


100%|██████████| 170498071/170498071 [00:04<00:00, 40368857.19it/s]
Extracting ../data/cifar-10-python.tar.gz to ../data
WARNING:matplotlib.image:Clipping input data to the valid range for imshow with RGB data ([0..1] for floats or [0..255] for integers).
# TODO
class Net(nn.Module):
def __init__(self):
super(Net, self).__init__()
self.conv1 = nn.Conv2d(3, 32, 3, 1)
self.conv2 = nn.Conv2d(32, 64, 3, 1)
self.dropout1 = nn.Dropout(0.25)
self.dropout2 = nn.Dropout(0.5)
self.fc1 = nn.Linear(12544, 128)
self.fc2 = nn.Linear(128, 10)

def forward(self, x):


x = F.relu(self.conv1(x))
x = F.relu(self.conv2(x))
x = F.max_pool2d(x, 2)
x = self.dropout1(x)
x = torch.flatten(x, 1)
x = F.relu(self.fc1(x))
x = self.dropout2(x)
x = self.fc2(x)
return F.log_softmax(x, dim=1)

def train(model, device, train_loader, optimizer, epoch):


model.train()
for batch_idx, (data, target) in enumerate(train_loader):
data, target = data.to(device), target.to(device)
optimizer.zero_grad()
output = model(data)
loss = F.nll_loss(output, target)
loss.backward()
optimizer.step()
if batch_idx % 20 == 0:
print('Train Epoch: {} [{}/{} ({:.0f}%)]\tLoss: {:.6f}'.format(epoch,
batch_idx * len(data), len(train_loader.dataset),
100. * batch_idx / len(train_loader), loss.item()))

def test(model, device, test_loader):


model.eval()
test_loss = 0
correct = 0
with torch.no_grad():
for data, target in test_loader:
data, target = data.to(device), target.to(device)
output = model(data)
test_loss += F.nll_loss(output, target, reduction='sum').item()
pred = output.argmax(dim=1, keepdim=True)
correct += pred.eq(target.view_as(pred)).sum().item()
test_loss /= len(test_loader.dataset)
print('\nTest set: Average loss: {:.4f}, Accuracy: {}/{} ({:.2f}%)\n'.format(test_loss,
correct, len(test_loader.dataset), 100. * correct / len(test_loader.dataset)))

keyboard_arrow_down Execution
model = Net().to(device)
optimizer = torch.optim.Adadelta(model.parameters(), lr=1.0)
scheduler = torch.optim.lr_scheduler.StepLR(optimizer, step_size=1, gamma=0.7)
for epoch in range(5):
train(model, device, train_loader, optimizer, epoch)
test(model, device, test_loader)
scheduler.step()

Train Epoch: 0 [0/50000 (0%)] Loss: 2.318360


Train Epoch: 0 [10000/50000 (20%)] Loss: 2.096740
Train Epoch: 0 [20000/50000 (40%)] Loss: 1.758216
Train Epoch: 0 [30000/50000 (60%)] Loss: 1.719453
Train Epoch: 0 [40000/50000 (80%)] Loss: 1.709096

Test set: Average loss: 1.6616, Accuracy: 4220/10000 (42.20%)


Train Epoch: 1 [0/50000 (0%)] Loss: 1.760314
Train Epoch: 1 [10000/50000 (20%)] Loss: 1.550476
Train Epoch: 1 [20000/50000 (40%)] Loss: 1.445359
Train Epoch: 1 [30000/50000 (60%)] Loss: 1.463669
Train Epoch: 1 [40000/50000 (80%)] Loss: 1.380227

Test set: Average loss: 1.3196, Accuracy: 5411/10000 (54.11%)

Train Epoch: 2 [0/50000 (0%)] Loss: 1.328120


Train Epoch: 2 [10000/50000 (20%)] Loss: 1.367199
Train Epoch: 2 [20000/50000 (40%)] Loss: 1.333000
Train Epoch: 2 [30000/50000 (60%)] Loss: 1.279678
Train Epoch: 2 [40000/50000 (80%)] Loss: 1.285211

Test set: Average loss: 1.1399, Accuracy: 5940/10000 (59.40%)

Train Epoch: 3 [0/50000 (0%)] Loss: 1.179847


Train Epoch: 3 [10000/50000 (20%)] Loss: 1.217231
Train Epoch: 3 [20000/50000 (40%)] Loss: 1.180811
Train Epoch: 3 [30000/50000 (60%)] Loss: 1.109936
Train Epoch: 3 [40000/50000 (80%)] Loss: 1.142391

Test set: Average loss: 1.0717, Accuracy: 6223/10000 (62.23%)

Train Epoch: 4 [0/50000 (0%)] Loss: 1.094708


Train Epoch: 4 [10000/50000 (20%)] Loss: 1.169327
Train Epoch: 4 [20000/50000 (40%)] Loss: 1.099559
Train Epoch: 4 [30000/50000 (60%)] Loss: 1.019827
Train Epoch: 4 [40000/50000 (80%)] Loss: 1.086240

Test set: Average loss: 1.0068, Accuracy: 6468/10000 (64.68%)

Entrenamiento 1 con 5 epocas Inicia con 46.06% y termina en 63.90%

Entrenamiento 2 con 20 epocas Inicia con 45.05% y termina en 67.61%

Entrenamiento 3 con 50 epocas Inicia con 42.57% y termina en 71.98%

Reportar el desempeño alcanzado con la red convolucional en el conjunto de datos CIFAR10.

CONCLUSION: en 3 entrenamientos no alcanzo niveles optimos, pero al estar en una plataforma free, cuando necesitaba entrenar a 100 para
ver el comportamiento me salio que ya no podia seguir entrenando por limites de conexión a GPU. Tambien pude apreciar que desde mi
maquina si se ejecuta pero el tiempo de respuesta es muy lento.

Despues de la ejecución del tercer entrenamiento , solo para presentar el taller volvi a enviar uno de 5 epocas. ya que el anterior se borro .

...

You might also like