Scribd
Upload
35 views

Tensor Board For Model Debugging and Visualization

Using TensorBoard Debugger 2, developers can track NaN values in TensorFlow models during training to help diagnose issues. The debugger allows viewing tensor values and gradients to identify where NaNs are being introduced. To invoke it, use tf.debugging.experimental.enable_dump_debug_info() when running the model. This will log debugging information to a logdir that can then be analyzed in TensorBoard's Debugger 2 dashboard.

Uploaded by

anhtu12st
Copyright
© © All Rights Reserved
Available Formats
Download as DOCX, PDF, TXT or read online on Scribd
35 views

Tensor Board For Model Debugging and Visualization

Using TensorBoard Debugger 2, developers can track NaN values in TensorFlow models during training to help diagnose issues. The debugger allows viewing tensor values and gradients to identify where NaNs are being introduced. To invoke it, use tf.debugging.experimental.enable_dump_debug_info() when running the model. This will log debugging information to a logdir that can then be analyzed in TensorBoard's Debugger 2 dashboard.

Uploaded by

anhtu12st
Copyright
© © All Rights Reserved
Available Formats
Download as DOCX, PDF, TXT or read online on Scribd
You are on page 1/ 5

1.

Using TensorBoard Debugger


During model training using Tensorflow, events which involve NANs can affect the training
process leading to the non-improvement in model accuracy in subsequent steps. TensorBoard
2.3+ (together with TensorFlow 2.3+) provides a debugging tool known as Debugger 2. This
tool will help track NANs in a Neural Network written in Tensorflow.

Here’s an example provided in the TensorFlow Github which involves training the mnist
dataset and capturing NANs then analyzing our results on TensorBoard debugger 2. 

First, Navigate to project root on your terminal and run the below command.

python -m tensorflow.python.debug.examps.v2.debug_mnist_v2 --dump_dir


/tmp/tfdbg2_logdir --dump_tensor_debug_mode FULL_HEALTH
Copy
Once the above command runs, you will notice non-improvement in model accuracy displayed
on the terminal. This is caused by NANs, you can launch a TensorBoard debugger 2 by the
following command. 

tensorboard --logdir /tmp/tfdbg2_logdir


Copy
Output

To invoke the debugger on your model,


use tf.debugging.experimental.enable_dump_debug_info()  which is the API entry point of
Debugger V2. It takes parameters:

1. logdir which is log directory


2. tensor_debug_mode – controls information debugger extract from each aeger or in-
graph tensor
3. circular_buffer_size – controls number of tensor events to be saved. Default is 1000 and
unsetting it, use value -1
tf.debugging.experimental.enable_dump_debug_info(
logdir,
tensor_debug_mode="FULL_HEALTH",
circular_buffer_size=-1)
Copy
If you don’t have NANs in your model, you will see a blank Debugger 2 on your TensorBoard
dashboard.

2.How to display image data in TensorBoard


If you want to visualize layer weights, generated tensors or input data, TensorFlow Image
Summary API helps you view them on TensorBoard Images.

Visualizing a single image in TensorBoard


The shape of a single image in our data set is (28,28) known as a rank-2 tensor, this represents
the height and width of the image. You can examine this by running the following code

print(X_train[0].shape)
Copy
Since we are going to use `tf.summary.image()` which expects  rank-4 tensor, we have to
reshape using the `numpy` reshape method. Our output should contain (batch_size, height,
width, channels) .

Since we are logging a single image and our image is grayscale, we are setting both batch_size
and ‘channels’ values as 1.

First, let’s delete old logs and create a file writer.

rm -rf ./logs/
logdir = "logs/single-image/"
file_writer = tf.summary.create_file_writer(logdir)
Copy
Next, log the image to TensorBoard

import numpy as np

with file_writer.as_default():
image = np.reshape(X_train[4], (-1, 28, 28, 1))
tf.summary.image("Single Image", image, step=0)
Copy
Launch tensorboard

tensorboard --logdir logs/single-image


Copy
Output 

Visualizing multiple images in TensorBoard


You can visualize multiple images, by setting a range as follows:

import numpy as np

with file_writer.as_default():
images = np.reshape(X_train[5:20], (-1, 28, 28, 1))
tf.summary.image("Multiple Digits", images, max_outputs=16,
step=0)
Copy
Visualizing actual images in TensorBoard
From the above two examples, you have been visualizing mnist tensors. However, using
`matplotlib`, you can visualize the actual images by logging them in TensorBoard.

Clear previous logs

rm -rf logs
Copy
Import `matplotlib` library and create class names and initiate ‘tf.summary.create_file_writer’.

import io
import matplotlib.pyplot as plt

class_names =
['Zero','One','Two','Three','Four','Five','Six','Seven','Eight','Nine
']
logdir = "logs/actual-images/"
file_writer = tf.summary.create_file_writer(logdir)
Copy
Write a function that will create grid of mnist images

def image_grid():
figure = plt.figure(figsize=(12,8))

for i in range(25):
plt.subplot(5, 5, i + 1)
plt.xlabel(class_names[y_train[i]])
plt.xticks([])
plt.yticks([])
plt.grid(False)
plt.imshow(X_train[i], cmap=plt.cm.coolwarm)

return figure
Copy
Next, let’s write a function that will convert this 5X5 grid into a single image that will be
logged in to Tensorboard logs

def plot_to_image(figure):
buf = io.BytesIO()
plt.savefig(buf, format='png')
plt.close(figure)
buf.seek(0)
image = tf.image.decode_png(buf.getvalue(), channels=4)
image = tf.expand_dims(image, 0)
return image
Copy
Launch Tensorboard

tensorboard --logdir logs/actual-images


Copy

You might also like