AI by Hand ✍

25 Exercise Workbook Series

1. Dot Product
2. Matrix Multiplication
3. Linear Layer
4. Activation
5. Artificial Neuron
6. Batch
7. Connection
8. Hidden Layer
9. Deep
10.Wide
11.Softmax
12.Gradient

More to come …

http://by-hand.ai/workbook
© 2024 Tom Yeh
 Gradient
Exercise 1
dW = 0.1

X X
2 2
W 3 6 Z = W⋅X W 3.1 6.2 Z

dZ =

dZ
= = = X
dW 0.1

AI by Hand ✍© 2024 Tom Yeh 0.2 2

 Gradient
Exercise 2
dW = 0.1

X X
3 3
W 3 9 Z = W⋅X W 3.1 9.3 Z

dZ =

dZ
= = = X
dW 0.1

AI by Hand ✍© 2024 Tom Yeh 0.3 3

 Gradient
Exercise 3
dW = 0.1

X X
3 3
W 2 6 Z = W⋅X W 2.1 Z

dZ =

dZ
= = 3 = X
dW 0.1

AI by Hand ✍© 2024 Tom Yeh 0.3 6.3

 Gradient
Exercise 4
dW = 0.1

X X
4 4
W 2 8 Z = W⋅X W 2.1 Z

dZ = 0.4

dZ 0.4
= = = X
dW 0.1

AI by Hand ✍© 2024 Tom Yeh 8.4 4

 Gradient
Exercise 5
dW = 0.1

X X

W 2 * Z = W⋅X W 2.1 * Z

dZ = 0.6

dZ 0.6
= = = X
dW 0.1

AI by Hand ✍© 2024 Tom Yeh 6

 Gradient
Exercise 6
dW = 0.1

X X
9 9
W 2 * Z = W⋅X W 2.1 * Z

dZ =

dZ
= = = X
dW 0.1

AI by Hand ✍© 2024 Tom Yeh 0.9 9

 Gradient
Exercise 7
dW = -0.1

X X
2 2
W 3 6 Z = W⋅X W 2.9 5.8 Z

dZ =

dZ
= = = X
dW -0.1

AI by Hand ✍© 2024 Tom Yeh -0.2 2

 Gradient
Exercise 8
dW = 0.2

X X
2 2
W 3 6 Z = W⋅X W 3.2 6.4 Z

dZ =

dZ
= = = X
dW 0.2

AI by Hand ✍© 2024 Tom Yeh 0.4 2

 Gradient
Exercise 9
dW = 2
X X
2 2
W 3 6 Z = W⋅X W 5 10 Z

dZ =

dZ
= = = X
dW
2

AI by Hand ✍© 2024 Tom Yeh 4 2

 Gradient
Exercise 10
dX = 0.1

X X
2 2.1

W 3 6 Z = W⋅X W 3 6.3 Z

dZ =

dZ
= = = W
dX 0.1

AI by Hand ✍© 2024 Tom Yeh 0.3 3

 Gradient
Exercise 11
dX = 1

X X
2 3
W 3 6 Z = W⋅X W 3 Z

dZ =

dZ
= = 3 = W
dX
1

AI by Hand ✍© 2024 Tom Yeh 9 3

 Gradient
Exercise 12
dX = 0.1

X X
2 2.1

W 4 * Z = W⋅X W 4 * Z

dZ =

dZ
= = = W
dX 0.1

AI by Hand ✍© 2024 Tom Yeh 0.4 4

 Gradient
Exercise 13
dX = -2

X X
2 0
W 4 * Z = W⋅X W 4 * Z

dZ =

dZ
= = = W
dX
-2

AI by Hand ✍© 2024 Tom Yeh -8 4

 Gradient
Exercise 14
dW = 0.1

X X
2 2
ReLU ReLU
W 3 6 ≈ 6 W 3.1 6.2 ≈
Z A Z A

dZ = 0.2

dA =

AI by Hand ✍© 2024 Tom Yeh 6.2 0.2

 Gradient
Exercise 15
dW = 0.1

X X
2 2
ReLU ReLU
W 3 6 ≈ 6 W 3.1 6.2 ≈ 6.2
Z A Z A

dZ = 0.2

dA =

dA
= =
dZ 0.2

AI by Hand ✍© 2024 Tom Yeh 0.2 1

 Gradient
Exercise 16
dW = 0.1

X X
-2 -2
ReLU ReLU
W 3 -6 ≈ 0 W 3.1 -6.2 ≈
Z A Z A

dZ = 0.2

dA =

AI by Hand ✍© 2024 Tom Yeh 0 0

 Gradient
Exercise 17
dW = 0.1

X X
-2 -2
ReLU ReLU
W 3 -6 ≈ 0 W 3.1 -6.2 ≈ 0
Z A Z A

dZ = 0.2

dA =

dA
= =
dZ 0.2

AI by Hand ✍© 2024 Tom Yeh 0 0

 Gradient
Exercise 18
dW = 0.1

X X
4 4
ReLU ReLU
W 3 12 ≈ 12 W 3.1 12.4 ≈ 12.4

Z A Z A

dZ = 0.4

dA =

dA
= =
dZ 0.4

AI by Hand ✍© 2024 Tom Yeh 0.4 1

 Gradient
Exercise 19
dW = 0.1

X X
5 5
ReLU ReLU
W 3 15 ≈ * W 3.1 15.5 ≈ *
Z A Z A

dZ = 0.5

dA =

dA
= =
dZ 0.5

AI by Hand ✍© 2024 Tom Yeh 0.5 1

 Gradient
Exercise 20
dW = 0.1

X X
9 9
ReLU ReLU
W 3 27 ≈ * W 3.1 * ≈ *
Z A Z A

dZ = *

dA = *

dA *
= =
dZ
*

AI by Hand ✍© 2024 Tom Yeh 1

 Gradient
Exercise 21
dW = 0.1

X X
-1 -1
ReLU ReLU
W 3 -3 ≈ * W 3.1 * ≈ *
Z A Z A

dZ = *

dA = *

dA *
= =
dZ
*

AI by Hand ✍© 2024 Tom Yeh 0

 Gradient
Exercise 22
dW = 0.1

X X
4 4
ReLU ReLU
W 3 12 ≈ 12 W 3.1 12.4 ≈ 12.4

Z A Z A

dZ =

dA = 0.4
dA 0.4
= = 1
dZ
dA 0.4
= =
dW
dZ 0.1
= = 4
dW 0.1

AI by Hand ✍© 2024 Tom Yeh 0.4 4

 Gradient
Exercise 23
dW = 0.1

X X
-2 -2
ReLU ReLU
W 3 -6 ≈ 0 W 3.1 -6.2 ≈ 0
Z A Z A

dZ = -0.2

dA =
dA
= = 0
dZ -0.2 dA
= =
dW
dZ -0.2 0.1
= = -2
dW 0.1

AI by Hand ✍© 2024 Tom Yeh 0 0

 Gradient
Exercise 24
dX = 0.1

X X
4 4.1
ReLU ReLU
W 3 12 ≈ 12 W 3 12.3 ≈ 12.3

Z A Z A

dZ =

dA = 0.3
dA 0.3
= = 1
dZ
dA 0.3
= =
dX
dZ 0.1
= = 3
dX 0.1

AI by Hand ✍© 2024 Tom Yeh 0.3 3

 Gradient
Exercise 25
dX = 0.1

X X
3 3.1
ReLU ReLU
W -2 -6 ≈ 0 W -2 -6.2 ≈ 0
Z A Z A

dZ = -0.2

dA =
dA
= = 0
dZ -0.2 dA
= =
dX
dZ -0.2 0.1
= = -2
dX 0.1

AI by Hand ✍© 2024 Tom Yeh 0 0