AIN SHAMS UNIVERSITY FACULTY OF ENGINEERING Computer Engineering and Software Systems Program Fall, 2023/2024 Course Code: CSE 485 Time allowed: 1 Hr. Deep Learning The Exam Consists of Three Questions in Two Pages. Maximum Marks: 20Marks 1/2 Important Rules Wala olay + Having a mobile, Smart Watch or earphones inside the examination | 41, (5341 datas «461M clalall pant) fue © halls forbidden and is consideved as a cheating behavior. | eaten * Its forbidden to have any references, notes, books, or any other | materials even if it is not related to the exam content with you in the | Rall ots lal sl dle sl iS gl Ipbuy maa © all agin sad Ule pia ln dig examination hall. EVE pes Ulally Question (1): [10 marks] (1) What is the effect of L,Regulation on the weights ? (2) Why is the activation function of crucial importance to the power of neural networks? (3) What is the problem of using sigmoid and tanh activation functions in deep networks? (4) (true/false) Drop-out is used to reduce the number of parameters of a network. Justify your answer. (5) Which operation is a non-linear one during training and is consideraed a linear one during testing? (6) Which concepts does ADAM optimizer combine that makes the training more efficient? (7) (True/False) Stochastic gradient descent will typically provide a more accurate estimate of the gradient of a loss function than the full gradient calculated over all examples. (8) How GoogLeNet network makes the computation more faster? (9) What is the advatange of using Bottleneck Residual Block rather than using basic Residual basic Block in Residual networks. (10) What is the advantage of using Cross-Entropy loss function versus SVM Loss function?AIN SHAMS UNIVERSITY, FACULTY OF ENGINEERING Computer Engineering and Software Systems Program, COMPUTER AND SYSTEMS ENGINEERING Fall, 2022/2023, Course Code: CSE 485 Time Allowed: 1 Hr. Deep Learning The Exam Consists of Three Questions in Two Pages. 2/2 Question (2): [5 marks] Consider a neural net for a binary classification which has one hidden layer as shown in the figure. We use a linear activation function h(z) = cz at hidden units and a sigmoid activation function g(z) = ed <== at the output unit to learn the function for Ply=1|x, w) where x = (xy,x2)? and w = (Wy,We,..-,Wa)?- a) What is the output P(y = 1 | x, w) from the above neural net? Express it in terms of x; ,c and weights w; .What is the final classification boundary? b) Draw a neural net with no hidden layer which is equivalent to the given neural net, and write weights w~ of this new neural net in terms of c and w; - c) Is it true that any multi-layered neural net with linear activation functions at hidden layers can be represented as a neural net without any hidden layer? Briefly explain your answer Question (3): [5 marks] (a) Ina convolutional neural network, the image is of dimension 100x100 and one of the learned filters is of dimension 10x10 with a stride of 5, What is the dimension of the resulting feature map? (b) Assume we have an error function E and modify our cost function C by adding an L2-weight penalty, or specifically C = E +4) a} Whatis the value of w at which the cost function is minimized? (c) Derive the gradient with regard to the inputs of a softmax function when cross entropy loss is used for evaluation, i.e. find the gradients with respect to the softmax input vector, 6 , when the prediction is made by 9 = softmax(é). Remember the cross entropy function is C(Y, y)=— Lid log %, where y is the one-hot label vector, and ¥ is the predicted probability vector for all classes. END of Exam, Good Luck Examination Committee: Prof. Mahmoud Khalil Exam. Date : 14"*of November, 2022