Take the Deep Learning — Lecture 2, Set 05: 20 multiple-choice questions (30 minutes) with instant scoring and a full answer review. Covers 20 Deep…
The first 5 questions of this test, with the correct answer and explanation.
Q1. Which loss is used for regression, and what is its formula?
- A. Mean squared error, the average of (y - yhat) squared ✓
- B. Binary cross-entropy, minus y log yhat minus (1-y) log(1-yhat)
- C. Categorical cross-entropy, minus the sum of y_k log yhat_k
- D. Hinge loss, the maximum of 0 and 1 minus y times yhat
Answer: A. Mean squared error, the average of (y - yhat) squared
Explanation: Mean squared error for regression is L = (1/n) times the sum of (y_i - yhat_i) squared.
Q2. Which loss pairs with a sigmoid output for two classes?
- A. Mean squared error
- B. Binary cross-entropy ✓
- C. Categorical cross-entropy
- D. Mean absolute error
Answer: B. Binary cross-entropy
Explanation: Binary cross-entropy, L = -[y log yhat + (1-y) log(1-yhat)], is used for two classes with a sigmoid output.
Q3. What exactly is the loss number Keras prints during training, such as loss: 0.2524?
- A. The loss on the single hardest sample in the batch
- B. The cumulative loss since training began
- C. The mean loss over the samples in the batch ✓
- D. The loss on the 10,000 test images
Answer: C. The mean loss over the samples in the batch
Explanation: The batch loss is the mean over its samples; this is the number Keras prints.
Q4. For yhat = (0.66, 0.24, 0.10) and a one-hot target y = (1, 0, 0), what is the error signal dL/dz, and what is notable about it?
- A. (0.34, -0.24, -0.10), and it sums to zero
- B. (0.66, 0.24, 0.10), and it sums to one
- C. (1, 0, 0), and it equals the target
- D. (-0.34, 0.24, 0.10), and it sums to zero ✓
Answer: D. (-0.34, 0.24, 0.10), and it sums to zero
Explanation: dL/dz = yhat - y = (-0.34, 0.24, 0.10): the correct score is pushed up, the others down in proportion to their wrongly assigned probability, and the signal sums to zero.
Q5. What is the equivalent result for sigmoid paired with binary cross-entropy?
- A. dL/dz = sigma(z) - y ✓
- B. dL/dz = y - sigma(z) squared
- C. dL/dz = sigma(z) times (1 - sigma(z))
- D. dL/dz = log sigma(z) - y
Answer: A. dL/dz = sigma(z) - y
Explanation: The same holds for sigmoid with binary cross-entropy: dL/dz = sigma(z) - y.
Take the timed test (free, no sign-up) to see your score and the answer and explanation for every question.
Q6. The derivative is introduced as f(x + epsilon) is approximately f(x) + a times epsilon. What is a, and what does it let you do?
- A. a is the learning rate, and it sets the step size directly
- B. a is the slope f'(x); to lower a function, step against its derivative
- C. a is the loss value, and it tells you when to stop
- D. a is the number of parameters, and it sets the batch size
Q7. What does the partial derivative of the loss with respect to one weight mean?
- A. The derivative taken after all other weights are set to zero
- B. The average derivative across all weights in the layer
- C. The derivative with respect to that one weight, treating all the others as constants
- D. The second derivative of the loss at that weight
Q8. For L(w1, w2) = w1 squared + 3 w2, what is the gradient at the point (1, 0)?
- A. (1, 3)
- B. (2, 3)
- C. (2, 0)
- D. (0, 3)
Q9. The lecture notes that L(W - epsilon times grad L) is approximately L(W) minus epsilon times the squared norm of the gradient. What does this justify?
- A. That the loss always reaches zero eventually
- B. That the gradient must be recomputed only once per epoch
- C. That momentum is unnecessary for small networks
- D. That the step lowers the loss whenever the gradient is non-zero, but only for small epsilon, which is why the learning rate must be small
Q10. What is the chain rule, as stated in the lecture?
- A. If y = f(u) and u = g(x), then dy/dx = (dy/du) times (du/dx)
- B. If y = f(u) and u = g(x), then dy/dx = dy/du plus du/dx
- C. If y = f(u) and u = g(x), then dy/dx = f'(x) times g'(x)
- D. If y = f(u) and u = g(x), then dy/dx = (du/dy) times (dx/du)
Q11. What is the correct chain of computations in the two-layer MNIST network?
- A. x to h to z1 to z2 to yhat to L
- B. x to z1 to h = relu(z1) to z2 to yhat = softmax(z2) to L
- C. x to yhat to z2 to h to z1 to L
- D. x to L to z1 to h to z2 to yhat
Q12. What is the formula for the error signal at the hidden layer, delta-1?
- A. delta-1 = yhat - y
- B. delta-1 = W2 times delta-2
- C. delta-1 = (W2 transposed times delta-2) element-wise multiplied by the derivative of relu at z1
- D. delta-1 = relu(z1) times x transposed
Q13. What happens when the error signal is passed back through a layer whose activation derivative is zero?
- A. The signal is amplified, causing exploding gradients
- B. The signal is blocked, which is the vanishing-gradient problem
- C. The weights of that layer are reinitialised automatically
- D. The forward pass is recomputed from the start
Q14. Why is the backward pass roughly as cheap as the forward pass?
- A. Because it only updates the last layer
- B. Because it uses a smaller batch size
- C. Because the gradients are approximated rather than computed exactly
- D. Because each layer needs only its input, its output, and the error signal from the layer after it
Q15. What do frameworks do with the backward pass, and where does this run?
- A. They compute it automatically (automatic differentiation); this is what runs inside network.fit()
- B. They require the user to supply gradient formulas by hand
- C. They approximate it numerically by perturbing each weight
- D. They skip it entirely and use random search instead
Q16. In step 1 of the training loop, how are the weights and biases initialised?
- A. Both weights and biases at zero
- B. Every W with small random numbers so units start out different, and every b with zeros
- C. Both weights and biases with large random numbers
- D. Weights copied from a pretrained network, biases at one
Q17. What happens to the training set at every epoch?
- A. It is augmented with new synthetic images
- B. It is reduced to the hardest examples only
- C. It is shuffled and cut into batches of 128 images
- D. It is merged with the test set for better coverage
Q18. Over how many updates does the MNIST network go from about 10% to about 98% test accuracy?
- A. 469 updates
- B. 2,345 updates
- C. 60,000 updates
- D. 407,050 updates
Q19. What distinguishes overfitting from underfitting, and what handles both?
- A. Overfitting means both losses stay high, underfitting means training loss falls; dropout handles both
- B. Overfitting means the test set is too small, underfitting means the model is too large; more data handles both
- C. Overfitting means training is too slow, underfitting means it is too fast; changing the learning rate handles both
- D. In overfitting training loss falls while validation loss rises; in underfitting both stay high; early stopping handles both
Q20. What does L2 regularisation, also called weight decay, do to each update, and what value of lambda is suggested as a start?
- A. It shrinks every weight by a factor before the usual gradient step; lambda around 0.001
- B. It sets the smallest weights to zero after each step; lambda around 0.5
- C. It adds random noise to every weight; lambda around 0.1
- D. It freezes the weights of the first layer; lambda is not used