Take the Deep Learning — Lecture 2, Set 03: 20 multiple-choice questions (30 minutes) with instant scoring and a full answer review. Covers 20 Deep…
The first 5 questions of this test, with the correct answer and explanation.
Q1. What is the loss, in one sentence?
- A. A single number measuring the error: zero when the prediction matches the label, larger the further off it is ✓
- B. The number of images the network got wrong
- C. The time taken to complete one epoch
- D. The difference between training and test accuracy
Answer: A. A single number measuring the error: zero when the prediction matches the label, larger the further off it is
Explanation: The loss is a single number: zero when the prediction matches the label and larger the further off it is. Training becomes one task: make this number as small as possible.
Q2. In the foggy-hillside analogy, what do height, position and slope correspond to?
- A. Height is the accuracy, position is the dataset, slope is the batch size
- B. Height is the loss, position is the weights, slope is the gradient ✓
- C. Height is the learning rate, position is the epoch, slope is the loss
- D. Height is the gradient, position is the loss, slope is the weights
Answer: B. Height is the loss, position is the weights, slope is the gradient
Explanation: Height is the loss, your position is the weights, and the slope you can feel is the gradient.
Q3. Which loss is used with a softmax output over K classes?
- A. Mean squared error
- B. Binary cross-entropy
- C. Hinge loss
- D. Categorical cross-entropy ✓
Answer: D. Categorical cross-entropy
Explanation: Categorical cross-entropy, L = -sum over k of y_k log yhat_k, pairs with a softmax output for K classes.
Q4. With one-hot labels, the categorical cross-entropy sum collapses to what?
- A. The mean of all 10 predicted probabilities
- B. The negative log of the probability the network gave to the correct digit ✓
- C. The squared difference between prediction and label
- D. The number of incorrectly classified digits
Answer: B. The negative log of the probability the network gave to the correct digit
Explanation: Only the true class has y = 1, so the sum collapses to L = -log yhat for the correct class.
Q5. If the true digit is 3, a predicted probability of 0.9 gives a loss of 0.105 and a probability of 0.1 gives 2.303. What does this illustrate?
- A. That the loss is always between 0 and 1
- B. That accuracy and loss always move together
- C. That confident mistakes are punished about 20 times harder ✓
- D. That the loss is unaffected by how confident the network is
Answer: C. That confident mistakes are punished about 20 times harder
Explanation: -log 0.9 = 0.105 and -log 0.1 = 2.303, so confident mistakes are punished roughly 20 times harder.
Take the timed test (free, no sign-up) to see your score and the answer and explanation for every question.
Q6. Why is accuracy only reported rather than optimised directly?
- A. It is too slow to compute during training
- B. It is not differentiable, so it cannot be optimised directly
- C. It is only meaningful on the test set
- D. It cannot be computed for multi-class problems
Q7. Why is squared error a poor loss for classification with softmax outputs?
- A. It produces negative loss values
- B. It cannot be used with one-hot labels
- C. It is more expensive to compute than cross-entropy
- D. Its gradient is near zero exactly when the prediction is confidently wrong, while cross-entropy's gradient stays large there
Q8. The lecture says cross-entropy is maximum likelihood in disguise. What does minimising it mean?
- A. Choosing the weights under which the observed labels were most likely
- B. Choosing the weights that give the smallest possible network
- C. Choosing the weights that maximise the training accuracy directly
- D. Choosing the weights that make all probabilities equal
Q9. For softmax with cross-entropy, what is the derivative of the loss with respect to the raw scores z?
- A. The learning rate times the weights
- B. Prediction minus target, that is yhat - y
- C. Target minus prediction squared
- D. The transpose of the weight matrix
Q10. What does the gradient of the loss tell us, and what shape does it have?
- A. It gives the training accuracy, and is a single number
- B. It gives the batch size, and is a vector of length 128
- C. It points in the direction in which the loss increases fastest, and has the same shape as W
- D. It points towards the global minimum directly, and has the shape of the input
Q11. What is the gradient-descent update rule, and what is eta?
- A. W becomes W + eta times the gradient; eta is the momentum
- B. W becomes W times eta; eta is the decay factor
- C. W becomes W minus the gradient; eta is not used
- D. W becomes W minus eta times the gradient of L; eta is the learning rate
Q12. What happens if the learning rate is too large?
- A. Training crawls and takes far too many epochs
- B. Each step overshoots the valley floor and the loss grows or oscillates
- C. The gradient becomes exactly zero
- D. The network stops using the bias terms
Q13. In mini-batch SGD with batch_size=128 on MNIST, what is computed and how many batches make one epoch?
- A. The exact gradient on all 60,000 images; one batch per epoch
- B. A noisy gradient estimate on a random batch; 469 batches make one epoch
- C. The gradient on a single image; 60,000 batches make one epoch
- D. The gradient on the test set; 10,000 batches make one epoch
Q14. What does momentum do to the update?
- A. It shrinks the learning rate after every epoch
- B. It averages the weights of the last few epochs
- C. It keeps a running velocity, so steps accumulate where the gradient is consistent and shrink where it changes sign
- D. It randomly resets weights that stop changing
Q15. What do RMSprop and Adam do in addition to momentum?
- A. They divide the learning rate, per parameter, by a running average of recent gradient magnitudes
- B. They increase the batch size automatically
- C. They remove the bias terms from every layer
- D. They compute the exact second derivative of the loss
Q16. In the hand-worked example (w = 0.5, b = 0, x = 2, y = 3, eta = 0.05), the forward pass gives yhat = 1.0 and L = 4.0. After one update, what are w and b?
- A. w = 0.1 and b = -0.2
- B. w = 0.5 and b = 0.05
- C. w = 0.9 and b = 0.2
- D. w = 1.3 and b = 0.4
Q17. In the same example, what does the loss become after that single update, and what does using eta = 0.5 instead show?
- A. The loss becomes 0; a larger eta would simply converge faster
- B. The loss becomes 2.0; a larger eta has no effect in this example
- C. The loss stays at 4.0; the learning rate only matters for deep networks
- D. The loss falls from 4.0 to 1.0; with eta = 0.5 the step overshoots to yhat = 11 and L = 64
Q18. Reading the backpropagation formulas, what is the gradient of a weight matrix always equal to?
- A. The learning rate times the weight matrix itself
- B. The error signal of its output times its input, transposed
- C. The sum of all activations in the layer
- D. The derivative of the activation function alone
Q19. Why can weights not all start at zero, and what scale is recommended for the first MNIST layer?
- A. Zero weights make the loss undefined; the scale should be 1/784
- B. Zero weights overflow the gradient; the scale should be 1.0
- C. Every unit would stay identical forever; with fan-in 784, the standard deviation should be about 1/sqrt(784) = 0.036
- D. Zero weights prevent the bias from updating; the scale should be 0.5
Q20. What happens during dropout, and what happens at test time?
- A. Every hidden unit is dropped with probability p during training and survivors are scaled by 1/(1-p); at test time nothing is dropped
- B. Units are dropped at test time only, to speed up inference
- C. Weights are set to zero permanently once they stop changing
- D. Half the training images are discarded at every epoch