Take the Deep Learning — Lecture 2, Set 02: 20 multiple-choice questions (30 minutes) with instant scoring and a full answer review. Covers 20 Deep…
The first 5 questions of this test, with the correct answer and explanation.
Q1. A Dense layer computes output = relu(dot(W, input) + b). Which three operations does that one line contain?
- A. A dot product with W, an addition of the bias b, and the element-wise function relu ✓
- B. A convolution, a pooling step and a normalisation
- C. A reshape, a transpose and a broadcast
- D. A softmax, a cross-entropy and a gradient step
Answer: A. A dot product with W, an addition of the bias b, and the element-wise function relu
Explanation: The line contains a dot product between input and W, an addition of the bias vector b, and the element-wise relu(x) = max(x, 0).
Q2. Which four families of tensor operation are said to describe any network?
- A. Addition, subtraction, multiplication and division
- B. Convolution, pooling, normalisation and dropout
- C. Forward pass, backward pass, update and evaluation
- D. Element-wise operations, broadcasting, the tensor dot, and reshaping ✓
Answer: D. Element-wise operations, broadcasting, the tensor dot, and reshaping
Explanation: Element-wise operations, broadcasting, the tensor dot (matrix product) and reshaping describe any network.
Q3. What defines an element-wise operation?
- A. It combines entries across different axes into weighted sums
- B. The same function is applied to every entry independently, and the output has exactly the shape of the input ✓
- C. It changes the number of entries in the tensor
- D. It can only be applied to 2D tensors
Answer: B. The same function is applied to every entry independently, and the output has exactly the shape of the input
Explanation: The same function is applied to every entry independently, so the output has exactly the shape of the input; relu, sigmoid and tanh are all element-wise.
Q4. What does broadcasting do when a bias b of shape (512,) is added to a batch X of shape (128, 512)?
- A. It raises a shape error
- B. It adds b only to the first row of X
- C. It adds the same 512 biases to every one of the 128 samples ✓
- D. It averages b across the 128 samples first
Answer: C. It adds the same 512 biases to every one of the 128 samples
Explanation: Broadcasting repeats the smaller tensor along new axes, so X[i, :] + b for every row i: the same 512 biases are added to every sample.
Q5. Using the broadcasting rule, which of these additions fails?
- A. (128, 512) + (512,)
- B. (32, 10) + (1, 10)
- C. (60000, 784) + (784,)
- D. (3, 4) + (5,) ✓
Answer: D. (3, 4) + (5,)
Explanation: Comparing from the last axis backwards, each pair of axes must be equal, or one must be 1 or missing. 4 and 5 are neither, so (3, 4) + (5,) is an error.
Take the timed test (free, no sign-up) to see your score and the answer and explanation for every question.
Q6. Why does X + b cost the same as adding two tensors of equal shape?
- A. Because broadcasting never copies data in memory; it is only a rule for how indices are matched
- B. Because NumPy caches the result of the first addition
- C. Because the bias vector is always small enough to ignore
- D. Because the GPU performs the copy for free
Q7. What is the shape rule for the tensor dot product?
- A. Both tensors must have identical shapes
- B. The last axis of the first tensor must equal the first axis of the second; those two axes disappear
- C. The first axis of each tensor must match
- D. The result always has the shape of the larger tensor
Q8. What is the result of [[1, 2], [3, 4]] . [5, 6]?
- A. [5, 12]
- B. [11, 25]
- C. [17, 39]
- D. [8, 10]
Q9. What does the lecture say about the commutativity of the dot product?
- A. A . B always equals B . A
- B. A . B equals B . A only for square matrices
- C. The dot product is commutative but not associative
- D. It is not commutative: A . B is generally not B . A, and B . A may not even be defined
Q10. In the MNIST network, a batch X of shape (128, 784) is multiplied by W of shape (784, 512). What is the result shape, and what does it represent?
- A. (128, 512): 512 weighted sums of the 784 pixels, for each of 128 images
- B. (784, 512): the weights after one update
- C. (128, 784): the batch unchanged
- D. (512, 784): the transposed weight matrix
Q11. Why does the lecture call the dot product the reason GPUs make deep learning fast?
- A. Because GPUs have more memory than CPUs
- B. Because it is the single most expensive operation in a network, and GPUs are built for matrix products
- C. Because only GPUs can compute dot products correctly
- D. Because the dot product cannot be parallelised on a CPU at all
Q12. What stays constant when a tensor is reshaped?
- A. The number of axes
- B. The length of each axis
- C. The count of entries
- D. The data type changes to float32
Q13. Geometrically, what kind of transformation is W . x + b?
- A. A non-linear fold
- B. A random permutation
- C. A probability distribution
- D. An affine transformation: a linear map plus a translation
Q14. What does relu do geometrically, and why is it singled out?
- A. It rotates space, and is the only step that preserves distance
- B. It folds the negative half of every axis onto zero, and is the only step that bends space
- C. It scales space uniformly, and is the only step that changes the origin
- D. It projects space onto a lower dimension, and is the only step that loses information
Q15. How many trainable parameters does the two-layer MNIST network have in total?
- A. 401,920
- B. 5,130
- C. 407,050
- D. 784,512
Q16. Substituting h = W1 x + b1 into y = W2 h + b2 gives y = (W2 W1) x + (W2 b1 + b2). What does this prove?
- A. That two layers are always better than one
- B. That the bias terms can be removed without loss
- C. That the weights must be initialised to zero
- D. That any number of stacked linear layers has exactly the expressive power of a single one
Q17. What kind of decision boundary can an affine function draw, and what can it NOT represent?
- A. Only flat boundaries such as lines, planes and hyperplanes; it cannot represent XOR or a circle
- B. Only curved boundaries; it cannot represent a straight line
- C. Any boundary at all, given enough layers
- D. Only boundaries that pass through the origin
Q18. What does the universal approximation result say, and what role does depth play?
- A. Depth alone guarantees any function can be learned; width is irrelevant
- B. One hidden layer with enough units can approximate any continuous function on a bounded region; width makes it possible, depth makes it efficient
- C. Only infinitely deep networks can approximate continuous functions
- D. Approximation is only possible with sigmoid units, not relu
Q19. What is the derivative of ReLU, and why does this matter for deep networks?
- A. It is always 0, so the network never changes
- B. It is sigma(x)(1 - sigma(x)), which keeps gradients bounded
- C. It is 1 if x > 0 and 0 otherwise, so the gradient does not shrink for positive inputs
- D. It is 1 - tanh squared, which centres the output at zero
Q20. For scores z = (2.0, 1.0, 0.1), softmax gives approximately (0.66, 0.24, 0.10). What property means softmax(z + c) = softmax(z)?
- A. Only the differences between scores matter, so shifting all scores by the same constant changes nothing
- B. The exponential function is linear for small inputs
- C. The probabilities are rounded to two decimal places
- D. The largest score is always mapped to exactly 1