Take the Research Methodology — How to Read a Paper (Keshav): 30 multiple-choice questions (40 minutes) with instant scoring and a full answer review.…
The first 5 questions of this test, with the correct answer and explanation.
Q1. Who is the author of "How to Read a Paper", and with which institution is he affiliated?
- A. S. Keshav, David R. Cheriton School of Computer Science, University of Waterloo ✓
- B. Timothy Roscoe, ETH Zurich
- C. Simon Peyton Jones, Microsoft Research
- D. George Whitesides, Harvard University
Answer: A. S. Keshav, David R. Cheriton School of Computer Science, University of Waterloo
Explanation: The paper is by S. Keshav of the David R. Cheriton School of Computer Science, University of Waterloo, Canada.
Q2. According to the abstract, what problem motivates this article?
- A. Research papers are too long to publish
- B. Reading is a skill that is rarely taught, leading to much wasted effort ✓
- C. Journals charge too much for access
- D. Too few papers are published each year
Answer: B. Reading is a skill that is rarely taught, leading to much wasted effort
Explanation: The abstract notes that researchers spend a great deal of time reading papers, but the skill is rarely taught, leading to much wasted effort.
Q3. Which reasons does Keshav give for why researchers must read papers?
- A. To increase their citation count and h-index
- B. To fulfil a university attendance requirement
- C. To review them for a conference or class, to keep current in their field, or for a literature survey of a new field ✓
- D. To find spelling mistakes made by other authors
Answer: C. To review them for a conference or class, to keep current in their field, or for a literature survey of a new field
Explanation: The introduction lists reviewing for a conference or class, keeping current in the field, and doing a literature survey of a new field.
Q4. How much time does a typical researcher spend reading papers, according to the introduction?
- A. A few hours every year
- B. About ten hours every year
- C. One full working month every year
- D. Hundreds of hours every year ✓
Answer: D. Hundreds of hours every year
Explanation: The introduction states that a typical researcher will likely spend hundreds of hours every year reading papers.
Q5. What is the key idea of the three-pass approach?
- A. Read the paper three times from beginning to end
- B. Read the paper in up to three passes, instead of starting at the beginning and plowing to the end ✓
- C. Read only the abstract three separate times
- D. Divide the paper among three readers
Answer: B. Read the paper in up to three passes, instead of starting at the beginning and plowing to the end
Explanation: The key idea is reading in up to three passes rather than reading straight through from start to finish.
Take the timed test (free, no sign-up) to see your score and the answer and explanation for every question.
Q6. What does each of the three passes accomplish?
- A. All three passes give the same general overview at increasing speed
- B. First gives the details, second the general idea, third the references
- C. First gives a general idea, second lets you grasp the content but not the details, third helps you understand the paper in depth
- D. First checks grammar, second checks figures, third checks citations
Q7. How long should the first pass take?
- A. About five to ten minutes
- B. About thirty minutes
- C. Up to one hour
- D. Four to five hours
Q8. Which of the following is NOT one of the four steps of the first pass?
- A. Carefully read the title, abstract and introduction
- B. Read the section and sub-section headings
- C. Study the proofs and experimental methodology in detail
- D. Read the conclusions
Q9. What should you do with the references during the first pass?
- A. Read every referenced paper before continuing
- B. Glance over them, mentally ticking off the ones you have already read
- C. Ignore them entirely until the third pass
- D. Count them to judge the paper's quality
Q10. At the end of the first pass you should be able to answer the five Cs. What are they?
- A. Cost, Citation, Clarity, Conclusion, Contribution
- B. Context, Content, Correctness, Credibility, Conclusion
- C. Category, Context, Correctness, Contributions, Clarity
- D. Category, Citation, Content, Comparison, Clarity
Q11. Under the five Cs, the question "What type of paper is this? A measurement paper? An analysis of an existing system? A description of a research prototype?" belongs to which C?
- A. Category
- B. Context
- C. Clarity
- D. Contributions
Q12. Which of these is a reason Keshav gives for choosing NOT to read further after the first pass?
- A. The paper was published more than a year ago
- B. The paper has too many figures
- C. The authors make invalid assumptions
- D. The paper is written by a single author
Q13. For which kind of paper does Keshav say the first pass alone is adequate?
- A. Papers you are formally reviewing for a conference
- B. Papers that form the core of your thesis
- C. Papers you intend to re-implement
- D. Papers that aren't in your research area, but may someday prove relevant
Q14. Knowing that most reviewers make only one pass, what writing advice does Keshav give?
- A. Choose coherent section and sub-section titles and write concise, comprehensive abstracts
- B. Place the most important results only in the appendix
- C. Use as many technical acronyms as possible
- D. Write a very long introduction to impress the reviewer
Q15. What does Keshav say happens if a reviewer cannot understand the gist after one pass, or a reader cannot understand the highlights after five minutes?
- A. The reviewer will request a major revision
- B. The paper will likely be rejected, and will likely never be read, respectively
- C. The paper will be forwarded to another reviewer
- D. The paper will be accepted but not cited
Q16. In the second pass, what should you ignore, and what technique does Keshav recommend?
- A. Ignore the figures; recommend reading the conclusion twice
- B. Ignore the abstract; recommend memorising the references
- C. Ignore details such as proofs; jot down key points or make comments in the margins
- D. Ignore the related work; recommend skipping to the third pass
Q17. When examining graphs in the second pass, what should you check for?
- A. Whether the colours match the journal's style guide
- B. Whether the axes are properly labeled and results are shown with error bars so conclusions are statistically significant
- C. Whether the graphs were drawn with professional software
- D. Whether the graphs appear before the tables
Q18. What should you do with references during the second pass?
- A. Mark relevant unread references for further reading
- B. Delete references older than ten years from your notes
- C. Read every reference in full before continuing
- D. Ignore the reference list completely
Q19. How long should the second pass take?
- A. Five to ten minutes
- B. Up to an hour
- C. Two to three hours
- D. Four to five hours
Q20. What should you be able to do after completing the second pass?
- A. Re-implement the paper's system from memory
- B. List every citation in the bibliography
- C. Summarize the main thrust of the paper, with supporting evidence, to someone else
- D. Identify every implicit assumption the authors made
Q21. The level of detail reached by the second pass is described as appropriate for which kind of paper?
- A. A paper you are interested in, but which does not lie in your research speciality
- B. A paper you have been asked to formally review
- C. A paper completely outside your field of interest
- D. A paper you plan to extend in your own thesis
Q22. Which of the following is NOT listed as a reason you might still fail to understand a paper after the second pass?
- A. The subject matter is new to you, with unfamiliar terminology and acronyms
- B. The paper is poorly written with unsubstantiated assertions and numerous forward references
- C. The paper has been published in a top-tier conference
- D. It is late at night and you are tired
Q23. If you still do not understand the paper after the second pass, what three choices does Keshav offer?
- A. Email the authors, post on a forum, or abandon the field
- B. Re-read the abstract, re-read the conclusion, or re-read the figures
- C. Ask a supervisor, ask a classmate, or ask a librarian
- D. Set the paper aside, return to it later after reading background material, or persevere and go on to the third pass
Q24. What is the key to the third pass?
- A. To attempt to virtually re-implement the paper, making the same assumptions as the authors and re-creating the work
- B. To rewrite the paper in your own words for publication
- C. To translate the paper into your own language
- D. To read every cited reference in full
Q25. During the third pass, what attitude should you take toward the paper's statements?
- A. Accept the authors' assumptions, since the paper was peer-reviewed
- B. Identify and challenge every assumption in every statement
- C. Focus only on the conclusions and skip the body
- D. Compare the paper only against your own previous work
Q26. How long does the third pass take for beginners compared with experienced readers?
- A. One hour for beginners and five minutes for experienced readers
- B. Two hours for beginners and thirty minutes for experienced readers
- C. About four or five hours for beginners and about an hour for an experienced reader
- D. The same amount of time for both
Q27. At the end of the third pass, what should you be able to do?
- A. Recite the abstract word for word
- B. Decide whether to read the paper at all
- C. List the paper's references in alphabetical order
- D. Reconstruct the entire structure of the paper from memory and identify its strong and weak points
Q28. What is the first step of using the three-pass approach to do a literature survey?
- A. Use an academic search engine such as Google Scholar or CiteSeer with well-chosen keywords to find three to five recent papers, do one pass on each, and read their related work sections
- B. Read the five most cited papers of all time in the field
- C. Attend the top conference in the field
- D. Write a draft survey from memory and then verify it
Q29. In the second step of the literature survey, what identifies the key papers and researchers in an area?
- A. The papers with the longest reference lists
- B. Shared citations and repeated author names in the bibliography
- C. The papers published most recently
- D. The papers with the most figures and tables
Q30. What is the third step of the literature survey method, and what do you do with the papers it produces?
- A. Email the key researchers and ask them for a reading list, then do one pass on each reply
- B. Go to the websites of the top conferences, scan their recent proceedings for high-quality related work, and make two passes through the resulting set of papers
- C. Read every paper in the field in full, starting with the oldest
- D. Publish a preliminary survey and wait for reviewer feedback